Software Engineer
Build and optimize Ray Data, a Python-native data processing engine for large-scale AI workloads. The role focuses on distributed systems performance, scalable data pipelines, production training solutions, and fault tolerance while partnering with AI-focused customers.
About the job
Responsibilities
- Improve the performance of Ray Data and multimodal batch inference use cases.
- Ensure efficient scaling across different stages of data pipelines in heterogeneous environments.
- Build data-loading solutions for production training workloads.
- Focus on stability and fault tolerance at high scale.
- Work with customers and AI-native companies to scale their AI workloads.
Requirements
- 3–4 years of relevant work experience.
- Strong background building scalable and fault-tolerant distributed systems.
- Experience with data processing and database internals.
- Passion for large-scale systems and AI performance.
Compensation
- Annual salary: $215,000–$230,000.
Skills
Python, Ray, Ray Data, Distributed Systems, Data Processing, Database Internals, Batch Inference, Machine Learning, Ai Workloads, Fault Tolerance
Similar jobs
Data Engineering jobs
OpenAI
Mountain View, CABuild and operate scalable monetization data platforms, pipelines, models, and quality systems spanning product, financial, and operational data. The role partners with Product Engineering, Finance, Accounting, Analytics, and GTM teams to deliver reliable, observable data products.
Applied Intuition
Sunnyvale, CA
Builds scalable data pipelines and data engine architecture for machine learning, integrating foundation models to automate labeling and discovery. The role requires 5+ years of experience, modern ML infrastructure expertise, and U.S. citizenship with security-clearance eligibility.
The Voleon Group
New York, NYBuild and evolve reliable analytics infrastructure, pipelines, schemas, and foundational datasets supporting quantitative research across strategies. The role requires strong Python and SQL skills, distributed data-platform experience, and ownership of observability, performance, and reproducibility.
Abridge
San Francisco, CA
Builds and optimizes scalable data pipelines, storage, and OLAP databases for ML training, analytics, and product features. Requires 5+ years in data engineering, proficiency in Python/SQL/cloud platforms, and distributed systems experience.
Thinking Machines Lab
San Francisco, CA
Own end-to-end data sourcing and vendor operations that help researchers train and evaluate frontier AI models. The role requires strong judgment, communication, problem-solving, and comfort managing ambiguous, fast-changing projects.