AnyscaleSan Francisco, CABuild, optimize, and scale Ray Data, a Python-native data processing engine for AI/ML workloads. Improve performance, ensure fault tolerance at high scale, and support production training and multi-modal batch inference for AI-native companies.
Designs and scales distributed data infrastructure for large-scale multimodal training and evaluation at OpenAI. Collaborates with researchers to build reliable, high-performance systems handling massive data volumes in a fast-paced environment.
Build and manage data pipelines and canonical datasets for product metrics, safety systems, and business decisions. Collaborate with cross-functional teams including Data Science and Research; requires 3+ years data engineering experience with Spark, ETL tools, and distributed systems.
As a Data Engineering Manager, you will lead the Data & ML Platform team, owning platforms for analytics, experimentation, and machine learning. You will guide the evolution towards a streaming-first, ML-ready architecture and partner with Data Science to operationalize models.
Software Engineer building scalable data infrastructure, cataloging, versioning, and lineage tools to support ML research and production workflows at an AI-driven hedge fund. Requires 3+ years experience, strong software design skills, and expertise in a modern language like Python or Java.
Designs and implements dataset infrastructure for OpenAI's large-scale LLM training stack, including standardized APIs for multimodal data, scaling pipelines across GPU fleets, and performance debugging. Requires strong distributed systems experience and collaboration with researchers.