OpenAISan Francisco, CADesigns and scales distributed data infrastructure for large-scale multimodal training and evaluation at OpenAI. Collaborates with researchers to build reliable, high-performance systems handling massive data volumes in a fast-paced environment.
Build and manage data pipelines and canonical datasets for product metrics, safety systems, and business decisions. Collaborate with cross-functional teams including Data Science and Research; requires 3+ years data engineering experience with Spark, ETL tools, and distributed systems.
Build, optimize, and scale Ray Data, a Python-native data processing engine for AI/ML workloads. Improve performance, ensure fault tolerance at high scale, and support production training and multi-modal batch inference for AI-native companies.
Software Engineer building scalable data infrastructure, cataloging, versioning, and lineage tools to support ML research and production workflows at an AI-driven hedge fund. Requires 3+ years experience, strong software design skills, and expertise in a modern language like Python or Java.
As a Data Engineering Manager, you will lead the Data & ML Platform team, owning platforms for analytics, experimentation, and machine learning. You will guide the evolution towards a streaming-first, ML-ready architecture and partner with Data Science to operationalize models.
Designs and implements dataset infrastructure for OpenAI's large-scale LLM training stack, including standardized APIs for multimodal data, scaling pipelines across GPU fleets, and performance debugging. Requires strong distributed systems experience and collaboration with researchers.