Software Engineer, Research Data Platform
Build data pipelines, APIs, libraries, and web interfaces that help AI researchers manage, query, and analyze training and evaluation data. The role requires significant software engineering experience and close collaboration with technical users; experience with large-scale data systems and ML tooling is advantageous.
About the job
Responsibilities
- Build and operate data pipelines that extract data from research training runs and land it in storage systems that are easy and fast to query.
- Work closely with researchers to design and build APIs, libraries, and web interfaces that support data management, exploration, and analysis.
- Develop dataset management, data cataloging, and provenance tooling for researchers’ day-to-day work.
- Embed with research teams to understand workflows, identify high-leverage tooling opportunities, and ship solutions quickly.
- Collaborate with adjacent teams to build on existing systems rather than reinventing them.
Requirements
- Significant software engineering experience, particularly building data-intensive applications or internal tooling.
- Enjoy working directly with users, gathering requirements iteratively, and shipping adopted tools.
- Results-oriented, flexible, and impact-focused.
- Willingness to learn more about machine learning research.
- Care about the societal impacts of the work.
Nice-to-haves
- Large-scale ETL, columnar storage formats, and query engines such as Spark, BigQuery, or DuckDB.
- High-volume time-series data ingestion, storage, and efficient querying.
- Data cataloging, lineage, or metadata management systems.
- ML experiment tracking or metrics platforms.
- Experience partnering closely with quantitative users in research labs, trading firms, observability companies, or analytics startups.
- Complex data visualization and full-stack web application development.
Compensation and Benefits
- Annual salary: $320,000–$405,000 USD.
- Minimum education: bachelor’s degree or an equivalent combination of education, training, and experience.
- Competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office collaboration space.
- Hybrid policy: staff are expected to work from an office at least 25% of the time; some roles may require more office time.
- Visa sponsorship may be available depending on the role and candidate.
Skills
Python, ETL, Spark, BigQuery, Duckdb, Parquet, Columnar Storage, Time-Series Data, Data Cataloging, Data Lineage, Ml Experiment Tracking, Data Visualization, Web Application Development, APIs, Query Engines
Similar jobs
Fullstack Engineering jobs
Thinking Machines Lab
San Francisco, CA
Build and ship AI-powered web products end to end, translating emerging model capabilities into user experiences while owning architecture and collaborating with research, product, and design teams. Requires 5+ years of full-stack software engineering experience.
OpenAI
San Francisco, CA
Build and ship net-new full-stack product experiences that turn emerging model capabilities into impactful user products. The role requires strong product intuition, autonomy, rapid prototyping, and effective cross-functional collaboration in ambiguous environments.
OpenAI
San Francisco, CA
Builds zero-to-one, AI-native financial products across the frontend and backend, including enterprise memory, APIs, integrations, and professional workflows. The role requires 5+ years of software engineering experience, TypeScript and React expertise, backend development skills, and strong product judgment.
AfterQuery
San Francisco, CA
Build full-stack product systems for expert acquisition, onboarding, matching, activation, and retention. The role requires 3–5 years of software engineering experience, strong product judgment, and the ability to ship measurable improvements with cross-functional partners.
Runway
United States
Build full-stack AI-powered video creation and collaboration features, including rich interfaces, backend APIs, LLM experiences, and production systems. The role requires 5+ years of software engineering experience, strong JavaScript or TypeScript fluency, and experience integrating third-party AI APIs.