Staff Software Engineer, Data Platform
Leads architecture and development of large-scale data platforms for AI, including storage, streaming, caching, and indexing. Requires 8+ years experience with databases, streaming tools, Kubernetes, and distributed systems.
About the job
Responsibilities
- Drive the architecture, design, implementation, and reliability of foundational data platforms and systems, working closely with stakeholders.
- Collaborate with cross-functional teams to define, design, and deliver new features.
- Proactively identify opportunities for improvements to programming practices, processes, and tools.
- Present technical information to teams and stakeholders, providing guidance on development processes and technologies.
- Provide technical leadership, including upholding engineering standards and mentoring junior engineers.
Requirements
- 8+ years of full-time engineering experience in back-end systems, specifically large-scale data storage, streaming, and warehousing.
- Extensive experience with database technologies (MongoDB, Postgres), streaming/processing (Kinesis, Flink, Spark), indexing/caching (ElasticSearch, Redis), and query engines (Trino, Presto, Snowflake).
- Track record of mentoring and leading teams in successful projects.
- Excellent communication and collaboration skills to translate complex technical concepts.
- Experience with containerization & deployment technologies like Kubernetes and public cloud offerings.
- Deep understanding of distributed systems, cloud platforms, and data systems.
- Experience driving cross-functional collaboration.
Nice-to-Haves
- Strong knowledge of software engineering best practices and CI/CD tooling (CircleCI).
- Experience with performance tuning and cost optimizations of cloud-based data platforms.
- Experience defining data lifecycle strategy and tooling for data privacy (e.g., GDPR).
- Experience scaling products at hyper-growth startups.
- Excitement to work with AI technologies.
Skills
MongoDB, Postgres, Kinesis, Flink, Spark, Elasticsearch, Redis, Trino, Presto, Snowflake
Similar jobs
Data Engineering jobs
Anyscale
San Francisco, CA
Staff Software Engineer responsible for designing and scaling Ray Data’s distributed data-processing infrastructure for large-scale AI training and inference. Requires 6+ years of production software and architectural ownership experience, plus deep distributed-systems expertise and strong Python skills.
Haus
San Francisco, CAStaff-level engineer leading backend services and data-platform architecture, including large-scale ingestion, distributed systems, and trustworthy BigQuery/dbt warehouse models. Requires 10+ years of software engineering experience, expert Python, deep SQL/dbt expertise, and strong technical leadership.
Vercel
San Francisco, CAStaff Data Platform Engineer leading the architecture and development of financial data infrastructure for revenue reporting, billing, forecasting, and compliance. Requires 8+ years of data engineering or architecture experience, strong streaming and warehouse expertise, and the ability to mentor engineers and partner with Finance and Audit leaders.
Vanta
Remote
Leads the re-platforming of Vanta’s compliance data layer from MongoDB to schema-aware PostgreSQL across high-throughput Kafka and S3 pipelines. The role requires staff-level distributed systems expertise, migration leadership, and strong experience with relational and document data modeling.
Snowflake
Menlo Park, CA
Leads the design and operation of highly available distributed data platforms and pipelines at Snowflake, while providing technical leadership across teams. Requires 12+ years of distributed-systems experience, cloud expertise, and strong database and system-design depth.