Skip to content
The Voleon GroupThe Voleon GroupBerkeley, CA

Staff Software Engineer, Batch and Realtime Streaming

Architect and build Voleon’s batch and realtime streaming platform that powers ML research and production trading systems. Requires 10+ years experience building scalable data infrastructure with Python, Go, and distributed systems.

315k – 405k
Remote10+ YOEData Engineering

About the role

Responsibilities

  • Architect, design, and implement core components for a Batch and Realtime Streaming platform, including data ingestion pipelines, storage systems, serving layers, and API interfaces
  • Ship new features by collaborating across research, legal, trading, finance operations data, and infra teams for trading systems
  • Collaborate with ML researchers and data scientists to understand their workflows and design intuitive interfaces (APIs, SDKs, UIs) for seamless feature discovery, access, and reuse
  • Ensure data quality, consistency, and lineage for features, building robust mechanisms for versioning, monitoring, and governance
  • Optimize data pipelines and storage for high performance, scalability, and reliability, considering both batch and real-time use cases
  • Drive adoption of the feature store across teams by producing documentation, onboarding materials, and developer support
  • Mentor junior engineers and contribute to team best practices and technical excellence

Requirements

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field (or equivalent experience)
  • 10+ years of software engineering experience, with a strong background in distributed computing and data infrastructure
  • Experience in programming languages such as Python and Go
  • Understanding of database technologies (Postgres, MySQL, Cassandra, DuckDB) and experience with APIs (REST/gRPC)
  • Strong problem-solving skills, with a focus on delivering high-quality, maintainable, and well-documented solutions
  • Excellent communication and collaboration skills; ability to work closely with both engineering and research teams

Preferred Qualifications

  • Experience building large-scale data pipelines and storage systems (e.g., Airflow, Spark, Ray, Iceberg, etc)
  • Exposure to modern Python data science tooling (pandas, polars, dask, duckdb, etc)
  • Experience with monitoring and observability tools for distributed systems (e.g., Prometheus, Grafana, ELK Stack)
  • Prior experience working with feature stores (e.g., Feast, Hopsworks, or custom solutions)

Skills

PythonGoAirflowSparkRayIcebergPostgresMySQLCassandraDuckdbRestgRPCpandasPolarsDask

Similar roles

Data Engineering jobs
Anthropic

Staff+ Software Engineer, Capacity Engineering

AnthropicSan Francisco, CA +2

Build and operate production data pipelines, observability tools, and planning systems to maximize utilization, efficiency, and attribution of Anthropic's large-scale multi-cloud accelerator and CPU fleet. Requires strong Python/SQL, cloud operations, and Kubernetes experience in a high-ambiguity environment.

320k – 485k
Hybrid7+ YOEData Engineering
Anthropic

Staff+ Software Engineer, Databases

AnthropicSan Francisco, CA +2

Build and scale the core database infrastructure powering Claude at Anthropic, including data plane/control plane, data movement (CDC, migrations), and caching systems that support millions of users and frontier AI research across multi-cloud environments. Requires deep expertise in distributed databases and production storage systems.

320k – 485k
Hybrid10+ YOEData Engineering
Reddit

Senior Staff Software Engineer, Indexing & Retrieval Platform

RedditUnited States

Leads development and scaling of ML indexing and retrieval infrastructure for Reddit's recommendation systems, integrating lexical/vector search and GenAI. Requires 10+ years in software engineering with 3+ in technical leadership, expertise in distributed systems, stream processing, and cloud-native architectures.

279k – 391k
Remote10+ YOEData Engineering
Runway

Member of Technical Staff, Research Engineer (Datasets)

RunwayUnited States

Research Engineer owning datasets for training world simulation AI models, designing multimodal datasets, running experiments, and building data pipelines to enhance model capabilities across tasks like robotics and creative tools. Requires 4+ years in ML with experience in generative models and frameworks like PyTorch or JAX.

270k – 370k
Remote4+ YOEData Engineering
Scale AI

Staff Software Engineer, Data Platform

Scale AISan Francisco, CA +2

Leads architecture and development of large-scale data platforms for AI, including storage, streaming, caching, and indexing. Requires 8+ years experience with databases, streaming tools, Kubernetes, and distributed systems.

248k – 311k
Hybrid8+ YOEData Engineering