Skip to content
RedditRedditUnited States

Senior Staff Software Engineer, Indexing & Retrieval Platform

Leads development and scaling of ML indexing and retrieval infrastructure for Reddit's recommendation systems, integrating lexical/vector search and GenAI. Requires 10+ years in software engineering with 3+ in technical leadership, expertise in distributed systems, stream processing, and cloud-native architectures.

279k – 391k
Remote10+ YOEData Engineering

About the role

What You’ll Do

  • Lead the technical strategy, architecture, and implementation of Reddit’s next-generation ML Indexing & Retrieval engine, integrating capabilities across lexical and vector indexing, low-latency retrieval, and emerging GenAI applications.
  • Partner closely with product engineers across Content Understanding, Search, Feeds, Ads, Growth, and Safety to deliver high-quality experiences.
  • Define best practices for observability, reliability, and operational excellence in large-scale distributed systems.
  • Mentor and guide engineers in designing scalable infrastructure and adopting robust DevOps and SRE principles.
  • Collaborate with infrastructure, and ML teams to ensure the platform evolves to meet the needs of Reddit’s growing user base and diverse content ecosystem.

Who You Might Be

  • 10+ years of experience in software engineering, specializing in Indexing and Retrieval systems.
  • 3+ years in technical leadership, architecting and scaling distributed systems in production environments.
  • Deep expertise in large-scale data platforms, including batch indexing and stream processing.
  • Proven experience designing and operating large-scale, low-latency retrieval services.
  • Expertise in lexical and vector search retrieval technologies, such as Milvus, Vespa, or Elasticsearch.
  • Skilled in designing cloud-native architectures and managing containerized workloads using Kubernetes and AWS/GCP.
  • Adept at translating complex technical challenges into clear, actionable strategies.
  • Strong communicator and mentor who leads through collaboration, influence, and technical excellence.

Languages: Go, Java, Python, or any object oriented programming language Frameworks: Flink, Airflow, Spark for large scale batch & stream processing Databases: Familiarity with Vector, Lexical & Key-Value Databases Tools: Kubernetes, Docker, AWS, GCP

Skills

GoJavaPythonFlinkAirflowSparkKubernetesDockerAWSGCPMilvusVespaElasticsearch
Runway

Member of Technical Staff, Research Engineer (Datasets)

RunwayUnited States

Research Engineer owning datasets for training world simulation AI models, designing multimodal datasets, running experiments, and building data pipelines to enhance model capabilities across tasks like robotics and creative tools. Requires 4+ years in ML with experience in generative models and frameworks like PyTorch or JAX.

270k – 370k
Remote4+ YOEData Engineering
Scale AI

Staff Software Engineer, Data Platform

Scale AISan Francisco, CA +2

Leads architecture and development of large-scale data platforms for AI, including storage, streaming, caching, and indexing. Requires 8+ years experience with databases, streaming tools, Kubernetes, and distributed systems.

248k – 311k
Hybrid8+ YOEData Engineering
Airbnb

Senior Staff Data Engineer

AirbnbUnited States

Lead multi-year vision and architecture for data governance and quality at scale. Define best practices, tooling, and culture while coaching senior engineers and influencing executive strategy on compliance and data integrity.

248k – 310k
Remote12+ YOEData Engineering
Discord

Staff Data Engineer, Ads

DiscordUnited States

Discord is seeking a Staff Data Engineer to lead technical vision and strategy for ads data infrastructure. This role involves building and maintaining sophisticated data pipelines, datasets, and analytical tools, and mentoring other engineers.

248k – 279k
Remote7+ YOEData Engineering
Discord

Staff Data Engineer, Analytics

DiscordSan Francisco, CA

Leads technical strategy for analytics data infrastructure, owns metric definitions, builds curated datasets and data models using SQL/Python, implements quality monitoring, and collaborates cross-functionally. Requires 7+ years in analytics/data engineering.

248k – 279k
On-site7+ YOEData Engineering