# Staff Software Engineer - Data Platform

**Company:** [Idme](https://hotfix.jobs/companies/idme)
**Location:** Mountain View, CA, McLean, VA
**Role:** Data Engineering
**Salary:** $218k – $260k/yr
**Experience:** 8+ years
**Skills:** Go, Python, Java, Postgres, Kafka, Google Pub/Sub, Dataflow, Temporal, Airflow, Terraform, Kubernetes, Helm, AWS, GCP
**Posted:** 2026-05-05

> Leads design, build, and operation of core data infrastructure including event streaming, relational databases, and batch processing systems. Ensures scalability, reliability, and compliance while mentoring engineers and optimizing pipelines for analytics and ML.

## Job Description

## Responsibilities
- Own and operate core data infrastructure, including event streaming, relational database, and batch processing platforms.
- Design and implement highly reliable, observable, and scalable data systems that enable real-time and batch data processing.
- Develop automation and guardrails for data governance, retention, and compliance, ensuring auditability and consistency across services.
- Partner with application, platform, and SRE teams to improve data access patterns, reliability SLAs, and recovery processes.
- Establish standards for data infrastructure monitoring, alerting, and capacity planning, ensuring proactive issue detection.
- Drive operational excellence by improving resilience, reducing toil, and implementing self-healing or automated recovery mechanisms.
- Evolve and optimize data pipelines that support downstream analytics, identity verification, and machine learning systems.
- Evaluate, implement, and operate event-driven and batch data platforms such as Kafka, Google Pub/Sub, Dataflow, or Temporal.
- Lead incident response and root cause analysis for production data systems, contributing to postmortems and platform improvements.
- Mentor engineers and advocate for reliability-focused engineering culture across teams.
- Design and build the data lake storage and compute topology (object storage, partitioning, lifecycle, tiering) to support batch and streaming workloads.

## Minimum Qualifications
- Bachelor's or Graduate degree in Computer Science, Software Engineering, or a related technical field.
- 8+ years of professional experience in data engineering, software engineering, or distributed systems development.
- 6+ years of programming experience in one or more languages such as **Go**, **Python**, or **Java**, with emphasis on automation and data system integration.

## Preferred Qualifications
- Deep expertise in building and operating data systems—including relational databases, streaming, and batch platforms—in production environments.
- Hands-on experience administering and optimizing **PostgreSQL** or other relational databases in the cloud (**AWS RDS**, **CloudSQL**, or **AlloyDB**).
- Solid understanding of reliability engineering principles, including observability, **SLOs**, capacity management, and operational readiness.
- Experience managing cloud infrastructure (**AWS** or **GCP**) using infrastructure-as-code tools like **Terraform**, **Kubernetes**, or **Helm**.
- Experience operating event streaming platforms such as **Kafka** or **Google Pub/Sub**.
- Experience with batch and stream processing systems, including **Dataflow**, **Temporal**, or **Airflow**.
- Strong knowledge of data pipeline orchestration, change data capture, and schema management.
- Background in automation, incident response, and data platform observability.
- Familiarity with data governance and regulatory compliance frameworks (e.g., **FedRAMP**, **GDPR**, **NIST**).
- Contributions to open-source data infrastructure projects or strong community engagement in the data reliability space.
- Passion for performance engineering, system design, and mentoring others to deliver operational excellence at scale.
- AI-assisted development — Demonstrable experience using AI developer tools (e.g., code generation, test generation, query synthesis).
- Data-aware LLM usage — Ability to safely use large language models for tasks such as SQL generation, data lineage summarization, and runbook drafting.

## Similar roles

- [Sr. Staff Software Engineer - Unity Catalog Data Governance](https://hotfix.jobs/jobs/db9ddfa4-c0eb-4240-a845-9c775b4a5fef) - Databricks - Bellevue, WA - $217k – $312k/yr
- [Member Of Technical Staff](https://hotfix.jobs/jobs/f24e3fa6-e241-415c-9654-d96b03a4d2c9) - Perplexity - San Francisco, CA - $220k – $405k/yr
- [Staff Data Engineer](https://hotfix.jobs/jobs/2bd10078-9817-42e6-ba4e-be0a897ec441) - Upside - Remote - $215k – $250k/yr
- [Staff Software Engineer, Data Governance & Foundations](https://hotfix.jobs/jobs/25ee533b-89ae-4df8-9a50-ebe183b36b5f) - Instacart - Remote - $221k – $280k/yr
- [Staff Software Engineer, Ads Data Platform](https://hotfix.jobs/jobs/f66ebc13-2514-4665-a55e-89645c54da3b) - Instacart - Remote - $221k – $280k/yr

**Apply:** https://hotfix.jobs/jobs/f4656882-92bc-4d93-bc0f-a3396f044dd1
**Canonical:** https://hotfix.jobs/jobs/f4656882-92bc-4d93-bc0f-a3396f044dd1