# Staff Engineer, Data Platform

**Company:** [Shield AI](https://hotfix.jobs/companies/shield-ai)
**Location:** San Diego, CA
**Role:** Data Engineering
**Salary:** $150k – $230k/yr
**Experience:** 7+ years
**Skills:** Go, Python, Kubernetes, Linux, Networking, Distributed Systems, Data Modeling, API Design, Graph Databases, Object Storage, Terraform, Helm, GitOps, Apache Kafka, Openapi
**Posted:** 2026-08-26

> Leads the architecture and hands-on development of a knowledge-graph-centered data platform for AI and autonomy workflows. The role requires deep distributed data systems expertise, strong Go or Python engineering skills, and experience with storage, APIs, infrastructure, and production reliability.

## Job Description

## Responsibilities
- Architect and implement a knowledge graph and multimodal Graph API for human, service, and agentic workflows.
- Research, optimize, and maintain storage, indexing, query, ingestion, and compute infrastructure across the data lifecycle.
- Establish patterns for schema modeling, relationships, lineage, schema evolution, observability, security, reliability, disaster recovery, and lifecycle management.
- Build agent-access APIs for structured, connected, explainable context and develop reference architectures, deployment patterns, benchmarks, and operational guidance.
- Partner with autonomy, ML, test, infrastructure, product, and customer-facing teams to turn workflows into reusable platform capabilities.
- Deliver integrations for simulation, testing, training, and edge-device data; create self-service APIs, SDKs, tools, examples, and diagnostics.
- Evaluate emerging technologies and make build-versus-buy decisions while remaining hands-on in implementation and debugging.

## Requirements
- Significant experience designing and operating distributed data solutions, storage systems, or data-intensive backend services.
- Strong production software engineering experience with languages such as Go and Python.
- Deep understanding of data modeling, API design, schema evolution, identity, consistency, indexing, query planning, and data lifecycle management.
- Experience with relational or graph databases, object storage, analytical or columnar systems, and file storage.
- Experience designing reliable ingestion and access paths for high-volume or operationally important data.
- Strong understanding of Kubernetes, Linux, networking, security, storage, observability, and distributed systems.
- Experience deploying data infrastructure in cloud or customer-managed environments using Infrastructure as Code and platform engineering practices.
- Ability to evaluate technologies through prototypes, benchmarks, operational requirements, and lifecycle cost.
- Experience defining architecture and technical standards while implementing and debugging production systems.
- Ability to collaborate across ML, autonomy, test, platform, and product teams and communicate complex architecture clearly.

## Nice-to-haves
- Graph, OLAP, or other specialized modern databases; graph-backed retrieval; structured RAG; agent tooling; provenance-aware or explainable retrieval.
- S3-compatible APIs, cloud object storage, content-addressable storage, multipart transfer, and large-file lifecycle management.
- Apache Arrow, Parquet, columnar formats, time-series data, and high-performance analytical query systems.
- OpenAPI, AsyncAPI, WebSockets, generated SDKs, and long-lived public API contracts.
- Kubernetes storage and data operators, Terraform, Helm, GitOps, and repeatable platform distribution.
- Ray or other distributed execution technologies integrated with data lineage and artifact management.
- Kafka, NATS, Redpanda, or comparable event-driven ingestion systems.
- Robotics or AI data systems; ML data lifecycle systems; experiment tracking; dataset management; evaluation infrastructure; feature or artifact stores; model versioning.
- Observability, distributed tracing, benchmarking, data security, authorization, governance, retention, classification, and auditability.

## Similar jobs

- [Senior/Staff Analytics Engineer](https://hotfix.jobs/jobs/3aa8e105-4ff4-4272-b3fe-576a9bd1aa08) - Setpoint.io - Austin, TX - $150k – $170k/yr
- [Staff Data Engineer](https://hotfix.jobs/jobs/d8e6a779-ce3c-4f81-956a-c9daad6fab39) - DAT Freight & Analytics - Denver, CO - $134k – $175k/yr
- [Staff Software Engineer (L4) Data Platform](https://hotfix.jobs/jobs/700b0ec6-2fce-4997-8651-32d8f38bfdb9) - Twilio - Remote - $171k – $214k/yr
- [Staff Software Engineer, Workflow Platform](https://hotfix.jobs/jobs/01070583-7fc6-401d-a3d7-6063b42f2b26) - Pinterest - Remote - $177k – $365k/yr
- [Staff Software Engineer, Data Warehouse Foundation](https://hotfix.jobs/jobs/1c0e9e86-69f5-4dc2-bb6b-3897d5c99b75) - Pinterest - Remote - $177k – $365k/yr

**Apply:** https://hotfix.jobs/jobs/5ca414e4-1917-489d-a5cd-e9ccce552d3e
**Canonical:** https://hotfix.jobs/jobs/5ca414e4-1917-489d-a5cd-e9ccce552d3e