Skip to content
Black Canyon ConsultingBlack Canyon ConsultingBethesda, MD

Senior Platform Engineer (DataOps)

Builds and maintains enterprise DataOps platform using Kubernetes, cloud services, and data processing tools. Requires 7+ years experience, strong coding skills, and expertise in containerization, GCP/AWS/Azure, Kafka, and Airflow.

Salary not listed
Hybrid7+ YOEData Engineering

About the role

Duties & Responsibilities

The DataOps Platforms team:

  • Develops and continuously improves DataOps platform.
  • Develops and maintains common tools and libraries.
  • Evaluates new technologies and practices.
  • Helps NCBI developers with adoption of platform.
  • Ensures compliance with the Federal application security regulations and standards by providing automated solutions and compliance pipelines.
  • Embraces agile development and continuous improvement.
  • Encourages growth mindset and offers leadership opportunities at any level.

Required Skills

  • 7+ years of experience in the field
  • Strong coding skills in at least one programming language (Python, C++, ...)
  • Kubernetes, containerization
  • Google Cloud Platform (GCP), Amazon Web Services (AWS), Microsoft Azure or equivalent cloud services
  • Apache Kafka, Google Cloud Pub/Sub or equivalent
  • Apache AirFlow or equivalent
  • Experience with data processing applications and modern cloud-based data processing infrastructure
  • Linux command-line skills

Bonus Skills

  • Google Anthos
  • Docker
  • GitOps tools: ArgoCD or equivalent
  • Infrastructure as code tools: Terraform or equivalent
  • GitLab, GitHub, Bitbucket, Teamcity, Artifactory, or equivalent products for management of Git source control, CI/CD pipelines and artifact lifecycle management
  • Modern observability and logging tools: Prometheus, EFK (ElasticSearch, fluentd, Kibana), TIGK (Telegraf, InfluxDB, Grafana, Kapacitor), DataDog, Sensu, Jaeger, Sentry, OpsGenie, PagerDuty, Splunk, or equivalent
  • Secret Management tools such as Hashicorp Vault, CyberArk, Azure Key Vault, Google Cloud Secret Manager or equivalent
  • Data transfer tools: AWS DataSync, Aspera, MinIO, CloudSoda or equivalent
  • Apache Pulsar, RabbitMQ, Amazon Kinesis, Apache Flume, Apache Storm, Apache Spark Streaming, Google Cloud Pub/Sub
  • Experience with best-practice design patterns in coding and architecture
  • Experience working in Agile environment

Skills

KubernetesDockerGCPAmazon Web ServicesMicrosoft AzureApache KafkaGoogle Cloud Pub/SubApache AirflowTerraformArgo CDPrometheusGrafanaHashicorp VaultPythonLinux
HappyRobot

Data Operations Lead

HappyRobotSan Francisco, CA

Lead data operations for AI audio models at HappyRobot. Translate research needs into data specs, manage end-to-end acquisition/annotation with internal and vendor teams, optimize tooling/workflows, and scale labeling operations to deliver high-quality datasets on time.

180k – 240k/yr
Hybrid5+ YOEData Engineering
Ambient.ai

Senior Software Engineer, AI Data Systems & Database Infrastructure

Ambient.aiRedwood City, CA

Senior Platform Engineer designing, building, and scaling database infrastructure for production and AI systems, including relational, analytical, and vector stores. Requires 7+ years experience with distributed systems, high-availability databases, and supporting AI workloads like vector search and RAG.

168k – 205k/yr
Hybrid7+ YOEData Engineering
Vanta

Senior Data Engineer

VantaUnited States

Senior Data Engineer responsible for designing and deploying scalable data infrastructure, orchestration models, and analytics tooling to enable data-driven decisions, ML products, and enterprise reporting at Vanta. Requires 4+ years data experience, software engineering mindset, modern data stack proficiency, and passion for secure, compliant data systems.

190k – 224k/yr
Remote4+ YOEData Engineering
Cloudflare

Senior Data Engineer

CloudflareAustin, TX

Senior Data Engineer building scalable data pipelines, services, and AI-ready data layers in Go/Scala/ClickHouse to power internal products, analytics, and agentic AI for go-to-market, engineering, and product teams. Requires 5+ years experience in production data systems, strong programming, SQL, and databases.

Salary not listed
On-site5+ YOEData Engineering
Trexquant

Senior Data Engineer

TrexquantStamford, CT +1

Senior Data Engineer responsible for building scalable ingestion pipelines, normalizing, and maintaining large-scale financial and alternative datasets from global vendors to support quantitative research and alpha generation. Requires 5+ years data engineering experience in finance/quant environments, strong Python/SQL/Linux skills, and deep knowledge of market/tick/reference data across asset classes.

150k – 200k/yr
On-site5+ YOEData Engineering