Senior Platform Engineer (DataOps)
Builds and maintains enterprise DataOps platform using Kubernetes, cloud services, and data processing tools. Requires 7+ years experience, strong coding skills, and expertise in containerization, GCP/AWS/Azure, Kafka, and Airflow.
About the job
Duties & Responsibilities
The DataOps Platforms team:
- Develops and continuously improves DataOps platform.
- Develops and maintains common tools and libraries.
- Evaluates new technologies and practices.
- Helps NCBI developers with adoption of platform.
- Ensures compliance with the Federal application security regulations and standards by providing automated solutions and compliance pipelines.
- Embraces agile development and continuous improvement.
- Encourages growth mindset and offers leadership opportunities at any level.
Required Skills
- 7+ years of experience in the field
- Strong coding skills in at least one programming language (Python, C++, ...)
- Kubernetes, containerization
- Google Cloud Platform (GCP), Amazon Web Services (AWS), Microsoft Azure or equivalent cloud services
- Apache Kafka, Google Cloud Pub/Sub or equivalent
- Apache AirFlow or equivalent
- Experience with data processing applications and modern cloud-based data processing infrastructure
- Linux command-line skills
Bonus Skills
- Google Anthos
- Docker
- GitOps tools: ArgoCD or equivalent
- Infrastructure as code tools: Terraform or equivalent
- GitLab, GitHub, Bitbucket, Teamcity, Artifactory, or equivalent products for management of Git source control, CI/CD pipelines and artifact lifecycle management
- Modern observability and logging tools: Prometheus, EFK (ElasticSearch, fluentd, Kibana), TIGK (Telegraf, InfluxDB, Grafana, Kapacitor), DataDog, Sensu, Jaeger, Sentry, OpsGenie, PagerDuty, Splunk, or equivalent
- Secret Management tools such as Hashicorp Vault, CyberArk, Azure Key Vault, Google Cloud Secret Manager or equivalent
- Data transfer tools: AWS DataSync, Aspera, MinIO, CloudSoda or equivalent
- Apache Pulsar, RabbitMQ, Amazon Kinesis, Apache Flume, Apache Storm, Apache Spark Streaming, Google Cloud Pub/Sub
- Experience with best-practice design patterns in coding and architecture
- Experience working in Agile environment
Skills
Kubernetes, Docker, GCP, Amazon Web Services, Microsoft Azure, Apache Kafka, Google Cloud Pub/Sub, Apache Airflow, Terraform, Argo CD, Prometheus, Grafana, Hashicorp Vault, Python, Linux
Similar jobs
Data Engineering jobs
Addepar
United States
Staff Software Engineer building scalable frameworks for high-performance financial data ingestion/distribution and AI-native products. Requires 7+ years experience with distributed systems, microservices, and data architectures; partners with product teams to drive technical direction.
Alpaca
Remote
Build and operate scalable lakehouse infrastructure, streaming and CDC pipelines, query systems, and self-serve BI capabilities. Requires 5+ years of data engineering experience, strong Kubernetes and infrastructure-as-code expertise, and hands-on experience with distributed data platforms.
NinjaTrader
Chicago, IL
Leads database architecture, performance, reliability, and developer-tooling initiatives for high-volume trading applications. Requires 8+ years of software engineering experience, expert MySQL skills, backend development expertise, and strong knowledge of distributed systems and database operations.
Deepgram
United States
Own the company’s metric governance program by defining canonical metrics, enforcing them in semantic and catalog systems, improving data quality, and validating AI-agent outputs. Requires 5+ years in analytics or analytics engineering, strong SQL, production semantic-layer ownership, and experience with AI evaluation and data governance.
Aleph
United States
Own and scale transformation pipelines that convert diverse financial and operational data into reliable FP&A-ready models. The role requires strong SQL and dbt expertise, data integrity and performance skills, and effective collaboration across Engineering and Customer Success.