# Senior Data Engineer

**Company:** [Socure](https://hotfix.jobs/companies/socure)
**Location:** Remote
**Role:** Data Engineering
**Salary:** $160k – $195k/yr
**Experience:** 5+ years
**Skills:** Python, Scala, SQL, Spark, AWS, Emr, AWS Lambda, S3, Kafka, Kinesis, Airflow, Step Functions, CI/CD, Generative AI, LLMs
**Posted:** 2026-03-26

> Designs and builds scalable batch/streaming data pipelines for identity verification products, owning end-to-end data initiatives using cloud-native tech. Requires 5+ years data engineering with Spark, AWS, Python/SQL; streaming/orchestration experience preferred.

## Job Description

## What You'll Do
- Design and build batch and streaming data pipelines to support automated data ingestion, ML feature engineering and analytics across multiple product domains.
- Own end-to-end delivery of complex, ambiguous data initiatives, including architecture, implementation, testing, deployment, monitoring, and documentation.
- Develop and evolve the data platform to support large-scale data processing using modern cloud-native technologies.
- Automate data operations (validation, quality checks, alerting, backfills, and recovery workflows) to reduce manual effort and improve consistency.
- Optimize cost, performance, and reliability of data workloads.
- Partner closely with cross-functional teams (Data Science, Product, Engineering) to understand requirements, translate them into technical solutions.
- Evaluate and adopt new technologies (new processing engines, storage formats, orchestration tools, GenAI-assisted ingestion) to keep the platform modern and efficient.

## What You Bring
- 5+ years of hands-on data engineering experience, building and maintaining production-grade data platforms and pipelines.
- Strong programming skills in general-purpose language (such as **Python** or **Scala**) for data processing, and **SQL** for data analytics.
- Deep experience with distributed data processing frameworks, such as **Apache Spark**, including performance tuning and optimization.
- Proven experience building data solutions using services on **AWS** (**EMR**, **Lambda**, **S3**, etc).
- Strong understanding of data modeling and data warehousing concepts, including partitioning, schema design for large-scale datasets.
- Experience operating and supporting production pipelines, including monitoring, alerting, incident response, and improving reliability over time.
- Solid foundation in software engineering practices, including version control, **CI/CD**, testing strategies, and code review.
- Strong communication and collaboration skills, with the ability to work effectively with both technical and non-technical stakeholders.

## Preferred Qualifications
- Experience with streaming or near-real-time data processing (**Kafka**, **Kinesis**, etc).
- Hands-on experience with data orchestration tools (**Airflow**, **Step Functions**, etc).
- Familiarity with modern data platform patterns such as Data Lakehouse, Data Mesh, and large-scale data sharing across teams.
- Experience with prompt engineering using modern **GenAI**, **Large Language Models (LLM)**.
- Experience mentoring other engineers and contributing to engineering-wide standards, best practices.

## Similar roles

- [Senior Data Engineer](https://hotfix.jobs/jobs/64a0cd61-061e-499f-b50f-9cf9ecba5702) - Metropolis - New York, NY - $160k – $220k/yr
- [Senior Software Engineer, Data Platform](https://hotfix.jobs/jobs/ef60f693-15c8-46f7-906d-1fe09ef13a24) - Lithic - Remote - $160k – $200k/yr
- [Senior Data Engineer](https://hotfix.jobs/jobs/f3005c82-d257-45c0-94eb-8947692f4f6d) - Together AI - San Francisco, CA - $160k – $240k/yr
- [Senior Backend Engineer, Data Platform](https://hotfix.jobs/jobs/ee2e4160-f027-4d0c-8479-8ffbdad3d4eb) - Launchdarkly - Remote - $158k – $257k/yr
- [Senior Software Engineer- Big Data & Java](https://hotfix.jobs/jobs/f96e3c65-24b9-4991-81b8-1cb20b410fd9) - PointClickCare - Remote - $158k – $176k/yr

**Apply:** https://hotfix.jobs/jobs/30334333-414f-465c-99b1-0018d78648e0
**Canonical:** https://hotfix.jobs/jobs/30334333-414f-465c-99b1-0018d78648e0