# Staff Data Engineer, Core Migrations

**Company:** [Machinify](https://hotfix.jobs/companies/machinify)
**Location:** Remote
**Role:** Data Engineering
**Salary:** $180k – $220k/yr
**Experience:** 8+ years
**Skills:** Python, SQL, Spark, Apache Airflow, AWS, S3, ETL, Ssis, T-Sql, Ai Coding Agents
**Posted:** 2026-06-30

> Staff Data Engineer owning end-to-end client data migrations at Machinify. Reverse-engineer legacy healthcare systems, architect and build Spark/Airflow pipelines, ensure data fidelity via automated reconciliation, and coordinate cross-functional teams from discovery through go-live. Requires 8+ years hands-on data engineering, strong Python/SQL/Spark/Airflow, and AI-assisted development experience.

## Job Description

## What You’ll Do

- Lead discovery & technical due diligence: analyze poorly documented legacy systems (ETL, stored procedures, reporting layers, file feeds), reconstruct business logic, and capture it in lineage maps, mapping specs, and risk analyses.
- Reverse-engineer complex legacy systems using agents: reconstruct intent from undocumented systems (SSIS packages, stored procedures) to build modern equivalents.
- Drive ambiguity to resolution: identify unknowns early, pull answers from clients, SMEs, and Operations.
- Architect & build migration pipelines: convert legacy logic into production-grade Airflow DAGs and Spark jobs, handling edge cases, payer-specific rules, from ingestion through reconciliation to handoff.
- Make and document architectural decisions: own pipeline design, partitioning strategy, validation approach, and document reasoning.
- Prove correctness at scale: build automated reconciliation frameworks to ensure migrated output matches source data.
- Own the program end-to-end: scope, sequence, track work; surface risks; align cross-functional teams and client; drive UAT to sign-off.
- Raise the bar for the practice: codify runbooks and retrospectives, mentor engineers, turn pain points into reusable tooling.

## What You Bring

- 8+ years as a hands-on Data Engineer or Software Engineer, independently owning complex, multi-stakeholder technical projects.
- Strong Python and SQL, including complex legacy code.
- Deep Apache Spark expertise: distributed processing, performance tuning, partitioning, debugging at scale.
- Advanced Apache Airflow: designing and authoring production DAG architectures from scratch.
- AI-assisted development: actively uses coding agents (Claude, Copilot, Cursor); prompt-engineering for code, debugging, documentation; critically evaluates AI output.
- Experience with legacy ETL stacks: reconstruct intent from SSIS, SQL Server stored procedures, T-SQL and translate to modern stack.
- Data validation & reconciliation at scale: designing automated frameworks for high-confidence data matching.
- Proficient with AWS: S3, partitioned object storage, secure cross-account access, cost/performance trade-offs.
- Cross-functional coordination across engineering, operations, and client-facing teams.
- Clear technical communication: writing discovery specs, architecture docs, stakeholder updates for technical and non-technical audiences.
- Data-first mindset with experience building automated reconciliation frameworks guaranteeing data fidelity.

## Similar roles

- [Staff Data Engineer, Corporate Engineering](https://hotfix.jobs/jobs/1145710f-0512-426f-b3dc-11fae8c5c550) - Reddit - Remote - $180k – $252k/yr
- [Staff Data Engineer](https://hotfix.jobs/jobs/2bec16ea-0906-42ff-b5d3-6e2a924385eb) - Komodo Health - Remote - $181k – $282k/yr
- [Staff Software Engineer, Batch Processing Platform](https://hotfix.jobs/jobs/9c400aa6-4baa-415a-9465-835083a27e99) - Pinterest - Remote - $177k – $365k/yr
- [Staff Data Engineer](https://hotfix.jobs/jobs/5ad59978-de00-44e6-bab9-07e1f9889b20) - NexHealth - San Francisco, CA - $177k – $226k/yr
- [Senior Staff Enterprise Architect, Data](https://hotfix.jobs/jobs/ffe430b2-4b1c-43db-b3b1-0e3750cf22c8) - MongoDB - Palo Alto, CA - $177k – $349k/yr

**Apply:** https://hotfix.jobs/jobs/6a217df9-0582-4916-a04b-ae102cfa5a33
**Canonical:** https://hotfix.jobs/jobs/6a217df9-0582-4916-a04b-ae102cfa5a33