# Principal Data Engineer

**Company:** [Octus](https://hotfix.jobs/companies/octus)
**Location:** Remote
**Role:** Data Engineering
**Experience:** 8+ years
**Skills:** Python, SQL, AWS, Mwaa, Apache Airflow, AWS Lambda, Amazon Ecs, Amazon Sqs, Aws Glue, Amazon S3, Amazon Redshift, Terraform, Snowflake, Databricks, Docker
**Posted:** 2026-08-24

> Leads the strategy, architecture, and hands-on development of scalable AWS data ingestion and transformation platforms. Requires expert Python and SQL skills, Terraform and cloud-native pipeline experience, and 8+ years in data engineering or backend development, with technical leadership responsibilities.

## Job Description

## Responsibilities
- Lead the technical strategy for the data platform and align infrastructure with business goals.
- Design and develop scalable, efficient, and reliable data ingestion and transformation pipelines across APIs, web data, and internal feeds.
- Provide technical leadership and mentorship to senior data platform engineers, including architecture, design, and implementation guidance.
- Architect and manage workflows using AWS MWAA (Airflow), Lambda, ECS, and SQS.
- Implement and maintain infrastructure as code with Terraform.
- Partner with data analysts, data scientists, and backend engineers to improve data consistency, discoverability, and reliability.
- Apply data modeling, schema design, and ETL/ELT best practices for high-volume structured and semi-structured data.
- Establish automated testing, monitoring, alerting, and lineage practices to ensure data quality.
- Promote code reviews, observability, continuous improvement, and knowledge sharing.
- Align data platform development with technology leadership, business strategy, and product goals.

## Requirements
- Strong software engineering foundations, including SOLID design, modularity, and scalability.
- Expert Python proficiency for data pipelines and automation.
- Advanced SQL skills, including complex query and data model optimization.
- Experience designing and maintaining cloud-native AWS data pipelines using services such as MWAA/Airflow, Lambda, ECS, SQS, Glue, S3, and Redshift.
- Experience with data warehousing or lakehouse technologies such as Redshift, Snowflake, or Databricks.
- Experience managing Terraform or similar infrastructure-as-code frameworks.
- Strong understanding of data ingestion, transformation, orchestration, and AI/ML pipeline patterns.
- Familiarity with CI/CD, automated testing, and modern DevOps practices.
- 8+ years of experience in data engineering or backend development focused on scalable data solutions.
- Technical leadership experience, including mentoring and leading complex data infrastructure projects end to end.
- Familiarity with Docker and workflow orchestration best practices.
- Excellent communication, collaboration, and problem-solving skills.

## Nice to Have
- Experience with streaming technologies such as Kafka, Kinesis, or Flink.
- Experience with data quality and observability tools such as Great Expectations or Monte Carlo.
- Familiarity with Scrapy, BeautifulSoup, or other data extraction frameworks.

## Compensation and Benefits
- Health benefits.
- Matched 401(k) and pension plans.
- Paid time off.
- Generous parental leave.
- Gym subsidies.
- Educational reimbursement for career development.
- Recognition programs.

## Similar jobs

- [Staff / Principal Data Engineer](https://hotfix.jobs/jobs/3cd01bf0-55d5-4091-8b73-777132dbbe4a) - Orum - Austin, TX
- [Senior Principal Software Engineer](https://hotfix.jobs/jobs/d4a33ad4-8c30-49d4-9b29-529e276693e2) - ZoomInfo - Remote - $164k – $257k/yr
- [Principal Engineer, Big Data Platform](https://hotfix.jobs/jobs/3d5442e1-f116-44d4-88dd-4ebf241f721a) - Pinterest - Remote - $243k – $500k/yr
- [Senior Staff Data Engineer](https://hotfix.jobs/jobs/29501416-f1fb-4cbb-9039-726cd3359598) - SmithRx - Remote - $179k – $252k/yr
- [Staff Software Engineer, Data Catalog](https://hotfix.jobs/jobs/ddc84115-d511-4356-a04b-4a4a1c821859) - Airbnb - Remote - $212k – $265k/yr

**Apply:** https://hotfix.jobs/jobs/77f1b622-25e9-4550-9a03-44d84d33938b
**Canonical:** https://hotfix.jobs/jobs/77f1b622-25e9-4550-9a03-44d84d33938b