Sr. Data Engineer
Design and maintain scalable data pipelines and lake architecture on GCP/AWS to power analytics, trading tools, and ML initiatives. Requires 5+ years experience, strong SQL/Python, dbt, orchestration tools, and cloud infrastructure experience.
About the job
Responsibilities
- Design, build, and maintain robust data pipelines and data lake architecture for both batch and real-time streaming use cases, including high-volume, low-latency data processing
- Improve observability, alerting, and SLOs across data systems
- Optimize ETL/ELT workflows for performance, scalability, and fault tolerance
- Develop dbt workflows to onboard Evaluation Partners and create end-of-day reporting
- Build and support event-driven architectures and scalable platform components
- Contribute to the orchestration and automation of workflows
- Integrate complex financial APIs and third-party data sources into internal systems
- Collaborate with analytics, product, and ML engineers to develop and deploy reliable data products
- Work on feature pipelines and model-ready data to support ML engineers
- Promote high standards in code quality, testing, and platform reliability
- Participate in Agile ceremonies
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field
- 5+ years of experience in data engineering, platform engineering, or backend development
- Strong skills in SQL and Python
- Hands-on experience with GCP and GCP data products (BigQuery, Cloud SQL, Cloud Storage)
- Experience with AWS cloud services (S3, Glue, Athena, Kinesis); bonus for EMR
- Experience with CI/CD pipelines, infrastructure-as-code, and version-controlled deployment workflows (e.g., Terraform, GitOps)
- Hands-on dbt experience building and maintaining dbt projects
- Proficiency with workflow orchestration tools (e.g., Airflow, Prefect)
- Knowledge of data lake architecture, including file formats (Parquet, Avro) and open table formats (Apache Iceberg)
- Familiarity with event-driven and service-oriented architecture
- Track record of building automated, well-tested, and observable data systems
- Comfortable working independently and collaboratively in a fast-paced Agile environment
Nice-to-Haves
- Hands-on Kubernetes experience, especially around data workloads
- Experience with streaming technologies (e.g., Kafka, Spark Streaming, Flink)
- Experience with change data capture tools (e.g., Debezium, Kafka Connect)
- Experience with BI tools like Looker Studio or QuickSight
- Experience with observability and monitoring tooling (e.g., Datadog, Grafana, Prometheus)
- Background in fintech, trading, or derivatives
Compensation & Benefits
- Salary range: $100,000 - $150,000 USD
- Annual target bonus of 10%
- 401k with up to 3.5% company match
- 18 days PTO per year + 7 paid holidays
- Health, Vision, Dental Coverage
- Life and Disability Insurance covered 100%
- Paid Parental Bonding Leave
Skills
SQL, Python, GCP, BigQuery, Cloud Sql, Cloud Storage, AWS, S3, Glue, Athena, Kinesis, Terraform, dbt, Airflow, Prefect
Similar jobs
Data Engineering jobs
SimplePractice
United States
The Senior Analytics Engineer will architect and operate marketing data infrastructure, productionize predictive models, and enable attribution, experimentation, and customer activation. The role requires strong Snowflake, dbt, Python, SQL, and Segment expertise, plus experience with marketing data and cross-functional analytics initiatives.
Twilio
United StatesBuilds agentic AI, automated data workflows, and BI solutions for complex telecommunications datasets. The role requires 5+ years of technical data and automation experience, strong SQL and Python skills, and expertise in data governance and LLM-based tools.
Aleph
United States
Own and scale transformation pipelines that convert diverse financial and operational data into reliable FP&A-ready models. The role requires strong SQL and dbt expertise, data integrity and performance skills, and effective collaboration across Engineering and Customer Success.
Shield AI
United States
Builds governed Silver and Gold data models, semantic assets, and reusable transformation patterns on Databricks for business domains such as finance, RevOps, marketing, and HR. Requires 5+ years in analytics or data engineering, strong SQL and dimensional modeling expertise, and effective stakeholder collaboration.
NinjaTrader
Chicago, IL
Leads database architecture, performance, reliability, and developer-tooling initiatives for high-volume trading applications. Requires 8+ years of software engineering experience, expert MySQL skills, backend development expertise, and strong knowledge of distributed systems and database operations.