Software Engineer L3 Data Substrate
As a Software Engineer on the Data & Analytics Platform team, you will design, build, and optimize the data platform to support various data-driven initiatives. You will work with cross-functional teams to architect scalable solutions and implement data infrastructure using modern data technologies.
About the job
Responsibilities
In this role, you’ll:
- Design, build, and maintain infrastructure and scalable frameworks to support data ingestion, processing, and analysis.
- Collaborate with stakeholders, analysts, and product teams to understand business requirements and translate them into technical solutions.
- Architect and implement data solutions using modern data technologies such as Kafka, Spark, Hive, Hudi, Presto, Airflow, and cloud-based services like AWS Lakeformation, Glue and Athena.
- Design and implement frameworks and solutions for performance, reliability, and cost-efficiency.
- Ensure data quality, integrity, and security throughout the data lifecycle.
- Stay current with emerging technologies and best practices in big data technologies
- Mentor early in career engineers and contribute to a culture of continuous learning and improvement
Qualifications
Required:
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 5+ years of experience in software development or a related field.
- Strong proficiency in programming languages such as Python, Java, or Scala.
- Strong experience with building frameworks for big data technologies such as Spark, Kafka, Hive, and distributed computing systems.
- Experience with AWS technologies at scale
- Solid understanding of software engineering principles, including object-oriented and functional programming paradigms, design patterns, and code quality practices.
- Excellent problem-solving and analytical skills.
- Strong verbal & written communication skills, with the ability to work effectively in a cross-functional team environment.
Desired:
- Bias to action, ability to iterate and ship rapidly
- Passion to build data products, prior projects in this area
Compensation
The estimated pay ranges for this role are as follows:
- Based in Colorado, Hawaii, Illinois, Maryland, Massachusetts, Minnesota, Vermont or Washington D.C. : $138,700 - $173,400
- Based in New York, New Jersey, Washington State, or California (outside of the San Francisco Bay area): $146,800 - $183,600
- Based in the San Francisco Bay area, California: $163,100 - $203,900.
This role may be eligible to participate in Twilio’s equity plan and corporate bonus plan. All roles are generally eligible for the following benefits: health care insurance, 401(k) retirement account, paid sick time, paid personal time off, paid parental leave.
Skills
Kafka, Spark, Hive, Hudi, Presto, Airflow, Aws Lakeformation, Glue, Athena, Python, Java, Scala
Similar jobs
Data Engineering jobs
Sigma
New York, NY
Build and scale data architecture, governance, and ETL pipelines across Snowflake, Databricks, and cloud platforms. The role requires 3+ years of data engineering experience, strong API and pipeline expertise, and the ability to collaborate across technical and business teams.
OnePay
United States
The Analytics Engineer will own OnePay’s analytical data foundation, building trusted dbt models, tests, documentation, dashboards, and semantic metrics on Databricks. The role requires at least three years of analytics engineering experience, expert SQL and dbt skills, strong data-quality practices, and hands-on use of AI coding tools.
Stuut
San Francisco, CABuild and own Stuut’s foundational data platform, including ingestion pipelines, canonical models, semantic layers, and observability. The role requires 3+ years of production data pipeline experience with Python, SQL, cloud warehouses, and ETL/ELT tooling.
Ontic
United States
Build scalable analytics engineering infrastructure, SaaS data models, and AI-enabled workflows that support enterprise decision-making. The role requires 3–6 years of hands-on analytics or data engineering experience, strong SQL and modern data modeling expertise, and cloud data warehouse experience.
Underdog Fantasy
United States
Build and operate the data platform supporting automated regulatory reporting for a prediction markets business. The role combines SQL and dbt development, end-to-end data investigation, automated validation, and cross-functional ownership under strict deadlines.