Senior/Staff Software Engineer, Data Engineering Team
Builds scalable data ingestion pipelines, processing systems, and tooling to support AI/ML research and trading operations at a quantitative finance firm. Requires 5+ years experience in robust software engineering with modern languages and data infrastructure expertise.
About the job
Responsibilities
- Engage and collaborate in diverse software development work, including design and implementation of data processing technologies, sourcing and delivery systems and pipelines, development of data related tools and libraries, and more.
- Support trading operations and promote research effort through reliable delivery of high-quality data.
- Build scalable and robust ingestion and distribution systems, and fault-tolerant production-critical pipelines.
- Lead complex projects from start to finish, including gathering requirements, creating a robust software design, reasoning about supporting or dependent technologies, and communicating effectively with stakeholders, collaborators, and teammates.
- Provide technical guidance to engineering and research staff.
- Provide mentorship and support to help grow your teammates and up-level the team.
Requirements
- Computer Science / Engineering bachelor’s degree (or equivalent)
- 5+ years of relevant software engineering experience
- Proven track record of software design and implementation with focus on correctness, robustness, efficiency, and scale
- Experience working with large codebases and building modular, extensible, and maintainable software
- Expertise in a modern programming language, such as Python, Go, Java or C++
- Hands-on experience developing in a Linux/UNIX environment
- Design and implementation of scalable services, highly-available systems, and/or robust data infrastructure
- Strong communication skills and a knack for explaining complex ideas with clarity and simplicity
Preferred Qualifications
- Experience with data storage and management technologies (e.g. PostgreSQL, Artifactory, Ceph, Redis)
- Cluster management and containerization technologies (e.g. Kubernetes, Docker)
- Job scheduling and orchestration technologies (e.g. Airflow, Slurm)
Skills
Python, Go, Java, C++, Linux, Kubernetes, Docker, Postgres, Airflow, Redis
Similar jobs
Data Engineering jobs
OpenAI
San Francisco, CAAnalytics Engineer supporting Go-to-Market teams by building scalable data models, metrics, pipelines, visualizations, and self-service products. The role requires 10+ years of data experience, deep SQL expertise, Python proficiency, and strong business judgment.
Harvey
San Francisco, CAStaff Software Engineer on the central data platform team, responsible for architecture, ingestion, orchestration, streaming, governance, and self-service data tooling. Requires 10+ years building production data infrastructure and deep experience with warehouses, CDC, streaming, orchestration, Python, and SQL.
Snowflake
Menlo Park, CA
Leads the design and operation of highly available distributed data platforms and pipelines at Snowflake, while providing technical leadership across teams. Requires 12+ years of distributed-systems experience, cloud expertise, and strong database and system-design depth.
Airbnb
United States
Staff Software Engineer leading development of data catalog and metadata infrastructure for discovery, governance, lineage, and quality across Airbnb’s data ecosystem. Requires 9+ years of software engineering experience focused on data infrastructure and strong programming and distributed data technology skills.
Anyscale
San Francisco, CA
Staff Software Engineer responsible for designing and scaling Ray Data’s distributed data-processing infrastructure for large-scale AI training and inference. Requires 6+ years of production software and architectural ownership experience, plus deep distributed-systems expertise and strong Python skills.