Engineering Manager - Streaming
Engineering manager leading Spark Structured Streaming development, enhancing stream processing with advanced features and performance improvements. Requires 5+ years in big data/Spark, team leadership, and distributed systems expertise.
About the job
Key Responsibilities
- Lead engineering team in developing and promoting Spark™ Structured Streaming in OSS and Databricks platform
- Oversee recruitment of top talent and upskill team members
- Implement processes to execute product vision, strategy, and roadmap
- Build high-quality, operable software
- Drive stream processing adoption across Databricks products
- Manage technical debt and long-term architecture decisions
Requirements
- 5+ years experience in big-data systems, Apache Spark™, or database internals
- Passion for database systems, storage, distributed systems, language design, or performance optimization
- Experience ensuring high-quality infrastructure with testing, SLAs; leading teams in complex domains like distributed data systems
- Ability to attract, hire, coach engineers; manage distributed teams (preferred)
- Comfortable working cross-functionally with product management and customers
Skills
Spark, Structured Streaming, Distributed Systems, Stream Processing, Big Data, Performance Optimization, State Management
Similar jobs
Engineering Management jobs
Permitflow
New York, NY
Leads and grows an engineering team building new products from initial market hypothesis through commercial launch. The role requires at least five years of software engineering experience, engineering leadership, strong architecture judgment, and comfort operating in ambiguous 0-to-1 environments.
Permitflow
New York, NY
Leads the engineering team responsible for production agentic infrastructure and evaluation systems, combining people management with hands-on technical direction. The role requires 5+ years of software engineering experience, engineering leadership, and experience shipping AI systems.
Permitflow
New York, NY
Leads and scales an engineering team while remaining hands-on with architecture, system design, code review, and technical execution. The role requires 5+ years of software engineering experience, engineering leadership experience, and strong communication in ambiguous, high-growth environments.
Duolingo
New York, NY
Engineering manager leading a cross-functional product team responsible for strategy, experimentation, reliable infrastructure, and measurable user impact. Requires experience hiring and developing software engineers, delivering multi-engineer projects, and partnering across product, design, and data.
Applied Intuition
Ann Arbor, MI
Leads and mentors a software engineering team building reliable, high-performance simulation products, while guiding technical execution, prioritization, and hiring. Requires a bachelor’s degree, programming experience in C++, Python, or Java, people-management experience, and eligibility for a US Government security clearance.