Sr. Staff Software Engineer - Managed Tables
Senior engineer building autonomous analytical data systems for Unity Catalog Managed Tables, focusing on Predictive Optimization, Liquid Clustering, and large-scale storage innovations. Requires 15+ years building distributed systems and strong expertise in databases and performance optimization.
About the job
Responsibilities
- Design and implement next-generation autonomous analytical data systems for enterprise-grade performance
- Develop storage lifecycle and background optimizations for Predictive Optimization
- Create efficient storage structures and innovations for storage layout solutions in ML environments
- Influence Data format and Catalog interface unification efforts
- Enable strong ecosystem interoperability for ManagedTables
- Solve real business needs at large scale through software engineering and collaboration with Product partners
- Deliver a highly scalable, available, and interoperable architecture for Managed Tables
- Perform low-level systems debugging, performance measurement, and optimization on large production clusters
- Build architecture design, influence product roadmap, and take ownership of new projects
- Introduce tools to enable greater automation and operability of services
- Prevent and investigate production issues
- Plan and lead complicated technical projects across multiple teams
- Mentor and up-level engineers on the team
Requirements
- 15+ years of industry experience building and supporting large-scale distributed systems
- Strong passion for database systems, storage systems, distributed systems, language design, or performance optimization
- Experience working towards a multi-year vision with incremental deliverables
- Strong foundation in algorithms and data structures with real-world application
- BS/MS/PhD in Computer Science or related majors, or equivalent experience
Nice-to-Haves
- Experience driving company initiatives towards customer satisfaction
- Motivation to deliver customer value and impact
Skills
Distributed Systems, Database Systems, Storage Systems, Performance Optimization, Algorithms, Data Structures, Large-Scale Systems, Systems Debugging, Architecture Design, Data Formats
Similar jobs
Backend Engineering jobs
Axion
New York, NYStaff Software Engineer leading backend and data-intensive systems, including scalable services, AI-powered workflows, cloud infrastructure, and reliability initiatives. Requires 8+ years of software engineering experience and expertise in distributed systems, data processing, and containerized applications.
Fivetran
Oakland, CA
Leads the design and implementation of scalable distributed data pipelines and related infrastructure, while improving performance, reliability, and security. Requires 5+ years of software experience, systems programming expertise, cloud and containerization experience, and senior staff-level technical leadership.
Fivetran
Oakland, CA
Leads performance engineering for high-volume data systems by diagnosing bottlenecks, designing improvements, and building resilient performance platforms. Requires 10+ years of software experience, strong Java and database expertise, and experience with distributed backend systems.
Idme
McLean, VA
Staff Software Engineer responsible for the reporting data model, customer-facing data delivery APIs and pipelines, and revenue-critical usage metering and billing systems. The role requires 8+ years of backend and data-systems experience, strong SQL and API expertise, and technical leadership across teams.
Snowflake
Menlo Park, CA
Staff Software Engineer on Snowflake’s Snowtrail team, designing and operating query-processing and distributed infrastructure that improves engineering velocity and production quality. Requires expertise in databases, query optimization, compiler implementation, and large-scale systems.