Staff Software Engineer, Metrics and Logging

The impact you'll have:

• Build the future of logging at Databricks by designing and scaling our next-generation logging platform that processes petabytes of logs daily.
• Develop and optimize log delivery pipelines to support low-latency, high-throughput log ingestion and querying, ensuring seamless observability across all Databricks services.
• Enhance log accessibility and usability, developing tools that enable engineers to efficiently search, analyze, and derive insights from logs.
• Collaborate with teams across Databricks to define best practices for structured logging, standardizing formats and improving the developer experience.
• Improve reliability and cost-efficiency by optimizing log retention, indexing, and query performance to reduce operational overhead.
• Mentor and uplevel engineers, fostering a culture of technical excellence within the team and broader observability community.

What we look for:

• BS (or higher) in Computer Science, or a related field.
• 7+ years of production-level experience in one of: Scala, Rust, Go, Python, Java, C++, or similar languages.
• Deep experience in software development, in large-scale distributed systems.
• Experience driving complex projects involving multiple teams and stakeholders.
• Familiarity with log collection, health monitoring, and observability tools.