Senior Software Engineer
As a Senior Software Engineer, you will advance the Tableflow project by developing and maintaining multi-tenant, highly available compute infrastructure and core components for table materialization and background maintenance at scale. You will collaborate with cross-functional teams to integrate storage features and infrastructure.
About the job
About the Role:
We are a team of passionate engineers who love solving complex distributed storage systems and infrastructure problems. Through the Tableflow project, we are building the next generation of storage solutions for Kora, transforming it into a true multi-modal storage engine. Our goal is to enable streaming data to be consumed both as a table and as a stream seamlessly. If you're interested in learning more about the challenges we are tackling, check out our blog post: Tableflow: The Stream/Table, Kafka/Iceberg Duality.
As a Senior Software Engineer on the team, you will focus on advancing key layers of the Tableflow project, including its multi-tenant, highly available compute infrastructure, and the core components responsible for table materialization and background maintenance. You will tackle challenges such as developing strategies for efficient table materialization and maintenance at massive scale across all of Confluent Cloud, ensuring high availability and scalability for background storage tasks.
What You Will Do:
- Storage Engine Development: Design and implement innovative strategies for table schematization, materialization, and compaction at massive scale.
- Platform Engineering: Build and maintain multi-tenant, highly available infrastructure that powers background computational tasks for Kora’s long-term storage. Ensure that the platform is optimized for scalability, reliability, and resilience, in alignment with organizational objectives.
- Cross-Functional Collaboration: Collaborate with product management, design, and other engineering teams to ensure seamless integration of storage features and infrastructure with the broader organization.
What You Will Bring:
- BS, MS, or PhD in computer science or a related field, or equivalent work experience
- Excellent communication and collaboration skills, with the ability to influence and guide stakeholders at all levels.
What Gives You an Edge:
- Familiarity with or has experience contributing to the following open-source technologies: Apache Kafka, Apache Iceberg, Apache Flink
- Experience/knowledge with public clouds (AWS, Azure or GCP)
Skills
Apache Kafka, Apache Iceberg, Apache Flink, AWS, Azure, GCP, Distributed Systems, Platform Engineering, Storage Engine Development
Similar jobs
Backend Engineering jobs
Duolingo
Pittsburgh, PA
Build and operate reliable, scalable backend services and infrastructure for Duolingo’s advertising business. The role requires strong backend fundamentals, systems thinking, programming experience in Java, Python, or Kotlin, and a bachelor’s degree in a technical field.
Duolingo
New York, NYBuild and operate reliable, scalable backend services or infrastructure for Duolingo products, owning complex projects from design through production. The role requires strong backend fundamentals, a bachelor’s degree, and experience with Java, Python, or Kotlin and database technologies.
Maybern
New York, NY
Senior software engineer building core platform components, developer tools, and reliability capabilities that improve engineering productivity. Requires 4+ years of software engineering experience, cloud or platform exposure, and strong knowledge of databases, distributed systems, observability, and production reliability.
Cantina
Sunnyvale, CABuild and optimize high-performance, real-time speech, audio, video, and WebRTC infrastructure for conversational AI across mobile and web platforms. The role requires professional C or C++ experience, concurrent systems knowledge, and at least three years of software engineering experience.
Beacon AI
San Carlos, CA
Leads the design, development, and operation of secure, scalable backend services, APIs, distributed systems, and data pipelines for a mission-critical flight safety platform. Requires 5+ years of backend experience and strong Python and JavaScript/TypeScript skills.