Staff Software Engineer
Staff Software Engineer designs and implements core components of Alluxio's distributed data orchestration platform, optimizing caching, metadata, and I/O for large-scale AI/ML workloads across multi-cloud environments. Requires strong distributed systems expertise in Java/C++/Go and technical leadership.
About the job
What You’ll Own
- Cache and metadata consistency - advance Alluxio’s intelligent caching framework for multi-tenant environments (TTL policies, write-back consistency, invalidation protocols, and distributed metadata scaling).
- High-throughput data I/O optimization - profile and optimize Alluxio’s data path across S3, GCS, HDFS, and POSIX interfaces using adaptive prefetching, async I/O, and tier-aware scheduling.
- Scaling for AI and analytics workloads - evolve the coordination layer to efficiently serve distributed AI training clusters, accelerating model load and shuffle operations across regions and clouds.
- Observability and performance insights - build fine-grained metrics and tracing for cache efficiency, throughput, and latency across storage tiers.
- Open-source leadership - drive design discussions, mentor contributors, and represent Alluxio’s core-systems direction within the OSS community.
What You’ll Do
- Design and implement core components of Alluxio’s distributed file and object-access layer.
- Optimize performance for large-scale, high-throughput environments using advanced concurrency and caching techniques.
- Build scalable metadata and coordination systems that ensure strong consistency, high availability, and minimal latency.
- Collaborate cross-functionally with product, solution-engineering, and research teams to drive roadmap and customer success.
What We’re Looking For
- Strong computer-science fundamentals and a passion for large-scale distributed systems.
- Professional experience developing in Java, C++, or Go.
- Deep understanding of concurrency, replication, fault tolerance, and performance optimization.
- Experience with distributed storage, data-access layers, or cloud infrastructure (e.g., Spark, Presto, Hadoop, Kubernetes).
- Bachelor’s or advanced degree in Computer Science or related technical field (or equivalent experience).
- Demonstrated technical leadership: defining architecture, mentoring peers, or driving major projects from design through release.
Skills
Java, C++, Go, Kubernetes, Spark, Presto, Hadoop, S3, Gcs, Hdfs
Similar jobs
Backend Engineering jobs
Square
Seattle, WAStaff software engineer leading hands-on development and technical direction for a global card-issuing platform. The role requires deep card-network expertise, direct connectivity and certification experience, and strong software engineering experience in highly reliable payments systems.
Altana
Brooklyn, NYLeads technical design, development, deployment, and engineering excellence for scalable backend and data-driven products. Requires 12+ years of professional software experience, cloud deployment expertise, and experience leading engineers on complex projects.
Leads architecture and implementation of large-scale, low-latency serving platforms supporting Ads and Core experiences. The role requires 8+ years of backend or distributed-systems experience, strong technical leadership, and expertise in scalable production systems.
Airbnb
United States
Senior technical IC focused on backend architecture and AI technologies for Airbnb's communication and connectivity platform. Partners with senior leaders and contributes code while providing technical leadership across teams.
Axion
New York, NYStaff Software Engineer leading backend and data-intensive systems, including scalable services, AI-powered workflows, cloud infrastructure, and reliability initiatives. Requires 8+ years of software engineering experience and expertise in distributed systems, data processing, and containerized applications.