Staff Software Engineer
Staff Software Engineer designs and implements core components of Alluxio's distributed data orchestration platform, optimizing caching, metadata, and I/O for large-scale AI/ML workloads across multi-cloud environments. Requires strong distributed systems expertise in Java/C++/Go and technical leadership.
About the job
What You’ll Own
- Cache and metadata consistency - advance Alluxio’s intelligent caching framework for multi-tenant environments (TTL policies, write-back consistency, invalidation protocols, and distributed metadata scaling).
- High-throughput data I/O optimization - profile and optimize Alluxio’s data path across S3, GCS, HDFS, and POSIX interfaces using adaptive prefetching, async I/O, and tier-aware scheduling.
- Scaling for AI and analytics workloads - evolve the coordination layer to efficiently serve distributed AI training clusters, accelerating model load and shuffle operations across regions and clouds.
- Observability and performance insights - build fine-grained metrics and tracing for cache efficiency, throughput, and latency across storage tiers.
- Open-source leadership - drive design discussions, mentor contributors, and represent Alluxio’s core-systems direction within the OSS community.
What You’ll Do
- Design and implement core components of Alluxio’s distributed file and object-access layer.
- Optimize performance for large-scale, high-throughput environments using advanced concurrency and caching techniques.
- Build scalable metadata and coordination systems that ensure strong consistency, high availability, and minimal latency.
- Collaborate cross-functionally with product, solution-engineering, and research teams to drive roadmap and customer success.
What We’re Looking For
- Strong computer-science fundamentals and a passion for large-scale distributed systems.
- Professional experience developing in Java, C++, or Go.
- Deep understanding of concurrency, replication, fault tolerance, and performance optimization.
- Experience with distributed storage, data-access layers, or cloud infrastructure (e.g., Spark, Presto, Hadoop, Kubernetes).
- Bachelor’s or advanced degree in Computer Science or related technical field (or equivalent experience).
- Demonstrated technical leadership: defining architecture, mentoring peers, or driving major projects from design through release.
Skills
Java, C++, Go, Kubernetes, Spark, Presto, Hadoop, S3, Gcs, Hdfs
Similar jobs
Backend Engineering jobs
Leads architecture and implementation of large-scale, low-latency serving platforms supporting Ads and Core experiences. The role requires 8+ years of backend or distributed-systems experience, strong technical leadership, and expertise in scalable production systems.
Airbnb
United States
Senior technical IC focused on backend architecture and AI technologies for Airbnb's communication and connectivity platform. Partners with senior leaders and contributes code while providing technical leadership across teams.
Axion
New York, NYStaff Software Engineer leading backend and data-intensive systems, including scalable services, AI-powered workflows, cloud infrastructure, and reliability initiatives. Requires 8+ years of software engineering experience and expertise in distributed systems, data processing, and containerized applications.
Anthropic
San Francisco, CA
Design and operate high-QPS backend systems on Claude’s token-generation path, owning latency, reliability, safe deployments, and incident response. The role requires 8+ years of software engineering experience, strong distributed-systems expertise, and production ownership of mission-critical services.
GitLab
United StatesBuild and operate Go-based automation and a PostgreSQL-as-a-service platform for high-throughput, always-on production systems. The role requires deep PostgreSQL production experience, backend development expertise, infrastructure automation, and staff-level technical leadership.