Skip to content
FalFalSan Francisco, CA

Staff Technical Lead for Inference & ML Performance

Leads team to build and optimize high-performance ML inference systems for generative models. Drives hands-on optimizations across the performance stack, collaborates with research teams, and mentors engineers to exceed industry benchmarks.

Salary not listed
On-siteML Engineering

About the role

Responsibilities

  • Set technical direction for team working on kernels, applied performance, ML compilers, and distributed inference to build high-performance inference solutions.
  • Provide hands-on IC leadership by contributing to critical inference performance enhancements and optimizations.
  • Collaborate with research and applied ML teams to influence model inference strategies and deployment techniques.
  • Drive advanced performance optimizations including model parallelism, kernel optimization, and compiler strategies.
  • Mentor and scale team of performance-focused engineers.

Requirements

  • Deep experience in ML performance optimization for large-scale generative models in production.
  • Expertise in full ML performance stack: PyTorch, TensorRT, TransformerEngine, Triton, CUTLASS kernels.
  • Expert knowledge of inference techniques: quantization, kernel authoring, compilation, model parallelism (TP, context/sequence parallel, expert parallel), distributed serving, profiling.
  • Lead from the front as a respected IC who enjoys hands-on problem-solving.
  • Thrive in cross-functional collaboration with ML teams, researchers, and stakeholders.

Nice-to-haves

  • Experience building inference engines for diffusion and generative media models.
  • Track record of industry-leading performance improvements (papers, open-source, benchmarks).
  • Leadership experience in scaling technical teams.

Skills

PyTorchTensorRTTransformerengineTritonCutlassQuantizationModel ParallelismKernel AuthoringMl CompilersDistributed Serving

Similar roles

ML Engineering jobs
Databricks

Staff Software Engineer, Agentic Applications

DatabricksMountain View, CA

Staff Software Engineer owning architecture and delivery of LLM-powered agentic workflows for marketing content creation, publishing, and reliability at scale. Requires 12+ years experience building production LLM systems, human-in-the-loop designs, stakeholder collaboration, and mentoring.

198k – 273k/yr
On-site12+ YOEML Engineering
xAI

Member of Technical Staff

xAIPalo Alto, CA

Build and optimize the RL training framework and infrastructure for large-scale workloads at SpaceXAI, from ablations to production runs. Requires experience with distributed systems and proficiency in Python, JAX, Rust, or C++.

180k – 440k/yr
On-site5+ YOEML Engineering
Chime

Staff Software Engineer, AI & App Experience

ChimeNew York, NY +1

Staff Software Engineer building and scaling Jade, Chime's LLM-powered AI financial assistant. Sets technical direction for agent systems, evals, and guardrails while staying hands-on with prototyping, backend services, and cross-functional product delivery. Requires 8+ years production experience, deep AI/LLM fluency, and technical leadership.

223k – 308k/yr
Hybrid8+ YOEML Engineering
Anthropic

Staff + Senior Software Engineer, Cloud Inference Launch Engineering

AnthropicSan Francisco, CA

Build and own validation pipelines, CI/CD infrastructure, and platform integrations to launch frontier models and inference features reliably across AWS, GCP, and Azure. Requires strong large-scale distributed systems experience and track record improving release velocity.

320k – 485k/yr
Hybrid7+ YOEML Engineering
Coinbase

Senior Staff Software Engineer, Legal Automation

CoinbaseUnited States

Senior Staff Software Engineer building an AI agent platform and automated workflows to transform Coinbase's Legal organization. Architect production-grade LLM and multi-agent systems that replace manual legal processes such as agreement redlining and governance.

254k – 299k/yr
Remote12+ YOEML Engineering