Skip to content
FalFalSan Francisco, CA

Staff Software Engineer, ML Performance & Systems

Designs and implements novel model serving architectures on in-house inference engine to maximize throughput and minimize latency for generative media models. Develops performance tools and collaborates with ML teams on Nvidia-based systems optimizations.

180k – 250k/yr
On-siteML Engineering

About the role

Key Responsibilities

  • Help fal maintain its frontier position on model performance for generative media models.
  • Design and implement novel approaches to model serving architecture on top of our in-house inference engine, focusing on maximizing throughput while minimizing latency and resource usage.
  • Develop performance monitoring and profiling tools to identify bottlenecks and optimization opportunities.
  • Work closely with our Applied ML team and customers (frontier labs on the media space) and make sure their workloads benefit from our accelerator.

Requirements

  • Strong foundation in systems programming with expertise in identifying and fixing bottlenecks.
  • Deep understanding of cutting edge ML infrastructure stack (PyTorch, TensorRT, TransformerEngine, Nsight), including model compilation, quantization, and serving architectures.
  • Fundamental view of underlying hardware (Nvidia based systems), including custom GEMM kernels with CUTLASS.
  • Proficient in Triton or comparable experience in lower-level accelerator programming.
  • Experience with multi-dimensional model parallelism (TP with context/sequence parallel).
  • Familiar with internals of Ring Attention, FA3, FusedMLP implementations.

Compensation

$180,000 - $250,000 + equity + comprehensive benefits package

Skills

PyTorchTensorRTTransformerengineNsightTritonCutlassNvidia HardwareModel CompilationQuantizationModel ServingRing AttentionFa3Fusedmlp

Similar roles

ML Engineering jobs
xAI

Member of Technical Staff

xAIPalo Alto, CA

Build and optimize the RL training framework and infrastructure for large-scale workloads at SpaceXAI, from ablations to production runs. Requires experience with distributed systems and proficiency in Python, JAX, Rust, or C++.

180k – 440k/yr
On-site5+ YOEML Engineering
LlamaIndex

Member of Technical Staff, Applied Research

LlamaIndexSan Francisco, CA

Build and productionize vision-language models for document understanding at LlamaIndex. Focus on training, fine-tuning, synthetic data, benchmarking, and turning research prototypes into accurate, low-latency production systems for real-world PDFs, tables, and enterprise docs. Requires 3+ years ML engineering/applied research experience with strong PyTorch skills.

180k – 250k/yr
Hybrid3+ YOEML Engineering
xAI

Member of Technical Staff, Inference

xAIPalo Alto, CA

Build and optimize the high-performance inference platform serving Grok at massive scale. Design distributed serving infrastructure, low-level GPU optimizations, quantization, speculative decoding, and CI/CD for production reliability and low latency.

180k – 440k/yr
On-site5+ YOEML Engineering
Black Forest Labs

Member of Technical Staff

Black Forest LabsSan Francisco, CA

Research Engineer embedded in large-scale multimodal model training at Black Forest Labs (creators of FLUX). Own performance, stability, and low-precision optimizations for production training runs; profile, debug distributed systems, implement GPU kernels, and partner with researchers to ship frontier generative models.

180k – 290k/yr
Hybrid7+ YOEML Engineering
xAI

Member of Technical Staff - X Search

xAIPalo Alto, CA

Develops and operates large-scale search engine infrastructure, including retrieval algorithms, indexing, and ML ranking models integrated with Grok AI. Requires experience with search systems, vector databases, and production ML in Python, Go, or Rust.

180k – 440k/yr
On-siteML Engineering