Skip to content
BasetenBasetenSan Francisco, CA

Software Engineer, Model Performance Tooling

Builds performance benchmarking, diagnostic, and optimization tools for LLM inference on GPU clusters. Early-career role requiring Python proficiency, systems curiosity, and interest in AI hardware—no prior experience needed.

160k – 200k
On-siteEntry levelML Engineering

About the role

The Opportunity

Early-career Software Engineers to build automated performance benchmarking and diagnostic tools for AI infrastructure at the intersection of high-performance computing and LLM engineering. Focus on tearing apart models to optimize performance on hardware.

Responsibilities

  • Performance Benchmarking: Run and automate standard LLM quality benchmarks (GSM8K, MMLU) alongside custom performance suites for specific workloads (e.g., long-context window, KV cache reuse).
  • Infrastructure Validation: Create automated acceptance tests for new GPU clusters across x86 and ARM systems, measuring GPU memory bandwidth, networking throughput, and multi-node networking performance.
  • Model Dev Experience: Develop and maintain internal GPU-enabled development environments (similar to GitHub Codespaces).
  • Tool Development: Build and contribute to tools such as InferenceMAX and genai-bench to automate model evaluation and optimization.
  • Deep Hardware Profiling: Use PyTorch Profiler and NVIDIA Nsight Systems to collect performance profiles, identify bottlenecks, and debug the NVIDIA compute/networking stack.
  • Monitoring & Observability: Develop real-time dashboards and alerts to monitor system health, model startup times, and runtime performance.
  • Continuous Integration: Automate performance testing via CI/CD pipelines to catch regressions in model setups before they hit production.
  • Optimization Automation: Build tools to find the "Pareto frontier"—identifying the absolute best configuration (latency vs. cost vs. quality) for a given model and workload.

What We're Looking For

Fresher-friendly role emphasizing curiosity and technical depth over experience.

  • Love for systems & hardware (GPU memory, InfiniBand).
  • Automation mindset and passion for stress-testing.
  • Mathematical curiosity in Transformers, FLOPs, memory.
  • Interest in optimization (quantization, speculative decoding).
  • Technical Toolkit: Python; eagerness for NVIDIA stack; C++ nice-to-have.

Benefits

  • Competitive compensation, including meaningful equity.
  • 100% coverage of medical, dental, and vision insurance for employee and dependents.
  • Generous PTO policy including company wide Winter Break.
  • Paid parental leave.
  • Company-facilitated 401(k).

Skills

PythonPyTorchNvidia Nsight SystemsGPUInfiniBandCI/CDLLMsBenchmarkingKubernetesC++

Similar roles

ML Engineering jobs
Nuro

Software Engineer, ML Infrastructure, Optimization

NuroMountain View, CA

Build and optimize ML infrastructure for autonomous vehicles, focusing on model optimization, compilers, and deployment across the autonomy stack. Requires 2+ years in ML optimization and strong Python/C++/CUDA skills.

160k – 241k
On-site2+ YOEML Engineering
Harvey

Software Engineer, Agents

HarveySan Francisco, CA +1

Builds AI agents for legal workflows, optimizing performance via prompt engineering, model selection, tools, and evaluations. Partners with PMs and customers to ship low-latency, high-quality agent systems using Python and LLM APIs. Requires 2-5 years experience.

161k – 242k
Hybrid2+ YOEML Engineering
Abridge

Software Engineer

AbridgeSan Francisco, CA +1

Early-career Software Engineer building and iterating on agentic LLM systems, RAG pipelines, evaluation frameworks, and backend/frontend components for AI-powered clinical documentation at Abridge. Requires CS degree or equivalent plus hands-on GenAI experience from projects, internships or coursework.

157k – 184k
On-siteEntry levelML Engineering
Pinterest

Machine Learning Engineer, Monetization Engineering

PinterestSan Francisco, CA +2

Develops and evolves machine learning technology stack for Pinterest's Ads monetization, building personalized recommendation systems using deep learning and LLMs. Requires 2+ years ML experience, CS degree, and expertise in large-scale systems.

163k – 286k
Hybrid2+ YOEML Engineering
Airbnb

Machine Learning Engineer, Relevance and Personalization

AirbnbUnited States

Build and productionize cutting-edge ML models and ranking algorithms for Airbnb's search, recommendation, and personalization systems at scale. Requires PhD (new grad) or 2+ years applied ML experience with strong programming and ML systems skills.

166k – 195k
Remote2+ YOEML Engineering