# Software Engineer, Model Performance Tooling

**Company:** [Baseten](https://hotfix.jobs/companies/baseten)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $160k – $200k/yr
**Experience:** 0+ years
**Skills:** Python, PyTorch, Nvidia Nsight Systems, GPU, InfiniBand, CI/CD, LLMs, Benchmarking, Kubernetes, C++
**Posted:** 2026-01-07

> Builds performance benchmarking, diagnostic, and optimization tools for LLM inference on GPU clusters. Early-career role requiring Python proficiency, systems curiosity, and interest in AI hardware—no prior experience needed.

## Job Description

## The Opportunity

Early-career Software Engineers to build automated performance benchmarking and diagnostic tools for AI infrastructure at the intersection of high-performance computing and LLM engineering. Focus on tearing apart models to optimize performance on hardware.

## Responsibilities

- **Performance Benchmarking**: Run and automate standard LLM quality benchmarks (GSM8K, MMLU) alongside custom performance suites for specific workloads (e.g., long-context window, KV cache reuse).
- **Infrastructure Validation**: Create automated acceptance tests for new GPU clusters across x86 and ARM systems, measuring GPU memory bandwidth, networking throughput, and multi-node networking performance.
- **Model Dev Experience**: Develop and maintain internal **GPU-enabled development environments** (similar to GitHub Codespaces).
- **Tool Development**: Build and contribute to tools such as **InferenceMAX** and **genai-bench** to automate model evaluation and optimization.
- **Deep Hardware Profiling**: Use **PyTorch Profiler** and **NVIDIA Nsight Systems** to collect performance profiles, identify bottlenecks, and debug the NVIDIA compute/networking stack.
- **Monitoring & Observability**: Develop real-time dashboards and alerts to monitor system health, model startup times, and runtime performance.
- **Continuous Integration**: Automate performance testing via CI/CD pipelines to catch regressions in model setups before they hit production.
- **Optimization Automation**: Build tools to find the "Pareto frontier"—identifying the absolute best configuration (latency vs. cost vs. quality) for a given model and workload.

## What We're Looking For

Fresher-friendly role emphasizing curiosity and technical depth over experience.

- Love for systems & hardware (GPU memory, InfiniBand).
- Automation mindset and passion for stress-testing.
- Mathematical curiosity in Transformers, FLOPs, memory.
- Interest in optimization (quantization, speculative decoding).
- **Technical Toolkit**: Python; eagerness for NVIDIA stack; C++ nice-to-have.

## Benefits

- Competitive compensation, including meaningful equity.
- 100% coverage of medical, dental, and vision insurance for employee and dependents.
- Generous PTO policy including company wide Winter Break.
- Paid parental leave.
- Company-facilitated 401(k).

## Similar roles

- [Software Engineer, ML Infrastructure, Optimization](https://hotfix.jobs/jobs/a6b5f42b-e6f7-497c-adda-7a35b24c3eb4) - Nuro - Mountain View, CA - $160k – $241k/yr
- [Software Engineer, Agents](https://hotfix.jobs/jobs/c48c3d5d-396e-4051-938f-a677c38c294d) - Harvey - San Francisco, CA - $161k – $242k/yr
- [Software Engineer](https://hotfix.jobs/jobs/823e27ee-a56e-4d3e-ac4d-dacbce6de23d) - Abridge - San Francisco, CA - $157k – $184k/yr
- [Machine Learning Engineer, Monetization Engineering](https://hotfix.jobs/jobs/717f63ae-bbd1-483c-9c49-b2a025ed411d) - Pinterest - San Francisco, CA - $163k – $286k/yr
- [Machine Learning Engineer, Relevance and Personalization](https://hotfix.jobs/jobs/e55bff4d-9948-4fd5-8f5e-51f0f5c17db4) - Airbnb - Remote - $166k – $195k/yr

**Apply:** https://hotfix.jobs/jobs/26ede8a1-a8ce-494b-bcb9-87270ba2d333
**Canonical:** https://hotfix.jobs/jobs/26ede8a1-a8ce-494b-bcb9-87270ba2d333