# Machine Learning Engineer - Inference

**Company:** [Together AI](https://hotfix.jobs/companies/together-ai)
**Location:** San Francisco, CA
**Role:** ML Engineering
**Salary:** $160k – $230k/yr
**Experience:** 3+ years
**Skills:** PyTorch, Python, CUDA, Triton, vLLM, Tensorrt-Llm, Tgi, Optimum, Rust, Cython
**Posted:** 2026-02-03

> Optimizes and builds production inference systems for large language models at scale using PyTorch and high-performance tooling. Requires 3+ years experience in production code, OS concepts, and AI inference systems.

## Job Description

## Responsibilities
- Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale.
- Develop and optimize runtime inference services for large-scale AI applications.
- Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world.
- Conduct design and code reviews to ensure high standards of quality.
- Create services, tools, and developer documentation to support the inference engine.
- Implement robust and fault-tolerant systems for data ingestion and processing.

## Requirements
- 3+ years of experience writing high-performance, well-tested, production-quality code.
- Proficiency with Python and PyTorch.
- Demonstrated experience in building high performance libraries and tooling.
- Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale.
- **Preferred:** Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum.
- **Preferred:** Knowledge of AI inference techniques such as speculative decoding.
- **Preferred:** Knowledge of CUDA/Triton programming.
- **Nice to have:** Knowledge of Rust, Cython and compilers.

## Compensation
US base salary range: $160,000 - $230,000 + equity + benefits. Salary determined by location, level, experience, skills, and job-related knowledge.

## Similar roles

- [Research Scientist II](https://hotfix.jobs/jobs/4ac4da1a-a497-413d-b8fa-6b3b58cef7e1) - Pindrop - Remote - $160k – $185k/yr
- [AI Engineer - Database Engineering](https://hotfix.jobs/jobs/e2167614-4d69-4c69-bb69-40e1c6ba4571) - Snowflake - Menlo Park, CA - $160k – $230k/yr
- [Software Engineer, Cortex AI Infrastructure](https://hotfix.jobs/jobs/cfcfa4ed-3661-45be-8d97-71e231e17de7) - Snowflake - Menlo Park, CA - $160k – $225k/yr
- [ML Infrastructure Engineer](https://hotfix.jobs/jobs/ad8cfc6e-fc74-4f2a-ad7c-a2f5127c6461) - Mach9 - San Francisco, CA - $160k – $200k/yr
- [ML Ops Infrastructure Engineer](https://hotfix.jobs/jobs/3e53824c-d6f1-4156-892e-cca11a6a394d) - Deepgram - Remote - $160k – $220k/yr

**Apply:** https://hotfix.jobs/jobs/04d49af2-cb2c-4acb-8395-099621366be7
**Canonical:** https://hotfix.jobs/jobs/04d49af2-cb2c-4acb-8395-099621366be7