Skip to content
Together AITogether AISan Francisco, CA

Machine Learning Engineer - Inference

Optimizes and builds production inference systems for large language models at scale using PyTorch and high-performance tooling. Requires 3+ years experience in production code, OS concepts, and AI inference systems.

160k – 230k/yr
On-site3+ YOEML Engineering

About the role

Responsibilities

  • Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale.
  • Develop and optimize runtime inference services for large-scale AI applications.
  • Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world.
  • Conduct design and code reviews to ensure high standards of quality.
  • Create services, tools, and developer documentation to support the inference engine.
  • Implement robust and fault-tolerant systems for data ingestion and processing.

Requirements

  • 3+ years of experience writing high-performance, well-tested, production-quality code.
  • Proficiency with Python and PyTorch.
  • Demonstrated experience in building high performance libraries and tooling.
  • Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale.
  • Preferred: Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum.
  • Preferred: Knowledge of AI inference techniques such as speculative decoding.
  • Preferred: Knowledge of CUDA/Triton programming.
  • Nice to have: Knowledge of Rust, Cython and compilers.

Compensation

US base salary range: $160,000 - $230,000 + equity + benefits. Salary determined by location, level, experience, skills, and job-related knowledge.

Skills

PyTorchPythonCUDATritonvLLMTensorrt-LlmTgiOptimumRustCython

Similar roles

ML Engineering jobs
Pindrop

Research Scientist II

PindropUnited States

Research Scientist II building and improving fraud risk models and scam detection systems using audio, behavioral, and metadata signals. Requires an advanced degree and 3+ years of applied ML experience with Python and modern ML frameworks.

160k – 185k/yr
Remote3+ YOEML Engineering
Snowflake

AI Engineer - Database Engineering

SnowflakeMenlo Park, CA

As an AI Engineer, you will own the full AI engineering lifecycle, from design to optimization, for Snowflake Database Engineering products. You will build agentic workflows, coding harnesses, and evaluation pipelines, working with a high-powered engineering team.

160k – 230k/yr
On-site5+ YOEML Engineering
Snowflake

Software Engineer, Cortex AI Infrastructure

SnowflakeMenlo Park, CA

Build and scale backend infrastructure powering agentic AI products including orchestration engines, RAG systems, evals infrastructure, and production AI workflows. Requires 4+ years distributed systems experience and deep Python plus Go/Java proficiency.

160k – 225k/yr
On-site4+ YOEML Engineering
Mach9

ML Infrastructure Engineer

Mach9San Francisco, CA

Builds and maintains ML infrastructure for training pipelines handling massive 3D data and real-time inference serving integrated with CAD software. Requires 3+ years experience with Python, PyTorch, ML orchestration tools, data versioning, and inference optimization.

160k – 200k/yr
On-site3+ YOEML Engineering
Deepgram

ML Ops Infrastructure Engineer

DeepgramCalifornia

Build and maintain ML infrastructure pipelines to deploy research models to production at scale, including CI/CD, A/B testing, monitoring, and optimization for low-latency voice AI serving. Requires 4+ years MLOps experience with Python, Docker, Kubernetes.

160k – 220k/yr
Remote4+ YOEML Engineering