Skip to content
CohereCohereNew York, NY

Member of Technical Staff, Model Efficiency

Engineers on this team optimize LLM inference for lower latency and higher throughput by identifying bottlenecks, developing optimizations across the execution stack, and collaborating with modeling teams. Requires 5+ years high-performance coding in C++/Python and LLM inference experience.

Salary not listed
Remote5+ YOEML Engineering

About the role

Responsibilities

  • Work across the inference stack to improve core performance metrics by diving deep into model execution, identifying bottlenecks, and developing innovative optimizations.
  • Collaborate closely with modeling and systems teams to experiment, measure, and ship improvements that meaningfully accelerate inference.
  • Build expertise in advanced performance techniques, including GPU/CUDA optimizations, kernel-level improvements, and model execution strategies for MoE and large-scale architectures.

Requirements

  • 5+ years of experience writing high-performance, production-quality code.
  • Strong programming skills in C++ or Python (Rust/Go also welcome).
  • Experience working with large language models and familiarity with the LLM inference ecosystem (e.g., vLLM, SGLang, etc.).
  • Ability to diagnose and resolve performance bottlenecks across the model execution stack.
  • A strong bias for action — ship fast, measure impact, and iterate.

Nice-to-Haves

  • Experience with GPU programming, CUDA, or low-level systems optimization.
  • Language modeling with transformers (MoE, speculative decoding, KV-cache optimizations).
  • Scaling performance-critical distributed systems (e.g., computation, search, storage).

Skills

C++PythonRustGoLLMsvLLMSglangCUDAGPUTransformersMoeKv-Cache

Similar roles

ML Engineering jobs
xAI

Member of Technical Staff

xAIPalo Alto, CA

Build and optimize the RL training framework and infrastructure for large-scale workloads at SpaceXAI, from ablations to production runs. Requires experience with distributed systems and proficiency in Python, JAX, Rust, or C++.

180k – 440k/yr
On-site5+ YOEML Engineering
Chime

Staff Software Engineer, AI & App Experience

ChimeNew York, NY +1

Staff Software Engineer building and scaling Jade, Chime's LLM-powered AI financial assistant. Sets technical direction for agent systems, evals, and guardrails while staying hands-on with prototyping, backend services, and cross-functional product delivery. Requires 8+ years production experience, deep AI/LLM fluency, and technical leadership.

223k – 308k/yr
Hybrid8+ YOEML Engineering
Anthropic

Staff + Senior Software Engineer, Cloud Inference Launch Engineering

AnthropicSan Francisco, CA

Build and own validation pipelines, CI/CD infrastructure, and platform integrations to launch frontier models and inference features reliably across AWS, GCP, and Azure. Requires strong large-scale distributed systems experience and track record improving release velocity.

320k – 485k/yr
Hybrid7+ YOEML Engineering
Coinbase

Senior Staff Software Engineer, Legal Automation

CoinbaseUnited States

Senior Staff Software Engineer building an AI agent platform and automated workflows to transform Coinbase's Legal organization. Architect production-grade LLM and multi-agent systems that replace manual legal processes such as agreement redlining and governance.

254k – 299k/yr
Remote12+ YOEML Engineering
Pinterest

Staff Machine Learning Engineer, Computer Vision

PinterestSan Francisco, CA

Staff Machine Learning Engineer developing state-of-the-art visual encoders and multimodal models at Pinterest Labs. Prototype visual reasoning tools, train billion-scale models on rich visual-text data, ship to production for recommender systems and VLMs, publish research, and mentor juniors. Requires strong CV/ML background, publications, and PhD or equivalent.

189k – 390k/yr
Hybrid7+ YOEML Engineering