# AI Inference Engineer

**Company:** [Perplexity](https://hotfix.jobs/companies/perplexity)
**Location:** San Francisco, CA, Palo Alto, CA, New York, NY
**Role:** ML Engineering
**Salary:** $220k – $405k/yr
**Skills:** Python, Rust, C++, PyTorch, Triton, CUDA, Kubernetes, TensorFlow, Onnx
**Posted:** 2024-06-10

> Develops APIs and optimizes large-scale ML model inference using Python, Rust, C++, PyTorch, and CUDA. Benchmarks performance, improves reliability, and implements LLM optimizations on GPU architectures.

## Job Description

## Responsibilities
- Develop APIs for AI inference that will be used by both internal and external customers
- Benchmark and address bottlenecks throughout our inference stack
- Improve the reliability and observability of our systems and respond to system outages
- Explore novel research and implement LLM inference optimizations

## Qualifications
- Experience with ML systems and deep learning frameworks (e.g. **PyTorch**, **TensorFlow**, **ONNX**)
- Familiarity with common LLM architectures and inference optimization techniques (e.g. continuous batching, quantization, etc.)
- Understanding of GPU architectures or experience with GPU kernel programming using **CUDA**

## Similar roles

- [AI Engineer, Applied ML](https://hotfix.jobs/jobs/fa2a14c5-a9f8-41ed-b414-a6dc50f0126d) - Perplexity - San Francisco, CA - $220k – $405k/yr
- [Machine Learning Infrastructure Engineer- Model Inference](https://hotfix.jobs/jobs/a358c1a7-73bf-4c2b-b887-7030d504b58b) - Abridge - San Francisco, CA - $221k – $260k/yr
- [ML Systems Engineer, Robotics](https://hotfix.jobs/jobs/fd6494c5-b63a-412e-87fe-48b825f54495) - Scale AI - San Francisco, CA - $218k – $273k/yr
- [ML Research Engineer, ML Systems](https://hotfix.jobs/jobs/c814c5a3-7e79-4c03-8f4b-89eb8352da30) - Scale AI - San Francisco, CA - $218k – $273k/yr
- [Machine Learning Research Engineer, Agent Data Foundation - Enterprise GenAI](https://hotfix.jobs/jobs/c04650e4-6722-4170-ae84-943c4d2aac4f) - Scale AI - San Francisco, CA - $218k – $273k/yr

**Apply:** https://hotfix.jobs/jobs/76655dad-b39d-48cd-8de2-290b953d6d0c
**Canonical:** https://hotfix.jobs/jobs/76655dad-b39d-48cd-8de2-290b953d6d0c