# Engineering Manager - Model Performance

**Company:** [Baseten](https://hotfix.jobs/companies/baseten)
**Location:** San Francisco, CA, New York, NY
**Role:** Engineering Management
**Salary:** $260k – $380k/yr
**Experience:** 5+ years
**Skills:** PyTorch, TensorRT, CUDA, Python, C++, Go, Docker, Kubernetes, LLMs
**Posted:** 2024-09-12

> Leads a team optimizing ML model inference and performance, focusing on frameworks like PyTorch, TensorRT, and CUDA. Requires 5+ years software engineering experience with 2+ years technical leadership, plus expertise in production AI/ML scaling.

## Job Description

## Responsibilities
- Lead, mentor, and manage a team of engineers focused on developing and optimizing ML model inference and performance.
- Oversee technical strategy and architecture decisions, driving improvements across our engineering organization.
- Collaborate with cross-functional teams to ensure seamless integration and scalability of ML models in production environments.
- Dive into the codebase of frameworks like **TensorRT**, **PyTorch**, **CUDA**, and others to identify and solve complex performance bottlenecks.
- Drive the development and deployment of large-scale optimization techniques for various ML models, especially large language models (**LLMs**).
- Own the full lifecycle of projects from inception through delivery, including planning, execution, and resource management.
- Foster a collaborative, inclusive team environment that encourages continuous learning and growth.

## Requirements
- Bachelor’s, Master’s, or Ph.D. in Computer Science, Engineering, or a related field.
- 5+ years of professional experience in software engineering, with at least 2 years in a technical leadership role.
- Proven experience managing and mentoring teams of engineers.
- Expertise in one or more programming languages, such as **Python**, **C++**, or **Go**.
- In-depth understanding of ML model performance optimization, especially using libraries such as **PyTorch**, **TensorRT**, and **CUDA**.
- Strong knowledge of containerization (**Docker**) and orchestration systems (**Kubernetes**).
- Experience with production-level AI/ML solutions, including scaling and deploying large models.
- Ability to balance hands-on technical work with team leadership and project management.

## Bonus Points
- Experience enhancing the performance of large language models (**LLMs**) or similar AI systems.
- Familiarity with LLM optimization techniques such as quantization, speculative decoding, or continuous batching.
- Deep knowledge of GPU architecture and performance tuning.
- Previous experience in a high-growth startup environment.

## Benefits
- Competitive compensation, including meaningful equity.
- 100% coverage of medical, dental, and vision insurance for employee and dependents.
- Generous PTO policy including company wide Winter Break.
- Paid parental leave.
- Company-facilitated 401(k).
- Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.

## Similar roles

- [Engineering Manager - Forward Deployed Engineering (LLM)](https://hotfix.jobs/jobs/51279c3d-a4d8-41f6-aeed-4f95a288b3cd) - Baseten - San Francisco, CA - $260k – $380k/yr
- [Engineering Manager, Product](https://hotfix.jobs/jobs/8a722d90-e6aa-4ccb-b89f-2e7a699a0614) - Harvey - New York, NY - $260k – $330k/yr
- [Engineering Manager, Product](https://hotfix.jobs/jobs/e0d3eeef-bb77-4f32-8dc1-00801b6d7af0) - Harvey - San Francisco, CA - $260k – $330k/yr
- [Manager, Software Engineering](https://hotfix.jobs/jobs/08b204a0-1a8e-4bf2-b23c-6f8cf3a6c5b2) - Figma - Remote - $258k – $376k/yr
- [Manager, Software Engineering - Storage Platform](https://hotfix.jobs/jobs/3a3793f3-3772-4c0f-8d0d-7afdd4408eb7) - Figma - Remote - $258k – $376k/yr

**Apply:** https://hotfix.jobs/jobs/96a49fbd-49b3-4757-983f-d40719320379
**Canonical:** https://hotfix.jobs/jobs/96a49fbd-49b3-4757-983f-d40719320379