# Member of Technical Staff, Evals & Post-Training Product

**Company:** [Fireworks AI](https://hotfix.jobs/companies/fireworks-ai)
**Location:** San Mateo, CA
**Role:** Fullstack Engineering
**Skills:** Llm Evaluations, Post-Training, Fine-Tuning, Sft, Rft, APIs, SDKs, Full-Stack, Generative AI, GPU, Inference Optimization
**Posted:** 2026-03-24

> Develops evaluation workflows and fine-tuning products for LLM model improvement, working across full-stack from APIs to web interfaces. Requires 1-7 years software engineering with hands-on LLM evals/post-training experience and user-centric product skills.

## Job Description

## Key Responsibilities

* Build internal eval workflows: Design and scale evaluation tooling used by internal teams to measure model quality, compare model changes, and inform post-training decisions.
* Own fine-tuning product experiences: Build and improve user-facing product workflows for post-training, including fine-tuning experiences across SFT, RFT, and related model-improvement capabilities.
* Work closely with users: Partner with customers and internal stakeholders to understand evaluation and fine-tuning needs, support high-priority engagements, triage issues, and convert bespoke workflows into productized solutions.

## Minimum Requirements

* 1 - 7 years of software engineering experience (We are hiring at multiple levels for this role).
* Hands-on experience with LLM evaluations and/or post-training methods: How to design useful evals and use their results to guide model improvement.
* Product Engineering Skills: The ability to work across backend systems and developer-facing product surfaces. Comfortable shipping full-stack features when needed.
* Understanding of the GenAI Lifecycle: You understand the end-to-end workflow—from prompting a base model to curating a dataset, fine-tuning, and productionizing agents—and how these steps interconnect.
* User-Centric Mindset: Willing to talk to users, triage GitHub issues for open-source projects, and build products from scratch to serve emerging needs.

## Preferred Qualifications

* 3+ years of software engineering experience.
* Domain-Specific Evaluation Experience: Strong familiarity with designing and running evaluations for domain-specific use cases (e.g. medical, legal, coding, or custom internal datasets).
* Open Source Contributions: Prior contributions to developer tools or AI/ML repositories.
* Inference & Hardware Knowledge: Interest in the hardware side of AI—understanding GPU constraints, inference optimization techniques, and how they relate to model performance.
* Startup DNA: Experience in fast-paced environments where you own features end-to-end.

## Similar roles

- [Staff Software Engineer](https://hotfix.jobs/jobs/a083e5c9-e532-4a65-90bd-4c7152d0d7e9) - Commure - Mountain View, CA - $210k – $275k/yr
- [Staff Product Engineer](https://hotfix.jobs/jobs/796207a1-77e1-4f86-bef0-da9658cea69a) - Aurelian - Seattle, WA - $200k – $300k/yr
- [Senior Staff Software Engineer, Client Architecture](https://hotfix.jobs/jobs/29a99e15-fdc9-4209-8905-7203695bf08c) - Reddit - Remote - $233k – $326k/yr
- [Staff+ Software Engineer, Safeguards Review Tooling](https://hotfix.jobs/jobs/9232ee3d-7cd8-4e17-b2a4-affdb44459af) - Anthropic - San Francisco, CA - $320 – $485k
- [Staff Software Engineer, RCM Billing](https://hotfix.jobs/jobs/23c036cd-de44-4541-b0cd-229ea3ec9123) - Commure - Mountain View, CA - $170k – $240k/yr

**Apply:** https://hotfix.jobs/jobs/a1a8a21b-8ceb-4383-b369-ffc43719942a
**Canonical:** https://hotfix.jobs/jobs/a1a8a21b-8ceb-4383-b369-ffc43719942a