# Senior Machine Learning Engineer, Applied AI Quality

**Company:** [Square](https://hotfix.jobs/companies/square)
**Location:** California
**Role:** ML Engineering
**Salary:** $195k – $343k/yr
**Experience:** 7+ years
**Skills:** LLMs, AI Agents, Machine Learning, Evaluation Systems, Retrieval Architectures, Python, Large-Scale Ai Systems, AI Infrastructure, Systems Architecture
**Posted:** 2026-07-15

> Senior ML Engineer leading architecture and strategy for LLM- and agent-powered quality evaluation systems at Block. Builds scalable AI frameworks to measure product behavior, detect issues, and generate insights across millions of interactions to improve reliability and decision-making.

## Job Description

## You Will
- Lead the technical strategy and architecture for AI-driven quality and evaluation systems used across products and teams.
- Drive the development of scalable systems that use LLMs, agents, and behavioral signals to evaluate quality, detect regressions, and generate product insights.
- Define long-term approaches for evaluation, measurement, and quality intelligence across complex product surfaces.
- Translate ambiguous organizational needs into clear technical direction, roadmap priorities, and platform capabilities.
- Influence engineering standards and best practices for building reliable, measurable, and trustworthy AI systems.
- Lead complex cross-functional initiatives spanning product, infrastructure, data, and applied AI teams.
- Mentor and level up engineers across the organization through technical leadership, design reviews, and systems thinking.
- Identify leverage opportunities where AI systems can fundamentally improve how teams understand, debug, and improve product behavior.

## You Have
- 5+ years of experience in software engineering, machine learning engineering or applied AI.
- Deep experience designing and shipping large-scale AI/ML systems in production environments.
- Strong expertise with LLMs, agents, evaluation systems, retrieval architectures, and modern AI infrastructure.
- Proven ability to lead ambiguous, high-impact technical initiatives from concept through adoption across multiple teams.
- Strong systems thinking and architectural judgment, with the ability to balance experimentation, scalability, and operational rigor.
- Experience defining technical strategy and influencing roadmaps beyond your immediate team.
- Excellent communication and cross-functional leadership skills, with the ability to align engineering, product, and organizational priorities.
- A track record of creating leverage through platforms, frameworks, and systems that enable other engineers and teams to move faster and make better decisions.

## Similar roles

- [Senior AI Product Engineer](https://hotfix.jobs/jobs/f7bd5db3-c830-40ef-8d16-ccb47ad56d74) - TruckSmarter - San Francisco, CA - $195k – $225k/yr
- [Senior Software Engineer, ML Infrastructure](https://hotfix.jobs/jobs/fd26a76a-7098-47cd-a074-bbea82b2dfba) - Nuro - Mountain View, CA - $194k – $291k/yr
- [Technical Lead, Evaluation Infrastructure](https://hotfix.jobs/jobs/b6babc41-afd4-46cd-95fb-a21895bd696e) - Nuro - Mountain View, CA - $194k – $291k/yr
- [Senior Software Engineer, Perception ML Data](https://hotfix.jobs/jobs/26a84452-76f9-4466-afdf-9bd549c4add7) - Nuro - Mountain View, CA - $194k – $291k/yr
- [Senior Machine Learning Engineer](https://hotfix.jobs/jobs/bb887dd9-d892-4716-9554-22f5c5b3a0b5) - Airbnb - Remote - $196k – $227k/yr

**Apply:** https://hotfix.jobs/jobs/5c7dc4a3-fd38-4849-89ec-06e3d7a37661
**Canonical:** https://hotfix.jobs/jobs/5c7dc4a3-fd38-4849-89ec-06e3d7a37661