Member of Technical Staff, QA
Owns end-to-end QA for Vapi's voice AI platform, building Playwright e2e tests, testing integrations (LLMs, STT, TTS, telephony), filing bugs, and partnering with engineers on releases. Requires 5+ years QA/SDET experience, TypeScript/JavaScript proficiency, and AI coding tools usage.
About the job
What You’ll Do
First 30 Days
- Build and deploy real assistants on Vapi, run live calls and simulations across our main provider integrations (LLMs, STT, TTS, telephony), and break the product in at least a few interesting ways.
- Get up and running in our existing e2e test framework and ship your first Playwright tests against the dashboard.
- File bugs that engineers fix, and develop a working point of view on where our coverage is weakest.
By 60 Days
- Own release testing for at least one major product area (assistant builder, simulations, monitoring, knowledge base, or versioning), and earn engineers' trust to gate it.
- Meaningfully improve automated coverage in your area.
- Partner with engineers on test strategy for new features before they ship.
- Use AI coding agents to multiply your throughput on test scenarios and regression coverage.
By 90 Days
- Operate as a full peer to engineers on the team, owning QA across the product surface without handholding.
- Extend our testing practices and contribute patterns the rest of the team picks up.
- Be involved in new features from the start, with the automation suite meaningfully larger on your watch.
Who You Are
Must-Haves
- 5+ years in QA, SDET, Test Engineering, or product engineering with a strong testing focus.
- Hands-on with at least one modern e2e framework — Playwright preferred; Cypress, Selenium, or equivalent are fine.
- Comfortable reading and writing TypeScript / JavaScript well enough to extend our test suite and file bugs with a hypothesis attached.
- Track record testing complex, distributed systems with real failure modes — APIs, integrations, real-time behavior, anything where "correct" isn't a fixed string.
- Daily use of AI coding agents (Claude Code, Cursor, or similar) with opinions about where they help and where they don't.
Nice-to-Haves
- Background in voice AI, conversational AI, LLM applications, or telephony.
- Experience with LLM evals or testing non-deterministic systems.
- Worked on a developer-facing product — APIs, SDKs, dashboards — where the audience is technical.
Skills
Playwright, TypeScript, JavaScript, Cypress, Selenium, Ai Coding Agents, APIs, Llm Evals, Distributed Systems, E2E Testing
Similar jobs
QA Engineering jobs
Baseten
San Francisco, CABuild and own the testing frameworks, integration environments, performance tooling, and resilience capabilities that enable reliable AI inference infrastructure. The role requires strong Go or Python skills, distributed-systems testing experience, Kubernetes and Docker expertise, and the ability to improve test reliability at scale.
Discord
San Francisco, CA
Owns automated QA suite development and maintenance for Oracle ERP Cloud and Salesforce, integrating Tosca-based regression testing into CI/CD pipelines and quarterly release cycles. Requires 3–5+ years of QA automation experience, Tosca expertise, enterprise ERP testing, and scripting or pipeline skills.
OnePay
United States
Build and scale shared test automation frameworks and execution infrastructure for web, mobile, and backend systems. The role requires 5+ years in test automation or platform engineering, strong TypeScript/Node.js skills, and experience with Kubernetes, AWS, and cross-stack testing.
Deepgram
United States
Build automated and exploratory testing systems for products, APIs, AI models, and data platforms. The role requires 5+ years of software or QA engineering experience, strong scripting skills, and the ability to evaluate metrics, reliability, and model behavior.
Skydio
San Mateo, CA
Owns requirements, architecture, integration, verification, testing, and failure analysis for autonomous flying robots. The role requires 3+ years of experience with reliable electromechanical systems, cross-domain troubleshooting, test development, and Python or SQL-based data analysis.