# Senior Software Development Engineer in Test

**Company:** [Dialpad](https://hotfix.jobs/companies/dialpad)
**Location:** Vancouver, Canada
**Role:** QA Engineering
**Salary:** CA$151k – CA$175k/yr
**Experience:** 6+ years
**Skills:** Python, Java, JavaScript, GCP, Cloud Run, App Engine, Kubernetes, Datastore, Redis, Elasticsearch, Vue 3, React, LLM APIs, Livekit, GitHub Actions
**Posted:** 2026-08-24

> Owns automated quality engineering for AI Voice Agent services, spanning UI, backend, APIs, and audio/text interactions. The role requires 6+ years of software engineering or SDET experience, strong programming skills, cloud-native testing expertise, and hands-on experience with LLM systems and evaluation.

## Job Description

## Responsibilities
- Own end-to-end quality for agentic features and workflows, including strategy, development, execution, and release qualification.
- Design and build automation tooling and frameworks for AI/LLM-driven systems, including prompt flows, agent orchestration, and tool integrations.
- Develop and maintain evaluation frameworks to measure response quality, accuracy, and hallucination rates.
- Drive automation coverage of 80%+ for critical AI workflows using deterministic and probabilistic validation approaches.
- Integrate AI quality checks into CI/CD pipelines with PR validation feedback cycles under 15 minutes.
- Build tooling for LLM observability and debugging, including prompt tracing and response analysis.
- Partner with Applied AI teams on prompt engineering, model selection, and evaluation strategies.
- Design and execute performance and load tests for AI services, including latency, throughput, and cost efficiency.
- Identify and mitigate risks related to hallucinations, bias, safety, and edge cases.
- Define and track AI quality KPIs such as task success rates, precision, recall, and latency.
- Participate in design and architecture reviews to ensure systems are testable, observable, and resilient.
- Mentor engineers and contribute to AI quality engineering practices.

## Requirements
- 6+ years of experience in software engineering or SDET roles, with an emphasis on software development.
- Strong programming skills in Python, Java, or JavaScript.
- Experience testing distributed, cloud-native SaaS systems and APIs.
- Experience coding with AI agents to accelerate development and improve code quality.
- Hands-on exposure to LLMs or AI/ML systems, such as OpenAI, Claude, or Gemini.
- Understanding of non-deterministic systems and probabilistic testing approaches.
- Experience building test frameworks and scalable automation systems.
- Familiarity with AI evaluation techniques, including benchmarking, golden datasets, and human-in-the-loop validation.
- Experience with CI/CD pipelines such as Jenkins or GitHub Actions.
- Strong collaboration skills across distributed teams and time zones.
- Bachelor’s degree in Computer Science or equivalent practical experience.

## Technical Stack
- **Backend:** Python, Go, Google Cloud Platform, Cloud Run, App Engine, Kubernetes, Datastore, Redis, Elasticsearch
- **Frontend:** Vue 3, React
- **AI:** LLM APIs, LiveKit, prompt orchestration frameworks, evaluation tooling

## Compensation and Benefits
- Base salary range for British Columbia, Canada: **$150,500–$175,250 CAD**.
- Salary range reflects base salary only and excludes bonus, equity, and benefits.
- Competitive benefits, training, AI tools, and growth opportunities.

## Similar jobs

- [Senior Software Development Engineer in Test](https://hotfix.jobs/jobs/9dd83fa4-8f54-447b-a9c3-a931af3991e9) - Dialpad - Kitchener, Canada - CA$149k – CA$174k/yr
- [Senior Quality Engineer](https://hotfix.jobs/jobs/7af0e7ee-6cfd-4b65-9ef1-1bf0e4bc2c7b) - Circle - Remote - $120k – $130k/yr
- [Software Engineer - Testing Frameworks](https://hotfix.jobs/jobs/e2c14dc9-5b7e-41e9-819d-bd621388ce52) - Baseten - San Francisco, CA - $165k – $330k/yr
- [Ingénieur de Test en Fabrication, Solutions Urbains](https://hotfix.jobs/jobs/6ce144b9-ade5-418f-b12c-bd65871ba381) - Lyft - Longueuil, Canada

**Apply:** https://hotfix.jobs/jobs/86218182-10dd-447f-a264-9fb9ea0f417d
**Canonical:** https://hotfix.jobs/jobs/86218182-10dd-447f-a264-9fb9ea0f417d