Senior QA Engineer

Responsibilities

• Review requirements specifications and technical design documents to provide timely and meaningful feedback.
• Create detailed, comprehensive, and well-structured test plans and test cases; analyze and update existing test cases.
• Lead and assist junior QA Analysts.
• Estimate, prioritize, plan, and coordinate testing activities.
• Design, develop, and execute automation scripts using TypeScript, Playwright, or comparable frameworks such as Cypress, Selenium, or WebdriverIO.
• Use AI coding assistants and agentic tools, such as Claude Code, to accelerate test authoring, maintain automation scripts, and triage defects.
• Build, customize, and maintain reusable AI skills and agentic workflows for test generation, regression checks, and defect documentation.
• Apply spec-driven development practices, including the AI-Driven Development Lifecycle, to guide AI-assisted test generation and validation.
• Design and run evaluations for AI-powered features using defined test datasets and quality metrics.
• Apply non-deterministic testing approaches for AI/LLM features using statistical thresholds, sampling, and repeated-run analysis.
• Configure and monitor AI observability and evaluation tools such as Arize AX/Phoenix and LangSmith to track performance, detect drift, and flag regressions.
• Identify, record, document, track, and close bugs.
• Perform regression testing when bugs are resolved.
• Transfer knowledge through documentation and QA presentations, including AI-assisted testing practices and tooling.

Requirements

• Proven experience in manual and automated QA testing of web applications.
• Proficiency in TypeScript or JavaScript with hands-on experience in Playwright or a comparable automation framework.
• Practical experience using AI coding assistants and agentic tools such as Claude Code or GitHub Copilot for test automation.
• Working understanding of spec-driven development methodologies such as AI-DLC.
• Ability to build and maintain custom AI skills, prompts, or agent workflows.
• Understanding of non-deterministic testing for AI/LLM outputs.
• Experience designing evaluation datasets and metrics including accuracy, relevance, hallucination, toxicity, and latency.
• Familiarity with AI observability and evaluation platforms such as Arize AX/Phoenix, LangSmith, or Braintrust.
• Solid grounding in QA methodologies, the SDLC, and test planning and estimation.
• Strong analytical, written, and verbal communication skills, including the ability to mentor junior QA Analysts.
• Bachelor's degree in a technical field and 5+ years of experience, or equivalent experience.

Compensation and Benefits

• Fully remote-first work with flexible work arrangements.
• Comprehensive health and wellness benefits, including retirement savings programs, eligibility for two bonus plans, paid time off, and medical and dental benefits based on country of location.
• New hire equipment allowance and monthly flex allowance.
• Career growth and internal mobility opportunities.
• Employee-driven diversity, equity, and inclusion programs.