Software Test Engineer

Build automated and exploratory testing systems for products, APIs, AI models, and data platforms. The role requires 5+ years of software or QA engineering experience, strong scripting skills, and the ability to evaluate metrics, reliability, and model behavior.

About the job

Responsibilities

Define and execute test plans across products, APIs, SDKs, model-powered features, and data platforms.
Design, build, and maintain automated functional, integration, end-to-end, regression, API, browser, and service-level test suites and frameworks.
Translate requirements and customer acceptance criteria into test strategies, repeatable test cases, and release gates.
Build customer-focused and adversarial datasets, fixtures, and test environments covering edge cases, failure modes, and system limits.
Validate speech-to-text, text-to-speech, and other AI behavior using metrics, expected outputs, human review, and regression coverage.
Build test harnesses, reusable scripts, test-data tooling, result pipelines, dashboards, and visualizations.
Integrate automated tests, quality checks, canaries, and release validation into CI/CD.
Test data ingestion, processing, annotation, and quality-control workflows for integrity, completeness, representativeness, deduplication, leakage, and readiness.
Execute staging and production validation, load and reliability testing, cross-browser testing, customer-workflow testing, and user acceptance testing.
Maintain test repositories, automation coverage, documentation, and release-readiness reporting.
Write precise bug reports and participate in triage and escalation.
Contribute to code reviews, test-design reviews, and technical discussions.

Requirements

Bachelor's, master's, or doctoral degree in Computer Science, AI, Applied Math, or a related field, or equivalent experience.
5+ years of professional software or QA engineering experience, including shipped test infrastructure or evaluation systems.
Backend and scripting experience with Python, Rust, Go, or a similar language.
Experience designing automated test pipelines, evaluation frameworks, or data-processing systems.
Strong analytical skills and ability to reason about metrics, thresholds, statistical variation, and regression noise.
Ability to lead ambiguous technical challenges and communicate across research, engineering, and product teams.

Nice to Have

Experience testing or evaluating LLMs, RAG pipelines, agents, or multimodal models.
Experience with voice, audio, speech recognition, or real-time systems and metrics such as WER, MOS, latency, and time-to-first-byte.
Experience building test, evaluation, benchmarking, or ML infrastructure used by multiple teams or external users.
Experience with React Native, mobile applications, or cross-platform test tooling.
Familiarity with cloud infrastructure, containers, ephemeral test environments, CI/CD, Grafana, canaries, and anomaly detection.
Open-source contributions, reviews, maintenance, or community engagement.
Experience bridging product, QA, evaluation, training, inference, data, and agent-framework teams.
Appreciation for correctness, reproducibility, determinism, and consistency across environments.

Skills

Python, Rust, Go, Automated Testing, Test Automation, CI/CD, Api Testing, Integration Testing, End-To-End Testing, Data Processing, Llm Evaluation, RAG, Grafana, React Native, Load Testing