Lead Software Development Engineer in Test - AI
Leads quality strategy for the AI vertical, including scalable automation frameworks and testing for RAG, hallucination, drift, and bias. The role requires 7+ years in QA/SDET work, strong technical leadership, and experience with AI testing methodologies.
About the job
Responsibilities
- Define the overall QA strategy and architect large-scale, reusable, cross-platform automation frameworks.
- Drive organization-wide quality initiatives and own the quality charter for the AI vertical.
- Design scalable testing approaches for RAG, hallucination, drift, and bias in AI tools and models.
- Own decisions on test strategy, tools, frameworks, and AI testing methodologies.
- Coach and upskill the entire team, establish best practices, and build a talent pipeline.
- Influence cross-functional teams, participate in architecture discussions, and align testing with business strategy.
- Own end-to-end quality strategy at the product or vertical level.
- Shape the AI quality roadmap, reduce organization-wide risk, and improve scalability and reliability.
Requirements
- 7+ years of overall experience in QA or SDET roles.
- Deep expertise in AI tools and model testing.
- Experience designing scalable test approaches for RAG, hallucination, drift, and bias.
- Proven experience architecting large-scale, reusable, cross-platform automation frameworks.
- Strong leadership skills, including coaching teams, setting best practices, and building talent pipelines.
- Ability to influence cross-functional teams and participate in architecture discussions.
- Strong problem-solving skills and the ability to make decisions about test strategy, tools, frameworks, and AI testing methodologies.
Nice-to-Haves
- Experience with CI/CD pipelines and cloud platforms such as AWS, Google Cloud, or Azure.
- Familiarity with MLOps pipelines and monitoring AI systems.
- Exposure to performance and scalability testing of AI workloads.
Skills
Ai Testing, Model Testing, RAG, Hallucination Testing, Drift Testing, Bias Testing, Test Automation, CI/CD, AWS, GCP, Azure, MLOps, Ai Monitoring, Performance Testing, Scalability Testing
Similar jobs
QA Engineering jobs
Circle
Remote
Senior Quality Engineer partnering across product teams to assess risk, build web and API test automation, investigate technical failures, and improve quality practices. Requires strong Playwright and TypeScript experience, exploratory testing skills, CI/CD knowledge, and the ability to debug application code.
Fivetran
Bengaluru, India
Designs and maintains scalable Java-based test automation frameworks for data pipelines, databases, and distributed systems. The role also develops AI/LLM-assisted testing tools, strengthens CI/CD validation, and requires 6+ years of software industry experience.
Celonis
Bengaluru, India
Build AI-powered quality engineering and automation for a high-scale accounts receivable product, validating Salesforce–Celonis integrations across UI, APIs, data pipelines, and performance. The role requires 6 years of technical QA experience, strong Playwright expertise, and familiarity with LLM-based testing tools.
Fivetran
Bengaluru, IndiaThe Senior SDET designs scalable Java automation frameworks, test strategies, and CI/CD validation for distributed data systems. The role also develops AI-assisted testing and debugging tools while improving reliability, automation efficiency, and engineering productivity.
Okta
Bengaluru, India
Owns test strategy, automation frameworks, and quality assurance for the Okta Verify macOS application. The role requires senior-level software engineering in test experience, native macOS testing expertise, and proficiency with Swift or related programming languages and CI/CD tooling.