Senior Software Development Engineer In Test - AI Cluster Networking And Security
Leads automated validation of large-scale AI clusters across distributed software, networking, hardware, security, reliability, and observability. The role requires 10+ years of testing experience, strong coding and debugging skills, and expertise in datacenter networking and infrastructure.
About the job
Responsibilities
- Design and execute tests for large-scale AI infrastructure deployments.
- Define optimized test strategies and methodologies for distributed systems.
- Break large distributed-system challenges into components that can be unit tested.
- Use an automation-first approach, targeting highly automated coverage of cluster features.
- Test high availability, failure scenarios, performance, stress, and security.
- Champion cluster security, reliability for 99.9999% uptime, and observability.
- Test AI cluster components, including Kubernetes, Prometheus, Grafana, ML wafer-scale accelerators, CPU runtime nodes, SwarmX interconnects, and MemoryX interconnects.
- Qualify cluster networking solutions, including high-speed switches, routers, and optics from multiple vendors.
- Qualify cluster security features, including operating-system security, network security, cloud compliance, user access, and security certifications.
Requirements
- Bachelor's or master's degree in computer science, electrical engineering, AI, data science, or a related field.
- 10+ years of experience testing enterprise software, distributed systems, datacenter hardware, or datacenter software.
- Experience with enterprise or cloud networking infrastructure, high-speed switches, routers, and firewalls.
- Experience qualifying networking platforms from Juniper, Arista, or Cisco and network test equipment such as Ixia or Spirent.
- Experience with datacenter technologies including BGP, ECN, and PFC.
- Experience testing network security, compliance, and firewalls.
- Strong coding skills in Python, Golang, or C/C++.
- Strong debugging skills for large distributed systems, hardware, and software.
- Experience with debugging tools such as GDB, strace, and network monitors.
- Understanding of operating-system internals, including memory management, filesystems, security fundamentals, and performance.
- Understanding of datacenter layouts and device performance characteristics, including PCIe, networking, and storage.
- Experience with cloud technologies such as AWS, Kubernetes, and Docker.
Nice-to-haves
- Experience with Grafana and Prometheus.
- Understanding of ML model training and inference.
- Experience with ML hardware accelerators, including GPUs and custom accelerator ASICs.
Skills
Python, Go, C++, Kubernetes, Prometheus, Grafana, AWS, Docker, BGP, Ecn, Pfc, Juniper, Arista, Cisco, Firewalls
Similar jobs
QA Engineering jobs
Circle
Remote
Senior Quality Engineer partnering across product teams to assess risk, build web and API test automation, investigate technical failures, and improve quality practices. Requires strong Playwright and TypeScript experience, exploratory testing skills, CI/CD knowledge, and the ability to debug application code.
Fivetran
Bengaluru, India
Designs and maintains scalable Java-based test automation frameworks for data pipelines, databases, and distributed systems. The role also develops AI/LLM-assisted testing tools, strengthens CI/CD validation, and requires 6+ years of software industry experience.
Celonis
Bengaluru, India
Build AI-powered quality engineering and automation for a high-scale accounts receivable product, validating Salesforce–Celonis integrations across UI, APIs, data pipelines, and performance. The role requires 6 years of technical QA experience, strong Playwright expertise, and familiarity with LLM-based testing tools.
Fivetran
Bengaluru, IndiaThe Senior SDET designs scalable Java automation frameworks, test strategies, and CI/CD validation for distributed data systems. The role also develops AI-assisted testing and debugging tools while improving reliability, automation efficiency, and engineering productivity.
Okta
Bengaluru, India
Owns test strategy, automation frameworks, and quality assurance for the Okta Verify macOS application. The role requires senior-level software engineering in test experience, native macOS testing expertise, and proficiency with Swift or related programming languages and CI/CD tooling.