Member of Technical Staff (Software Engineer)
Develops and optimizes Kubernetes-based infrastructure for high-performance AI inference services, including deployment, scaling, debugging, and integration with ML workflows. Requires Master's in CS and 1+ year experience with Docker, Kubernetes, Python, and related tools.
About the job
Job Duties
- Implement infrastructure to support high-performance, low-latency inference service.
- Deploy and configure Kubernetes services to ensure scalability and reliability of inference workloads.
- Optimize resource allocation and auto-scaling policies to handle variable inference demand while minimizing operational costs.
- Integrate inference services with containerized environments using Docker and Kubernetes for orchestration.
- Ensure high availability and fault tolerance by implementing multi-region deployments and disaster recovery strategies.
- Develop Python-based scripts and APIs to streamline data preprocessing, inference execution, and post-processing for real-time inference tasks.
- Collaborate with machine learning engineers to validate inference accuracy and performance against functional and latency requirements.
- Triage and resolve defects in the service by analyzing logs, metrics, and distributed traces.
- Debug issues related to model deployment, container orchestration, or networking configurations, documenting steps to reproduce and root-cause defects.
- Collaborate with cross-functional teams to address performance regressions, scalability issues, or integration failures in the inference pipeline.
- Develop automated scripts to detect and mitigate common failure modes, improving system reliability.
- Author detailed technical documentation for infrastructure configurations, inference workflows, and APIs, ensuring clarity for internal teams and external customers.
- Work with product management and user experience teams to define requirements for inference service interfaces, including configuration, monitoring, and event logging.
- Document and track defects, enhancements, and release notes using tools like Jira and Git, ensuring version control and traceability.
- Participate in release planning and prioritization discussions to align infrastructure development with customer needs and business objectives.
Minimum Requirements
Master’s degree or foreign equivalent in Computer Science or related field and 1 year of experience as Software Developer, Student/Intern (Software Developer), Member of Technical Staff (Software Engineer), Software Engineer, or related occupation.
Required Skills:
- Docker and Kubernetes
- Java or C++
- ActiveMQ and Kafka
- Python or Groovy
- JavaScript or TypeScript
- Linux
- SQL, OracleDB, and Redis
- Git
Salary Range
$169,600 - $175,000 per year
Skills
Kubernetes, Docker, Python, Java, C++, Kafka, Activemq, Linux, Redis, SQL, Git, TypeScript, JavaScript, Groovy, Oracledb
Similar jobs
DevOps / SRE jobs
Baseten
San Francisco, CA
Leads global GPU capacity management across acquisition, orchestration, infrastructure automation, and incident response. The role requires 5+ years of experience, deep Kubernetes expertise, production Go or Python skills, and the ability to balance reliability with unit economics.
Ramp
New York, NYProduction Engineer responsible for building and operating scalable infrastructure, driving reliability and architecture initiatives, and enabling product teams through platform tooling. Requires software engineering experience, distributed-systems expertise, cloud experience, and cross-team technical leadership.
Benchling
San Francisco, CABuild developer-experience tooling and release systems within Benchling’s Platform team, helping engineering teams develop, test, package, and ship high-quality software rapidly. The role requires 4+ years of software engineering experience, web framework expertise, strong problem-solving, and effective cross-functional communication.
Roboflow
New York, NYInfrastructure Engineer responsible for securing, scaling, and operating cloud infrastructure and machine-learning platforms across a distributed startup. The role requires Kubernetes, infrastructure-as-code, cloud operations, CI/CD, programming, observability, and security experience.
Baseten
San Francisco, CABuild and operate continuous delivery infrastructure for Kubernetes deployments across global regions, including progressive rollouts, automated health evaluation, and rollback systems. The role requires strong Go or Python skills, large-scale Kubernetes experience, and familiarity with GitOps tooling.