Senior Backend Engineer - AI Agents
Builds scalable, low-latency backend systems and orchestration frameworks for production AI agents in real-time enterprise environments. Requires 5+ years of backend, distributed systems, or platform engineering experience, with strong API, cloud, and microservices expertise.
About the job
Responsibilities
- Design and build scalable backend systems powering real-time AI Agents in enterprise environments.
- Develop agent orchestration frameworks for multi-step reasoning, tool usage, and decisioning workflows.
- Build systems for agent memory, context management, and state persistence across interactions.
- Architect low-latency inference pipelines integrating LLMs, SLMs, and external tools and services.
- Implement evaluation frameworks to measure agent performance, accuracy, and reliability.
- Enable continuous improvement loops from feedback through retraining and deployment.
- Design and manage event-driven, asynchronous workflows for complex agent tasks.
- Optimize systems for high throughput, low latency, and cost-efficient inference at scale.
- Build and maintain REST and gRPC APIs and service layers for agent capabilities.
- Partner with Applied AI and ML teams to productionize models and agent behaviors.
- Collaborate with Product and Solutions teams to translate customer workflows into agentic systems.
- Drive best practices in observability, monitoring, safety, and guardrails for AI systems.
- Contribute to architecture decisions for scaling multi-tenant, enterprise-grade AI platforms.
Requirements
- 5+ years of experience in backend engineering, distributed systems, or platform engineering.
- Strong experience building high-scale, production-grade backend systems.
- Experience designing systems for real-time processing, streaming, or event-driven architectures.
- Strong understanding of API design, including REST and gRPC, and microservices architectures.
- Experience with SQL and NoSQL databases and data modeling for high-scale systems.
- Hands-on experience with Docker, Kubernetes, and cloud platforms such as AWS, Google Cloud, or Azure.
- Strong fundamentals in system design, concurrency, and performance optimization.
Nice-to-Haves
- Experience working with LLMs, conversational AI, or AI-powered products in production.
- Familiarity with agent frameworks, tool calling, or multi-step reasoning systems.
- Experience building or integrating RAG pipelines, vector databases, or retrieval systems.
- Exposure to offline or online evaluation systems and A/B testing for AI systems.
- Understanding of prompting strategies, context windows, and model behavior optimization.
- Experience with real-time decisioning systems or workflow orchestration engines.
Skills
Python, Rest, gRPC, Microservices, SQL, NoSQL, Docker, Kubernetes, AWS, GCP, Azure, Distributed Systems, LLMs, RAG, Vector Databases
Similar jobs
Backend Engineering jobs
ZoomInfo
Bengaluru, India
Leads the architecture and scaling of distributed backend systems for AI-powered talent search, matching, and agentic workflows. Requires 7+ years of backend experience, strong cloud and data expertise, and proficiency in a modern object-oriented language.
CommandLink
United StatesSenior individual contributor responsible for designing, building, operating, and improving large-scale backend services, APIs, and telemetry pipelines in Go and Python. The role requires production systems ownership, distributed-systems expertise, incident leadership, mentoring, and technical design leadership.
Navan
Bengaluru, India
Senior backend software engineer building and scaling Java and Spring Boot services for a global expense management platform. The role requires 4–6 years of experience with distributed systems, cloud platforms, production ownership, and AI-driven development.
Phantom
Remote
Senior backend engineer designing scalable services and APIs that connect Phantom Cash to blockchain networks. The role requires Node.js/NestJS, REST API, event-driven, and distributed-systems expertise, with blockchain or fintech experience preferred.
Rippling
Bengaluru, India
Senior software engineer responsible for designing reliable, distributed tax remittance systems operating globally across currencies, agencies, and compliance requirements. Requires 6+ years of programming experience and strong backend architecture and distributed-systems expertise, with Python, Django, React, MongoDB, and AWS preferred.