Senior Software Engineer building and operating Coinbase's central LLM and AI agent platform infrastructure. Owns LLM Gateway, knowledge bases, agent runtime, FinOps, evaluation tooling, and cross-team applied AI strategy. Requires 5+ years building ML/AI infrastructure at scale.
186k – 219k
Remote5+ YOEML Engineering
About the role
What you'll do
Own the architecture and delivery of core platform systems including the LLM Gateway (60+ models, auth, PII redaction, fallbacks, cost optimization), AI Hub, and agent runtime with microVM sandboxes and governed MCP gateway
Drive the design and implementation of Knowledge Base infrastructure, connecting data sources to auto-provisioned vector and markdown stores queryable by any agent
Lead AI FinOps capabilities including spend attribution, governance, and cost optimization across all AI workloads company-wide
Partner across engineering, security, legal, finance, product, and external partners at frontier labs and major cloud providers to ship high-impact platform capabilities
Build evaluation and observability tooling including LLM-as-judge harnesses, full tracing, and feedback loops that let subject matter experts refine production agents
Shape applied AI strategy by embedding with teams to build production agents, applying fine-tuning and traditional ML where it outperforms prompting
Required Skills and Experience
5+ years of software engineering experience with demonstrated expertise building ML/AI infrastructure, LLM systems, or distributed platform services at scale
Demonstrated ability to lead multi-quarter initiatives end-to-end, from technical design through production operation, with measurable platform-wide impact
Deep expertise in LLM orchestration patterns, agent frameworks, vector databases, and model serving infrastructure across multiple providers
Proven track record of cross-functional partnership with security, legal, and product teams to ship governed, production-grade AI systems
Fluency in Python and at least one systems language (Go, Rust, or C++), with production experience building APIs, microservices, and runtime environments
Utilizes generative AI responsibly, maintaining human oversight to deliver business-ready outputs and drive measurable improvements in workflow efficiency, cost, and quality.
Skills
PythonGoRustC++Llm GatewayLlm OrchestrationAgent FrameworksVector DatabasesModel ServingKnowledge BasesMicrovm SandboxesObservability ToolingFine-TuningDistributed Systems
Senior Software Engineer - AI Code Gen - Agent Engineer
DatadogNew York, NY
Build AI-assisted tools using LLMs to generate tests, fix bugs, and boost developer productivity at Datadog. Requires 5+ years backend experience in Go/Java/Python and shipping LLM software to production.
187k – 240k
Hybrid5+ YOEML Engineering
Senior AI Engineer - APM Experiences
DatadogNew York, NY
Leads development of LLM- and agent-based features for Datadog's APM to detect, resolve, and prevent performance issues using traces, metrics, and logs. Requires 4+ years in backend/ML systems, production AI experience, and distributed systems expertise.
187k – 240k
On-site4+ YOEML Engineering
Sr Software Engineer, Cribl AI
CriblUnited States
Build and productionize AI integrations across Cribl's observability platform. Requires 5+ years building AI features end-to-end with fullstack experience in React, TypeScript/JavaScript, and backend systems plus hands-on LLM/agent work.
185k – 215k
Remote5+ YOEML Engineering
Sr Machine Learning Engineer, AI Research
CriblUnited States
Design, train, and productionize ML models and pipelines for AI-enabled security/observability platforms. Requires 4+ years experience with Python, PyTorch/TensorFlow, and modern ML techniques.
185k – 215k
Remote4+ YOEML Engineering
Senior Machine Learning Engineer - Scene Understanding
ZooxFoster City, CA +1
Develops advanced Vision-Language-Action models for robotaxi scene understanding, detecting hazards and enabling safe driving. Leads data strategies, post-training of large models, and deployment using PyTorch and production ML pipelines. Requires MS/PhD in CS and deep learning expertise.