Skip to content

Software Engineer, Cortex AI Infrastructure

Build and scale backend infrastructure powering agentic AI products including orchestration engines, RAG systems, evals infrastructure, and production AI workflows. Requires 4+ years distributed systems experience and deep Python plus Go/Java proficiency.

160k – 225kMenlo Park, CAML EngineeringOnsite4+ YOE

About the role

What you will do in this role

  • Build Agentic Runtimes: Help build and scale the orchestration engines that execute complex agentic workflows, ensuring low-latency tool execution and robust state management.
  • Scale Context Engineering Infra: Develop and design high-performance systems for RAG (Retrieval-Augmented Generation), including vector database integration, scalable search indexing, query processing, and semantic caching.
  • Develop the "Evals Engine": Build the automated infrastructure required to run massive-scale golden set simulations, error analysis pipelines, and experiments.
  • Productionize AI Workflows: Collaborate with the modeling team to take raw LLM capabilities and turn them into hardened, multi-tenant microservices with strict guardrails and observability.
  • Optimize Performance: Implement system optimizations for model routing, prompt caching, and token optimization to ensure maximum efficiency.

Requirements

  • Education: Bachelor's degree in Computer Science or a related technical field; Masters or PhD preferred.
  • Experience: 4+ years of industry experience designing, building, and supporting distributed systems, high-throughput APIs, machine learning platforms, or data-intensive systems.
  • Technical Stack: Deep proficiency in Python (for AI orchestration) and strong experience with Go or Java (for systems). Familiarity with C++ is a plus.
  • Systems Thinking: Strong understanding of database internals, distributed state management, and cloud-native architecture (e.g., Kubernetes, FoundationDB). Experience setting up and maintaining CI/CD pipelines is a plus.
  • Domain Expertise: Familiarity with the "plumbing" of AI, such as vector indices, agent platforms, building scalable data pipelines, and working with frameworks like PyTorch, TensorFlow, or XGBoost.
  • Mindset: A growth mindset and excitement about breaking the status quo by seeking innovative solutions.

Bonus

  • Query optimization and SQL engine internals.
  • Designing multi-tenant systems that handle sensitive enterprise data at scale.
  • Developing search infrastructure for large-scale applications.

Skills

PythonGoJavaKubernetesFoundationdbRAGVector DatabasesPyTorchTensorFlowXgboost

Similar roles

ML Engineering jobs

Research Scientist II

Research Scientist II building and improving fraud risk models and scam detection systems using audio, behavioral, and metadata signals. Requires an advanced degree and 3+ years of applied ML experience with Python and modern ML frameworks.

160k – 185kUnited StatesML EngineeringRemote3+ YOELLMsKeras

Applied Data Scientist / Machine Learning Engineer

Build and ship ML models (forecasting, recommendation, ranking, optimization) into customer-facing SaaS products. Own end-to-end pipelines from experimentation through production deployment, monitoring, and iteration.

160k – 170kUnited StatesML EngineeringRemote5+ YOESQLdbt

Audio ML Engineer II

Build, tune, and deploy state-of-the-art audio deepfake detection models. Requires Master's/PhD and 3+ years experience with ML/DL frameworks and audio processing.

160k – 210kNew York, NYML EngineeringRemote3+ YOEJAXPython

AI Engineer - Database Engineering

As an AI Engineer, you will own the full AI engineering lifecycle, from design to optimization, for Snowflake Database Engineering products. You will build agentic workflows, coding harnesses, and evaluation pipelines, working with a high-powered engineering team.

160k – 230kMenlo Park, CAML EngineeringOn-site5+ YOEGodbt

Research Engineer – Evals

Builds evaluation systems to measure Firecrawl's web data extraction quality across diverse websites and workflows. Designs metrics, pipelines, benchmarks, and LLM judges; integrates into CI/CD and model training loops. Requires 3+ years in ML engineering or data quality with production systems.

160k – 240kSan Francisco, CAML EngineeringHybrid3+ YOERLHFCI/CD