Staff Softare Engineer, Cortex AI Infrastructure

Responsibilities

• Architect Agentic Runtimes: Build and scale the orchestration engines that execute complex agentic workflows, ensuring low-latency tool execution and robust state management.
• Scale Context Engineering Infra: Design high-performance systems for RAG (Retrieval-Augmented Generation), including vector database integration, scalable and efficient search indexing, query processing, and result ranking, semantic caching, and automated metadata extraction.
• Build the "Evals Engine": Develop the automated infrastructure required to run massive-scale golden set simulations, error analysis pipelines, and "hillclimbing" experiments.
• Productionize AI Workflows: Collaborate with the modeling team to take raw LLM capabilities and turn them into hardened, multi-tenant microservices with strict guardrails and observability.
• Optimize Performance & Cost: Direct the infra strategy for model routing, prompt caching, and token optimization to ensure Snowflake’s AI features are the most efficient in the industry.

Requirements

• Education: Bachelor’s degree in Computer Science or a related technical field.
• Experience: 5+ years of experience building distributed systems, high-throughput APIs, or backend infrastructure for AI/ML products.
• Technical Stack: Deep proficiency in Go or Java (for systems) and Python (for AI orchestration).
• Systems Thinking: Strong understanding of database internals, distributed state management, and cloud-native architecture (Kubernetes, FoundationDB, etc.).
• Domain Expertise: Familiarity with the "plumbing" of AI: vector indices, agent platforms, and building scalable data pipelines.

Bonus

• Experience with query optimization and SQL engine internals.
• Designing multi-tenant systems that handle sensitive enterprise data at scale.
• Developing search infrastructure for large-scale applications.
• Direct experience with any of the subsystems outlined above.