Skip to content
SimplePracticeSimplePracticeUnited States

Senior Software Engineer II, AI Developer Foundations

Build and own AI-assisted engineering tools, conventions, context layers, and evaluation frameworks inside a large Rails monolith to scale AI usage across engineering, product, and design teams. Requires 7+ years production software experience with recent Rails and LLM-backed system expertise.

208k – 260k/yr
Remote7+ YOEML Engineering

About the role

Responsibilities

  • Own the context layer agents read: rules files, docs conventions, grounding data, with CI and named owners keeping it current
  • Publish the standards for how we prompt coding agents, how we supply context, and what a spec needs to contain before it's worth handing to an agent
  • Define the review gates any AI-generated artifact clears before a human spends time on it, and get what reviewers catch flowing back into the conventions
  • Take the prototypes running today and turn the ones that hold up into supported workflows that run for someone other than their author
  • Evaluate harnesses, models and vendors against criteria and eval sets
  • Work the product-to-engineering handoff with PMs, designers and EMs
  • Teach. Documentation, worked examples, office hours, and time sitting with a squad while they work with the tools we build
  • Help us draw the boundaries for where an agent runs unattended, where a human signs off, and where we don't use one at all

Requirements

  • BS/MS in Engineering, Computer Science, or related field, or equivalent experience
  • 7+ years building production software and then maintaining it. You’ve shipped something, lived with the decisions, and found out whether a design decision held up
  • Strong problem-solving and communication skills; comfortable in fast-paced, cross-functional environments
  • Recent, direct Ruby on Rails experience in a large codebase
  • Ability to read unfamiliar application code quickly and judge whether what an agent produced in it is any good
  • LLM-backed systems you've shipped to users and kept running, past the point where better prompting stops helping
  • Experience building context or memory handling, and writing evals
  • Judgment about what belongs in code and what belongs in the model
  • Track record evaluating tools with evidence
  • Internal tools, platforms, conventions or workflows that other engineers chose to use
  • CLI and harness-shaped software, and comfort connecting systems over APIs
  • Strong writing. Much of this role is getting other people to work differently, which happens through docs and worked examples rather than announcements
  • Comfort working outside engineering, explaining technical constraints to people who don't share your background and taking their pushback seriously

Nice-to-Haves

  • Agent harnesses, MCP servers, orchestration frameworks
  • Healthcare or another HIPAA-regulated environment
  • AWS, Terraform, Kubernetes, Docker
  • Experience on a platform or infrastructure team where the customers were other engineers
  • Having overbuilt a platform once before the workflows existed, and learning to spot the warning signs earlier

Our Stack

Ruby on Rails monolith, Aurora MySQL, Redis, Sidekiq, Ember.js, AWS (EKS, Bedrock and more). Linear, GitHub, Semaphore, Datadog, LaunchDarkly.

Benefits

  • Medical, dental, vision, life & disability insurance
  • 401(k) plan with company match
  • Flexible Time Off (FTO), wellbeing days, paid holidays, and summer Fridays
  • Mental health resources
  • Paid parental leave & Backup Care
  • Tuition reimbursement
  • Employee Resource Groups (ERGs)

Skills

Ruby on RailsLLMsAWSTerraformKubernetesDockerMySQLRedissidekiqember.jsGitHubDatadog

Similar roles

ML Engineering jobs
Zoox

Senior Software Systems Engineer

ZooxFoster City, CA +1

Develop metrics and large-scale evaluation pipelines to assess fidelity of Zoox's GenAI-powered autonomous vehicle simulator. Requires 5+ years in quantitative evaluation or robotics, strong Python/data analysis skills, and statistics knowledge.

208k – 300k/yrHybrid5+ YOEML Engineering
Confido Legal

Senior ML Ops Engineer

Confido LegalNew York, NY

Be the first dedicated owner of Confido's ML platform, owning end-to-end ML pipelines, infrastructure for training/inference/agentic workloads, and providing reproducible environments for the AI/ML team in a fast-growing CPG AI startup.

210k – 300k/yrOn-site5+ YOEML Engineering
Confido Legal

Senior ML / AI Engineer

Confido LegalNew York, NY

Build and deploy AI/ML systems including LLM-powered document extraction, agentic workflows, sales forecasting, anomaly detection, and promotion optimization for CPG brands. Requires 2+ years applied AI/ML and 6+ years production software engineering experience.

210k – 280k/yrOn-site2+ YOEML Engineering
Instacart

Senior Machine Learning Engineer II, Ads Response Prediction

InstacartUnited States

Lead research on pCTR and conversion models for Instacart Ads. Tackle bias mitigation, calibration, multi-task learning, and generative retrieval systems. Requires 6+ years ML experience and advanced degree.

201k – 254k/yrRemote6+ YOEML Engineering
Instacart

Senior Applied Scientist II, Ads Optimization

InstacartUnited States

Leads algorithmic development for Instacart's ads optimization systems, including real-time bidding, budget pacing, and auction mechanics using constrained optimization and control theory. Requires MS/PhD, 8+ years experience deploying production systems at scale, and proficiency in Go/Java/C++/Python.

201k – 254k/yrRemote8+ YOEML Engineering