Backend Software Engineer (Evals)

In this role, you will:

• Design eval pipelines that are reliable, reproducible, and extendable
• Build infrastructure for continuous eval monitoring frameworks (regression/drift monitoring, robust golden datasets) with feedback loops
• Design, build, and maintain backend services and APIs for intelligent automation and knowledge systems
• Integrate and structure data across internal platforms for downstream systems and AI workflows
• Collaborate with data, research, and engineering teams to integrate OpenAI models into workflows
• Own the full development lifecycle of new backend systems
• Build with scale and maintainability in mind while iterating rapidly

You might be a great fit if you have:

• 4+ years of backend engineering experience at product-driven companies (excluding internships)
• Proficiency in backend technologies: Python, FastAPI, Postgres
• Experience designing and scaling distributed systems, APIs, or data processing pipelines
• Experience building AI agents or applications, including evals and performance improvement via prompting or scaffolding
• Familiarity with LLM evaluation methods and patterns like multi-agent workflows, tool use, or long context
• Experience creating production evals and/or measuring ML/LLM model performance at scale
• Pragmatic mindset for iterative shipping toward long-term vision