Research Engineer
Research Engineer building self-improving AI agent systems at Console. Develop eval/optimization loops, fine-tune specialist models, and improve agent reasoning over enterprise context using production data to drive measurable gains in quality, latency, and reliability.
About the job
What You'll Do
- Build and improve the eval and optimization loop for core agents, turning real production usage into measurable improvements in quality, latency, and reliability.
- Systematically improve agent behavior across prompts, programs, routing logic, constraints, and model adaptations, applying techniques like DSPy and GEPA where useful.
- Fine-tune, adapt, and evaluate specialist models for repeatable, high-volume agent tasks where there is clear production feedback or verifiable quality signals.
- Work across the stack when needed, from traces and eval infrastructure to agent orchestration, product workflows, and customer-facing AI behavior.
Requirements
- Strong technical background in software engineering, machine learning, or applied AI, demonstrated through an advanced degree and/or equivalent experience building production AI systems.
- Strong software engineering fundamentals and good judgment for designing, building, and debugging complex systems.
- Experience building evals for AI systems, including datasets, judges, metrics, offline replay, tracing, or regression testing.
- Practical understanding of modern model adaptation and post-training methods, including LoRA/QLoRA, SFT, distillation, preference optimization, reward modeling, DPO/GRPO, and reinforcement learning from verifiable feedback.
- Ownership mindset: drive projects end-to-end, move quickly from real usage, and care about shipping measurable improvements.
- Enjoy following SOTA research into new models, agent architectures, evals, post-training methods, and optimization techniques.
Nice-to-Haves
- Experience with techniques like DSPy and GEPA.
- Background in improving how agents reason over complex enterprise context (users, apps, devices, tickets, licenses, policies, customer-specific data).
Skills
Machine Learning, Ai Systems, Evals, Model Adaptation, Lora, Qlora, Sft, Distillation, Dpo, Grpo, Reinforcement Learning, Dspy, Gepa, Prompt Optimization, Fine-Tuning
Similar jobs
ML Engineering jobs
ClickUp
United States
Build and operate large-scale ranking and retrieval systems that power search relevance, including hybrid lexical/vector search, embeddings, query understanding, and permission-aware retrieval. Requires a bachelor's degree and 5+ years of ML engineering experience in ranking or information retrieval.
Atomicmachines
Emeryville, CA
Build and operate production ML infrastructure spanning training, deployment, serving, monitoring, data pipelines, and feedback-driven retraining. The role requires strong MLOps and DevOps experience, Python and SQL proficiency, and ownership of reliable cloud-based systems.
Tessera Labs
San Jose, CABuild and scale post-training, reinforcement-learning, evaluation, and inference systems for long-horizon agents operating over complex enterprise software. The role requires strong Python and PyTorch or JAX skills, distributed GPU experience, empirical rigor, and the ability to take research results into production.
Tessera Labs
San Jose, CABuild and operate production AI agents that transform enterprise processes, data, and code. The role focuses on tool layers, retrieval, context management, evaluations, monitoring, auditability, and guardrails, requiring strong Python and TypeScript plus experience with production LLM systems and traditional machine learning.
Confido
New York, NY
Build and productionize applied AI/ML systems for document understanding, agentic workflows, and demand forecasting using rich, messy enterprise data. The role requires 3+ years of production AI/ML experience, strong evaluation and monitoring practices, and a STEM master’s degree.