Software Engineer 3

Responsibilities

• Build and maintain agent skills and the infrastructure to validate, evaluate, publish, and maintain them.
• Design evaluation datasets and workflows that compare agent behavior against a baseline and produce actionable quality signals.
• Build agent metrics and observability for skill selection and routing, success and failure outcomes, tool calls, latency, and token usage.
• Design safety and quality gates for agent-authored content, including rule packs, static analysis, confidence thresholds, structured verdicts, and bounded suppression.
• Create CLIs, libraries, and MCP integrations for local development and CI adoption.
• Integrate tooling into GitHub Actions and other CI workflows, including secrets, annotations, exit codes, and artifacts.
• Build code-generation quality checks, including anti-pattern catalogs and linting for AI-generated MongoDB code.
• Investigate nondeterministic results, false positives, and unsafe generated guidance, and turn recurring failures into reusable improvements.
• Collaborate with engineers, security partners, and product teams; communicate trade-offs, risks, and ownership across teams.

Requirements

• 2+ years of experience building production software, developer tools, internal platforms, or automation systems.
• Strong software engineering fundamentals in API design, testing, error handling, and maintainability.
• Experience building CLIs, libraries, test infrastructure, static analysis, or CI/CD workflows.
• Ability to design systems that are usable by developers and reliable in automation.
• Experience reasoning about correctness and safety with ambiguous input, nondeterministic output, false positives, or untrusted content.
• Comfort in an evolving R&D environment where abstractions emerge through prototypes and feedback.
• Strong written and verbal communication, including explaining technical trade-offs and aligning stakeholders.

Nice to Have

• Experience with agentic systems, LLM applications, prompt or rubric-based evaluation, or AI-assisted development.
• Experience building evaluation harnesses, benchmark datasets, quality metrics, LLM-as-judge workflows, or human-review tooling.
• Experience with Go, Python, JavaScript/TypeScript, Java, or C#.
• Experience with GitHub Actions security, secret handling, static rule engines, or policy enforcement.
• Experience moving prototypes into production.

Compensation and Benefits

• Base salary range in Canada: $108,000–$149,000 CAD.
• Benefits may include equity, employee stock purchase program participation, flexible paid time off, 20 weeks of fully paid gender-neutral parental leave, fertility and adoption assistance, RRSP with employer match, mental health counseling, backup child and elder care, and health, dental, and vision benefits.