Product Specialist - Concierge
Own the daily quality and operations loop for Bilt’s AI concierge across multiple product surfaces. The role requires strong diagnostics, cross-functional communication, experience with conversational AI, and comfort reviewing code and editing structured files.
About the job
Responsibilities
- Monitor AI concierge conversation quality daily across rentals, dining, travel, payments, mortgages, customer experience, and other verticals across app, web, SMS, and email.
- Diagnose failures to the responsible layer, including prompts, knowledge, tool output, or upstream services, and drive fixes to completion.
- Ship and validate changes to prompt YAML, knowledge files, and tool-output shaping by replaying real conversations before opening pull requests.
- Review concierge pull requests as a code owner, including bot-authored and cross-pod contributions.
- Triage internal requests and enable product pods to own their concierge surfaces.
- Build automations and repository skills for recurring investigations, monitoring, and knowledge updates.
- Test in staging, including SMS/iMessage, email, cross-channel tracing, and cross-pod quality assurance.
- Monitor regressions from model changes and design escalation logic for human handoff.
- Provision phone numbers, email addresses, and DNS for merchants launching on SMS or email.
Requirements
- Approximately 3 years of relevant experience; candidates with approximately 1 year may be considered with a proven technical background.
- Comfort operating Git with AI tooling, reviewing pull requests, and editing YAML and Markdown files, or ability to learn quickly.
- Strong diagnostic and root-cause analysis skills across multiple systems.
- Exceptional written and verbal communication skills.
- Experience with LLM-powered products, prompting, or conversational AI.
- Ability to independently own a daily operating loop and prioritize by frequency and blast radius.
Nice to Have
- Experience building automations or agent workflows using Devin, Claude Code, or similar tools.
- Fintech, rewards, or loyalty experience.
- Startup experience.
- QA or test-plan writing experience.
Compensation and Benefits
- Salary range: $70,000–$120,000 annually, plus equity and an annual performance-based bonus.
- Health insurance, One Medical membership, wellness stipends, and family programs.
- 401(k) plan with company match.
- Commuter FSAs.
- Unlimited paid time off.
- Employee Bilt Points.
- Team events.
Skills
Git, AI Tools, Yaml, Markdown, Llm Products, Prompting, Conversational AI, Python, Automation, Agent Workflows, Qa Testing, DNS
Similar jobs
Product Operations jobs
Earnin
Mountain View, CA
Operationalize AI agents for customer care by developing procedures, evaluating bot quality, analyzing performance metrics, and driving improvement with Product, Engineering, QA, and vendors. Requires 3–5 years of relevant experience, strong analytical skills, and comfort with APIs and technical documentation.
Sleeper
Seattle, WA
Analyzes lifecycle and product-system requirements across mobile, web, email, and in-app experiences, translating them into reusable system components, interface specifications, prototypes, and implementation guidance. The role collaborates with Product, Engineering, Data, and Marketing and validates accessibility and quality.
Harvey
New York, NYManages cross-functional legal operations projects, workflows, systems, metrics, and AI enablement for an in-house legal team. Requires 3–6 years of operations or project management experience, meaningful legal-team exposure, strong systems thinking, and familiarity with legal technology.
Ramp
New York, NYOwn Juno’s product operating system and launch processes while supporting customers, investigating product issues, and translating recurring friction into actionable product insights. The role requires 3+ years in product operations, program management, support, or a related analytical customer-facing function.
Datadog
New York, NY
Own the operational infrastructure supporting Technical Solutions Enablement, including systems, knowledge management, reporting, governance, and process improvement. The role requires at least four years of operations experience and strong capabilities in scalable process design, business systems, and cross-functional project management.