# Senior Software Engineer, Trust & Safety
**Company:** [Replit](https://hotfix.jobs/companies/replit)
**Location:** Foster City, CA
**Salary:** $210K-$265K
**Experience:** 4+ years
**Skills:** Python, TypeScript, SQL, BigQuery, Machine Learning, Llm Guardrails, Prompt Injection Detection, GCP, Kubernetes, SAST, Sca
**Posted:** 2026-05-26
> Build and operate AI-powered abuse detection and response systems for a developer platform. Focus on LLM guardrails, fraud detection, and automated enforcement across millions of daily actions.
## Job Description
## Responsibilities
- Design and implement LLM guardrails that detect abuse scenarios in AI-generated code and agent interactions
- Build AI-powered detection systems that use LLMs to identify malicious patterns, classify threats, and automate response decisions
- Build and operate abuse detection systems that identify phishing, cryptomining, account takeover, and financial fraud across millions of daily user actions
- Design automated response mechanisms that enforce platform policies without manual intervention
- Own the full abuse response lifecycle: detection, investigation, enforcement, and handling appeals alongside Support and Legal
- Analyze attack patterns using BigQuery and Hex, turning investigation findings into new detection rules
- Maintain and extend internal detection tools (Slurper, Netwatch) that continuously monitor user activity
- Integrate and tune security scanners (SAST, SCA) in CI pipelines with tight performance SLAs
- Track abuse trends, measure detection effectiveness, and adapt defenses as attack patterns evolve

## Requirements
- 4+ years of experience in security engineering, anti-abuse, trust & safety, or fraud detection
- Strong programming skills in Python and/or TypeScript for building detection systems and automation
- Experience with SQL and data analysis at scale (BigQuery, Snowflake, or similar)
- Experience building or fine-tuning ML/LLM-based classifiers for security or abuse detection
- Familiarity with prompt injection, jailbreaking, and other LLM-specific attack vectors
- Ability to investigate complex abuse patterns and translate findings into automated defenses
- Familiarity with common attack patterns: phishing infrastructure, account takeover, credential stuffing, resource abuse
- Clear communication skills for working across Security, Support, Legal, and Engineering teams

## Nice to Have
- Experience at a platform company dealing with user-generated content or compute abuse (hosting providers, cloud platforms, developer tools)
- Background in fraud detection, payment abuse, or financial crime
- Familiarity with device fingerprinting, IP reputation, and email validation services
- Experience with CI/CD security tooling (SAST, SCA, Dependabot, Snyk)
- Knowledge of container security, Linux internals, or cloud infrastructure (GCP preferred)
- Prior work with abuse reporting pipelines, trust & safety tooling, or content moderation systems

## Tools & Tech Stack
- Languages: Python, TypeScript, Go, SQL
- Data: BigQuery, Hex
- Detection tools: Slurper, Netwatch, Stytch (device fingerprint), ClearOut (email reputation)
- CI/CD Security: Dependabot, Snyk, SAST/SCA scanners
- Infrastructure: GCP, Kubernetes
- Collaboration: Linear, Slack, Zendesk (for abuse reports)
**Apply:** https://hotfix.jobs/jobs/senior-software-engineer-trust-safety-at-replit-8e2a2866-a5d6-4d0b-8fb7-fea83c084a45
**Canonical:** https://hotfix.jobs/jobs/senior-software-engineer-trust-safety-at-replit-8e2a2866-a5d6-4d0b-8fb7-fea83c084a45