Skip to content
ExaExaSan Francisco, CA

Software Engineer, Web Crawling

Builds and scales Google-scale web crawlers to handle 100M+ pages daily, optimizing for politeness, dynamic content, anti-bot measures, and prioritization. Requires experience with high-performance languages like Rust/C++ and tools like TypeScript/Playwright.

150k – 300k/yr
On-siteBackend Engineering

About the role

Responsibilities

  • Build a distributed crawler that can handle 100M+ pages per day
  • Optimize crawl politeness and rate limiting across thousands of domains
  • Design systems to detect and handle dynamic content, JavaScript rendering, and anti-bot measures
  • Create intelligent crawl scheduling and prioritization algorithms for maximum coverage efficiency

Desired Experience

  • Extensive experience building and scaling web crawlers, or excited to ramp up quickly
  • Experience with high performance languages (C++, Rust, etc.)
  • Familiarity with TypeScript, Playwright, modern web design, CDP (Chrome DevTools Protocol)
  • Comfortable optimizing systems to an exceptional degree
  • Care about finding high quality knowledge

Skills

RustC++TypeScriptPlaywrightChrome Devtools ProtocolDistributed SystemsWeb CrawlingJavascript RenderingRate LimitingCrawl Scheduling
Cloudflare

Software Engineer, Workers Deploy & Config

CloudflareNew York, NY +4

Build and scale the control plane for Cloudflare Workers, the serverless edge platform. Design distributed systems, high-traffic APIs, storage schemas, and own production reliability for services powering Pages and R2. Requires strong Go experience plus knowledge of observability, databases, and Kubernetes.

150k – 206k/yr
On-site5+ YOEBackend Engineering
Doxel

Backend Software Engineer

DoxelSan Francisco, CA

Backend engineer building and scaling data pipelines, APIs, and storage systems (DuckDB, PostgreSQL, Airflow) that power construction-insight applications and integrate with CV/ML models.

150k – 200k/yr
Hybrid3+ YOEBackend Engineering
Resend

Deliverability Engineer, Core Sending

ResendUnited States

As a Deliverability Engineer, you will be responsible for building and maintaining core components for sending transactional and marketing emails. This role requires expertise in email deliverability, reputation, and high-volume email systems.

150k – 180k/yr
RemoteBackend Engineering
Resend

Backend Engineer , Core Sending

ResendUnited States

As a Backend Engineer on the Core Sending Squad, you will build and operate Resend’s email transport infrastructure, focusing on SMTP delivery, MTA systems, and routing. You will design systems for throttling, IP pools, and reputation-aware routing, and improve bounce classification.

150k – 170k/yr
RemoteBackend Engineering
Kodex

Software Engineer — Integrations

KodexUnited States

Build and operate customer-facing integrations (APIs, identity federation, webhooks) connecting Kodex with third-party systems. Backend-leaning generalist with 4+ years experience in TypeScript/Node.js/React and production API experience.

150k – 180k/yr
Remote4+ YOEBackend Engineering