Staff Scanning Engineer
Designs and operates Internet-scale scanning, DNS, attribution, and data pipelines, with deep ownership of distributed backend systems and production reliability. Requires 10+ years of software engineering experience, strong Go expertise, cloud and streaming infrastructure knowledge, and the ability to mentor engineers.
About the job
Responsibilities
- Design, build, and operate Internet-scale scanning, DNS, crawling, attribution, and data collection systems.
- Own backend services and pipelines that collect, process, validate, and enrich large Internet datasets.
- Improve the quality, freshness, reliability, and coverage of the Internet Map.
- Build systems for IP scanning, DNS lookups, protocol fingerprinting, asset attribution, and large-scale ingestion.
- Lead technical design for distributed services, streaming pipelines, storage systems, caching layers, scheduling systems, and scan orchestration infrastructure.
- Partner with Research, Product, Security, and Engineering teams to productionize research ideas.
- Make tradeoffs involving reliability, cost, scale, latency, data quality, and operational complexity.
- Participate in observability, incident response, and on-call rotation.
- Mentor engineers, review designs, and raise the technical bar for backend and scanning infrastructure.
Requirements
- 10+ years of software engineering experience building production systems.
- Strong backend engineering experience with data ingestion pipelines, scanning systems, crawlers, schedulers, APIs, storage systems, or high-throughput services.
- Experience designing and operating large-scale, high-volume, or Internet-scale systems.
- Strong programming experience; the primary language is Go.
- Experience with cloud infrastructure such as AWS, GCP, or Azure.
- Experience with message queues or streaming systems.
- Experience with distributed databases or large-scale storage systems.
- Strong understanding of reliability, observability, testing, deployment, and operational ownership.
- Ability to write understandable, testable, maintainable code.
- Strong technical communication skills.
- Good judgment when using AI-assisted development tools.
Nice-to-haves
- Experience with scanners, crawlers, DNS infrastructure, passive or active measurement systems, or Internet data collection platforms.
- Deep understanding of DNS, including resolution, delegation, record types, nameservers, and operational edge cases.
- Knowledge of Internet protocols, service discovery, network measurement, protocol fingerprinting, or large-scale crawling.
- Experience with attribution systems, asset mapping, cloud asset discovery, or linking Internet-facing services to organizations.
- Experience turning research prototypes into reliable production systems.
- Familiarity with gRPC, REST, Protobuf, MessagePack, or similar API and serialization technologies.
- Experience deploying and maintaining containerized services in Kubernetes.
- Experience with scan orchestration, scheduling, rate limiting, backpressure, retries, deduplication, or distributed task execution.
- Experience spanning research and engineering in security, infrastructure, networking, or Internet measurement.
Compensation and Benefits
- High cost-of-living US locations: $198,000–$233,000 USD, plus bonus eligibility and equity.
- Other US locations: $172,000–$216,000 USD, plus bonus eligibility and equity.
- Benefits include equity, health, dental and vision coverage, retirement contributions, parental leave, mental health and wellness benefits, flexible PTO, and a professional development stipend.
Skills
Go, AWS, GCP, Azure, Apache Kafka, Google Pub/Sub, Amazon Kinesis, Bigtable, Spanner, Cassandra, Postgres, Kubernetes, DNS, gRPC, Protobuf
Similar jobs
Backend Engineering jobs
Grafana Labs
United States
Staff backend engineer responsible for evolving Grafana into a scalable, multi-tenant observability application platform. The role requires production operations experience, distributed-systems expertise, strong communication, and familiarity with Go or willingness to learn it.
Turion Space
Irvine, CA
Leads architecture and development of mission-critical backend systems for spacecraft command, telemetry, mission planning, and operations. Requires 8+ years of software development experience, distributed-systems and cloud-native expertise, and technical leadership.
United States
Leads the technical direction and hands-on development of scalable backend platforms for Pinterest’s AI-driven products. Requires extensive backend and distributed-systems experience, strong Python skills, production LLM or ML experience, and cross-functional technical leadership.
United States
Leads the technical strategy, architecture, and operation of Pinterest’s large-scale storage infrastructure supporting SQL and graph workloads. Requires 8+ years of backend distributed-systems experience, technical leadership, and expertise in production reliability, performance tuning, and modern database technologies.
Motive
San Francisco, CA
Build and evolve scalable platform services, APIs, and product capabilities while making architecture decisions and improving reliability and operational excellence. The role requires substantial software engineering experience, distributed-systems expertise, strong technical judgment, and the ability to mentor engineers.