What You'll Do
- Build and optimize big data pipelines to extract and process signals from the web, job postings, and other sources
- Design and implement data architectures and storage solutions to efficiently handle massive data volumes
- Collaborate closely with data scientists to support and integrate ML models into data workflows
- Continuously improve data quality, performance, and scalability of our technographic data platform
- Drive technical strategy and roadmap for the data processing infrastructure
What We're Looking For
- Extensive experience building and scaling big data pipelines and architectures from scratch
- Deep expertise in big data frameworks (Hadoop, Spark) and the JVM stack (Java, Scala)
- Strong software engineering fundamentals and ability to write efficient, high-quality code
- Experience with entity recognition and NLP techniques a plus
- Proven track record delivering results and driving projects in a fast-paced environment
- Excellent collaboration and communication skills to work with data scientists, analysts and product teams
- Passion for leveraging huge datasets to power valuable insights
Ideal Background
- 8+ years of experience in software engineering roles
- Experience working with very large datasets and distributed systems
- Familiarity building data pipelines at large tech companies or data-driven organizations
- Bachelor's or advanced degree in Computer Science, Engineering or related technical field
Compensation: $140,000—$220,000 USD base salary. Additional compensation such as Bonus, Commission, Equity and other benefits may also apply.