Skip to content
Redpanda DataRedpanda DataSan Francisco, CA

Principal Software Engineer, AI Platform

Principal engineer architecting and building Redpanda's Agentic Data Plane, enabling AI agents to safely interact with enterprise data via streaming, query, and governance layers. Requires 10+ years building large-scale distributed systems and experience with agentic AI infrastructure.

245k – 290k/yr
Hybrid10+ YOEML Engineering

About the role

You Will

  • Track emerging research, frameworks, and industry developments in the agentic AI space, and translate the most relevant signals into concrete engineering proposals and product bets
  • Identify, prototype, and deliver high-leverage emerging capabilities for ADP, in partnership with the broader team
  • Define architectural patterns for new ADP capabilities, ensuring they integrate cleanly with the existing runtime, policy, and observability layers
  • Engage directly with strategic customers and the field to stress-test new capabilities against real enterprise deployment scenarios
  • Serve as a technical ambassador for ADP externally, publishing research, writing engineering blog posts, and speaking at conferences to establish Redpanda's credibility at the frontier of agentic infrastructure
  • Participate in on-call and help drive down operational toil, improving system observability and self-healing capabilities over time

You Have

  • 10+ years of software engineering experience, with a demonstrated track record of leading the design and delivery of complex, large-scale distributed systems
  • Experience building or contributing to emerging areas of agentic AI infrastructure, e.g. eval frameworks, orchestration layers, context management, AI gateways, or similar
  • Fluency with AI-assisted development at the level of agentic orchestration, i.e. composing multi-agent coding workflows with subagents to dramatically accelerate research and prototyping
  • Deep expertise in at least one systems language (Go preferred) and comfort working across the stack
  • Understanding of streaming data systems, messaging infrastructure, or data plane/control plane architectures, experience with Apache Kafka or Redpanda is a plus
  • Experience designing and operating systems that enforce access control, policy, and auditability at scale
  • Self-directed, strong written communicator, comfortable collaborating asynchronously in GitHub, Slack, and Google Docs
  • Excellent interpersonal and communication skills. Able to lead without authority across engineering teams and engage credibly with technical customers and external audiences

Nice to Have

  • Familiarity with AI safety, alignment, or responsible AI research as it applies to autonomous systems in enterprise environments
  • Experience with protocol buffers, protoreflect and codegen
  • Background in security, identity, or data governance systems relevant to enterprise environments
  • Experience running Kafka/Redpanda and/or Postgres at scale
  • Experience with Kubernetes and cloud-native infrastructure across AWS, GCP, or Azure
  • Prior experience in an early-stage or growth-stage startup, especially in a category-defining product area

Compensation

  • U.S. base salary range for this role is $245,000 - $290,000. Salary ranges are determined by role, level, and location.

Skills

GoDistributed SystemsStreaming DataApache KafkaAccess ControlPolicy EnforcementAi OrchestrationMulti-Agent SystemsKubernetesAWSGCPAzureProtocol BuffersPostgres

Similar roles

ML Engineering jobs
Pinterest

Principal Engineer, AI Platform

PinterestSan Francisco, CA

Principal Engineer setting technical vision and building AI/ML infrastructure for Generative AI and Recommender Systems at Pinterest, scaling to hundreds of millions of inferences per second. Requires deep expertise in distributed systems and proven cross-org technical leadership.

243k – 500k/yr
Hybrid7+ YOEML Engineering
Fivetran

Sr. Principal AI Systems Architect

FivetranOakland, CA

Designs and owns architecture for production-grade AI analyst systems, including prompts, tools, context, evaluation, and safety. Partners with teams to build reliable AI capabilities for analytical workflows, emphasizing scalability and quality.

252k – 315k/yr
HybridML Engineering
Everlaw

Staff/Principal AI Engineer

EverlawOakland, CA

Leads design and execution of AI strategy for legal discovery platform, guiding engineering teams on LLMs, NLP, and frontier models at massive scale. Requires 6+ years ML/AI experience and 10+ years software engineering.

228k – 340k/yr
Hybrid6+ YOEML Engineering
Upstart

Principal Machine Learning Engineer

UpstartUnited States

Leads engineering initiatives to build scalable ML platform infrastructure, including unified embeddings, feature pipelines, and continuous learning systems. Requires 7+ years in applied ML, Python/ML frameworks expertise, and Master's/PhD in quantitative field.

221k – 300k/yr
Remote7+ YOEML Engineering
Shield AI

Principal Engineer, State Estimation

Shield AIDallas, TX

Principal State Estimation Engineer developing advanced sensor fusion, inertial navigation, and real-time state estimation algorithms for autonomous UAVs in GPS-denied environments. Requires 12+ years experience with Kalman filters, C++, and shipping navigation products.

275k – 412k/yr
On-site12+ YOEML Engineering