Staff Software Engineer, AI Gateway

Responsibilities

• Set technical direction for core systems powering the AI Gateway 0 the model router, MCP gateway, and control plane - and make sure architectural decisions hold up as usage scales by 10x
• Consistently identify gaps in our infrastructure, process, or tooling before they become blockers, and drive the fix from idea to production
• Take a significant, hands-on role in defining, delivering, and implementing the Gateway's technical strategy, in close partnership with engineering leadership
• Own the hardest, highest-ambiguity problems in the system: dynamic model selection, auto-failover, cost-based routing, and policy enforcement that has to work end-to-end
• Act as the go-to expert for secure credential storage, session summarization, caching, and rate limiting - the capabilities everything else depends on
• Build the infrastructure that lets us, and our customers, evaluate, replay, and backtest agent and LLM behavior with confidence
• Mentor senior and mid-level engineers, and raise the technical bar for the team through code, design review, and example
• Partner with product, design, and leadership to shape not just what the control plane UI does, but where the product goes next
• Key Problems You’ll Help Solve
• Define the unified abstraction layer across diverse model and tool providers (OpenAI, Anthropic, Google, AWS Bedrock) that the rest of the org builds on

Languages