Senior Software Engineer, AI Gateway

Docker

Remote · United States$13,408–21,725 gross/mofull-timeDocker Careers

About Docker Docker has been one of the most loved brands in developer tooling, trusted by more than 20 million monthly users and over 20 billion container image pulls. From solo founders to the world's largest companies, developers rely on Docker to build, share, and run their applications across our suite of products including Docker Desktop, Docker Hub, and Docker Scout.

Responsibilities

Design and implement core systems powering the AI Gateway, including the model router, MCP gateway, and control plane
Build infrastructure that supports dynamic model selection/routing, auto-failover, cost-based routing, and policy enforcement
Own critical capabilities such as secure credential storage, session summarization, caching, and rate limiting
Develop APIs for developers building with OpenAI-compatible interfaces and the Model Context Protocol
Build the underlying infrastructure to support evaluation, telemetry, replay, and backtesting for agents and LLM workflows
Lead architectural decisions and mentor engineers as the team scales
Performance tuning - caching, session summarization, and rate limiting that keep the gateway fast under real production load
Contribute to roadmap planning, technical strategy, and cross-functional alignment
Key Problems You’ll Help Solve
Building a unified abstraction over fundamentally different model and tool providers without leaking their differences into the developer experience

Languages

Work format: Remote
Seniority: Senior
Posted: 15 Oct 2026
Last verified: In the last 5 days

Similar roles

Keep looking

More Software jobs in Armenia and remote All IT vacancies Internships and entry level