Software Engineer, Agent Optimization & Evaluation

Responsibilities

• Design and build Reddit's agent optimization harness for evaluating, benchmarking, and improving AI agents across real software engineering workflows.
• Develop Reddit-specific evaluation environments that accurately measure agent performance against meaningful engineering tasks.
• Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback.
• Design model routing and orchestration strategies that optimize for quality, latency, and cost.
• Define metrics and experimentation frameworks that measure developer productivity and engineering velocity.
• Build tooling to analyze agent behavior, investigate failures, and accelerate iteration.
• Partner closely with Platform Engineering, Infrastructure, and Developer Productivity teams to integrate agent capabilities into engineering workflows.
• Help shape Reddit's long-term strategy for a flexible, model-agnostic agent platform that gives us control over our AI capabilities and economics
• You Will Bring
• 5-12+ years of engineering experience

Languages