Senior Research Engineer (Agentic Behavior)

Responsibilities

• Build tools for agentic error analysis
• Design and implement tooling to systematically capture, classify, and analyse errors that AI coding agents make when generating Kotlin code.
• Build observability pipelines over agentic traces – mining patterns from agent sessions in JetBrains IDEs, Junie, Claude Code, Cursor, and other coding agents.
• Build evaluation pipelines
• Design, implement, and maintain evaluation pipelines that measure Kotlin code generation quality across dimensions, including correctness, idiomaticity, build success, framework usage, and test coverage.
• Build simulation environments where coding agents can be measured on realistic Kotlin developer tasks – from greenfield KMP projects and Gradle dependency management to migrating Spring applications from Java to Kotlin.
• Own evaluation infrastructure: metrics, experiment tracking, automated regression checks, and reproducible benchmarking.
• Research methods for improving agent and model behavior on Kotlin
• Collaborate with model providers (Anthropic, OpenAI, and Google) to translate Kotlin-specific findings into model improvements.
• Build public Kotlin benchmarks

Languages