All vacancies
Senior ML/Evaluation Engineer
EPAM Systems
Remote · Czechia, SlovakiaSalary not disclosedfull-timeEPAM Systems Careers (remote abroad)
We are building an Enterprise Agent Development Platform, a production-grade, cloud-native ecosystem that enables engineering teams to define, orchestrate, deploy, and observe AI agents at scale. The platform standardizes agent development across the organization.
Responsibilities
- AWS AgentCore Evaluation (on-demand mode for CI/CD gates, online mode for production sampling)
- LLM-as-judge evaluator design (built-in AgentCore evaluators — helpfulness, correctness)
- Custom code-based Lambda evaluators (Python — deterministic checks)
- Evaluation levels (TRACE for per-response, TOOL_CALL for per-invocation, SESSION for workflow)
- OTel spans from AWS AgentCore Observability as evaluation input
- Enterprise evaluation standard authoring (mandatory dimensions, pass/fail criteria)
Languages
- Work format
- Remote
- Seniority
- Senior
- Posted
- 29 Sept 2026
- Last verified
- In the last 5 days
Similar roles
- Data AnalystBrighty App · Remote · worldwide, UTC+4 accepted
- Data AnalystYeremyan Projects · Armenia · On-site
- Machine Learning / Computer Graphics Engineer (remote part/full time) - Junior/MImnoo AG · Remote · worldwide, UTC+4 accepted
- Senior Data Engineer / Analytics Engineer (AWS)DVT · Remote · worldwide, UTC+4 accepted
- Research Scientist Junior/Junior+Plata · Remote · worldwide
- AWS Data EngineerClera · Remote · worldwide, UTC+4 accepted