Sr. Staff Machine Learning Systems Engineer

Responsibilities

• Own the evaluation as a whole, not just a slice of it
• Set the technical direction for our evaluation systems; metric, judge and scorer design, the statistical methodology behind regression decisions, and the infrastructure that tracks and categorizes failures over time.
• Design and scale the data pipelines — ingestion, transformation, dataset versioning, labeling and calibration workflows — that both evaluation and downstream data science work depend on.
• Lead projects that span teams and quarters
• Own our approach to adversarial and red-team evaluation as a risk-reduction program, designing the test suites and failure taxonomies that catch safety and edge-case issues before they reach patients.
• Lead major platform improvements; re-architecting core systems, removing brittle logic, modernizing how things run with impact that's felt org-wide, not just on your own team.
• Build real working relationships across ML engineering, data science, platform engineering, clinical, legal, and product, the kind of trust that gets you looped in early, before decisions are locked in.

Languages