Senior System Software Engineer - Local AI

Responsibilities

• Partner across organizations with benchmarking teams, inference teams, and AI application developers to define requirements, design, and maintain shared infrastructure for deploying, benchmarking, and qualifying AI workloads across infernece backends such as llama.cpp, Ollama, PyTorch, vLLM, and WinML.
• Build Git-based CI infrastructure for automated builds, integration testing, and regression detection across hardware platforms, operating systems, and runtime configurations.
• Develop model management services to download, version, cache, and synchronize checkpoints and dependencies for reproducible execution.
• Design databases, backend services, and data pipelines to store and query model metadata, benchmark results, system telemetry, and execution artifacts.
• Create dashboards and reports that help engineers compare inference configurations, identify regressions, and investigate performance bottlenecks.
• Develop and operate server infrastructure, including environment provisioning, workload scheduling, resource management, and failure recovery.
• Develop autonomous AI agents for infrastructure management, workload orchestration, monitoring, and diagnostics, with appropriate access controls and execution safeguards.
• Maintain Git repositories and companion websites, keeping code, documentation, model recommendations, and evaluation results current.
• Collaborate with inference developers to implement instrumentation and debugging capabilities, resolve system issues, and improve infrastructure efficiency and scalability.
• What we need to see

Languages