Senior ML Engineer (Token Factory)

Responsibilities

• Token Factory is a part of Nebius Cloud, one of the world's largest GPU clouds, running tens of thousands of GPUs. We are building a high-performance inference and fine-tuning platform designed to push foundation models to their hardware limits. Our mission is to maximize throughput, minimise latency, and optimise cost-per-token across tens of thousands of GPUs.
• Inference engines support: Implement novel speculative decoding architectures, optimise components of various LLM designs (dense/MoE, autoregressive/parallel), and contribute to open-source inference engines.
• Low Precision Training & Inference: Design and productionise low-precision (FP8, NVFP4/MXFP4) training and inference pipelines with measurable gains in throughput and cost-efficiency.
• Strong communication and leadership abilities
• A track record of building and delivering products (not necessarily ML-related) in a dynamic startup-like environment.
• Strong engineering skills, including experience in developing large distributed systems or high-load web services.

Languages