Senior Software Engineer, LLM Inference

Responsibilities

• Implement and optimize inference algorithms for LLM and omnimodal architectures, including hybrid Mamba-Transformer and mixture-of-experts models
• Profile inference pipelines using NVIDIA's profiling and simulation tools. Correlate simulation predictions against real hardware across data center and edge devices
• Write and tune GPU kernels (CUDA, Triton) for operators like fused MoE layers, SSM state updates, and quantized GEMMs
• Solve distributed inference problems: expert parallelism, communication-compute overlap, collective tuning, multi-node deployment
• Build production-grade software inside major open-source libraries - vLLM, SGLang, Dynamo, FlashInfer
• Own optimization features end-to-end, from scoping through delivery, collaborating with research, product, and engineering teams worldwide
• What we need to see
• B.Sc., M.Sc., or equivalent experience in Computer Science or Computer Engineering
• 5+ years of hands-on software engineering experience in performance-critical systems
• Solid understanding of deep learning architectures (Transformers, SSMs, MoE, ...)

Languages