Senior Performance Engineer - LLM Inference Frameworks

Responsibilities

• Design, implement, and optimize high‐performance inference pipelines for large language models running on GPUs
• Profile and tune model execution across the stack - from scheduler design to kernel fusions and everything in-between
• Design and experiment with memory management strategies for improved memory bandwidth optimization and cache efficiency
• Innovate and
• Implement cutting-edge techniques such as Speculative Decoding, Context Caching, and FP8/INT4 quantization to push the boundaries of tokens-per-second-per-watt
• Develop and maintain benchmarking and testing systems that quantify latency, utilization, and efficiency
• What we need to see
• Bachelor's, Master's, or higher degree in Computer Engineering, Computer Science, Applied Mathematics, or related computing-focused degree (or equivalent experience)
• 5+ years of relevant software development experience.
• Excellent Python programming skills, software design, and software engineering skills

Languages