Deep Learning Model and System Optimization Engineering Intern - 2027

Responsibilities

• Analyze and optimize inference performance across model architecture, numerical precision, GPU kernels, memory systems, and end-to-end serving frameworks.
• Develop low-bit quantization techniques for model weights, activations, attention operands, and KV caches.
• Explore numerical formats and mixed-precision strategies involving FP8, NVFP4, MXFP4, and other emerging low-precision formats while preserving model accuracy and capabilities.
• Design and evaluate techniques for KV cache compression, sparsification, hierarchical storage, and reuse across inference requests.
• Optimize the inference of model architectures including MHA, GQA, MQA, MLA, sliding-window attention, sparse attention, linear attention, and state space models.
• Investigate model-system co-optimization techniques such as speculative decoding, state reuse, and computation reuse.
• Build performance models and optimization strategies for long-context inference and agentic workloads, with a focus on throughput, time to first token, inter-token latency, memory efficiency, and deployment cost.
• Collaborate closely with model, framework, kernel, library, system software, and GPU architecture teams to influence the design of future models, software platforms, and hardware architectures.
• What we need to see
• Pursuing an MS or PhD from a leading university in Computer Science, Computer Engineering, Electrical Engineering, Applied Mathematics, or a related field, with an expected graduation date in 2028.

Languages