Senior Deep Learning Communication Architect

Responsibilities

• The software architecture group at NVIDIA has openings for a Deep Learning Communication Architect. We scale the DNN models and training/inference frameworks to systems with hundreds of thousands of nodes.
• Optimizing communication performance
• Identify and eliminate bottlenecks in data transfer and synchronization during distributed deep learning training and inference.
• Designing efficient communication protocols
• Develop and implement communication algorithms and protocols tailored for deep learning workloads, minimizing communication overhead and latency.
• Hardware and software co-craft
• Collaborate with hardware and software teams to craft systems that effectively apply high-speed interconnects (e.g., NVLink, InfiniBand, SPC-X) and communication libraries (e.g., MPI, NCCL, UCX, UCC, NVSHMEM).
• Exploring innovative communication technologies
• Research and evaluate new communication technologies and techniques to enhance the performance and scalability of deep learning systems.
• Developing and implementing solutions

Languages