Software Engineering Intern, NCCL - 2027
Design, implement and maintain highly-optimized communication runtimes for Deep Learning frameworks (e.g. NCCL for TensorFlow/Pytorch) and HPC programming interfaces (e.g. UCX for MPI/OpenSHMEM) on GPU clusters.
- Shanghai, China
- Internship
- Posted Nov 13, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise Software
Responsibilities
- Design, implement and maintain highly-optimized communication runtimes for Deep Learning frameworks (e.g. NCCL for TensorFlow/Pytorch) and HPC programming interfaces (e.g. UCX for MPI/OpenSHMEM) on GPU clusters
- Participating in and contributing to parallel programming interface specifications like MPI/OpenSHMEM
- Design, implement and maintain system software that enables interactions among GPUs and interactions between GPUs and other system components
- Creating proof-of-concepts to evaluate and motivate extensions in programming models, new designs in runtimes and new features in hardware
Requirements
- Ph.D. in CE/CS/EE
- Experience in a High Performance Computing
- Experience in a Deep Learning
Nice to have: InfiniBand, RoCE, HPC applications, PyTorch, TensorFlow, JAX/XLA, vLLM/SGLang, Expert Parallelism (EP), TP, DP, PP, CUDA kernel optimization and profiling, large-scale model training and production inference software stack
Skills
C/C++ programmingLinuxparallel programming interfacescommunication runtimesNCCLTensorFlowPytorchUCXMPIOpenSHMEM
Full description
Company details are for members
Sign up for a free job alert to see NVIDIA's profile, website and LinkedIn, and the same for every employer on Referso Jobs.