Software Engineer, Systems ML: PyTorch Distributed
Develop and optimize distributed training infrastructure and tools for PyTorch and TorchTitan, enabling fast and reliable training of large AI models on modern hardware accelerators.
- Menlo Park, California, United States
- Full-time
- On-site
- Posted Nov 9, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareAI & Machine Learning
Responsibilities
- Improve PyTorch's state of the art training, post-training, and inference on modern AI hardware accelerators
- Enable new models in TorchTitan for pre-training and post-training
- Optimize the performance of TorchTitan’s pre-training and post-training of latest Gen AI models
- Develop the latest parallelism APIs, kernels, and tools for debugging/visualizing large distributed training jobs
- Build foundational distributed training infrastructure including communication collectives, profiling, memory allocation/management, etc
Requirements
- Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
- Experience in a SaaS & Enterprise Software
- Experience in a AI & Machine Learning
Nice to have: Prompt/context engineering, Agent orchestration, Emerging AI technologies, System performance optimizations, Runtime analysis of latency, memory bandwidth, I/O access, compute utilization
Skills
Python programmingAI framework developmentDeep learning model acceleration on hardware architecturesResponsible AI practicesGPU/CPU/AI hardware accelerator architecturesPyTorchCaffe2TensorFlowONNXTensorRTCUDA programmingOpenMPOpenCLAI hardware accelerator kernel programmingcuBLAScuDNNCUTLASSHIPROCmPyTorchTorchTitanCUDAOpenMPOpenCLcuBLAScuDNNCUTLASSHIPROCm
More jobs nearby
Full description
MetaCompany details are for members
Sign up for a free job alert to see Meta's profile, website and LinkedIn, and the same for every employer on Referso Jobs.