- Menlo Park, California, United States
- On-site
- Full-time · Mid
Develop and optimize distributed training infrastructure and tools for PyTorch and TorchTitan, enabling fast and reliable training of large AI models on modern hardware accelerators.
Responsibilities
- Improve PyTorch's state of the art training, post-training, and inference on modern AI hardware accelerators
- Enable new models in TorchTitan for pre-training and post-training
- Optimize the performance of TorchTitan’s pre-training and post-training of latest Gen AI models
- Develop the latest parallelism APIs, kernels, and tools for debugging/visualizing large distributed training jobs
- Build foundational distributed training infrastructure including communication collectives, profiling, memory allocation/management, etc
Requirements
- Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience
- Nice to have: Prompt/context engineering
- Nice to have: Agent orchestration
- Nice to have: Emerging AI technologies
- Nice to have: System performance optimizations
