- Palo Alto
- $200K–$400K per year
- 3+ years
- Visa sponsorship
Design, build, and operate distributed training, rollout, and orchestration systems for large-scale LLM and multimodal post-training across multi-GPU, multi-node environments.
Compensation
Depending on background, skills, and experience, the expected annual salary range for this position is $200,000 to $400,000, plus equity.
Responsibilities
- Design, build, and operate distributed training, rollout, and orchestration systems for large-scale LLM and multimodal post-training across multi-GPU, multi-node environments.
- Profile and optimize performance across the full-stack — model implementation, parallelism strategies, communication libraries, and GPU kernels — to improve throughput, latency, memory efficiency, hardware utilization, and cost.
- Investigate numerical correctness and low-precision issues in distributed training and inference, including train–inference consistency for reinforcement learning.
- Improve the reliability of long-running workloads through checkpointing, fault recovery, observability, and operational tooling.
- Build supporting infrastructure for reinforcement learning and agentic post-training, including asynchronous rollout, trajectory collection, sandboxed execution, evaluation harnesses, and data pipelines.
Requirements
- Nice to have: RL algorithms such as PPO, GRPO
- Nice to have: modern post-training frameworks such as Miles, slime, AReaL, verl, Prime-RL
- Nice to have: key open-source contributions to training or inference frameworks such as SGLang, vLLM, Megatron-LM
