Principal Software Engineer - AI Runtime & Distributed Inference Systems
International Recruiting LLC · Posted Nov 7, 2026
- San Jose, California, United States
- Principal
- 12+ years
Lead the architecture of large-scale AI inference systems and distributed runtimes.
Responsibilities
- Define the software architecture for large-scale AI inference systems
- Lead the evolution of the distributed runtime stack for dense and MoE foundation models
- Establish long-term technical strategy for performance, scalability, reliability, and programmability
- Influence hardware-software co-design decisions across future accelerator generations
- Architect runtime systems supporting tensor, pipeline, expert, data parallelism and multi-rack inference execution
Requirements
- BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering
- Nice to have: large language model serving systems
- Nice to have: vLLM
- Nice to have: SGLang
- Nice to have: TensorRT-LLM