- Shanghai, Shanghai, China
- Senior
Senior member of the LLM inference framework team responsible for building and optimizing production-grade single-node and distributed inference runtimes for large language models on AMD GPUs.
Responsibilities
- Architect and optimize distributed LLM inference runtimes based on in-house LLM engines or open-source stacks such as vLLM, SGLang, and llm-d
- Design and improve TP / PP / EP (MoE) hybrid execution, including KV-cache management, attention dispatch, and token scheduling
- Implement and optimize multi-node inference pipelines using RCCL, RDMA, and collective-based execution
- Drive throughput, latency, and memory efficiency across single-GPU and multi-GPU clusters
- Optimize continuous batching, speculative decoding, KV-cache paging, prefix caching, and multi-turn serving
Requirements
- Master’s in Computer Science
- PhD in Computer Science
- Master’s in Computer Engineering
- PhD in Computer Engineering
- Master’s in Electrical Engineering