- Shanghai, China
- On-site
- 4+ years
Lead LLM inference optimization for Qualcomm Cloud AI accelerators, working with customers and internal teams to deploy and tune generative AI workloads.
Responsibilities
- Deploy and optimize LLM and multimodal inference workloads on Qualcomm Cloud AI accelerator platforms
- Profile and optimize prefill and decode performance metrics
- Analyze model architecture and runtime bottlenecks across compute, accelerator memory, host memory, KV cache, interconnect, networking, and software stack
- Optimize model execution using quantization, batching, continuous/dynamic batching, KV-cache management, paged attention, prompt/context-length tuning, and speculative decoding
- Optimize distributed inference across tensor, pipeline, data, context, and expert parallelism
Requirements
- Master’s degree or higher in Computer Science, Electrical/Computer Engineering, AI/ML, or related technical field
- Nice to have: vLLM
- Nice to have: TensorRT-LLM
- Nice to have: SGLang
- Nice to have: Hugging Face Transformers