Cloud Inference Engineer
Deploy and tune models with optimizations like KV caching, paged attention, sequence packing, etc. Conduct model performance reviews, improve scheduler, batcher, autoscaling; profile latency, cost, utilization. Occasionally write kernels and engage in tasteful communication.
- San Francisco, California, United States
- Full-time
- On-site
- $150K–$250K per year
- Posted Nov 7, 2026
INDUSTRYSoftware & InternetData & Analytics
Responsibilities
- Deploy and tune models with optimizations like KV caching, paged attention, sequence packing, etc.
- Conduct model performance reviews
- Improve scheduler, batcher, autoscaling; profile latency, cost, utilization
- Occasionally write kernels and engage in tasteful communication
Skills
CUDAGPU inference optimizationvLLMSGLangTensorRT-LLMKV cachingpaged attentionbatchingtoken streamingdistributed computeGPUsCUDAvLLMSGLangTensorRT-LLM
More jobs nearby
Full description
LuminalCompany details are for members
Sign up for a free job alert to see Luminal's profile, website and LinkedIn, and the same for every employer on Referso Jobs.