- San Francisco, California, United States
- Principal
- 6+ years
Lead the design, API, and operational aspects of the AI infrastructure control plane, focusing on workload placement, scaling, and governance across Kubernetes clusters and accelerators.
Responsibilities
- Own the deployment and workload API, including resource model and lifecycle semantics
- Define placement, capacity, and autoscaling policies for GPU and accelerator workloads
- Design and enforce multi‑tenant isolation, quota, and fairness mechanisms
- Develop and maintain traffic routing, ingress, and private connectivity for model and agent endpoints
- Ensure governance, audit, and metering for customer usage
Requirements
- BS or MS in Computer Science or related field
- Nice to have: Service networking
- Nice to have: vLLM
- Nice to have: KV cache
- Nice to have: Batching