Inference Lead
The Real-Time Inference Engineering Lead will design and industrialize low-latency, resilient model-serving services for predictive AI use cases.
- Charlotte, North Carolina, United States
- Hybrid
- Posted Sep 4, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise Software
Responsibilities
- Architect low-latency online inference and real-time model-serving solutions.
- Develop scalable APIs, microservices, and deployment patterns for predictive models.
- Implement Kubernetes-based deployment, autoscaling, load balancing, and traffic-management strategies.
- Conduct benchmarking, performance tuning, capacity planning, and load testing.
- Optimize latency, throughput, resource consumption, availability, and cost.
- Define monitoring, alerting, SLOs, runbooks, and incident-response practices.
- Build CI/CD pipelines for repeatable model and service releases.
- Design resilience, failover, rollback, disaster recovery, and graceful-degradation patterns.
- Lead technical reviews and mentor inference and platform engineers.
Requirements
- Degree in computer science, engineering, or a related discipline
Nice to have: Experience with enterprise model-serving platforms and inference runtimes, Cloud, Kubernetes, SRE, or ML engineering certification
Skills
Online inference and real-time model-serving architectureREST/gRPC APIs and distributed microservicesKubernetes, containers, autoscaling, and traffic managementPerformance engineering, latency optimization, and load testingMonitoring, SLOs, capacity planning, and production operationsCI/CD and progressive-deployment approachesResilience and high-availability engineeringCloud and on-premises deployment experienceKubernetesRESTgRPCCI/CDcontainers
More jobs nearby
Full description
Kasmo Inc
Company details are for members
Sign up for a free job alert to see Kasmo Inc's profile, website and LinkedIn, and the same for every employer on Referso Jobs.