Nvidia GPU infrastructure Engineer/ HPC Engineer
Implement and manage multi-pod GPU clusters, focusing on infrastructure setup, operations, performance tuning, and incident response for large-scale AI/ML environments.
- Pune, Maharashtra, India
- Full-time
- On-site
- $0–$0
- Posted Nov 12, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareAI & Machine LearningData & Analytics
Responsibilities
- Stand up multi-pod GPU clusters including rack, power, cooling layouts, TOR/leaf connectivity, and InfiniBand/Ethernet host configurations
- Implement GPU partitioning using MIG/vGPU profiles and quota policies for multi-tenant environments
- Integrate cluster schedulers such as Slurm and Kubernetes with GPU device plugins, Node Feature Discovery, accounting, and quota management
- Manage Day 2 operations across firmware, drivers, DCGM, and NVML lifecycles
- Execute infrastructure change windows with zero or minimal downtime
- Perform capacity planning for GPU, CPU, memory, and NIC resources
- Implement bin-packing strategies for heterogeneous GPU SKUs
- Tune NCCL/UCX, GPU clocks and persistence, GPU Direct Storage, NUMA/locality, and CUDA runtime parameters
- Drive benchmarking and acceptance using HPL, HPL-AI, and MLPerf-like internal benchmark suites
- Monitor performance regressions and maintain performance SLOs
- Lead P0/P1 incident responses involving GPU, CUDA, and scheduler-related issues
- Perform detailed Root Cause Analysis (RCA) for network, GPU, CUDA, NCCL, RDMA, and scheduler-related failures
- Collaborate with platform and GPU engineering teams to resolve issues impacting CUDA jobs, Kubernetes scheduling, Slurm workload management, GPU resource allocation, and large-scale AI training environments
- Define golden images and implement node remediation processes, including cordon, drain, reimage, and auto-healing workflows
- Enforce GPU tenancy isolation using MIG, cgroups/device cgroups, and MPS daemon (mpsd)
- Ensure secure drivers and containers
- Implement SBOM and image scanning practices
- Publish operational runbooks and performance baselines
- Develop application tuning guides for LLM training and inference workloads
Requirements
- BE/B.Tech or equivalent degree in Computer Science or Electronics & Communication
- Experience in a AI/ML environments
Nice to have: LLM training and inference workloads
Skills
GPU cluster infrastructureGPU operationsCUDANCCLUCXInfiniBand/RDMAKubernetesSlurmGPU schedulingMulti-tenancyRoot Cause Analysis (RCA)NVIDIA GPUsDCGMNVMLMIGvGPUcgroups/device cgroupsMPS daemon (mpsd)HPLHPL-AIMLPerf
Benefits
- Operational runbooks and performance baselines
Full description
NR Consulting - India
Company details are for members
Sign up for a free job alert to see NR Consulting - India's profile, website and LinkedIn, and the same for every employer on Referso Jobs.