Back to search

Nvidia GPU infrastructure Engineer/ HPC Engineer

Implement and manage multi-pod GPU clusters, focusing on infrastructure setup, operations, performance tuning, and incident response for large-scale AI/ML environments.

  • Pune, Maharashtra, India
  • Full-time
  • On-site
  • $0–$0
  • Posted Nov 12, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareAI & Machine LearningData & Analytics

Responsibilities

  • Stand up multi-pod GPU clusters including rack, power, cooling layouts, TOR/leaf connectivity, and InfiniBand/Ethernet host configurations
  • Implement GPU partitioning using MIG/vGPU profiles and quota policies for multi-tenant environments
  • Integrate cluster schedulers such as Slurm and Kubernetes with GPU device plugins, Node Feature Discovery, accounting, and quota management
  • Manage Day 2 operations across firmware, drivers, DCGM, and NVML lifecycles
  • Execute infrastructure change windows with zero or minimal downtime
  • Perform capacity planning for GPU, CPU, memory, and NIC resources
  • Implement bin-packing strategies for heterogeneous GPU SKUs
  • Tune NCCL/UCX, GPU clocks and persistence, GPU Direct Storage, NUMA/locality, and CUDA runtime parameters
  • Drive benchmarking and acceptance using HPL, HPL-AI, and MLPerf-like internal benchmark suites
  • Monitor performance regressions and maintain performance SLOs
  • Lead P0/P1 incident responses involving GPU, CUDA, and scheduler-related issues
  • Perform detailed Root Cause Analysis (RCA) for network, GPU, CUDA, NCCL, RDMA, and scheduler-related failures
  • Collaborate with platform and GPU engineering teams to resolve issues impacting CUDA jobs, Kubernetes scheduling, Slurm workload management, GPU resource allocation, and large-scale AI training environments
  • Define golden images and implement node remediation processes, including cordon, drain, reimage, and auto-healing workflows
  • Enforce GPU tenancy isolation using MIG, cgroups/device cgroups, and MPS daemon (mpsd)
  • Ensure secure drivers and containers
  • Implement SBOM and image scanning practices
  • Publish operational runbooks and performance baselines
  • Develop application tuning guides for LLM training and inference workloads

Requirements

  • BE/B.Tech or equivalent degree in Computer Science or Electronics & Communication
  • Experience in a AI/ML environments

Nice to have: LLM training and inference workloads

Skills

GPU cluster infrastructureGPU operationsCUDANCCLUCXInfiniBand/RDMAKubernetesSlurmGPU schedulingMulti-tenancyRoot Cause Analysis (RCA)NVIDIA GPUsDCGMNVMLMIGvGPUcgroups/device cgroupsMPS daemon (mpsd)HPLHPL-AIMLPerf

Benefits

  • Operational runbooks and performance baselines

Full description

Title: Nvidia GPU infrastructure Engineer/ HPC Engineer Location: Pune Exp: 7-12 Yrs Job Description: Key Responsibilities Implementation Stand up multi-pod GPU clusters, including rack, power, cooling layouts, TOR/leaf connectivity, and InfiniBand/Ethernet host configurations. Implement GPU partitioning using MIG/vGPU profiles and quota policies for multi-tenant environments. Integrate cluster schedulers such as Slurm and Kubernetes with GPU device plugins, Node Feature Discovery, accounting, and quota management. Operations Manage Day 2 operations across firmware, drivers, DCGM, and NVML lifecycles. Execute infrastructure change windows with zero or minimal downtime. Perform capacity planning for GPU, CPU, memory, and NIC resources. Implement bin-packing strategies for heterogeneous GPU SKUs. Performance & Optimization Tune NCCL/UCX, GPU clocks and persistence, GPU Direct Storage, NUMA/locality, and CUDA runtime parameters. Drive benchmarking and acceptance using HPL, HPL-AI, and MLPerf-like internal benchmark suites. Monitor performance regressions and maintain performance SLOs. Reliability & Incident Management Lead P0/P1 incident responses involving GPU, CUDA, and scheduler-related issues. Act as the primary technical lead during production outages, coordinating with Networking, GPU Operations, Platform Engineering, Storage, and Application teams to restore services within SLA targets. Perform detailed Root Cause Analysis (RCA) for network, GPU, CUDA, NCCL, RDMA, and scheduler-related failures. Identify underlying issues and implement preventive and corrective actions. Collaborate with platform and GPU engineering teams to resolve issues impacting CUDA jobs, Kubernetes scheduling, Slurm workload management, GPU resource allocation, and large-scale AI training environments. Define golden images and implement node remediation processes, including cordon, drain, reimage, and auto-healing workflows. Security & Compliance Enforce GPU tenancy isolation using MIG, cgroups/device cgroups, and MPS daemon (mpsd). Ensure secure drivers and containers. Implement SBOM and image scanning practices. Documentation & Enablement Publish operational runbooks and performance baselines. Develop application tuning guides for LLM training and inference workloads. Experience & Educational Requirements Qualifications and Experience Educational Qualifications: BE/B.Tech or equivalent degree in Computer Science or Electronics & Communication. Relevant Experience: Relevant experience in GPU cluster infrastructure, GPU operations, and large-scale AI/ML environments. Hands-on experience with NVIDIA GPUs, CUDA, NCCL, UCX, InfiniBand/RDMA, and GPU resource management. Experience with Kubernetes, Slurm, GPU scheduling, multi-tenancy, and performance optimization. Experience in production incident management, troubleshooting, and Root Cause Analysis (RCA).

Nvidia GPU infrastructure Engineer/ HPC Engineer at NR Consulting - India | Referso Jobs