Knowledge of LLM evaluation frameworks jobs
Skill:Knowledge of LLM evaluation frameworks Senior Consultant | Full Stack | Mumbai | Engineering | Platform Development & IntegrationNew
deloitte · Mumbai, India+1 moreHybrid · Full-time · Senior
+10K new in the last hour
deloitte · Posted Nov 13, 2026
- Mumbai, Maharashtra, India +1 more
- Hybrid
- Full-time · Senior
Lead enterprise digital transformation engagements by combining business consulting, product strategy, and AI solution architecture.
- Mumbai, Maharashtra, India
- Pune, Maharashtra, India
- Lead enterprise digital transformation engagements
- Partner with senior client stakeholders to define product vision
- Architect AI-powered solutions
- Lead delivery teams using modern Agentic AI frameworks and cloud platforms
- Bachelor's or Master's Degree in Computer Science, Engineering, Information Systems, or related field.
- Nice to have: Certifications in Azure, Google Cloud, AI Engineering, or Enterprise Architecture
- Nice to have: Experience designing and delivering enterprise AI, GenAI, or Agentic AI solutions at scale
- Nice to have: Proven consulting experience working with CXOs, business leaders, and technology stakeholders
Nice to have: Experience across banking, insurance, manufacturing, healthcare, or enterprise transformation programsDesigning, developing, and deploying Agentic AI solutionsBuilding and orchestrating multi-agent systemsImplementing Retrieval-Augmented Generation (RAG) solutionsKnowledge of agent memory architecturesExperience with LangGraph, Google Agent Development Kit (ADK), Microsoft Agent Framework (MAF), Semantic KernelIntegrating Large Language Models from OpenAI, Azure OpenAI, Anthropic Claude, Gemini, and open-source modelsKnowledge of LLM evaluation frameworksImplementing AI guardrails, safety controls, policy enforcement, content moderation, security controlsDesigning and leveraging Knowledge Graphs, graph databasesStrong proficiency in Python development, API integrations, tool development, workflow automationExperience with observability, tracing, monitoring, debugging of AI agentsOptimizing agent performance across accuracy, latency, scalability, cost efficiency, reliability, governanceLangGraphGoogle Agent Development Kit (ADK)Microsoft Agent Framework (MAF)Semantic KernelCrewAIAutoGenAzure AI SearchPineconeWeaviateQdrantChromaDBOpenAIAzure OpenAIAnthropic ClaudeGeminiOpen-Source LLMsPythonFastAPIREST APIsBackend EngineeringAzureGCP