- India
- Mid
- 4+ years
Test Analyst - AI Assurance role focusing on GenAI system architecture, RAG, and Agent-based systems, involving AI evaluation, Responsible AI validation, and observability.
Responsibilities
- Develop AI Assurance test strategy to assess AI systems for quality, safety, reliability and responsible AI behavior
- Design AI evaluation strategy for LLMs, RAG and agent evaluation across functional and non-functional dimensions
- Golden dataset curation & Synthetic dataset generation
- Implement AI evaluation frameworks like Ragas, DeepEval, etc.
- Use AI Evaluation tools like LangSmith, Arize, Langfuse, Galileo, Evidently, etc.