- India
- Hybrid
- Full-time · Mid
As an AI Software Development Engineer in Test, you will play a key role in redefining quality engineering for next-generation AI-powered applications and autonomous systems. You will go beyond traditional software testing to evaluate, validate, and improve large language models (LLMs), AI agents, copilots, chatbots, and intelligent workflows.
Responsibilities
- Design, implement, and optimise testing strategies for AI-powered applications, copilots, chatbots, Retrieval-Augmented Generation (RAG) pipelines, and autonomous AI agents.
- Develop AI evaluation (Evals) frameworks to assess hallucinations, groundedness, toxicity, bias, response quality, contextual relevance, prompt regression, consistency, and safety.
- Build automated validation pipelines for LLM outputs using structured assertions, semantic scoring, embeddings, and evaluation metrics.
- Create synthetic and benchmark datasets to support comprehensive AI testing scenarios.
- Test multi-agent orchestration workflows, including agent memory, context retention, planning, and execution.
Requirements
- Bachelor's degree in Computer Science, Engineering, Artificial Intelligence, Machine Learning, or a related discipline
- Master's degree in Computer Science, Engineering, Artificial Intelligence, Machine Learning, or a related discipline
- Nice to have: Robot Framework
- Nice to have: Playwright
- Nice to have: LangChain