Senior AI Agent Evaluation Engineer
Aditi Consulting · Posted Nov 19, 2026
- Bangalore, Karnataka, India
- Full-time · Senior
- 8–10 years
The Senior AI Agent Evaluation Engineer is a hands-on technical role responsible for defining, developing, and executing robust evaluation frameworks and test strategies for AI agents built using the Google Agent Development Kit (ADK). This role bridges AI development and reliable deployment, ensuring our agents are safe, ethical, effective, and meet high-quality performance standards.
Responsibilities
- Design, implement, and maintain scalable, repeatable evaluation datasets, metrics, and benchmarks to measure agent performance, robustness, safety, and alignment.
- Build synthetic testing environments and simulation strategies to stress-test agents under diverse real-world conditions.
- Develop and execute adversarial testing, jailbreaking, and red-teaming methodologies to identify potential harm, bias, toxicity, and unauthorized behavior in agent responses.
- Implement and measure adherence to established ethical guidelines, safety policies, and content-filtering mechanisms across all agent deployments.
- Partner with policy and legal teams to ensure agent evaluations cover regulatory compliance and fairness objectives.
Requirements
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or equivalent practical experience
- Nice to have: Pandas
- Nice to have: SQL