Lead Quality Engineer
Lead Quality Engineer specializing in AI-powered product applications and microservices, focusing on Responsible AI and safety metrics.
- Bangalore, Karnataka, India
- Full-time
- Posted Nov 13, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareAI & Machine LearningData & Analytics
Responsibilities
- Develop and execute robust evaluation frameworks for AI agents, focusing on Responsible AI and safety metrics such as bias, toxicity, and alignment failures
- Design synthetic testing environments and simulation strategies to stress-test agents under diverse real-world conditions
- Implement adversarial testing, red-teaming, and prompt injection methodologies to identify vulnerabilities and ensure agent robustness
- Integrate evaluation pipelines into CI/CD workflows for continuous quality assurance and rapid iteration
- Collaborate with policy and legal teams to ensure agent evaluations meet regulatory compliance and ethical standards
- Analyze evaluation results and provide actionable insights to development teams, prioritizing safety and risk mitigation
- Configure and execute automated test suites using Python and relevant AI/ML testing libraries (e.g., PyTest, DeepEval, Ragas, Giskard)
- Manage bug detection, documentation, and tracking using Jira, focusing on performance degradations and alignment failures
Requirements
- Bachelor's degree in Computer Science
- Bachelor's degree in Artificial Intelligence
- Bachelor's degree in Information Technology
- Bachelor's degree in Software Engineering
- Experience in a AI-powered product applications
- Experience in a microservices
Nice to have: Gemini models, Hugging Face, Garbage-in/Garbage-out (GIGO) testing tools, Data poisoning techniques, AI ethics principles and policy background, Visualization tools for evaluation results
Skills
Full description
BrillioCompany details are for members
Sign up for a free job alert to see Brillio's profile, website and LinkedIn, and the same for every employer on Referso Jobs.