- Indianapolis, Indiana, United States
- Mid
Validate the accuracy, reliability, completeness, and consistency of business context and AI generated responses against approved business requirements and ground truth answers.
Responsibilities
- Design and implement the testing strategy for the Domain Context Model, Business Context Repository, ontology mappings, and AI response evaluation.
- Develop representative ground truth question and answer test sets with defined expected results and tolerances, validated by data owners.
- Build and maintain automated evaluation tests for AI responses generated through the client’s Context Injection toolkit.
- Evaluate AI responses with and without domain context to establish baseline accuracy and measure improvements.
- Validate numeric answers, SQL and query results, business logic, aggregations, join relationships, and narrative responses.