Validation Engineer, AI & Data
Validate the accuracy, reliability, completeness, and consistency of business context and AI generated responses against approved business requirements and ground truth answers.
- Indianapolis, Indiana, United States
- Posted Nov 11, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise SoftwareAI & Machine LearningData & Analytics
Responsibilities
- Design and implement the testing strategy for the Domain Context Model, Business Context Repository, ontology mappings, and AI response evaluation.
- Develop representative ground truth question and answer test sets with defined expected results and tolerances, validated by data owners.
- Build and maintain automated evaluation tests for AI responses generated through the client’s Context Injection toolkit.
- Evaluate AI responses with and without domain context to establish baseline accuracy and measure improvements.
- Validate numeric answers, SQL and query results, business logic, aggregations, join relationships, and narrative responses.
- Identify and classify failures, including hallucinations, stale information, incorrect schemas, faulty joins, missing business context, and persona mismatches.
- Develop repeatable evaluation scripts, scoring rubrics, regression tests, and accuracy reporting dashboards or scorecards.
- Verify context repository data completeness, classification consistency, access control behavior, and changes introduced through automated context updates.
- Document test evidence, defects, remediation results, AI readiness assessments, and validation outcomes.
- Support release readiness, weekly demonstrations, and handover of reusable evaluation frameworks and documentation.
Requirements
- Experience in a AI
Skills
More jobs nearby
Full description
Company details are for members
Sign up for a free job alert to see RADcube's profile, website and LinkedIn, and the same for every employer on Referso Jobs.