Evaluation methodologies jobs Skill: Evaluation methodologies Quality Engineer New
Responsive · Coim, IndiaMid
Senior Research Scientist, Agent Evaluation New
ServiceNow · Hyderabad, IndiaSenior
Machine Learning Engineer
Arena Intelligence, Inc.
Learning Architect
Taylor Morrison · Remote, US Competitive Compensation· Remote · Full-time · Senior
Agentic AI Engineer
Booz Allen Hamilton · McLean, US $99K–$225K· Full-time
Evaluation and Learning Director
CYCSF · San Francisco, US $95K–$105K· Full-time · Director
Instructional Systems Designer
Booz Allen Hamilton · Washington, US $53K–$108K· Full-time · Senior
Lead Product Manager- AI
Wolters Kluwer · Coppell, US $108K–$188K
Agentic AI Engineer
Booz Allen Hamilton · Arlington, US $78K–$176K· Full-time · Entry
AI Validation, Senior Specialist
Vanguard · Charlotte, USHybrid · Full-time · Senior Visa
+10K new in the last hour
Responsive · Posted Nov 16, 2026
Coim, IN, India Mid 3–5 years
First 2 pages free · sign up to see every job
Product-minded Software Engineer with strong technical skills and a passion for building scalable solutions.
Participate in all stages of product development from requirements to release Design and execute detailed, comprehensive, and well-structured test plans and test cases Perform manual and automated testing of web applications Identify, document, and track bugs to closure using issue tracking tools (e.g., Jira) Collaborate closely with developers, product managers, and designers in agile teams Bachelor’s degree in Computer Science, IT, or related field Nice to have: SaaS experience Nice to have: B2B products experience ServiceNow · Posted Nov 16, 2026
Hyderabad, Telangana, India Senior 5+ yearsLead the design, development, and deployment of AI agent evaluation pipelines and scoring logic to validate agent performance before customer release.
Design evaluation methodologies and benchmarks for agent reasoning, planning, tool use, reliability, and safety across LLM-as-a-Judge, trajectory-based, and human evaluation Take problems from research question to prototype to shipped feature, owning them end to end Build and harden the pipelines and scoring logic behind customer-facing evaluation Curate synthetic and real-world datasets; measure the evaluator itself for consistency and agreement with human labels Nice to have: AI-assisted development tools Nice to have: Claude code Nice to have: Windsurf Arena Intelligence, Inc. · Posted Nov 14, 2026
Help scale and strengthen the core infrastructure that powers real-world AI evaluation, building and improving model benchmarking systems.
We offer competitive compensation aligned to the markets where our team members are based. The compensation range for each role is listed on its posting and will depend on the candidate's experience and work location.
Architect and build core modeling for data and evaluation products Own full stack data, model training, and eval pipelines Help grow a culture of rapid product iteration Conduct research into state-of-the-art evaluation methods Turn new ideas into reliable systems Taylor Morrison · Posted Nov 13, 2026
Remote, United States Remote Full-time · Senior Competitive Compensation 7+ yearsProvide strategy direction, standards, and measurement approach for priority learning initiatives, combining performance consulting, learning architecture, design governance, and impact measurement to create scalable learning solutions.
Partner with business leaders, HR, and stakeholders to identify performance gaps, capability needs, and workforce readiness risks Recommend the appropriate mix of learning, coaching, reference guides, performance support, manager reinforcement, and process improvements Embed learning in the flow of work through the LMS/LXP, collaboration tools, and job aids Design role-based learning paths, capability frameworks, upskilling/reskilling journeys, and assessment strategies Set the design operating model and standards for instructional designers, trainers, and Training Manager Bachelor's degree preferred Nice to have: AI-enabled learning practices Booz Allen Hamilton · Posted Nov 12, 2026
McLean, Virginia, United States Full-time $99K–$225K per year 1+ yearsDesign, develop, and deliver production‑grade agentic AI systems leveraging LLMs, RAG, and autonomous workflows.
The projected compensation range for this position is $99,000.00 to $225,000.00 (annualized USD).
Architect modular AI application patterns Integrate multiple model providers Apply GenAI stack capabilities Design solutions deployable across cloud and portable builds Prototype and iterate using AI‑assisted development tools Bachelor's degree Nice to have: LangChain Nice to have: AutoGen Nice to have: CrewAI Nice to have: LangGraph CYCSF · Posted Nov 10, 2026
San Francisco, California, United States Full-time · Director $95K–$105K per year 3+ yearsLead evaluation and learning initiatives to measure impact, enhance program effectiveness, and foster a culture of continuous improvement.
$95,000 - $105,000 per year plus comprehensive benefits
Develop and implement evaluation strategy Establish KPIs and data collection methods Conduct qualitative and quantitative analyses Ensure culturally responsive methods Translate findings into reports and recommendations Bachelor’s degree in social services, research, or a related field Nice to have: bilingual English and Cantonese/Mandarin or Spanish Booz Allen Hamilton · Posted Nov 10, 2026
Washington, District of Columbia, United States Full-time · Senior $53K–$108K per year 5+ yearsDesign and develop interactive training courses for intelligence analysis, applying instructional design principles and evaluation methodologies.
The projected compensation range for this position is $53,000.00 to $108,000.00 (annualized USD).
Capture customer training needs with SMEs Advise on learning solutions Design performance support tools Prepare training material following ADDIE model Recommend multimedia technologies and blended learning Bachelor's degree Nice to have: Captivate platform Nice to have: Intelligence Community experience Nice to have: Object-Based Intelligence Wolters Kluwer · Posted Nov 10, 2026
Coppell, Texas, United States $108K–$188K per year 8+ yearsLead Product Manager for an agentic audit product area focused on AI-enabled audit workflows.
$107,500.00 - $188,400.00 USD
Own the vision, roadmap, priorities, and measurable outcomes for an agentic audit product area. Define and manage product OKRs and KPIs aligned to adoption, retention, customer satisfaction, revenue growth, and value realization. Develop problem statements, business cases, product hypotheses, and investment recommendations. Identify where to play and how to win by evaluating customer needs, market opportunities, platform capabilities, and AI use cases. Prioritize investments using customer value, business impact, technical feasibility, and strategic alignment. Bachelor’s degree from an accredited university, or equivalent experience Nice to have: Computer Science degree Nice to have: Software Engineering degree Booz Allen Hamilton · Posted Nov 9, 2026
Arlington, Virginia, United States Full-time · Entry $78K–$176K per year 1+ yearsDesign, develop, and deliver production-grade agentic AI systems leveraging generative AI, LLMs, and autonomous workflows.
The projected compensation range for this position is $77,600.00 to $176,000.00 (annualized USD).
Architect modular AI application patterns Integrate multiple model providers Apply GenAI stack capabilities Design deployable solutions across contexts Prototype and iterate using AI-assisted tools Bachelor's degree Nice to have: LangChain Nice to have: AutoGen Nice to have: CrewAI Nice to have: LangGraph Vanguard · Posted Nov 9, 2026
Charlotte, North Carolina, United States Hybrid Full-time · Senior 7+ yearsVisa sponsorship Lead independent validation and challenge of AI/ML and generative AI models across Vanguard’s enterprise, ensuring sound governance and risk compliance.
Assess models and systems using generative AI throughout their lifecycle Provide effective challenge of model assumptions, methodologies, data, testing, implementation, monitoring and limitations Assess compliance with internal AI standards and regulatory requirements Ensure timely execution of model validation and high‑quality reports Develop, maintain and enhance validation procedures PhD in a quantitative discipline such as Computer Science, Mathematics, Statistics, Physics, Engineering or equivalent
comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs
Health Care - Medical/Dental/Vision/Prescription Drug Coverage 401(k) with Company Matching Contributions Flexible Spending Accounts Disability Programs Employee & Dependent Life Insurance
health life disability financial retirement
health life disability retirement paid leave
Medical Dental Vision 401(k) FSA/HSA
Health insurance Life insurance Disability insurance Financial benefits Retirement benefits