The AI Evaluation Specialist will evaluate enterprise AI tools and capabilities to determine their effectiveness, safety, usability, and readiness for business adoption.
Compensation
Estimated Salary/Wage USD $102,000.00/Yr. Up to USD $120,000.00/Yr.
Responsibilities
Plan and execute hands-on evaluations of AI tools such as OpenAI and life sciences models like Rosalind using representative business scenarios, workflows, prompts, datasets, and user personas.
Assess AI solutions for response quality, factual accuracy, relevance, consistency, reasoning, instruction following, reliability, latency, usability, accessibility, and overall task effectiveness.
Design and maintain standardized evaluation frameworks, scorecards, rubrics, and acceptance criteria for enterprise AI tools, models, agents, and AI-enabled applications.
Evaluate safety, privacy, security, data handling, governance, and responsible AI considerations in collaboration with security, privacy, legal, and governance stakeholders.
Conduct comparative evaluations and proof-of-concept assessments across multiple AI products, models, configurations, and vendors, including analysis of functionality, integration requirements, performance, and cost.
Evaluate enterprise AI tools and models for effectiveness, safety, usability, and readiness for business adoption.
Compensation
USD $102,000.00/Yr. Up to USD $120,000.00/Yr.
Responsibilities
Plan and execute hands‑on evaluations of AI tools such as OpenAI and life sciences models like Rosalind using representative business scenarios, workflows, prompts, datasets, and user personas.
Assess AI solutions for response quality, factual accuracy, relevance, consistency, reasoning, instruction following, reliability, latency, usability, accessibility, and overall task effectiveness.
Design and maintain standardized evaluation frameworks, scorecards, rubrics, and acceptance criteria for enterprise AI tools, models, agents, and AI‑enabled applications.
Evaluate safety, privacy, security, data handling, governance, and responsible AI considerations in collaboration with security, privacy, legal, and governance stakeholders.
Conduct comparative evaluations and proof‑of‑concept assessments across multiple AI products, models, configurations, and vendors, including analysis of functionality, integration requirements, performance, and cost.
AI Evaluation SpecialistChenega Naswik International, LLC · Remote, US