- Hyderabad, Telangana, India
- Hybrid
- Full-time
- 3–7 years
Design, build, and run evaluations of AI systems across the stack, develop evaluation datasets, metrics, and scoring rubrics, and collaborate with cross-functional teams to define acceptance criteria and evaluation standards for AI use-cases.
Responsibilities
- Design, build, and run evaluations of AI systems across the stack
- Develop evaluation datasets, test suites, metrics, and scoring rubrics
- Apply and adapt established evaluation frameworks and benchmarks
- Analyze and interpret evaluation results and communicate findings
- Collaborate with AI engineering, product, and risk and governance teams
Requirements
- Bachelor’s or master’s degree in computer science, Artificial Intelligence, Machine Learning, Data Science, Engineering, or a related science or technology discipline.
- Nice to have: AI testing and evaluation tools
- Nice to have: AI security and red teaming tools
- Nice to have: AI governance tools
- Nice to have: Qualitative and quantitative research methods