- Singapore, Singapore
- On-site
- Full-time · Mid
- $100K–$170K per year
- 2–4 years
- Visa sponsorship
Join a small, technical AI evaluation team building rigorous benchmarks for frontier AI agents. You will own the design and implementation of domain-specific evaluations that help research teams and customers understand how agents perform on realistic workflows.
Compensation
Base salary: $100,000 to $170,000 USD annually. Visa sponsorship is available.
Responsibilities
- Design, implement, and maintain internal benchmarks for evaluating frontier agents on domain-specific tasks
- Partner with subject-matter experts to translate real-world workflows into benchmark tasks and evaluation criteria
- Build reliable infrastructure to run models and agents against benchmark tasks at scale
- Develop metrics and analyses to assess benchmark difficulty, reliability, and failure modes
- Validate whether benchmark results align with real-world evaluations and user needs
Requirements
- Technical education or strong fundamentals in a technical field
- Nice to have: Reinforcement learning pipelines
- Nice to have: data generation
- Nice to have: agent evaluation