Software Engineer, Applied AI- Evals
Build and ship product features for Mercor’s evals platform, including APIs, data models, UI, and onboarding flows, while owning APEX launches and failure analysis.
- San Francisco, California, United States +2 more
- Full-time
- On-site
- Posted Nov 9, 2026
INDUSTRYSoftware & InternetSaaS & Enterprise Software
Locations (3)
- San Francisco, California, United States
- New York City, New York, United States
- London, England, United Kingdom
Responsibilities
- Build the product customers use to evaluate models: Ship the APIs, data models and UI that help customers run evals, compare models and share scores.
- Own APEX launches: Run every new frontier model on APEX and publish correct, defensible results to the public leaderboard.
- Explain why models fail: Build failure analysis that helps customers see where their models fail, with each pattern linked to the exact steps that went wrong.
- Make benchmarks easy to find and manage: Build dataset views, registries and versioning that help customers find the right benchmark or bring their own.
- Take customers from first login to self-serve: Build onboarding, roles, key management and usage views that help new teams run evals on their own.
- Work directly with customers: Join pilots with labs and enterprises and turn what they need into the roadmap.
Requirements
- Bachelor's Degree in Computer Science or a related field
Nice to have: LLM evals, benchmarks, AI developer tools, startup experience, B2B developer products
Skills
PythonGoTypeScriptReactfull-stackproduct sensedata-heavy productsAPIsdata modelsUIonboardingroleskey managementusage viewsPythonGoTypeScriptReact
Benefits
- Bi-annual performance bonus structure
- Generous equity grant vested over 4 years
- Up to $15k Relocation bonus
- $10K housing bonus (if you live within 0.5 miles of our office)
- $1.5K monthly stipend for meals
- Free Equinox membership
- $200 monthly laundry reimbursement
- $200 monthly personal wellness reimbursement
- Health, Dental, Vision insurance
More jobs nearby
Full description
MercorCompany details are for members
Sign up for a free job alert to see Mercor's profile, website and LinkedIn, and the same for every employer on Referso Jobs.