- Remote, United States
- Remote
- Full-time · Mid
A Technical Product Manager role focused on ensuring Medley’s AI-powered instructional scaffolds meet high standards for quality and user experience. The role sits at the intersection of pedagogy and AI product work.
Responsibilities
- Define what 'excellent' looks like by translating research-based practice in literacy, special education, and multilingual learner instruction into clear quality standards.
- Track and monitor quality with a robust evals system, building and maintaining datasets, rubrics, and eval suites.
- Conduct regular error analysis and trace review using human labeling, LLM-as-judge, and rule-based evals.
- Refine prompts and models; write, test, and run model comparisons and regression tests.
- Review signals from educators, data from support tickets, and product analytics to inform product decisions.
Requirements
- Graduate degree (preferred)
- Nice to have: Special education
- Nice to have: Reading/literacy
- Nice to have: ESL/ELD
- Nice to have: Bilingual education