- United States, United States
- Senior
- 5+ years
Design, build, and maintain the systems that measure and improve the quality of Bask Health's AI-powered applications, focusing on evaluation infrastructure, automated testing, and production reliability.
Responsibilities
- Design and build evaluation infrastructure for AI and machine learning applications across Bask Health.
- Develop automated testing systems for large language model applications and intelligent workflows.
- Establish measurable quality standards for production AI features.
- Create evaluation pipelines that assess accuracy, relevance, completeness, consistency, and reliability.
- Build and maintain representative datasets for testing AI application behavior.
Requirements
- Nice to have: LLM Evaluation Frameworks
- Nice to have: LangSmith
- Nice to have: DeepEval
- Nice to have: Ragas
- Nice to have: LLM-as-a-judge evaluation