- Remote, Indonesia +1 more
- Remote
- Full-time · Mid
- 5+ years
This role owns and scales the testing of Hupo’s AI pipelines, working closely with QA, Engineering, and Product as the platform grows.
Locations (2)
- New Delhi
- Jakarta, Jakarta Special Capital Region, Indonesia
Responsibilities
- Build and maintain automated testing for AI voice and chat agents, from single conversational turns to full roleplay flows
- Test LLM output quality - correctness, consistency, structured output, language fidelity, prompt regression - using evaluation harnesses (LLM-as-judge, golden datasets, tolerance-based assertions for non-determinism)
- Validate RAG pipelines: retrieval relevance, grounding/faithfulness, and answer quality
- Test voice pipelines: STT/TTS accuracy and real-time, low-latency behavior across languages
- Automate across UI/E2E, API, and AI output layers, and build CI/CD from scratch - lint, type-check, tests, evals, coverage gates, deployment checks