About the Role
Dailoqa is looking for an experienced Senior QA Engineer
– Agentic AI Testing & Evaluation to own the quality, testing, and
evaluation strategy for our Broccoli Platform. This role will be
responsible for establishing robust testing frameworks for AI agents,
multi-agent workflows, orchestration layers, tools, and enterprise-grade AI
applications.
The ideal candidate will have deep expertise in test
automation, quality engineering, AI/ML validation, security testing, and
performance engineering, with a strong understanding of Large Language
Models (LLMs), Agentic AI systems, and modern DevOps practices.
You will work closely with Capability Leads, AI
Architects, Product Teams, and Platform Engineers to ensure that Broccoli
delivers reliable, secure, scalable, and high-performing AI solutions.
Requirements
Required Skills & Qualifications
Technical Expertise
7–10
years of experience in Quality Assurance, Test Automation, or Quality
Engineering.
Strong
experience with automation frameworks using Python, Java, or
JavaScript/TypeScript.
Hands-on
experience with:
Selenium,
Playwright, Cypress, or Robot Framework
API
testing tools such as Postman, Rest Assured, or Karate
CI/CD
platforms including Azure DevOps, GitHub Actions, Jenkins, or GitLab
Experience
with cloud platforms such as Azure, AWS, or Google Cloud.
Agentic AI & LLM Testing
Understanding
of:
Large
Language Models (LLMs)
Retrieval-Augmented
Generation (RAG)
AI
Agents and Agent Orchestration Frameworks
Prompt
Engineering and Evaluation
Experience
with AI evaluation platforms and frameworks is preferred.
Knowledge
of hallucination detection, safety validation, guardrails, and model
benchmarking.
Performance & Security Testing
Experience
with performance testing tools such as JMeter, k6, or Locust.
Knowledge
of application security testing and threat modeling.
Familiarity
with AI security testing, adversarial testing, and governance controls.
Soft Skills
Strong
analytical and problem-solving skills.
Excellent
stakeholder management and communication abilities.
Experience
working in cross-functional Agile teams.
Ability
to lead quality initiatives in fast-paced, innovation-driven environments.
Preferred Qualifications
Experience
testing AI-powered enterprise applications.
Exposure
to Microsoft Azure AI, OpenAI, LangChain, LangGraph, CrewAI, Semantic
Kernel, or similar agent frameworks.
Certifications
such as:
ISTQB
Advanced Test Analyst/Test Manager
Certified
Agile Tester
Azure
AI Engineer Associate
Azure
DevOps Engineer Expert
Experience
working with BFSI, enterprise transformation, or regulated industry
environments.
QA Engineering Lead (AI)Meta · Menlo Park, US