AI Builders Global Conference 2026. October 14-16, 2026. Online.
LLM Evaluation, Testing & Quality Assurance
This description belongs to the 2026 topic catalog. The 2027 program is being prepared separately.
How do you know your AI is working? This emerging discipline, sometimes called "LLM unit testing", is critical for production AI. This workshop covers building evaluation suites for LLM applications, automated testing pipelines, benchmark design specific to your use case, regression detection across model versions, human-in-the-loop evaluation workflows, and the metrics that actually correlate with user satisfaction.
Questions explored in the 2026 topic catalog
- What metrics actually correlate with LLM application quality?
- How do you build automated eval pipelines for LLM apps?
- What does "LLM unit testing" look like in practice?
- How do you detect model regressions after updates?
- When is human evaluation necessary vs. automated evaluation sufficient?
- What open-source tools are available for LLM evaluation?
Who this topic is for
QA engineers, ML engineers, product managers, and anyone responsible for ensuring AI application quality and reliability.
Related topics