The Complete Guide to LLM Evaluation
Comments
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Ranked by a public rubric, not by votes: what a comment adds counts, not who wrote it. How comments are ranked · Comment rules
Summary
This guide walks through systematic LLM application evaluation: when to use code, reference-based, or LLM-judge scorers; how to write rubric-driven judges using QAG, G-Eval, and pairwise comparison; how to measure judge bias against human labels with Cohen's kappa; and how to evaluate RAG systems and agents component by component, both…