Introduction

Large Language Models can produce fluent, relevant, and human-readable responses. However, fluency does not prove factual correctness, faithfulness, safety, completeness, or usefulness. A response may sound confident while containing unsupported claims, incorrect citations, omitted requirements, invalid tool arguments, or fabricated information.

Traditional Machine Learning evaluation often compares a prediction with a structured reference label. LLM applications may produce long, open-ended outputs with several possible acceptable answers. This makes evaluation more complex.

Reliable LLM evaluation combines multiple methods. Deterministic checks can validate schemas, citations, tool calls, required terms, and executable outputs. Reference-based metrics can compare outputs with approved answers. Human evaluators can assess nuanced quality. An LLM-as-Judge can apply a structured rubric at scale, provided that its limitations and biases are controlled.

Learning Objectives

Prerequisites

python -m pip install \
  pandas \
  scikit-learn \
  deepeval \
  rouge-score \
  sacrebleu \
  bert-score \
  pydantic

What Is LLM Evaluation?

LLM evaluation is the systematic process of measuring whether a model or LLM application satisfies its defined quality, safety, operational, and business requirements.

Why LLM Evaluation Is Difficult

Evaluation Layers

Offline vs Online Evaluation

Evaluation Dataset

An evaluation dataset is a versioned collection of test cases representing the scenarios an LLM application must handle.

{
    "case_id": "support-001",
    "input": "How do I reset my account password?",
    "reference_answer": "Use the approved password-reset workflow.",
    "context": [
        "Approved password-reset policy content"
    ],
    "expected_behavior": {
        "must_reference_policy": true,
        "must_not_request_password": true,
        "requires_human_escalation": false
    },
    "tags": [
        "account-support",
        "grounded-answer",
        "security"
    ]
}

Evaluation Dataset Metadata

Deterministic Evaluation

Deterministic evaluation uses explicit rules or executable checks. It should be preferred when the required behavior can be tested reliably without subjective language judgment.

Validate Structured Output

import json
from typing import Literal

from pydantic import BaseModel
from pydantic import ValidationError


class SupportClassification(
    BaseModel
):
    category: Literal[
        "Bug",
        "Billing",
        "Access",
        "How-To",
        "Other"
    ]

    confidence: float

    explanation: str


def validate_structured_output(
    model_output
):
    try:
        parsed_output = json.loads(
            model_output
        )

        validated_output = (
            SupportClassification
            .model_validate(
                parsed_output
            )
        )

        if not (
            0.0
            <=
            validated_output.confidence
            <=
            1.0
        ):
            raise ValueError(
                "Confidence must be "
                "between zero and one."
            )

        return {
            "valid": True,
            "value": (
                validated_output.model_dump()
            ),
            "error": None
        }

    except (
        json.JSONDecodeError,
        ValidationError,
        ValueError
    ) as error:

        return {
            "valid": False,
            "value": None,
            "error": str(
                error
            )
        }

Reference-Based Text Metrics

Calculate ROUGE

from rouge_score import rouge_scorer


def calculate_rouge(
    reference_text,
    generated_text
):
    scorer = rouge_scorer.RougeScorer(
        [
            "rouge1",
            "rouge2",
            "rougeL"
        ],
        use_stemmer=True
    )

    scores = scorer.score(
        reference_text,
        generated_text
    )

    return {
        metric_name: {
            "precision": float(
                score.precision
            ),
            "recall": float(
                score.recall
            ),
            "fmeasure": float(
                score.fmeasure
            )
        }
        for metric_name, score
        in scores.items()
    }

Human Evaluation

Human evaluation uses qualified reviewers to assess model outputs according to an explicit rubric.

Example Evaluation Rubric

What Is LLM-as-Judge?

LLM-as-Judge uses a language model to evaluate another model's response against a prompt, reference, retrieved context, rubric, or expected behavior.

Pointwise vs Pairwise Judging

Design a Judge Prompt

You are evaluating an AI-generated response.

Evaluate only the criterion in the rubric.
Do not reward writing style unless the rubric requests it.
Use only the supplied reference and context as evidence.
If the evidence is insufficient, mark the relevant claim as unsupported.
Return valid JSON matching the required schema.

Input:
{input}

Reference answer:
{reference_answer}

Context:
{context}

Candidate response:
{candidate_response}

Rubric:
1. Correctness
2. Relevance
3. Completeness
4. Faithfulness to context

Required JSON:
{
  "correctness_score": 0,
  "relevance_score": 0,
  "completeness_score": 0,
  "faithfulness_score": 0,
  "unsupported_claims": [],
  "reason": ""
}

Validate Judge Output

from pydantic import BaseModel
from pydantic import Field


class JudgeResult(
    BaseModel
):
    correctness_score: int = Field(
        ge=0,
        le=4
    )

    relevance_score: int = Field(
        ge=0,
        le=4
    )

    completeness_score: int = Field(
        ge=0,
        le=4
    )

    faithfulness_score: int = Field(
        ge=0,
        le=4
    )

    unsupported_claims: list[
        str
    ]

    reason: str


def validate_judge_result(
    parsed_judge_output
):
    return JudgeResult.model_validate(
        parsed_judge_output
    )

Biases in LLM-as-Judge

Reduce Position Bias

def create_pairwise_orders(
    input_text,
    first_response,
    second_response
):
    return [
        {
            "input": input_text,
            "response_a": first_response,
            "response_b": second_response,
            "mapping": {
                "A": "first",
                "B": "second"
            }
        },
        {
            "input": input_text,
            "response_a": second_response,
            "response_b": first_response,
            "mapping": {
                "A": "second",
                "B": "first"
            }
        }
    ]

Calibrate the Judge

Judge calibration compares automated judgments with labels produced by qualified human evaluators.

Calculate Judge Agreement

from sklearn.metrics import accuracy_score
from sklearn.metrics import cohen_kappa_score
from sklearn.metrics import confusion_matrix


def calculate_judge_agreement(
    human_labels,
    judge_labels
):
    if len(
        human_labels
    ) != len(
        judge_labels
    ):
        raise ValueError(
            "Human and judge label counts "
            "must match."
        )

    return {
        "exact_agreement": float(
            accuracy_score(
                human_labels,
                judge_labels
            )
        ),
        "cohen_kappa": float(
            cohen_kappa_score(
                human_labels,
                judge_labels
            )
        ),
        "confusion_matrix": (
            confusion_matrix(
                human_labels,
                judge_labels
            ).tolist()
        )
    }

What Is Hallucination?

In practical LLM evaluation, hallucination refers to generated information that is fabricated, contradicted by reliable evidence, or unsupported by the evidence the application was expected to use.

Hallucination vs Faithfulness

Claim-Level Evaluation

Claim-level evaluation decomposes a response into checkable claims and evaluates each claim against context or reliable ground truth.

Calculate Claim-Level Metrics

def calculate_claim_metrics(
    claim_labels
):
    allowed_labels = {
        "supported",
        "unsupported",
        "contradicted"
    }

    invalid_labels = [
        label
        for label in claim_labels
        if label not in allowed_labels
    ]

    if invalid_labels:

        raise ValueError(
            "Unknown claim labels were found."
        )

    total_claims = len(
        claim_labels
    )

    if total_claims == 0:

        raise ValueError(
            "At least one evaluated claim "
            "is required."
        )

    supported_count = claim_labels.count(
        "supported"
    )

    unsupported_count = claim_labels.count(
        "unsupported"
    )

    contradicted_count = claim_labels.count(
        "contradicted"
    )

    return {
        "evaluated_claims": total_claims,
        "supported_claims": supported_count,
        "unsupported_claims": unsupported_count,
        "contradicted_claims": contradicted_count,
        "faithfulness": (
            supported_count
            /
            total_claims
        ),
        "unsupported_rate": (
            unsupported_count
            /
            total_claims
        ),
        "contradiction_rate": (
            contradicted_count
            /
            total_claims
        )
    }

Hallucination Evaluation Workflow

Hallucination Evaluation with DeepEval

from deepeval import evaluate
from deepeval.metrics import HallucinationMetric
from deepeval.test_case import LLMTestCase


test_case = LLMTestCase(
    input=(
        "What does the supplied policy "
        "say about password resets?"
    ),
    actual_output=(
        generated_response
    ),
    context=[
        approved_policy_context
    ]
)


hallucination_metric = (
    HallucinationMetric(
        threshold=0.5,
        include_reason=True
    )
)


evaluation_result = evaluate(
    test_cases=[
        test_case
    ],
    metrics=[
        hallucination_metric
    ]
)

Faithfulness for RAG

In a RAG system, faithfulness evaluates whether the generated response is supported by the retrieved context supplied to the model.

from deepeval.metrics import FaithfulnessMetric


faithfulness_metric = (
    FaithfulnessMetric(
        threshold=0.8,
        include_reason=True
    )
)


rag_test_case = LLMTestCase(
    input=user_question,
    actual_output=generated_answer,
    retrieval_context=(
        retrieved_contexts
    )
)


rag_evaluation = evaluate(
    test_cases=[
        rag_test_case
    ],
    metrics=[
        faithfulness_metric
    ]
)

RAG Evaluation Dimensions

Diagnose RAG Failures

Evaluate Tool Use

Deterministic Tool-Call Validation

def evaluate_tool_call(
    actual_call,
    expected_tool,
    required_arguments
):
    failures = []

    if actual_call.get(
        "tool_name"
    ) != expected_tool:

        failures.append(
            "Incorrect tool selected."
        )

    actual_arguments = actual_call.get(
        "arguments",
        {}
    )

    for argument_name, expected_value in (
        required_arguments.items()
    ):
        if argument_name not in actual_arguments:

            failures.append(
                f"Missing argument: {argument_name}"
            )

        elif (
            actual_arguments[
                argument_name
            ]
            != expected_value
        ):
            failures.append(
                f"Incorrect value for: "
                f"{argument_name}"
            )

    return {
        "passed": len(
            failures
        ) == 0,
        "failures": failures
    }

Evaluate Agent Trajectories

Agent evaluation may need to examine not only the final answer but also the observable sequence of planned actions, tool calls, tool results, retries, and state transitions.

Safety Evaluation

Multilingual Evaluation

Operational Evaluation

Composite Quality Scores

A composite score can summarize several dimensions, but it can also hide a critical failure. Use it for reporting only after preserving the individual criterion scores.

Create an Evaluation Gate

def evaluate_release_gate(
    measured_metrics,
    minimum_metrics,
    maximum_metrics=None,
    mandatory_failures=None
):
    maximum_metrics = (
        maximum_metrics
        or {}
    )

    mandatory_failures = (
        mandatory_failures
        or []
    )

    failures = list(
        mandatory_failures
    )

    for metric_name, minimum_value in (
        minimum_metrics.items()
    ):
        actual_value = measured_metrics.get(
            metric_name
        )

        if actual_value is None:

            failures.append(
                f"Missing metric: {metric_name}"
            )

        elif actual_value < minimum_value:

            failures.append(
                f"{metric_name} is below "
                "the approved minimum."
            )

    for metric_name, maximum_value in (
        maximum_metrics.items()
    ):
        actual_value = measured_metrics.get(
            metric_name
        )

        if actual_value is None:

            failures.append(
                f"Missing metric: {metric_name}"
            )

        elif actual_value > maximum_value:

            failures.append(
                f"{metric_name} exceeds "
                "the approved maximum."
            )

    return {
        "passed": len(
            failures
        ) == 0,
        "failures": failures
    }

Evaluation in CI/CD

name: Evaluate LLM Application

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  evaluate:
    runs-on: ubuntu-latest

    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          python -m pip install -r requirements.txt

      - name: Run deterministic tests
        run: |
          python -m pytest -v

      - name: Run offline LLM evaluation
        run: |
          python -m evaluation.run_suite \
            --dataset evaluation/dataset.jsonl \
            --configuration evaluation/config.json \
            --output artifacts/evaluation-results.json

      - name: Apply release gate
        run: |
          python -m evaluation.release_gate \
            --results artifacts/evaluation-results.json \
            --policy evaluation/release-policy.json

Continuous Production Evaluation

Production Evaluation Sampling

A/B and Canary Evaluation

Evaluation Reproducibility

{
    "evaluation_run_id": "generated-run-identifier",
    "application_version": "2.0.0",
    "candidate_model": "approved-model-identifier",
    "candidate_model_revision": "immutable-revision",
    "prompt_version": "3.1.0",
    "evaluation_dataset_version": "1.4.0",
    "judge_model": "approved-judge-identifier",
    "judge_prompt_version": "2.0.0",
    "rubric_version": "2.0.0",
    "generation_temperature": 0,
    "evaluation_library_version": "record-installed-version"
}

Security and Privacy

Responsible LLM Evaluation

Common LLM Evaluation Mistakes

Troubleshooting

Recommended LLM Evaluation Workflow

Hands-On Practice

Knowledge Check

LLM Evaluation Readiness Checklist

Lesson Summary