Introduction

Peer review is a structured process in which another person examines a project and provides evidence-based feedback before the work is finalized, published, merged, or deployed.

In a Machine Learning project, peer review should cover more than Python syntax or model accuracy. The reviewer should examine the problem definition, target, data quality, leakage risks, preprocessing, validation design, metrics, error analysis, reproducibility, deployment approach, security, privacy, fairness, documentation, and responsible-use boundaries.

Effective peer review is collaborative. Its purpose is to improve the work, identify overlooked risks, share knowledge, and strengthen the evidence supporting important project decisions.

Learning Objectives

Prerequisites

What Is Project Peer Review?

Project peer review is an independent examination of a work product by someone other than its primary author. The review may examine source code, notebooks, datasets, documentation, experiments, reports, deployment files, tests, and presentation materials.

Peer Review vs Model Evaluation

Review Roles

Principles of Effective Peer Review

Prepare a Review Package

The author should make the project reviewable before requesting feedback. Reviewers should not need to reconstruct the project from unexplained files.

Author's Review Request

## Review Request

### Project

Customer Churn Prediction

### Review Objective

Validate the final classification pipeline before the portfolio
release.

### Areas Requiring Review

- Churn target definition
- Temporal leakage controls
- Preprocessing pipeline
- Validation and test strategy
- Threshold selection
- Error analysis
- API input validation
- Model-card limitations
- Monitoring and rollback plan

### Changes Included

- Added data validation
- Added Logistic Regression and Random Forest baselines
- Added probability calibration analysis
- Added FastAPI inference endpoint
- Added automated tests
- Added responsible-use documentation

### Validation Completed

- Unit tests pass
- API tests pass
- Final metrics were recalculated
- Public-release privacy review was completed

### Known Limitations

- Intervention effectiveness is not estimated
- Test data represents a limited observation period
- Production monitoring is documented but not connected to a live service

Use a Pull Request for Review

A pull request creates a reviewable comparison between a proposed branch and the target branch. It provides a location for context, code changes, reviewer comments, suggested changes, decisions, and automated checks.

git switch -c review/capstone-release

git add .

git commit -m "review: prepare capstone release candidate"

git push -u origin review/capstone-release

Review the Business Problem

Review the Data

Review for Data Leakage

def review_split_overlap(
    training_ids,
    validation_ids,
    test_ids
):
    training_ids = set(
        training_ids
    )

    validation_ids = set(
        validation_ids
    )

    test_ids = set(
        test_ids
    )

    return {
        "train_validation_overlap": sorted(
            training_ids
            &
            validation_ids
        ),
        "train_test_overlap": sorted(
            training_ids
            &
            test_ids
        ),
        "validation_test_overlap": sorted(
            validation_ids
            &
            test_ids
        )
    }

Review Preprocessing and Features

from sklearn.pipeline import Pipeline


def verify_pipeline_structure(
    model
):
    if not isinstance(
        model,
        Pipeline
    ):
        return {
            "valid": False,
            "message": (
                "The model is not packaged as "
                "a scikit-learn Pipeline."
            )
        }

    step_names = [
        step_name
        for step_name, _
        in model.steps
    ]

    return {
        "valid": True,
        "steps": step_names
    }

Review Model Evaluation

Review Error Analysis

Review Source Code

Reproduce the Project

A reviewer should attempt to run the documented workflow in a clean environment when the project scope permits.

git clone replace-with-repository-address
cd replace-with-repository-directory

python -m venv .venv
source .venv/bin/activate

python -m pip install --upgrade pip
python -m pip install -r requirements.txt

python -m pytest -v
python -m src.train
python -m src.evaluate

Review Security and Privacy

Review Responsible AI

Review Deployment Readiness

Review the Monitoring Plan

Classify Review Findings

Write Actionable Feedback

Feedback Examples

Recognize Strong Work

Positive feedback is valuable when it identifies a specific practice that should be retained.

Receive Feedback Professionally

Peer Review Feedback Record

Machine-Readable Review Record

{
    "review_id": "PR-2026-001",
    "project": "Customer Churn Prediction",
    "reviewed_version": "1.0.0-rc1",
    "findings": [
        {
            "finding_id": "F-001",
            "area": "Model Evaluation",
            "severity": "major",
            "location": "reports/evaluation_report.md",
            "observation": "The report selects the threshold using test data.",
            "impact": "The final test result is no longer independent.",
            "recommended_action": "Select the threshold using validation data and rerun the final test evaluation once.",
            "status": "open",
            "resolution_evidence": null
        }
    ],
    "decision": "changes_requested"
}

Summarize Review Findings with Python

import pandas as pd


review_findings = pd.DataFrame(
    [
        {
            "finding_id": "F-001",
            "area": "Evaluation",
            "severity": "major",
            "status": "open"
        },
        {
            "finding_id": "F-002",
            "area": "Documentation",
            "severity": "minor",
            "status": "resolved"
        },
        {
            "finding_id": "F-003",
            "area": "Security",
            "severity": "critical",
            "status": "open"
        }
    ]
)


severity_summary = (
    review_findings
    .groupby(
        [
            "severity",
            "status"
        ],
        dropna=False
    )
    .size()
    .reset_index(
        name="finding_count"
    )
)


print(
    severity_summary
)

Re-Review Resolved Findings

git fetch origin

git switch review/capstone-release

git pull

python -m pytest -v

Review Decisions

Conduct a Review Meeting

Keep the Review Focused

Self-Review Before Peer Review

def self_review_project(
    checks
):
    failed_checks = [
        check_name
        for check_name, passed
        in checks.items()
        if not passed
    ]

    return {
        "ready_for_peer_review": (
            len(failed_checks)
            ==
            0
        ),
        "failed_checks": failed_checks
    }


project_checks = {
    "readme_complete": True,
    "tests_pass": True,
    "test_set_untouched": True,
    "leakage_review_complete": True,
    "privacy_review_complete": True,
    "metrics_recalculated": True,
    "limitations_documented": True,
    "rollback_documented": False
}


review_status = self_review_project(
    project_checks
)


print(
    review_status
)

Feedback and Improvement Cycle

Peer review is iterative. A correction may reveal new information, change model results, affect documentation, or require additional testing. Important changes should be reviewed at the appropriate level before closure.

Reflect After the Review

## Review Reflection

### Most Important Finding

Describe the finding that had the greatest effect on the project.

### Correction

Explain the technical or documentation change that resolved it.

### Evidence

Link the commit, test, chart, report, or decision record.

### Lesson Learned

Explain what you will do differently in future projects.

### Remaining Risk

Document any unresolved limitation or accepted risk.

Common Peer Review Mistakes

Recommended Peer Review Workflow

Hands-On Peer Review Tasks

Knowledge Check

Peer Review Evaluation Rubric

Final Peer Review Checklist

Lesson Summary