Introduction

A Machine Learning project changes continuously. Developers update Python scripts, notebooks, feature logic, configuration files, evaluation code, documentation, and deployment components. Without version control, it becomes difficult to identify what changed, understand why a result changed, restore a previous working state, or collaborate safely with other contributors.

Git is a distributed version control system. It records changes to files as a sequence of commits. Each contributor can maintain a local copy of the repository, create branches, review differences, combine approved changes, and synchronize work with a remote repository.

Git is highly effective for source code and small text-based files. However, Machine Learning projects may also contain large datasets, trained models, experiment outputs, and generated artifacts. These files usually require additional storage and versioning strategies rather than being committed directly to a normal Git repository.

Learning Objectives

Prerequisites

What Is Version Control?

Version control is a system for recording changes to files over time. It creates a structured history that helps developers inspect previous work, compare revisions, restore earlier versions, and collaborate without replacing one another's changes accidentally.

Git vs Remote Git Platforms

Git and a remote repository platform are related but different. Git is the version control software. A platform such as GitHub, GitLab, Azure Repos, or Bitbucket can host remote Git repositories and provide collaboration features.

Important Git Concepts

The Basic Git Workflow

Configure Git

Git associates an author name and email address with commits. Configure the identity required for the repositories you are authorized to use.

git config --global user.name "Your Name"
git config --global user.email "you@example.com"
git config --global init.defaultBranch main

Review the current Git configuration:

git config --list

Create a New ML Repository

Create a project directory and initialize it as a Git repository.

mkdir customer-churn-ml
cd customer-churn-ml
git init

Check the repository status:

git status

Git creates a hidden .git directory containing the repository metadata and history. Do not manually edit or delete this directory unless you fully understand the consequences.

Recommended ML Project Structure

A consistent repository structure helps contributors locate source code, configuration, tests, notebooks, documentation, and generated outputs.

customer-churn-ml/
|
|-- README.md
|-- .gitignore
|-- requirements.txt
|-- pyproject.toml
|-- configs/
|   `-- baseline.yaml
|-- data/
|   |-- raw/
|   |-- interim/
|   `-- processed/
|-- models/
|-- notebooks/
|   `-- 01_exploration.ipynb
|-- reports/
|   |-- figures/
|   `-- metrics/
|-- src/
|   |-- __init__.py
|   |-- data.py
|   |-- features.py
|   |-- train.py
|   `-- evaluate.py
|-- tests/
|   |-- test_data.py
|   `-- test_features.py
`-- scripts/
    `-- run_training.py

Create an ML-Specific .gitignore

A .gitignore file contains patterns for intentionally untracked files that Git should ignore. It is commonly committed so everyone using the repository shares the same ignore rules.

# Python bytecode and caches
__pycache__/
*.py[cod]
*$py.class

# Virtual environments
.venv/
venv/
env/

# Environment variables and secrets
.env
.env.*
!.env.example
*.pem
*.key
secrets/

# Testing and coverage
.pytest_cache/
.coverage
htmlcov/

# Type checker and linter caches
.mypy_cache/
.ruff_cache/

# Jupyter temporary files
.ipynb_checkpoints/

# IDE and operating-system files
.vscode/
.idea/
.DS_Store
Thumbs.db

# Logs and temporary files
*.log
*.tmp
*.temp
tmp/
cache/

# Build and package outputs
build/
dist/
*.egg-info/

# Local datasets
data/raw/*
data/interim/*
data/processed/*

# Keep documentation files inside data directories
!data/raw/README.md
!data/interim/README.md
!data/processed/README.md

# Generated model artifacts
models/*
!models/README.md
*.pkl
*.joblib
*.h5
*.keras
*.onnx
*.pt
*.pth
*.ckpt

# Generated experiment outputs
runs/
outputs/
artifacts/
mlruns/

# Large array and archive files
*.npy
*.npz
*.parquet
*.zip
*.tar
*.gz

Create the First Commit

Create the initial project files, review them, and add selected changes to the staging area.

git status
git add README.md .gitignore requirements.txt pyproject.toml
git status

Review the staged differences:

git diff --staged

Create the first commit:

git commit -m "Initialize ML project structure"

Review the commit history:

git log --oneline --decorate --graph

Write Meaningful Commit Messages

A useful commit message explains the purpose of a change. It should enable another contributor to understand the project history without opening every modified file.

Work with Branches

A branch provides an independent line of development. It allows a contributor to work on a feature, experiment, correction, or documentation update without immediately changing the main branch.

Create and switch to a feature branch:

git switch -c feature/add-baseline-model

After making and reviewing the changes:

git status
git add src/train.py tests/test_train.py
git commit -m "Add baseline logistic regression model"

Switch back to the main branch and merge the feature:

git switch main
git merge feature/add-baseline-model

Delete the local feature branch after the change has been merged and is no longer needed:

git branch -d feature/add-baseline-model

Connect to a Remote Repository

After creating an authorized empty remote repository, connect the local project to it.

git remote add origin REMOTE_REPOSITORY_URL
git remote -v
git push -u origin main

The -u option establishes an upstream relationship between the local and remote branches. Later pushes can normally use:

git push

To copy an existing repository:

git clone REMOTE_REPOSITORY_URL
cd REPOSITORY_DIRECTORY

Collaborative Git Workflow

git switch main
git pull

git switch -c feature/add-model-evaluation

git status
git diff
git add src/evaluate.py tests/test_evaluate.py
git diff --staged
git commit -m "Add classification evaluation report"

git push -u origin feature/add-model-evaluation

Resolve Merge Conflicts

A merge conflict can occur when Git cannot automatically combine competing changes. The developer must inspect the affected file, choose the correct final content, test the result, and complete the merge.

git status

After editing and testing the conflicted files:

git add PATH_TO_RESOLVED_FILE
git commit

Use Tags for Important Versions

A tag can mark an important repository state, such as a validated model release, experiment milestone, or deployment version.

git tag -a model-v1.0.0 -m "Validated baseline model"
git push origin model-v1.0.0

List available tags:

git tag

Why Git Alone Is Not Enough for ML

Traditional software repositories primarily contain source code and small configuration files. Machine Learning projects introduce additional versioned components.

Managing Large Data and Model Files

Large datasets and model binaries can make a normal Git repository slow and difficult to maintain. They may also contain confidential information that must not be stored in a source-code repository.

Git Large File Storage

Git Large File Storage, commonly called Git LFS, replaces selected large files in the repository with lightweight pointer files while storing the actual content in LFS-managed storage.

git lfs install
git lfs track "*.onnx"
git lfs track "*.pt"
git lfs track "*.joblib"
git add .gitattributes
git commit -m "Configure Git LFS for model artifacts"

Data Version Control

A data-versioning tool such as DVC can track metadata in Git while keeping large datasets and model artifacts in separate local or remote storage.

Version ML Experiments

A commit records file changes, but it does not automatically record every detail of an experiment. A reproducible experiment should connect the source-code version with its data, configuration, environment, metrics, and outputs.

Version Jupyter Notebooks Carefully

Jupyter notebooks are useful for exploration and communication, but they can create large and noisy Git differences because they contain code, outputs, execution counts, and metadata in one JSON document.

Protect Secrets and Sensitive Data

A Git repository must not be treated as a secret store. Removing a secret in a later commit does not automatically remove it from earlier repository history.

API_KEY = "real-secret-value"
DATABASE_PASSWORD = "real-password"

import os

api_key = os.environ["API_KEY"]
database_password = os.environ["DATABASE_PASSWORD"]

Inspect and Undo Changes Safely

Inspect Changes

git status
git diff
git diff --staged
git log --oneline --graph --decorate

Unstage a File

git restore --staged PATH_TO_FILE

Discard an Uncommitted File Change

git restore PATH_TO_FILE

Revert a Committed Change

git revert COMMIT_IDENTIFIER

Stash Temporary Work

Git stash can temporarily store uncommitted tracked-file changes when the developer needs a clean working directory.

git stash push -m "Temporary feature-engineering work"
git stash list
git stash pop

Pull Request Review Checklist

Common Git Mistakes in ML Projects

Essential Git Command Reference

Hands-On Practice

Knowledge Check

Lesson Summary