Table of Contents

Docker Basics for ML

Rumman Ansari September 16, 2026 57 views Subject Details

Model Deployment

Docker Basics for ML

Learn how to package a Machine Learning API, saved model, Python runtime, dependencies, configuration, and startup command into a versioned Docker image that can be tested and executed consistently across development, CI/CD, staging, cloud, and production environments.

Introduction

A Machine Learning deployment depends on more than a model file. The prediction service also requires the correct Python version, libraries, source code, model artifact, configuration, operating system packages, startup command, and runtime permissions.

Docker packages these components into a container image. The image can then be used to create one or more containers. Each container runs the packaged application as an isolated process while sharing the host operating system kernel.

This lesson containerizes the FastAPI customer-churn prediction API created in the previous lesson. The resulting image includes the application code, dependency manifest, model pipeline, model metadata, non-root runtime user, production server command, and health-check configuration.

Lesson goal: Create a secure, reproducible, testable, and versioned Docker image for a Machine Learning prediction service.

Learning Objectives

By the End of This Lesson

Explain what Docker is and why it is useful for ML
Distinguish images, containers, registries, volumes, and networks
Understand the role of a Dockerfile
Create a container-ready ML API structure
Create a focused .dockerignore file
Build a Docker image for a FastAPI prediction service
Run the application in a container
Publish a container port safely
Pass configuration through environment variables
Mount files and directories when appropriate
Configure a container health check
Run the application as a non-root user
Use immutable image tags and digests
Inspect running containers and logs
Test prediction behavior inside the image
Use Docker Compose for multiple services
Understand container registries
Scan images and dependencies
Apply resource limits and runtime controls
Use Docker images in CI/CD pipelines
Identify common containerization mistakes
Create a production-readiness checklist

Prerequisites

Recommended Knowledge

Python programming
Virtual environments and dependencies
Saving and loading trained models
FastAPI or Flask fundamentals
HTTP endpoints and JSON
Command-line usage
Git and source control
Basic operating-system concepts
Environment variables
Model deployment fundamentals

Required Tools

Docker Engine or an approved compatible container runtime
Docker CLI
Optional Docker Compose plugin
Git
A code editor
An existing Machine Learning API project

docker version
docker compose version
git --version

Organizational requirement: Use an approved container runtime, registry, base-image source, licensing process, security policy, and development environment.

What Is Docker?

Docker is a platform and toolset for building, distributing, and running containerized applications. A Docker image defines the application files and runtime requirements. A container is a running instance created from that image.

Containerization Flow

Application Code + Dependencies + Runtime + Configuration → Container Image → Running Container

Docker Concept	Meaning
Dockerfile	A text file containing instructions used to build an image
Image	An immutable package containing the application and runtime layers
Container	A running or stopped instance created from an image
Registry	A service used to store and distribute container images
Repository	A named collection of related image versions in a registry
Tag	A human-readable label that references an image version
Digest	A content-based identifier for an image manifest
Volume	Storage managed independently from a container's writable layer
Network	A mechanism through which containers and services communicate
Docker Compose	A declarative method for defining and running multiple services

Image vs Container

Docker Image

Built from Dockerfile instructions
Contains layered application content
Can be tagged and stored in a registry
Should be treated as an immutable release artifact

Docker Container

Created from an image
Runs the application process
Has a writable runtime layer
Can be started, stopped, restarted, and removed

Simple analogy: An image is the packaged definition. A container is one running instance created from that definition.

Containers vs Virtual Machines

Area	Container	Virtual Machine
Operating System	Shares the host kernel while isolating application processes	Includes a complete guest operating system
Package Size	Often smaller because a guest operating system is not included	Often larger because the guest operating system is included
Startup	Starts the packaged application process	Starts a complete guest operating system
Isolation Boundary	Process-level isolation managed by the container runtime	Virtualized machine boundary managed by a hypervisor
Typical ML Use	Packaging inference services and reproducible jobs	Hosting complete environments or specialized infrastructure

Security note: Containers provide isolation but are not automatically secure. Image content, runtime permissions, host configuration, secrets, networking, and workloads must still be protected.

Why Use Docker for Machine Learning?

Practical Benefits

Packages the API and model runtime together
Preserves the required Python environment
Reduces differences between testing and deployment
Supports repeatable CI/CD builds
Provides an immutable application-release artifact
Supports local, cloud, and orchestrated execution
Makes startup commands explicit
Supports health checks and resource controls
Separates application dependencies from the host
Supports versioned distribution through registries

Important limitation: Containers reduce environment differences, but they do not prevent data drift, concept drift, invalid inputs, weak monitoring, model misuse, or incorrect business integration.

ML Container Architecture

Containerized Inference Flow

Client → Published Host Port → Container Port → ASGI Server → FastAPI → Saved ML Pipeline

Container Component	Responsibility
Base Image	Provides the operating-system and language runtime foundation
Python Dependencies	Provide FastAPI, inference, and model libraries
Application Code	Defines schemas, endpoints, validation, and inference behavior
Model Artifact	Contains the approved fitted preprocessing and model
Metadata	Defines model version, threshold, schema, and intended use
Runtime User	Defines the operating-system identity running the application
Startup Command	Starts the production ASGI server
Health Check	Evaluates whether the containerized application responds

Recommended ML API Structure

docker-ml-api/
├── README.md
├── Dockerfile
├── compose.yaml
├── .dockerignore
├── .gitignore
├── requirements.txt
├── artifacts/
│   ├── customer_churn_pipeline.joblib
│   └── model_metadata.json
├── app/
│   ├── __init__.py
│   ├── main.py
│   ├── configuration.py
│   ├── inference.py
│   ├── schemas.py
│   └── routers/
│       ├── __init__.py
│       ├── health.py
│       └── predictions.py
├── sample-data/
│   └── request.json
└── tests/
    ├── conftest.py
    ├── test_health.py
    ├── test_predictions.py
    └── test_container.py

Build-context principle: Keep the Docker build context focused. Do not send raw datasets, virtual environments, secrets, notebooks, caches, or unrelated project files to the image builder.

Create requirements.txt

fastapi
uvicorn
pandas
numpy
scikit-learn
joblib

For a controlled release, record and test the exact package versions used by the approved serving environment.

Dependency rule: Include inference dependencies only. Training, notebook, plotting, and experimentation packages do not belong in the serving image unless the runtime genuinely requires them.

Create .dockerignore

# Source-control metadata
.git
.gitignore

# Python cache
__pycache__
*.py[cod]
.pytest_cache
.mypy_cache
.ruff_cache

# Virtual environments
.venv
venv

# Environment and secrets
.env
.env.*
*.pem
*.key

# Notebooks
.ipynb_checkpoints
notebooks

# Raw and processed data
data
*.csv
*.parquet

# Development reports
coverage.xml
htmlcov
reports

# Local logs and temporary files
*.log
tmp
temp

# Editor files
.vscode
.idea

# Docker development files not required in image
compose*.yaml

# Documentation
README.md

Important: Do not exclude the files required at runtime. The application code, dependency manifest, model artifact, and approved metadata must remain available to the build.

Dockerfile Fundamentals

Instruction	Purpose
`FROM`	Selects the base image
`WORKDIR`	Sets the working directory for later instructions
`COPY`	Copies files from the build context into the image
`RUN`	Executes a command while building the image
`ENV`	Defines an environment variable in the image
`ARG`	Defines a build-time variable
`USER`	Defines the runtime user
`EXPOSE`	Documents the application port
`HEALTHCHECK`	Defines a command used to evaluate container health
`CMD`	Defines the default container command
`ENTRYPOINT`	Defines the primary executable for the container

Create a Basic Dockerfile

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .

RUN python -m pip install \
    --no-cache-dir \
    -r requirements.txt

COPY app ./app
COPY artifacts ./artifacts

EXPOSE 8000

CMD [
    "uvicorn",
    "app.main:app",
    "--host",
    "0.0.0.0",
    "--port",
    "8000"
]

Layer-caching principle: Copy the dependency manifest before application code. Docker may reuse the dependency-installation layer when source code changes but dependencies remain unchanged.

Create a Safer Production Dockerfile

FROM python:3.12-slim

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
ENV PIP_NO_CACHE_DIR=1

WORKDIR /app

RUN groupadd \
        --system \
        mlgroup \
    && useradd \
        --system \
        --gid mlgroup \
        --create-home \
        mluser

COPY requirements.txt .

RUN python -m pip install \
    --upgrade \
    pip \
    && python -m pip install \
        -r requirements.txt

COPY --chown=mluser:mlgroup app ./app
COPY --chown=mluser:mlgroup artifacts ./artifacts

USER mluser

EXPOSE 8000

HEALTHCHECK \
    --interval=30s \
    --timeout=5s \
    --start-period=20s \
    --retries=3 \
    CMD python -c \
    "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=3)"

CMD [
    "uvicorn",
    "app.main:app",
    "--host",
    "0.0.0.0",
    "--port",
    "8000"
]

Non-root principle: The application should run with the minimum operating-system permissions required for inference.

CMD vs ENTRYPOINT

Instruction	General Role
`CMD`	Provides the default command or arguments and can be replaced at runtime
`ENTRYPOINT`	Defines the container's primary executable

For a simple ML API, a JSON-array CMD is often sufficient. It starts the ASGI server directly without invoking an unnecessary shell.

Weak Pattern Start the API development server with automatic reload inside a production image.

Better Pattern Start the production ASGI server directly and configure worker and resource behavior for the target environment.

Build the Docker Image

docker build \
  --tag customer-churn-api:1.0.0 \
  .

The final period represents the current directory as the Docker build context.

docker image ls

Versioning principle: Use an immutable release tag such as customer-churn-api:1.0.0. Do not depend only on a mutable tag such as latest.

Run the Container

docker run \
  --detach \
  --name customer-churn-api \
  --publish 8000:8000 \
  customer-churn-api:1.0.0

Argument	Meaning
`--detach`	Runs the container in the background
`--name`	Assigns a readable container name
`--publish 8000:8000`	Maps host port 8000 to container port 8000
`customer-churn-api:1.0.0`	Identifies the image used to create the container

Port rule: EXPOSE documents a port in the image. Publishing a host port is a separate runtime operation.

Inspect Running Containers

docker container ls

docker container ls --all

docker inspect customer-churn-api

Inspect For

Container status
Image identifier
Published ports
Runtime environment variables
Mounts
Network configuration
Health status
Restart count
Configured runtime user

View Container Logs

docker logs customer-churn-api

docker logs \
  --follow \
  --tail 100 \
  customer-churn-api

Logging principle: Containerized applications commonly write operational logs to standard output and standard error so the runtime platform can collect them.

Unsafe Logging Write complete customer requests, authentication tokens, or secret values to container logs.

Safer Logging Record approved operational information such as request ID, endpoint, status, latency, model version, and error category.

Check Container Health

docker inspect \
  --format='{{json .State.Health}}' \
  customer-churn-api

curl http://127.0.0.1:8000/health

curl http://127.0.0.1:8000/ready

Check	Purpose
Container Process	Confirms that the container's main process is running
Health Check	Confirms that the configured application check succeeds
Readiness Endpoint	Confirms that the approved model is loaded and ready
Smoke Prediction	Confirms end-to-end request validation and inference

Test the Containerized API

curl \
  --request POST \
  --header "Content-Type: application/json" \
  --data '{
    "tenure_months": 18,
    "monthly_charges": 64.5,
    "total_charges": 1161.0,
    "contract_type": "Monthly",
    "payment_method": "Electronic",
    "tech_support": "No"
  }' \
  http://127.0.0.1:8000/predictions

Synthetic-input notice: The values are invented for API-contract testing and do not describe a real customer.

Execute a Command Inside the Container

docker exec \
  customer-churn-api \
  python --version

docker exec \
  customer-churn-api \
  python -c "import sklearn; print(sklearn.__version__)"

docker exec \
  customer-churn-api \
  id

Diagnostic use: Runtime command execution can help troubleshoot development or authorized operational incidents. Restrict access to production container-management interfaces.

Stop and Remove the Container

docker stop customer-churn-api

docker rm customer-churn-api

Removing a container does not automatically remove the image from which it was created.

docker image rm customer-churn-api:1.0.0

Pass Environment Variables

docker run \
  --detach \
  --name customer-churn-api \
  --publish 8000:8000 \
  --env MODEL_PATH=/app/artifacts/customer_churn_pipeline.joblib \
  --env METADATA_PATH=/app/artifacts/model_metadata.json \
  --env MAXIMUM_BATCH_SIZE=100 \
  customer-churn-api:1.0.0

Unsafe Practice Include passwords, access keys, or tokens directly in the Dockerfile.

Required Practice Supply sensitive configuration through the approved secret management and workload-identity mechanisms of the target environment.

Use an Environment File Carefully

MODEL_PATH=/app/artifacts/customer_churn_pipeline.joblib
METADATA_PATH=/app/artifacts/model_metadata.json
MAXIMUM_BATCH_SIZE=100

docker run \
  --detach \
  --name customer-churn-api \
  --publish 8000:8000 \
  --env-file .env.runtime \
  customer-churn-api:1.0.0

Environment-file caution: Exclude local environment files from source control and protect their permissions. An environment file is not a replacement for a managed secret store.

Understand Container Storage

A container has a writable layer, but that layer is generally coupled to the container's lifecycle. Important persistent data should be stored using approved external storage or volumes.

Storage Type	Typical Use
Image Layer	Application code and immutable runtime assets
Container Writable Layer	Temporary runtime files
Named Volume	Data managed independently from a container
Bind Mount	Maps a specific host file or directory into a container
External Object Storage	Durable artifacts, batch inputs, and prediction outputs

Stateless-service principle: An online prediction API should normally avoid storing important application state inside its local container filesystem.

Mount a Model for Development

docker run \
  --detach \
  --name customer-churn-api \
  --publish 8000:8000 \
  --mount \
  type=bind,source="$(pwd)/artifacts",target=/app/artifacts,readonly \
  customer-churn-api:1.0.0

Development use: A read-only bind mount can simplify local artifact testing. A production release should use an approved artifact-distribution design with integrity and version controls.

Baked-In vs Mounted Model

Approach	Advantages	Considerations
Model Inside Image	One immutable package includes application and model	Every model update requires a new image
Model Mounted at Runtime	Application image and model can be versioned separately	Startup must verify availability, version, and integrity
Model Downloaded at Startup	Supports controlled retrieval from an artifact store	Requires identity, networking, retry, integrity, and readiness logic

Design principle: Select the model-distribution method from release frequency, artifact size, startup latency, security, traceability, rollback, and platform requirements.

Docker Networking

Containers can communicate through Docker networks. Services on the same user-defined network can generally reference one another by service or container name.

docker network create ml-network

docker run \
  --detach \
  --name customer-churn-api \
  --network ml-network \
  --publish 8000:8000 \
  customer-churn-api:1.0.0

Network principle: Publish only the ports required by approved clients. Internal service ports do not always need exposure on the host.

Docker Compose

Docker Compose defines multiple related services, networks, volumes, environment variables, health checks, and dependencies in a YAML file.

services:
  ml-api:
    build:
      context: .
      dockerfile: Dockerfile
    image: customer-churn-api:1.0.0
    ports:
      - "8000:8000"
    environment:
      MODEL_PATH: /app/artifacts/customer_churn_pipeline.joblib
      METADATA_PATH: /app/artifacts/model_metadata.json
      MAXIMUM_BATCH_SIZE: "100"
    restart: unless-stopped
    healthcheck:
      test:
        - CMD
        - python
        - -c
        - import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=3)
      interval: 30s
      timeout: 5s
      start_period: 20s
      retries: 3

Run with Docker Compose

docker compose up \
  --build \
  --detach

docker compose ps

docker compose logs \
  --follow \
  ml-api

docker compose down

Compose use: Compose is valuable for local development, integration testing, and controlled multi-service environments. Production orchestration requirements may require a managed platform or Kubernetes.

Multi-Service ML Architecture

services:
  ml-api:
    build: .
    ports:
      - "8000:8000"
    environment:
      CACHE_URL: redis://cache:6379
    depends_on:
      cache:
        condition: service_healthy

  cache:
    image: redis:7-alpine
    healthcheck:
      test:
        - CMD
        - redis-cli
        - ping
      interval: 10s
      timeout: 3s
      retries: 5

Architecture caution: Add a database, queue, cache, or feature service only when the use case requires it. Every additional component adds security, availability, configuration, monitoring, and operational work.

Understand Docker Image Layers

Dockerfile instructions create image layers. Build performance and image reuse improve when stable layers appear before frequently changing layers.

Simplified Layer Order

Base Image → System Setup → Python Dependencies → Application Code → Model Artifact → Startup Configuration

Improve Build Caching

Copy dependency files before source code
Combine related package-installation operations
Remove temporary package lists and caches
Keep the build context small
Do not invalidate stable layers unnecessarily
Pin and review dependencies

Multi-Stage Builds

A multi-stage build can use one stage to prepare dependencies and a later stage to create a smaller runtime image.

FROM python:3.12-slim AS builder

WORKDIR /build

COPY requirements.txt .

RUN python -m pip wheel \
    --no-cache-dir \
    --wheel-dir /wheels \
    -r requirements.txt


FROM python:3.12-slim AS runtime

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

WORKDIR /app

RUN groupadd \
        --system \
        mlgroup \
    && useradd \
        --system \
        --gid mlgroup \
        mluser

COPY --from=builder /wheels /wheels

RUN python -m pip install \
    --no-cache-dir \
    /wheels/* \
    && rm -rf /wheels

COPY --chown=mluser:mlgroup app ./app
COPY --chown=mluser:mlgroup artifacts ./artifacts

USER mluser

EXPOSE 8000

CMD [
    "uvicorn",
    "app.main:app",
    "--host",
    "0.0.0.0",
    "--port",
    "8000"
]

Multi-stage benefit: Build-only files and tools do not need to remain in the final runtime stage.

CPU and GPU Images

Image Type	Use	Considerations
CPU Image	Traditional ML and models with acceptable CPU latency	Usually simpler and smaller
GPU Image	Models requiring accelerator execution	Requires compatible drivers, runtime, libraries, and hardware
Architecture-Specific Image	Targets a specific CPU architecture	Must match the deployment environment
Multi-Platform Image	Publishes supported variants under one image reference	Each architecture must be built and tested

Hardware rule: Do not add GPU dependencies without measured inference benefit and a compatible runtime environment.

Run as a Non-Root User

docker exec \
  customer-churn-api \
  id

Additional Runtime Hardening

Run as a non-root user
Use a read-only root filesystem where supported
Drop unnecessary Linux capabilities
Prevent privilege escalation
Mount only required paths
Use read-only mounts where possible
Restrict network access
Apply CPU and memory limits
Use approved runtime security profiles

docker run \
  --detach \
  --name customer-churn-api \
  --publish 8000:8000 \
  --read-only \
  --tmpfs /tmp \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  customer-churn-api:1.0.0

Compatibility testing: Security restrictions can expose hidden write or permission requirements. Test the complete application before applying them in production.

Resource Limits

docker run \
  --detach \
  --name customer-churn-api \
  --publish 8000:8000 \
  --cpus 1.0 \
  --memory 1g \
  --memory-reservation 768m \
  customer-churn-api:1.0.0

Resource	Review Question
CPU	How much computation does one prediction require?
Memory	How much memory is required for the model and application?
Startup	How long does model loading require?
Concurrency	How many simultaneous requests can one container process reliably?
Temporary Storage	Does the application generate temporary files?
Accelerator	Does the workload require specialized hardware?

Capacity principle: Select resource requests and limits from representative load tests, model size, startup time, concurrency, and failure behavior.

Monitor Container Resources

docker stats customer-churn-api

Monitor

CPU utilization
Memory utilization
Network input and output
Block input and output
Process count
Restart frequency
Application latency
Prediction throughput
Model-loading duration

Monitoring distinction: Container resource metrics cannot replace model-quality, input-quality, prediction-drift, and business-outcome monitoring.

Tag the Image

docker tag \
  customer-churn-api:1.0.0 \
  registry.example.com/ml/customer-churn-api:1.0.0

docker image inspect \
  customer-churn-api:1.0.0

Useful Tagging Dimensions

Application version
Model version
Source-commit identifier
Release channel
CPU or GPU runtime
Target architecture

Tag caution: Tags can be changed to reference another image. Use immutable repository policies and image digests when exact content identity is required.

Inspect the Image Digest

docker image inspect \
  --format='{{json .RepoDigests}}' \
  registry.example.com/ml/customer-churn-api:1.0.0

Traceability requirement: Production release records should identify the application version, model version, image tag, image digest, source revision, environment, and approval status.

Push to a Registry

docker login registry.example.com

docker push \
  registry.example.com/ml/customer-churn-api:1.0.0

Registry Controls

Authenticated access
Role-based push and pull permissions
Immutable release tags
Image vulnerability scanning
Retention policies
Audit logging
Approved replication regions
Image-signing and verification policies where required

Placeholder registry: Replace the example registry address with an approved registry and follow its authentication and naming requirements.

Pull and Run an Image

docker pull \
  registry.example.com/ml/customer-churn-api:1.0.0

docker run \
  --detach \
  --name customer-churn-api \
  --publish 8000:8000 \
  registry.example.com/ml/customer-churn-api:1.0.0

Supply-chain rule: Pull images only from trusted registries and verify the approved image identity before deployment.

Scan the Image

docker scout quickview \
  customer-churn-api:1.0.0

docker scout cves \
  customer-churn-api:1.0.0

Scan Area	Purpose
Base Image	Identifies operating-system and runtime vulnerabilities
Python Packages	Identifies known dependency vulnerabilities
Secrets	Detects credentials accidentally included in image layers
Malware	Checks for known harmful content where supported
Licensing	Reviews dependency licensing obligations
Configuration	Detects unsafe Dockerfile and runtime settings

Scanning limitation: A clean scan does not prove that an image is secure. Access, business logic, custom code, model artifacts, runtime configuration, networking, and operational practices also require review.

Do Not Copy Secrets into Images

Unsafe Dockerfile Use ENV API_KEY=secret-value or copy a local .env file into an image.

Safer Design Obtain credentials at runtime through approved workload identity or secret-management mechanisms.

Never Bake These into an Image

Passwords
API keys
Private keys
Database connection secrets
Cloud credentials
Personal access tokens
Restricted configuration files
Production customer data

Layer-history warning: Deleting a secret in a later Dockerfile instruction may not remove it from an earlier image layer.

Protect the Model Artifact

Risk	Control
Model Tampering	Verify checksums, signatures, registry source, and image digest
Unsafe Deserialization	Load Pickle-based artifacts only from trusted sources
Model Theft	Restrict image and registry access
Version Confusion	Record model and image versions together
Unauthorized Replacement	Use immutable releases and controlled promotion
Metadata Exposure	Exclude confidential training and organizational information

Test the Image Before Deployment

docker build \
  --tag customer-churn-api:test \
  .

docker run \
  --detach \
  --name customer-churn-api-test \
  --publish 8001:8000 \
  customer-churn-api:test

curl \
  --fail \
  http://127.0.0.1:8001/health

curl \
  --fail \
  http://127.0.0.1:8001/ready

docker stop customer-churn-api-test

docker rm customer-churn-api-test

Test requirement: Build the exact image intended for release and test that image. Do not assume that successful execution outside the container proves that the packaged application works.

Test Prediction Consistency

import joblib
import numpy as np
import pandas as pd
import requests


record = {
    "tenure_months": 18,
    "monthly_charges": 64.5,
    "total_charges": 1161.0,
    "contract_type": "Monthly",
    "payment_method": "Electronic",
    "tech_support": "No"
}


local_model = joblib.load(
    "artifacts/customer_churn_pipeline.joblib"
)


local_probability = float(
    local_model.predict_proba(
        pd.DataFrame(
            [
                record
            ]
        )
    )[0, 1]
)


api_response = requests.post(
    "http://127.0.0.1:8000/predictions",
    json=record,
    timeout=10
)


api_response.raise_for_status()


container_probability = (
    api_response.json()[
        "positive_class_probability"
    ]
)


np.testing.assert_allclose(
    local_probability,
    container_probability,
    rtol=1e-10,
    atol=1e-12
)


print(
    "Container prediction matches "
    "the approved local model."
)

Docker in CI/CD

Container Release Flow

Code Change → Unit Tests → Build Image → Scan Image → Run Container Tests → Push Immutable Image → Deploy → Smoke Test

Pipeline Gate	Required Evidence
Source Review	Approved code and configuration changes
Unit Tests	Validation, inference, schema, and error tests
Image Build	Successful reproducible Docker build
Dependency Scan	Reviewed package and base-image findings
Secret Scan	No confidential values present in the image or repository
Runtime Test	Container health, readiness, and prediction tests
Artifact Verification	Expected model version and checksum
Registry Push	Immutable tag and recorded image digest
Deployment Approval	Authorized release decision
Post-Deployment Test	Health, readiness, version, and smoke prediction

Example GitHub Actions Workflow

name: Build and Test ML Container

on:
  push:
    branches:
      - main
  pull_request:

jobs:
  test-container:
    runs-on: ubuntu-latest

    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install test dependencies
        run: |
          python -m pip install --upgrade pip
          python -m pip install -r requirements.txt
          python -m pip install pytest requests

      - name: Run unit tests
        run: |
          python -m pytest -v

      - name: Build image
        run: |
          docker build \
            --tag customer-churn-api:test \
            .

      - name: Start container
        run: |
          docker run \
            --detach \
            --name customer-churn-api-test \
            --publish 8000:8000 \
            customer-churn-api:test

      - name: Wait for health
        run: |
          for attempt in $(seq 1 30); do
            if curl --fail http://127.0.0.1:8000/health; then
              exit 0
            fi
            sleep 2
          done
          docker logs customer-churn-api-test
          exit 1

      - name: Test readiness
        run: |
          curl --fail http://127.0.0.1:8000/ready

      - name: Show logs on failure
        if: failure()
        run: |
          docker logs customer-churn-api-test

      - name: Remove test container
        if: always()
        run: |
          docker rm \
            --force \
            customer-churn-api-test

Workflow security: Review third-party actions, use minimum permissions, protect registry credentials, and do not expose secrets through logs.

Container Rollback

Rollback replaces a problematic application image with a previously approved image and model combination.

docker stop customer-churn-api

docker rm customer-churn-api

docker run \
  --detach \
  --name customer-churn-api \
  --publish 8000:8000 \
  registry.example.com/ml/customer-churn-api:0.9.0

Rollback Requirements

Previous approved image remains available
Previous model artifact remains available
Configuration remains compatible
Database or schema changes are reversible where applicable
Rollback authority is documented
Health and smoke tests are defined
Incident evidence is preserved
Traffic restoration is verified

Responsible Containerized ML

Responsible-Use Checklist

Package only approved models
Include the correct decision threshold
Return the model and application versions
Validate every production request
Protect personal and confidential data
Do not log full sensitive payloads
Restrict access to the image and endpoint
Document intended and prohibited uses
Provide human review for consequential decisions
Monitor relevant groups and operating conditions
Preserve fallback and rollback paths
Retire vulnerable or obsolete image versions

Incorrect Assumption Containerization makes a Machine Learning model production-ready.

Correct Understanding Containerization packages the application. Production readiness still requires model validation, security, monitoring, scaling, governance, operational ownership, fallback, and rollback.

Common Docker Mistakes

Mistake Copy the entire project directory into the image.

Correction Use a focused build context and a carefully reviewed .dockerignore file.

Mistake Include training datasets and notebooks in the inference image.

Correction Include only runtime code, dependencies, approved artifacts, and required metadata.

Mistake Run the container as root without justification.

Correction Create and use a non-root runtime account with minimum permissions.

Mistake Use only the latest tag.

Correction Use immutable release versions and record the image digest.

Mistake Add secrets through Dockerfile instructions.

Correction Obtain secrets during runtime from approved identity and secret systems.

Mistake Use the API development server in the image.

Correction Start a production-oriented WSGI or ASGI server.

Mistake Consider a successful image build proof that the API works.

Correction Run the image and test health, readiness, schema validation, model version, and prediction consistency.

Mistake Leave old learning containers and images running indefinitely.

Correction Stop and remove authorized temporary resources and apply registry retention policies.

Mistake Store important prediction results only inside the container.

Correction Use approved external storage or downstream systems for durable business data.

Troubleshooting Docker Builds

Problem	Possible Investigation
Dependency Installation Fails	Review package versions, platform compatibility, indexes, and build dependencies
Image Is Very Large	Review base image, unnecessary packages, caches, data, and build-only tools
Application Module Not Found	Review `WORKDIR`, copied paths, package structure, and startup command
Model File Not Found	Review artifact path, build context, exclusions, mounts, and environment variables
Permission Denied	Review file ownership, runtime user, mount permissions, and writable paths
Port Is Unreachable	Review application binding, container port, host-port publication, and firewall rules
Container Exits Immediately	Review the main-process command and container logs
Health Check Fails	Review endpoint path, startup delay, dependency readiness, and timeout
Predictions Differ	Review artifact version, dependencies, feature order, preprocessing, and threshold
Out of Memory	Measure model size, worker count, concurrency, temporary arrays, and memory limits

Useful Docker Commands

Task	Command
List Running Containers	`docker container ls`
List All Containers	`docker container ls --all`
List Images	`docker image ls`
View Logs	`docker logs container-name`
Inspect Container	`docker inspect container-name`
Stop Container	`docker stop container-name`
Start Container	`docker start container-name`
Restart Container	`docker restart container-name`
Remove Container	`docker rm container-name`
Remove Image	`docker image rm image-name`
View Resource Use	`docker stats`
Run Command in Container	`docker exec container-name command`
Build Compose Services	`docker compose build`
Start Compose Services	`docker compose up --detach`
Stop Compose Services	`docker compose down`

Recommended Docker Workflow

Prepare the ML Service

Confirm that the API loads the approved model, validates input, exposes health and readiness, and passes automated tests.

Define Runtime Dependencies

Include only the Python and system packages required for inference.

Create the Dockerfile

Select an approved base image, install dependencies, copy runtime assets, create a non-root user, and define the startup command.

Control the Build Context

Exclude secrets, datasets, caches, notebooks, local environments, and unrelated files.

Build and Tag the Image

Build a versioned image and record its source revision and model version.

Run and Test the Container

Validate health, readiness, schemas, predictions, logs, permissions, resources, and shutdown behavior.

Scan and Verify

Review base image, packages, secrets, configuration, model integrity, and licensing.

Push to an Approved Registry

Publish the immutable image using controlled identity and record the resulting digest.

Deploy Gradually

Introduce the image through development, testing, staging, and controlled production release.

Monitor and Maintain

Track runtime health, security findings, resource use, model behavior, versions, cost, and rollback readiness.

Hands-On Practice

Create the Build Context

Copy only the API code, model artifact, metadata, dependency manifest, and required configuration into a focused project.

Create .dockerignore

Exclude secrets, datasets, virtual environments, caches, notebooks, logs, and unrelated files.

Build a Non-Root Image

Create a Dockerfile that runs the Machine Learning API using a dedicated runtime user.

Run and Inspect the Container

Review ports, environment variables, model paths, runtime user, image version, and container health.

Test Prediction Consistency

Compare output from the local approved model with output from the containerized API.

Add Docker Compose

Define the API service, ports, environment, health check, and restart behavior in compose.yaml.

Scan the Image

Review vulnerabilities, dependencies, secrets, licensing, and Dockerfile configuration.

Create a CI Container Test

Build the image, start the container, wait for health, test readiness, send a prediction, and remove the test container.

Knowledge Check

What is a Docker image?

Explain how it packages the application, runtime, dependencies, configuration, and files required to create containers.

What is a Docker container?

Explain how it is a running or stopped instance created from an image.

What is the purpose of a Dockerfile?

Explain how it defines the reproducible steps used to build an image.

Why is .dockerignore important?

Explain build-context size, secret protection, build speed, and prevention of unnecessary files entering image layers.

Why copy requirements.txt before source code?

Explain how stable dependency layers may be reused when the application source changes.

Why should the container run as a non-root user?

Explain least privilege and reduction of runtime permissions.

What is the difference between EXPOSE and port publishing?

Explain image-port documentation versus mapping a host port to a container port.

Why should production images use immutable versions?

Explain traceability, reproducibility, investigation, promotion, and rollback.

Why test predictions after containerization?

Explain dependency changes, file paths, feature order, preprocessing behavior, and runtime differences.

Does Docker make an ML system production-ready?

Explain that container packaging still requires security, model validation, monitoring, scaling, governance, fallback, and rollback.

Docker Readiness Checklist

Confirm Before Release

The model passed independent evaluation
The complete fitted pipeline is available
The model version and threshold are documented
The base image comes from an approved source
The base-image version is controlled
The serving dependencies are declared
Unnecessary training dependencies are excluded
The Docker build context is minimized
Secrets and restricted data are excluded
The application runs as a non-root user
The production server starts directly
Debug and automatic reload are disabled
Health and readiness endpoints are available
The model loads before readiness succeeds
The image builds successfully
The container starts successfully
Health and readiness tests pass
Prediction consistency is verified
Invalid input is rejected safely
Logs exclude sensitive payloads
Resource limits are tested
The image is scanned
Important findings are resolved or accepted formally
The image has an immutable version tag
The image digest is recorded
The registry uses controlled access
Runtime secrets use approved mechanisms
Monitoring and alert ownership are defined
The previous approved image remains available
The rollback procedure is tested

Lesson Summary

Docker Area	Main Outcome
Dockerfile	Defines how the Machine Learning image is built
Image	Packages the API, dependencies, runtime, and model assets
Container	Runs an isolated instance of the packaged ML service
.dockerignore	Controls which files enter the build context
Image Layers	Support caching and structured image construction
Non-Root User	Reduces runtime operating-system permissions
Port Publishing	Makes the containerized API reachable through a host port
Environment Variables	Provide environment-specific runtime configuration
Volumes and Mounts	Provide external or persistent files when required
Docker Compose	Defines related services and configuration declaratively
Health Check	Reports whether the containerized application responds
Registry	Stores and distributes versioned container images
Image Digest	Identifies exact image content
Security Scan	Identifies known package, base-image, and configuration risks
CI/CD	Automates building, testing, scanning, and publishing
Rollback	Restores a previous approved image and model combination

Key Takeaway

Docker for Machine Learning packages a prediction API, saved model, preprocessing, dependencies, runtime, and startup command into a versioned container image. A professional ML image uses a focused build context, approved base image, declared dependencies, non-root runtime user, production server, health checks, immutable tags, registry controls, vulnerability scanning, resource limits, and automated container tests. The containerized service must still validate input, protect secrets, preserve model and threshold versions, monitor service and model behavior, and maintain fallback and rollback procedures. Docker improves portability and reproducibility, but production reliability depends on the complete deployment and governance process.

Get Machine Learning updates by email

Short, practical lessons and course updates from Rumman Ansari. No spam.