1. The Scientific Computing Workflow

Scientific programming is not only about producing a final numerical answer. A good workflow should help you understand what happened, detect invalid assumptions, reproduce an experiment and determine why two runs produced different results.

A practical workflow can be viewed as six complementary layers:

2. Prerequisites

python -m pip install numpy pandas matplotlib pytest jupyter

For the optional PyTorch and TensorBoard examples:

python -m pip install torch tensorboard

3. Plots: Turn Numerical Behavior into Visual Evidence

Plots are diagnostic instruments. They help expose trends, outliers, nonlinear relationships, class imbalance, training instability and suspicious distributions that may be difficult to recognize from printed numbers alone.

3.1 A Simple Diagnostic Plot

import matplotlib.pyplot as plt

iterations = [1, 2, 3, 4, 5, 6]
loss = [0.92, 0.71, 0.56, 0.47, 0.41, 0.38]

plt.plot(iterations, loss, marker="o")
plt.xlabel("Iteration")
plt.ylabel("Loss")
plt.title("Training Loss")
plt.grid(True)
plt.show()

The code does more than produce decoration. It allows you to inspect whether loss decreases, plateaus, oscillates or suddenly becomes unstable.

3.2 Compare Training and Validation Curves

import matplotlib.pyplot as plt

epochs = [1, 2, 3, 4, 5, 6]
train_loss = [0.90, 0.69, 0.52, 0.40, 0.31, 0.24]
validation_loss = [0.94, 0.73, 0.60, 0.56, 0.58, 0.65]

plt.plot(epochs, train_loss, marker="o", label="Training")
plt.plot(epochs, validation_loss, marker="o", label="Validation")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.title("Training vs Validation Loss")
plt.legend()
plt.grid(True)
plt.show()

3.3 Distribution Plots for Data Debugging

import numpy as np
import matplotlib.pyplot as plt

rng = np.random.default_rng(42)
measurements = rng.normal(loc=50, scale=10, size=1000)

plt.hist(measurements, bins=30)
plt.xlabel("Measurement")
plt.ylabel("Frequency")
plt.title("Measurement Distribution")
plt.show()

A histogram can reveal skew, unexpected ranges, multiple populations or suspicious concentrations. During data preparation, always compare the plot with your domain expectations.

4. Logging: Make Runtime Behavior Observable

Temporary print() statements are useful during exploration, but a growing program needs structured runtime information. Python's logging module supports severity levels, timestamps, formatting and output destinations.

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s | %(levelname)s | %(name)s | %(message)s"
)

logger = logging.getLogger(__name__)

logger.info("Experiment started")

batch_size = 32
learning_rate = 0.001
logger.info(
    "Configuration: batch_size=%d learning_rate=%s",
    batch_size,
    learning_rate
)

logger.warning("Validation dataset is small")

4.1 Prefer Structured Messages

A useful log message contains enough context to reconstruct the failure without flooding the output with every variable in memory.

5. Assertions: Turn Assumptions into Executable Checks

Scientific code contains assumptions about shapes, ranges, missing values and relationships between objects. Assertions make those assumptions explicit and fail close to the point where they are violated.

import numpy as np

X = np.ones((100, 8))
y = np.zeros(100)

assert X.ndim == 2
assert y.ndim == 1
assert X.shape[0] == y.shape[0]
assert X.shape[1] == 8
assert np.isfinite(X).all()

5.1 Assertions for Numerical Preconditions

def standardize(x):
    x = np.asarray(x, dtype=float)

    assert x.ndim == 1, "Expected a one-dimensional vector"
    assert np.isfinite(x).all(), "Input contains NaN or infinity"

    std = x.std()
    assert std > 0, "Cannot standardize a constant vector"

    return (x - x.mean()) / std

Python can disable assert statements when optimization is enabled. For input validation that must always execute, raise explicit exceptions.

def require_positive_batch_size(batch_size: int) -> None:
    if batch_size <= 0:
        raise ValueError("batch_size must be greater than zero")

6. Python Classes: Organize State and Behavior

A class is useful when several values and operations belong to one conceptual object. Experiments commonly need configuration, accumulated metrics, model parameters or preprocessing state.

6.1 A Simple Experiment Class

class ExperimentTracker:
    """Keep experiment identity and metric history together."""

    def __init__(self, name: str):
        self.name = name
        self.metrics = {}

    def log_metric(self, metric_name: str, value: float) -> None:
        self.metrics.setdefault(metric_name, []).append(float(value))

    def latest(self, metric_name: str):
        values = self.metrics.get(metric_name, [])
        return values[-1] if values else None


experiment = ExperimentTracker("baseline-linear")
experiment.log_metric("loss", 0.84)
experiment.log_metric("loss", 0.62)

print(experiment.latest("loss"))  # 0.62

6.2 When a Class Is Better Than Loose Variables

Do not introduce classes merely because a project is scientific. Small stateless transformations are often clearer as functions. Use a class when persistent state and closely related behavior genuinely belong together.

7. PyTorch Models Are Classes

In PyTorch, models commonly subclass torch.nn.Module. Parameters and layers are defined in the class, while forward() describes the forward computation.

import torch
from torch import nn


class SmallRegressor(nn.Module):
    def __init__(self, input_features: int):
        super().__init__()
        self.network = nn.Sequential(
            nn.Linear(input_features, 16),
            nn.ReLU(),
            nn.Linear(16, 1)
        )

    def forward(self, x):
        return self.network(x)


model = SmallRegressor(input_features=8)
example = torch.randn(32, 8)
output = model(example)

print(example.shape)  # torch.Size([32, 8])
print(output.shape)   # torch.Size([32, 1])

8. Tests: Verify Behavior Automatically

Tests turn expected behavior into executable examples. They are especially valuable for preprocessing, numerical utilities and model components because a small refactor can silently change shapes or values.

8.1 Unit Test Example with pytest

Suppose math_utils.py contains:

import numpy as np


def normalize_rows(X):
    X = np.asarray(X, dtype=float)

    if X.ndim != 2:
        raise ValueError("X must be two-dimensional")

    norms = np.linalg.norm(X, axis=1, keepdims=True)

    if np.any(norms == 0):
        raise ValueError("Zero-length row cannot be normalized")

    return X / norms

Create test_math_utils.py:

import numpy as np
import pytest

from math_utils import normalize_rows


def test_normalize_rows_produces_unit_norms():
    X = np.array([
        [3.0, 4.0],
        [5.0, 12.0]
    ])

    result = normalize_rows(X)
    norms = np.linalg.norm(result, axis=1)

    np.testing.assert_allclose(norms, [1.0, 1.0])


def test_normalize_rows_preserves_shape():
    X = np.ones((4, 3))
    result = normalize_rows(X)

    assert result.shape == X.shape


def test_normalize_rows_rejects_zero_vector():
    X = np.array([
        [0.0, 0.0],
        [3.0, 4.0]
    ])

    with pytest.raises(ValueError, match="Zero-length"):
        normalize_rows(X)

pytest -q

8.2 What Should Scientific Tests Check?

9. Floating-Point Tests Need Tolerances

Floating-point calculations should normally be compared with a tolerance rather than exact decimal equality.

import numpy as np

actual = np.array([0.30000000000000004])
expected = np.array([0.3])

np.testing.assert_allclose(
    actual,
    expected,
    rtol=1e-7,
    atol=1e-9
)

Select tolerances based on the numerical method and required precision. Extremely loose tolerances can hide real defects, while unnecessarily strict tolerances can create fragile tests.

10. Experiment Notebooks: A Scientific Lab Notebook

Jupyter notebooks are valuable for exploration because code, outputs, plots and explanations can live together. A useful experiment notebook should tell the story of the experiment rather than become an unstructured collection of cells.

10.1 Record Reproducibility Information

import platform
import numpy as np
import pandas as pd

print("Python:", platform.python_version())
print("NumPy:", np.__version__)
print("pandas:", pd.__version__)

SEED = 42
rng = np.random.default_rng(SEED)
print("Seed:", SEED)

11. Keep Reusable Logic Outside the Notebook

Notebooks are excellent for investigation and communication, but reusable functions, classes and tests are usually easier to maintain in normal Python modules.

12. TensorBoard for Experiment Tracking

For iterative machine-learning experiments, TensorBoard can record metrics such as loss and accuracy over training steps. With PyTorch, SummaryWriter writes event data that TensorBoard can visualize.

from torch.utils.tensorboard import SummaryWriter

writer = SummaryWriter("runs/baseline")

for epoch, loss in enumerate([0.92, 0.70, 0.55, 0.46]):
    writer.add_scalar("Loss/train", loss, epoch)

writer.flush()
writer.close()

tensorboard --logdir=runs

A hierarchical naming convention helps group related metrics:

writer.add_scalar("Loss/train", train_loss, epoch)
writer.add_scalar("Loss/validation", validation_loss, epoch)
writer.add_scalar("Accuracy/train", train_accuracy, epoch)
writer.add_scalar("Accuracy/validation", validation_accuracy, epoch)

13. Logging and Assertions Work Together

Assertions detect violations. Logs give context. Used together, they make a failure easier to diagnose.

import logging
import numpy as np

logger = logging.getLogger(__name__)


def train_step(X, y):
    logger.debug("train_step X=%s y=%s", X.shape, y.shape)

    assert X.ndim == 2, "X must be a matrix"
    assert y.ndim == 1, "y must be a vector"
    assert X.shape[0] == y.shape[0], "Row counts must match"
    assert np.isfinite(X).all(), "X contains non-finite values"

    logger.info("Validated training batch with %d rows", X.shape[0])
    # Continue training...

14. Tests and Assertions Are Not the Same

15. A Small Integrated Scientific Example

The following example combines a class, assertions, logging and plotting. Tests can then verify the reusable class independently of the notebook or plotting code.

import logging
import numpy as np
import matplotlib.pyplot as plt

logger = logging.getLogger(__name__)


class LinearExperiment:
    def __init__(self, learning_rate=0.05, epochs=20):
        if learning_rate <= 0:
            raise ValueError("learning_rate must be positive")
        if epochs <= 0:
            raise ValueError("epochs must be positive")

        self.learning_rate = learning_rate
        self.epochs = epochs
        self.weight = 0.0
        self.loss_history = []

    def fit(self, x, y):
        x = np.asarray(x, dtype=float)
        y = np.asarray(y, dtype=float)

        assert x.ndim == 1
        assert y.ndim == 1
        assert x.shape == y.shape
        assert np.isfinite(x).all()
        assert np.isfinite(y).all()

        logger.info("Training with %d observations", x.size)

        for epoch in range(self.epochs):
            predictions = self.weight * x
            errors = predictions - y
            loss = np.mean(errors ** 2)
            gradient = 2 * np.mean(errors * x)

            self.weight -= self.learning_rate * gradient
            self.loss_history.append(float(loss))

            logger.debug(
                "epoch=%d loss=%.6f weight=%.6f",
                epoch,
                loss,
                self.weight
            )

        return self


x = np.array([0.0, 1.0, 2.0, 3.0])
y = np.array([0.0, 2.0, 4.0, 6.0])

experiment = LinearExperiment(learning_rate=0.05, epochs=30)
experiment.fit(x, y)

plt.plot(experiment.loss_history)
plt.xlabel("Epoch")
plt.ylabel("Mean Squared Error")
plt.title("Experiment Loss")
plt.show()

16. The Loss Equation in the Example

The example minimizes mean squared error:

The derivative with respect to the weight is:

A gradient-descent update is:

Here, \(\eta\) is the learning rate. The plot and logs provide two different views of how this iterative process behaves.

17. Test the Integrated Class

import numpy as np
import pytest


def test_linear_experiment_records_every_epoch():
    experiment = LinearExperiment(learning_rate=0.05, epochs=10)
    experiment.fit([0, 1, 2], [0, 2, 4])

    assert len(experiment.loss_history) == 10


def test_linear_experiment_loss_decreases_on_simple_data():
    experiment = LinearExperiment(learning_rate=0.05, epochs=30)
    experiment.fit([0, 1, 2, 3], [0, 2, 4, 6])

    assert experiment.loss_history[-1] < experiment.loss_history[0]


def test_linear_experiment_rejects_negative_learning_rate():
    with pytest.raises(ValueError, match="learning_rate"):
        LinearExperiment(learning_rate=-0.1)

18. A Practical Debugging Sequence

19. Common Mistakes

20. Recommended Experiment Record

For every meaningful experiment, record enough information to distinguish one run from another.

21. Concept Check

22. Final Mental Model