Table of Contents

Variance and Standard Deviation

Rumman Ansari June 26, 2026 24 views Subject Details

Statistics Fundamentals

Variance and Standard Deviation

Understand how data spreads around the mean and why variance and standard deviation are essential for statistics, data analysis, and Machine Learning.

Introduction

In statistics, only knowing the average of a dataset is not enough. Two datasets may have the same mean, but their data values may be spread very differently. This is why we need Variance and Standard Deviation.

Variance and Standard Deviation are measures of dispersion. Dispersion means how much the data values are spread out from the center, usually from the mean. These concepts help us understand whether the data is consistent, stable, scattered, or highly variable.

Simple Meaning: Mean tells us the center of data, but variance and standard deviation tell us how far the values are from that center.

Prerequisites

Before learning variance and standard deviation, students should understand the following topics:

Required Knowledge

Mean: Average value of a dataset.
Data Values: Individual observations present in a dataset.
Subtraction: Needed to calculate deviation from the mean.
Squaring Numbers: Needed because variance uses squared deviations.
Square Root: Needed because standard deviation is the square root of variance.
Basic Statistics: Dataset, observation, sample, and population.
Python Basics: Helpful for calculating variance and standard deviation programmatically.

Why Do We Need Variance and Standard Deviation?

Suppose two students have the same average marks, but one student scores almost the same marks in every subject, while the other student scores very high in some subjects and very low in others. The average alone cannot explain this difference.

Variance and standard deviation solve this problem by showing how spread out the values are. They help us understand consistency, reliability, and risk.

Dataset A

Marks: 48, 49, 50, 51, 52
Mean is around 50
Values are close to the mean
Low spread
More consistent performance

Dataset B

Marks: 10, 30, 50, 70, 90
Mean is also 50
Values are far from the mean
High spread
Less consistent performance

Both datasets may have the same mean, but their spread is different. Variance and standard deviation help us measure that spread.

What is Dispersion?

Dispersion means the spread or scattering of data values. If all values are close to the mean, dispersion is low. If values are far from the mean, dispersion is high.

Dispersion

Dispersion = How much data values are spread around the central value

Common measures of dispersion include:

Common Measures of Dispersion

Range: Difference between maximum and minimum value.
Variance: Average squared deviation from the mean.
Standard Deviation: Square root of variance.
Interquartile Range: Spread of the middle 50% of data.

What is Variance?

Variance measures how far each value in a dataset is from the mean. It is calculated by finding the difference between each value and the mean, squaring those differences, and then taking their average.

Definition

Variance is the average of squared differences from the mean.

Variance tells us how scattered the data is. A small variance means values are close to the mean. A large variance means values are spread far away from the mean.

Why Do We Square the Differences?

When we subtract the mean from each value, some deviations are positive and some are negative. If we add them directly, positive and negative values can cancel each other. To avoid this problem, we square each deviation.

Without Squaring

Positive and negative deviations may cancel out
Total deviation may become zero
Spread cannot be measured properly

With Squaring

All deviations become positive
Larger deviations get more weight
Spread becomes easier to measure mathematically

Population Variance

Population variance is used when we have data for the entire population. For example, if we know the marks of every student in a school, then we can calculate population variance.

POPULATION VARIANCE FORMULA

\( \sigma^2 = \frac{\sum_{i=1}^{N}(x_i - \mu)^2}{N} \)

Here:
\( \sigma^2 \) = Population variance
\( x_i \) = Each data value
\( \mu \) = Population mean
\( N \) = Total number of values in the population

Sample Variance

Sample variance is used when we have only a sample from a larger population. In real-world Machine Learning and data analysis, we usually work with samples instead of the entire population.

SAMPLE VARIANCE FORMULA

\( s^2 = \frac{\sum_{i=1}^{n}(x_i - \bar{x})^2}{n - 1} \)

Here:
\( s^2 \) = Sample variance
\( x_i \) = Each sample value
\( \bar{x} \) = Sample mean
\( n \) = Number of values in the sample

Important: Population variance divides by \( N \), while sample variance divides by \( n - 1 \). The \( n - 1 \) adjustment is used to better estimate population variability from sample data.

What is Standard Deviation?

Standard Deviation is the square root of variance. It measures how much the data values usually differ from the mean. Unlike variance, standard deviation is easier to interpret because it is expressed in the same unit as the original data.

Definition

Standard Deviation is the square root of variance.

POPULATION STANDARD DEVIATION

\( \sigma = \sqrt{\frac{\sum_{i=1}^{N}(x_i - \mu)^2}{N}} \)

SAMPLE STANDARD DEVIATION

\( s = \sqrt{\frac{\sum_{i=1}^{n}(x_i - \bar{x})^2}{n - 1}} \)

If standard deviation is small, values are close to the mean. If standard deviation is large, values are far from the mean.

Variance vs Standard Deviation

Variance and standard deviation are closely related, but they are not exactly the same. Variance is measured in squared units, while standard deviation is measured in the original unit of the data.

Basis	Variance	Standard Deviation
Meaning	Average squared deviation from mean	Square root of variance
Symbol	\( \sigma^2 \) or \( s^2 \)	\( \sigma \) or \( s \)
Unit	Squared unit	Same unit as original data
Interpretation	More mathematical	More practical and easier to understand
Use	Used in formulas and advanced statistics	Used for explaining spread in real-world terms

Steps to Calculate Variance and Standard Deviation

To calculate variance and standard deviation manually, follow these steps:

Calculation Steps

Find the mean of the dataset.
Subtract the mean from each data value.
Square each deviation.
Add all squared deviations.
Divide by \( N \) for population variance or \( n - 1 \) for sample variance.
Take the square root of variance to get standard deviation.

Complete Hand Calculation Example

Let us calculate population variance and standard deviation for the dataset:


2, 4, 6, 8

Step 1: Find the Mean

MEAN

\( \mu = \frac{2 + 4 + 6 + 8}{4} = \frac{20}{4} = 5 \)

Step 2: Find Deviations from Mean

Value \( x_i \)	Deviation \( x_i - \mu \)	Squared Deviation \( (x_i - \mu)^2 \)
2	2 - 5 = -3	9
4	4 - 5 = -1	1
6	6 - 5 = 1	1
8	8 - 5 = 3	9

Step 3: Add Squared Deviations

SUM OF SQUARED DEVIATIONS

\( 9 + 1 + 1 + 9 = 20 \)

Step 4: Calculate Variance

POPULATION VARIANCE

\( \sigma^2 = \frac{20}{4} = 5 \)

Step 5: Calculate Standard Deviation

STANDARD DEVIATION

\( \sigma = \sqrt{5} \approx 2.236 \)

So, for the dataset 2, 4, 6, 8:
Variance = 5
Standard Deviation ≈ 2.236

How to Interpret Standard Deviation?

Standard deviation helps us understand how much values usually vary from the mean. It gives practical meaning to data spread.

Low Standard Deviation

Values are close to the mean
Data is more consistent
Less variation
Lower risk or uncertainty

High Standard Deviation

Values are far from the mean
Data is more scattered
More variation
Higher risk or uncertainty

Same Mean, Different Standard Deviation

Two datasets can have the same mean but different spread. This is one of the most important reasons why standard deviation is useful.

Dataset	Values	Mean	Spread
A	48, 49, 50, 51, 52	50	Low spread
B	10, 30, 50, 70, 90	50	High spread

Both datasets have the same mean, but Dataset B is more spread out. This means Dataset B will have a higher variance and standard deviation.

Importance in Machine Learning

Variance and standard deviation are very important in Machine Learning because they help us understand the spread, quality, and behavior of data before training a model.

Uses in Machine Learning

Exploratory Data Analysis: Helps understand how features are distributed.
Outlier Detection: Values far from the mean can indicate possible outliers.
Feature Scaling: Standardization uses mean and standard deviation.
Model Stability: Features with very different scales may affect some algorithms.
Risk Understanding: High standard deviation means more uncertainty in data.
Feature Selection: Very low variance features may contain less useful information.
Model Evaluation: Error spread can help understand prediction consistency.

Connection with Feature Scaling

In Machine Learning, some algorithms work better when features are on a similar scale. Standardization is a common scaling method that uses mean and standard deviation.

STANDARDIZATION FORMULA

\( z = \frac{x - \mu}{\sigma} \)

Here:
\( z \) = Standardized value
\( x \) = Original value
\( \mu \) = Mean
\( \sigma \) = Standard deviation

Machine Learning Tip: Standardization transforms data so that the mean becomes 0 and the standard deviation becomes 1.

Low Variance and High Variance Features

In Machine Learning, each column in a dataset is called a feature. The variance of a feature tells us how much that feature changes across observations.

Low Variance Feature

Values are almost the same
May not provide much information
Can sometimes be removed during feature selection
Example: A column where almost all values are 1

High Variance Feature

Values change significantly
May contain useful patterns
Can also contain noise if variation is too extreme
Needs proper analysis before using in a model

Python Example

The following Python example calculates variance and standard deviation using Pandas.


import pandas as pd

# Sample dataset
marks = [2, 4, 6, 8]

# Convert list into pandas Series
data = pd.Series(marks)

# Population variance and standard deviation
population_variance = data.var(ddof=0)
population_std = data.std(ddof=0)

# Sample variance and standard deviation
sample_variance = data.var(ddof=1)
sample_std = data.std(ddof=1)

print("Population Variance:", population_variance)
print("Population Standard Deviation:", population_std)

print("Sample Variance:", sample_variance)
print("Sample Standard Deviation:", sample_std)

Output


Population Variance: 5.0
Population Standard Deviation: 2.23606797749979

Sample Variance: 6.666666666666667
Sample Standard Deviation: 2.581988897471611

In Pandas:
ddof=0 is used for population calculation.
ddof=1 is used for sample calculation.

NumPy Example

NumPy is commonly used for numerical calculations in data science and Machine Learning.


import numpy as np

data = np.array([2, 4, 6, 8])

variance = np.var(data)
standard_deviation = np.std(data)

print("Variance:", variance)
print("Standard Deviation:", standard_deviation)

Output


Variance: 5.0
Standard Deviation: 2.23606797749979

Real-World Examples

Student Marks

Low standard deviation means students scored similarly
High standard deviation means marks vary widely
Useful for understanding class consistency

Salary Data

Low spread means salaries are similar
High spread means large salary differences
Useful in HR and compensation analysis

Stock Market

High standard deviation means high volatility
Low standard deviation means more stable price movement
Useful for risk analysis

Quality Control

Low variation means consistent production quality
High variation may indicate process problems
Useful in manufacturing and testing

Practical ML Example: House Price Data

Suppose we are building a Machine Learning model to predict house prices. Standard deviation can help us understand how much house prices vary from the average price.

Questions Variance and Standard Deviation Can Answer

Are house prices tightly grouped or widely spread?
Are there extreme values or luxury houses acting as outliers?
Does one feature have much larger scale than another feature?
Should feature scaling be applied before model training?
Are model errors consistent or highly variable?

Advantages

Benefits of Variance and Standard Deviation

They help measure the spread of data.
They help compare consistency between datasets.
They are useful for outlier detection.
They support feature scaling and standardization.
They help understand risk and uncertainty.
They are used in statistics, Machine Learning, finance, research, and quality control.

Limitations

Important Limitations

Variance is difficult to interpret because it uses squared units.
Standard deviation can be affected by extreme outliers.
They work best for numerical data, not categorical data.
They do not show the complete shape of data distribution.
Two datasets can have similar standard deviation but different distribution shapes.

Common Mistakes Students Make

Avoid These Mistakes

Confusing variance with standard deviation.
Forgetting that variance is in squared units.
Using population formula when sample formula is required.
Not understanding why \( n - 1 \) is used in sample variance.
Ignoring outliers before interpreting standard deviation.
Thinking mean alone is enough to describe a dataset.
Using standard deviation for categorical data without proper encoding or analysis.

Quick Revision

Concept	Meaning	Formula / Rule
Variance	Average squared deviation from mean	\( \sigma^2 = \frac{\sum (x_i - \mu)^2}{N} \)
Sample Variance	Variance calculated from sample data	\( s^2 = \frac{\sum (x_i - \bar{x})^2}{n - 1} \)
Standard Deviation	Square root of variance	\( \sigma = \sqrt{\sigma^2} \)
Low Standard Deviation	Values are close to the mean	Data is consistent
High Standard Deviation	Values are far from the mean	Data is more spread out
ML Use	Used in scaling, EDA, outlier detection, and feature analysis	Helps understand data variability

Interview Questions

What is variance?

Variance is the average of squared differences from the mean. It tells us how much the values in a dataset are spread out.

What is standard deviation?

Standard deviation is the square root of variance. It tells us how much values typically differ from the mean.

Why do we square deviations in variance?

We square deviations so that negative and positive deviations do not cancel each other. Squaring also gives more weight to larger deviations.

What is the difference between variance and standard deviation?

Variance is measured in squared units, while standard deviation is measured in the same unit as the original data. Standard deviation is easier to interpret in real-world terms.

Why is standard deviation important in Machine Learning?

Standard deviation helps understand data spread, detect outliers, scale features, compare variability, and analyze model error consistency.

What does high standard deviation mean?

High standard deviation means data values are widely spread from the mean. It indicates more variation or uncertainty.

What does low standard deviation mean?

Low standard deviation means data values are close to the mean. It indicates more consistency and less variation.

Conclusion

Variance and Standard Deviation are important measures of dispersion. They help us understand how much data values vary from the mean. Variance calculates the average squared difference from the mean, while standard deviation gives the square root of variance and represents spread in the original unit of the data.

In Machine Learning, these concepts are useful for exploratory data analysis, feature scaling, outlier detection, model evaluation, and understanding feature variability. A strong understanding of variance and standard deviation helps students analyze data more deeply and build better Machine Learning models.

Final Takeaway

Mean tells us the center of data, but variance and standard deviation tell us how much the data spreads around that center. Without understanding spread, we cannot fully understand the behavior of a dataset.