Table of Contents

    K-Means Clustering

    Rumman Ansari June 26, 2026 15 views Subject Details
    MACHINE LEARNING

    K-Means Clustering

    The most popular unsupervised algorithm to group similar data points into K clusters.

    What is K-Means Clustering?

    K-Means is an unsupervised Machine Learning algorithm used to divide a dataset into K distinct, non-overlapping clusters based on similarity.

    In simple words — K-Means finds K groups in your data so that points within each group are as similar as possible.

    Why is it Called "K-Means"?

    • K = the number of clusters you want.
    • Means = the average (centroid) of each cluster.
    The algorithm tries to minimize the distance between each data point and the mean of its cluster.

    How Does K-Means Work?

    Step-by-Step Algorithm

    • Choose the number of clusters K.
    • Randomly initialize K centroids.
    • Assign each data point to its nearest centroid.
    • Calculate the mean of each cluster.
    • Update centroids using these new means.
    • Repeat steps 3–5 until centroids stop changing.
    • Final clusters are formed.

    Mathematical Formula

    K-Means tries to minimize the within-cluster sum of squares (WCSS):

    OBJECTIVE FUNCTION
    $$ J = \sum_{i=1}^{K} \sum_{x \in C_i} \| x - \mu_i \|^2 $$

    Where:

    • K = number of clusters
    • Cᵢ = cluster i
    • x = data point
    • μᵢ = centroid of cluster i

    Distance Used in K-Means

    K-Means uses Euclidean Distance to measure similarity between data points:

    EUCLIDEAN DISTANCE
    $$ d(p, q) = \sqrt{\sum_{i=1}^{n} (q_i - p_i)^2} $$

    How to Choose the Right K?

    Choosing the correct K is one of the most important steps. Common methods are:

    1

    Elbow Method

    Plot WCSS vs K, choose K at the “elbow point” where the curve bends.

    2

    Silhouette Score

    Measures how well points fit in their cluster vs other clusters (range: -1 to 1).

    3

    Davies-Bouldin Index

    Lower value indicates better clustering quality.

    Elbow Method — Python Example

    Prerequisites: Python 3.x, numpy, matplotlib, scikit-learn. 
    pip install numpy matplotlib scikit-learn
    import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs

# Generate sample data
X, _ = make_blobs(n_samples=300, centers=4, cluster_std=0.6, random_state=42)

# Compute WCSS for different K
wcss = []
for k in range(1, 11):
    km = KMeans(n_clusters=k, random_state=42, n_init=10)
    km.fit(X)
    wcss.append(km.inertia_)

# Plot Elbow Curve
plt.plot(range(1, 11), wcss, marker="o")
plt.title("Elbow Method")
plt.xlabel("Number of Clusters (K)")
plt.ylabel("WCSS")
plt.show()
    Output The point where the curve bends like an elbow indicates the optimal K.

    Full Python Example — K-Means in Action

    import numpy as np
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs

# Generate dataset
X, _ = make_blobs(n_samples=300, centers=4, cluster_std=0.7, random_state=42)

# Apply K-Means with K=4
kmeans = KMeans(n_clusters=4, random_state=42, n_init=10)
kmeans.fit(X)

labels = kmeans.labels_
centroids = kmeans.cluster_centers_

# Visualize
plt.scatter(X[:, 0], X[:, 1], c=labels, cmap="viridis", s=50)
plt.scatter(centroids[:, 0], centroids[:, 1], c="red", marker="X", s=200, label="Centroids")
plt.title("K-Means Clustering")
plt.xlabel("Feature 1")
plt.ylabel("Feature 2")
plt.legend()
plt.show()

print("Centroids:\n", centroids)
    Output K-Means successfully clusters the data into 4 distinct groups with their centroids.

    Before vs After K-Means

    Before K-Means After K-Means
    Unlabeled scattered points Organized clusters
    No pattern visible Clear grouping
    Hard to analyze Easy to interpret

    Real-Life Analogy

    K-Means = Party Group Photo

    At a party, people automatically gather in small groups based on shared interests (sports, music, food). K-Means works similarly — it groups data points into clusters based on similarity.

    Key Properties of K-Means

    • Requires specifying K in advance.
    • Centroids represent the cluster center.
    • Works best with numerical data.
    • Sensitive to initialization and outliers.
    • Produces spherical, similar-sized clusters.

    Initialization Methods

    1

    Random Initialization

    Centroids are placed randomly. Can lead to poor clustering if unlucky.

    2

    K-Means++ (Recommended)

    Smartly places centroids far from each other — gives stable results.

    KMeans(n_clusters=4, init="k-means++", random_state=42)

    Real-World Applications of K-Means

    Customer Segmentation

    • Group customers by buying patterns
    • Targeted marketing

    Image Compression

    • Group similar pixel colors
    • Reduce image size

    Document Clustering

    • Topic grouping
    • News article analysis

    Healthcare

    • Group patients by symptoms
    • Personalized treatment plans

    Fraud Detection

    • Find unusual clusters
    • Suspicious transactions

    Recommendation Systems

    • Group similar users
    • Movie/song suggestions

    Telecom

    • Group customers by data usage
    • Optimize networks

    Education

    • Group students by performance
    • Customized teaching

    Advantages of K-Means

    • Simple, fast, and easy to implement.
    • Works well on large datasets.
    • Scales well with high dimensions.
    • Produces clear, interpretable clusters.
    • Widely supported in ML libraries.

    Disadvantages

    Limitation 1 Must specify K in advance.
    Limitation 2 Sensitive to outliers.
    Limitation 3 Works only for spherical clusters.
    Limitation 4 Random initialization can affect results — fixed by K-Means++.
    Limitation 5 Doesn't handle categorical data directly.

    K-Means vs Other Clustering Algorithms

    Algorithm Requires K? Works on Non-Spherical Data? Outlier Friendly?
    K-MeansYesNoNo
    K-MedoidsYesPartialYes
    DBSCANNoYesYes
    HierarchicalNoYesPartial
    GMMYesYesPartial

    Common Evaluation Metrics

    MetricPurpose
    WCSS (Inertia)Within-cluster sum of squares
    Silhouette ScoreQuality of clustering (−1 to 1)
    Davies-Bouldin IndexCompactness & separation
    Elbow MethodFind optimal number of clusters

    Common Mistakes to Avoid

    Mistake 1 Not scaling features before applying K-Means.
    Mistake 2 Using K-Means on categorical data without encoding.
    Mistake 3 Choosing K randomly without using the Elbow Method.
    Mistake 4 Ignoring outliers — they distort centroid locations.

    Best Practices

    Quick Tips

    • Always standardize features.
    • Use K-Means++ for better initialization.
    • Try Elbow + Silhouette to choose K.
    • Visualize using PCA in 2D.
    • Remove outliers before clustering.
    • Use n_init=10 for stable results.

    Importance of K-Means

    Most Used Algorithm

    • Standard for clustering tasks
    • Easy to learn and apply

    Fast & Scalable

    • Works with millions of rows
    • Low computational cost

    Business Impact

    • Used in marketing analytics
    • Customer segmentation

    Foundation Algorithm

    • Used in many ML pipelines
    • Base of advanced clustering

    Golden Rule

    REMEMBER
    K Clusters + Means as Centers = K-Means Algorithm

    Key Takeaway

    K-Means Clustering is one of the simplest yet most powerful unsupervised algorithms. It groups data points into K clusters using their mean centroids, making it ideal for tasks like customer segmentation, anomaly detection, and pattern discovery in real-world datasets.

    Machine Learning Course  |  Chapter: Clustering Algorithms  |  Topic 2