Introduction

Memory optimization is the practice of reducing how much RAM your program uses — critical when handling large datasets, running on constrained devices, or scaling services to many users. Python prioritizes developer convenience over raw efficiency, so its objects carry overhead. The good news: with a few targeted techniques — __slots__, generators, efficient data structures, and object interning — you can dramatically shrink memory usage without rewriting your whole program.

Real-World Analogy

Prerequisites

Rule Zero: Measure First

Never optimize blindly — measure memory usage before and after with sys.getsizeof or profilers.

import sys

print(sys.getsizeof(42))         # int size in bytes
print(sys.getsizeof("hello"))    # string size
print(sys.getsizeof([1, 2, 3]))  # list size

# For deep/total size, use tools like tracemalloc or pympler

Technique 1: __slots__

By default, each object stores its attributes in a per-instance __dict__, which is memory-hungry. Defining __slots__ tells Python to use a fixed, compact layout instead — often cutting memory per object by more than half.

import sys

# Without __slots__ — uses a per-instance __dict__
class PointA:
    def __init__(self, x, y):
        self.x = x
        self.y = y

# With __slots__ — no __dict__, fixed layout
class PointB:
    __slots__ = ("x", "y")
    def __init__(self, x, y):
        self.x = x
        self.y = y

a = PointA(1, 2)
b = PointB(1, 2)
# PointB instances use significantly less memory each

Technique 2: Generators over Lists

A list holds every element in memory at once; a generator produces items one at a time — huge savings for large sequences.

import sys

# List comprehension — stores ALL values in memory
squares_list = [x * x for x in range(1_000_000)]
print(sys.getsizeof(squares_list))   # many megabytes

# Generator expression — stores only the recipe
squares_gen = (x * x for x in range(1_000_000))
print(sys.getsizeof(squares_gen))    # a few hundred bytes!

Technique 3: Leaner Data Structures

import array
import sys

# A list of ints — each int is a full Python object
py_list = [i for i in range(1000)]

# An array of typed ints — raw C-level storage
arr = array.array("i", range(1000))

print(sys.getsizeof(py_list))   # larger
print(sys.getsizeof(arr))       # much smaller

Technique 4: String Interning

Python reuses (interns) small strings and integers so identical values share one object in memory.

import sys

# Manually intern repeated strings to share memory
words = [sys.intern(w) for w in load_many_repeated_words()]
# Now identical words point to the SAME object,
# saving memory when the same value appears thousands of times

Technique 5: Process Data Lazily

Read and process large files line by line instead of loading everything into memory.

# Bad: loads the ENTIRE file into memory
with open("huge.log") as f:
    lines = f.readlines()        # all lines at once
    for line in lines:
        process(line)

# Good: streams one line at a time
with open("huge.log") as f:
    for line in f:               # lazy iteration
        process(line)

Quantifying the Savings

Suppose you have \(N\) objects, and an optimization reduces per-object size from \(s\) bytes to \(s'\) bytes. The total memory saved is:

\[ \Delta M = N \times (s - s') \]

This is why per-object optimizations like __slots__ matter so much: a small saving \(s - s'\) multiplied by millions of objects \(N\) becomes enormous.

Technique Summary

Best Practices

Common Mistakes

Interview Questions

Quick Revision