A list comprehension creates an entire list and stores all its values in memory. This is useful when every result is required immediately, but it may be unnecessary when values are processed only once or one at a time.

A generator expression creates a generator object that produces values when they are requested. It uses syntax similar to list comprehension, but it uses parentheses instead of square brackets.

Prerequisites

What is a Generator Expression?

A generator expression is a concise way to create a generator object. The generator produces values one at a time when the values are requested during iteration.

Unlike a list comprehension, a generator expression does not immediately create and store the complete sequence of generated values.

Basic Syntax

The condition is optional. The simplest generator expression is:

generator = (
    expression
    for item in iterable
)

A generator expression with a filtering condition is:

generator = (
    expression
    for item in iterable
    if condition
)

List Comprehension vs Generator Expression

List Comprehension

squares_list = [
    number ** 2
    for number in range(1, 6)
]

print(squares_list)

Output:

[1, 4, 9, 16, 25]

Generator Expression

squares_generator = (
    number ** 2
    for number in range(1, 6)
)

print(squares_generator)

Possible output:

<generator object <genexpr> at 0x...>

List Comprehension and Generator Expression Compared

Your First Generator Expression

Create a generator that produces squares from 1 through 5:

squares = (
    number ** 2
    for number in range(1, 6)
)

for square in squares:
    print(square)

Output:

1
4
9
16
25

The for loop requests values from the generator one at a time until no values remain.

Understanding Lazy Evaluation

Lazy evaluation means that the expression is evaluated only when the next generated value is requested.

def show_processing(number):
    print("Processing:", number)
    return number ** 2


squares = (
    show_processing(number)
    for number in range(1, 4)
)

print("Generator created")

for square in squares:
    print("Result:", square)

Output:

Generator created
Processing: 1
Result: 1
Processing: 2
Result: 4
Processing: 3
Result: 9

Retrieve Values with next()

The built-in next() function requests one value from a generator.

numbers = (
    number
    for number in range(10, 13)
)

print(next(numbers))
print(next(numbers))
print(next(numbers))

Output:

10
11
12

Each call continues from the generator's current position.

Generator Exhaustion

A generator is consumed as its values are requested. When all values have been produced, the generator is exhausted.

numbers = (
    number
    for number in range(1, 4)
)

print(list(numbers))
print(list(numbers))

Output:

[1, 2, 3]
[]

The first conversion consumes all generated values. The second conversion receives no values because the same generator has already been exhausted.

StopIteration

Calling next() after a generator is exhausted raises StopIteration.

numbers = (
    number
    for number in range(1, 3)
)

print(next(numbers))
print(next(numbers))

try:
    print(next(numbers))
except StopIteration:
    print("The generator is exhausted.")

Output:

1
2
The generator is exhausted.

Filter Values

Add an if clause to generate values only when a condition is true.

even_numbers = (
    number
    for number in range(1, 11)
    if number % 2 == 0
)

for number in even_numbers:
    print(number)

Output:

2
4
6
8
10

Transform and Filter Together

The generator expression can filter input values and transform the accepted values.

even_squares = (
    number ** 2
    for number in range(1, 11)
    if number % 2 == 0
)

print(list(even_squares))

Output:

[4, 16, 36, 64, 100]

Converting the generator with list() creates a list containing all remaining generated values.

Use if-else in the Expression

An inline conditional expression can select the generated value.

number_labels = (
    "Even" if number % 2 == 0 else "Odd"
    for number in range(1, 6)
)

print(list(number_labels))

Output:

['Odd', 'Even', 'Odd', 'Even', 'Odd']

Use a Generator Expression with sum()

Generator expressions are commonly passed directly to aggregation functions.

total = sum(
    number ** 2
    for number in range(1, 6)
)

print(total)

Output:

55

The generated square values are consumed by sum() without first creating a separate list.

Omit Extra Parentheses in a Function Call

When a generator expression is the only positional argument in a function call, another pair of parentheses is not required.

Concise Form

total = sum(
    number ** 2
    for number in range(1, 6)
)

Explicit Generator Variable

squares = (
    number ** 2
    for number in range(1, 6)
)

total = sum(squares)

Both examples calculate the same total.

Use Generator Expressions with any() and all()

Check Whether Any Value Matches

numbers = [3, 7, 12, 19]

contains_even_number = any(
    number % 2 == 0
    for number in numbers
)

print(contains_even_number)

Output:

True

Check Whether All Values Match

scores = [75, 82, 91, 68]

all_scores_passed = all(
    score >= 40
    for score in scores
)

print(all_scores_passed)

Output:

True

Use min() and max()

Built-in aggregation functions can consume generated values directly.

prices = [100, 250, 500, 800]

highest_discounted_price = max(
    price * 0.90
    for price in prices
)

lowest_discounted_price = min(
    price * 0.90
    for price in prices
)

print("Highest:", highest_discounted_price)
print("Lowest:", lowest_discounted_price)

Output:

Highest: 720.0
Lowest: 90.0

Use a Generator Expression with join()

The join() method can consume generated strings.

numbers = [10, 20, 30, 40]

formatted_numbers = ", ".join(
    str(number)
    for number in numbers
)

print(formatted_numbers)

Output:

10, 20, 30, 40

Convert a Generator to Another Collection

A generator can be consumed by collection constructors.

Convert to a List

generator = (
    number ** 2
    for number in range(1, 6)
)

result = list(generator)

print(result)

Output:

[1, 4, 9, 16, 25]

Convert to a Tuple

generator = (
    number ** 2
    for number in range(1, 6)
)

result = tuple(generator)

print(result)

Output:

(1, 4, 9, 16, 25)

Convert to a Set

values = [2, 2, 3, 3, 4, 4]

unique_squares = set(
    value ** 2
    for value in values
)

print(unique_squares)

Possible output:

{16, 9, 4}

Process a List of Dictionaries

Consider a list of employee records:

employees = [
    {
        "name": "Amina",
        "department": "IT",
        "active": True,
    },
    {
        "name": "Rahul",
        "department": "Finance",
        "active": False,
    },
    {
        "name": "David",
        "department": "IT",
        "active": True,
    },
]

active_employee_names = (
    employee["name"]
    for employee in employees
    if employee["active"]
)

for employee_name in active_employee_names:
    print(employee_name)

Output:

Amina
David

Process File Lines Lazily

File objects are iterable. A generator expression can process matching lines one at a time.

with open(
    "application.log",
    mode="r",
    encoding="utf-8"
) as log_file:

    error_lines = (
        line.strip()
        for line in log_file
        if "ERROR" in line
    )

    for error_line in error_lines:
        print(error_line)

Use Multiple for Clauses

A generator expression can represent nested loops.

coordinate_generator = (
    (row, column)
    for row in range(1, 3)
    for column in range(1, 4)
)

for coordinate in coordinate_generator:
    print(coordinate)

Output:

(1, 1)
(1, 2)
(1, 3)
(2, 1)
(2, 2)
(2, 3)

The order of the for clauses follows the order of the equivalent nested loops.

Flatten Nested Values Lazily

matrix = [
    [1, 2, 3],
    [4, 5, 6],
    [7, 8, 9],
]

flattened_values = (
    value
    for row in matrix
    for value in row
)

for value in flattened_values:
    print(value)

Output:

1
2
3
4
5
6
7
8
9

Build a Generator Pipeline

Multiple generators can be connected so that each stage receives values from the previous stage.

numbers = (
    number
    for number in range(1, 21)
)

even_numbers = (
    number
    for number in numbers
    if number % 2 == 0
)

squared_even_numbers = (
    number ** 2
    for number in even_numbers
)

for result in squared_even_numbers:
    print(result)

Output:

4
16
36
64
100
144
196
256
324
400

Generator Expression vs Generator Function

Both approaches create generator objects, but they are suitable for different levels of complexity.

Generator Expression

squares = (
    number ** 2
    for number in range(1, 6)
)

Generator Function

def generate_squares(limit):
    for number in range(1, limit + 1):
        yield number ** 2


squares = generate_squares(5)

Values Used by a Generator Expression

The iterable for the outermost for clause is obtained when the generator expression is created. The generated expression is evaluated as values are requested.

numbers = [1, 2, 3]

squares = (
    number ** 2
    for number in numbers
)

numbers.append(4)

print(list(squares))

Output:

[1, 4, 9, 16]

The generator iterates over the original list object. Because that list was modified before consumption, the added value is also observed.

Memory Behavior

A list comprehension stores every generated result in a list. A generator expression maintains the state required to produce subsequent values.

List Creation

squares_list = [
    number ** 2
    for number in range(1, 1000001)
]

Generator Creation

squares_generator = (
    number ** 2
    for number in range(1, 1000001)
)

Practical Example: Order Processing

Calculate the total value of completed orders without creating a separate list of order amounts.

orders = [
    {
        "order_number": "ORD-101",
        "amount": 750,
        "completed": True,
    },
    {
        "order_number": "ORD-102",
        "amount": 1500,
        "completed": True,
    },
    {
        "order_number": "ORD-103",
        "amount": 1800,
        "completed": False,
    },
    {
        "order_number": "ORD-104",
        "amount": 2200,
        "completed": True,
    },
]

completed_order_total = sum(
    order["amount"]
    for order in orders
    if order["completed"]
)

print("Completed order total:", completed_order_total)

Output:

Completed order total: 4450

Practical Example: Active Employee Names

employees = [
    {
        "name": "Amina",
        "active": True,
    },
    {
        "name": "Rahul",
        "active": False,
    },
    {
        "name": "David",
        "active": True,
    },
]

active_names = (
    employee["name"]
    for employee in employees
    if employee["active"]
)

print(", ".join(active_names))

Output:

Amina, David

Common Mistakes

When to Use Generator Expressions

When a Generator Expression is Not Ideal

Poor and Recommended Practices

Hands-On Practice

Knowledge Check

value = next(generator)

result = list(generator)

Generator Expression Quick Reference

# Basic generator expression
generator = (
    item
    for item in iterable
)

# Transform values
generator = (
    transform(item)
    for item in iterable
)

# Filter values
generator = (
    item
    for item in iterable
    if condition
)

# Transform and filter
generator = (
    transform(item)
    for item in iterable
    if condition
)

# Conditional output
generator = (
    value_if_true if condition else value_if_false
    for item in iterable
)

# Retrieve one value
value = next(generator)

# Consume with a loop
for value in generator:
    print(value)

# Convert to a list
result = list(generator)

# Aggregate generated values
total = sum(
    number ** 2
    for number in range(1, 6)
)

# Evaluate whether any value matches
matched = any(
    condition
    for item in iterable
)

# Evaluate whether all values match
matched = all(
    condition
    for item in iterable
)

Summary