Course topics

By WebNest Studio Engineering Team

Python Tutorial

The itertools and functools Modules

itertools and functools are two standard-library modules that make functional-style Python concise and fast. itertools provides building blocks for efficient looping — infinite counters, chaining, grouping, slicing iterators, combinations and permutations — all lazily evaluated. functools provides tools for working with functions — caching, partial application, reduction, ordering and decorators.

This lesson walks through the most useful functions of both modules with practical examples.

Why it matters

These two modules are the standard toolkit for working with iterables and functions. lru_cache can turn a slow recursive function into a fast one with a single line, partial fixes some of a function's arguments, and the itertools functions replace many hand-written loops with tested, memory-efficient code.

Infinite and Terminating Iterators

count(start, step), cycle(iterable) and repeat(x, n) generate values endlessly (combine with islice or zip). chain() joins iterables, islice() slices any iterator, accumulate() produces running totals, takewhile()/dropwhile() cut sequences by a condition, compress() filters by a selector, pairwise() yields overlapping pairs, batched() (3.12+) splits into fixed-size chunks, and zip_longest() pads uneven iterables.

Grouping and Combinatorics

groupby(iterable, key) groups consecutive items with the same key, so sort by the key first. product() gives the Cartesian product (nested loops), permutations() ordered arrangements, combinations() unordered selections, and combinations_with_replacement().

functools

lru_cache and cache memoise function results; partial() pre-fills some arguments of a function; reduce() folds a sequence into one value; wraps() preserves metadata in decorators; total_ordering completes comparison methods; cached_property caches a computed attribute; singledispatch dispatches on type; cmp_to_key adapts old-style comparison functions for sorting.

Examples

Infinite iterators and iterator tools

Python
from itertools import count, cycle, repeat, islice, chain, accumulate, takewhile, dropwhile, compress, pairwise, batched, zip_longest

print(list(islice(count(100, 5), 4)))
print(list(zip(["Mon", "Tue", "Wed", "Thu"], cycle(["on-call", "off"]))))
print(list(repeat("ab", 3)))
print(list(chain([1, 2], (3,), "ab")))
print(list(accumulate([100, 250, -50, 400])))
print(list(takewhile(lambda x: x < 5, [1, 3, 6, 2])), list(dropwhile(lambda x: x < 5, [1, 3, 6, 2])))
print(list(compress("ABCDEF", [1, 0, 1, 0, 1, 1])))
print(list(pairwise([10, 13, 11, 20])))
print(list(batched(range(1, 8), 3)))
print(list(zip_longest("abc", [1, 2], fillvalue="-")))
Output
[100, 105, 110, 115]
[('Mon', 'on-call'), ('Tue', 'off'), ('Wed', 'on-call'), ('Thu', 'off')]
['ab', 'ab', 'ab']
[1, 2, 3, 'a', 'b']
[100, 350, 300, 700]
[1, 3] [6, 2]
['A', 'C', 'E', 'F']
[(10, 13), (13, 11), (11, 20)]
[(1, 2, 3), (4, 5, 6), (7,)]
[('a', 1), ('b', 2), ('c', '-')]

groupby and combinatorics

Python
from itertools import groupby, product, permutations, combinations, combinations_with_replacement

sales = [("north", 100), ("south", 50), ("north", 70), ("south", 30), ("east", 90)]
sales.sort(key=lambda s: s[0])                     # groupby needs sorted input
for region, rows in groupby(sales, key=lambda s: s[0]):
    print(region, sum(amount for _, amount in rows))

print(list(product("AB", [1, 2])))
print(["".join(p) for p in permutations("abc", 2)])
print(list(combinations([1, 2, 3, 4], 2)))
print(len(list(combinations_with_replacement("xyz", 2))))
Output
east 90
north 170
south 80
[('A', 1), ('A', 2), ('B', 1), ('B', 2)]
['ab', 'ac', 'ba', 'bc', 'ca', 'cb']
[(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)]
6

functools: partial, reduce, lru_cache, cmp_to_key and wraps

Python
from functools import partial, reduce, lru_cache, cmp_to_key, wraps
import operator

def power(base, exponent):
    return base ** exponent

square = partial(power, exponent=2)
cube = partial(power, exponent=3)
print(square(7), cube(2))

print(reduce(operator.mul, range(1, 6)), reduce(lambda a, b: a + b, ["a", "b", "c"]))

@lru_cache(maxsize=128)
def slow_price(item):
    print(f"looking up {item}...")
    return len(item) * 100

print(slow_price("pen"), slow_price("pen"), slow_price.cache_info().hits)

def by_length_then_alpha(a, b):
    return (len(a) - len(b)) or ((a > b) - (a < b))

print(sorted(["kiwi", "fig", "apple", "date"], key=cmp_to_key(by_length_then_alpha)))

def shout(func):
    @wraps(func)
    def wrapper(*args):
        return func(*args).upper()
    return wrapper

@shout
def greet(name):
    """Return a greeting."""
    return f"hello {name}"

print(greet("asha"), greet.__name__, greet.__doc__)
Output
49 8
120 abc
looking up pen...
300 300 1
['fig', 'date', 'kiwi', 'apple']
HELLO ASHA greet Return a greeting.

Common Mistakes

  • Using groupby on unsorted data and getting several groups for the same key.
  • Materialising huge combinatoric iterators with list() — they grow factorially.
  • Caching functions with lru_cache whose arguments are unhashable (lists) or whose results change over time.
  • Forgetting functools.wraps in decorators, losing the original name and docstring.

Exercise

Use chain and islice to print the first three items of two lists joined together. Use partial to create a square function from power. Add lru_cache to fib so that fib(30) is computed quickly.

Expected output: [1, 2, 3], 25, 832040 (one per line)

Python · starter code
from functools import lru_cache, partial
from itertools import chain, islice

# TODO: print the first three items of [1, 2] followed by [3, 4]


def power(base, exponent):
    return base ** exponent


# TODO: create square from power using partial, and print square(5)


# TODO: add a cache to this function
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)


print(fib(30))
Show hints
  • islice(chain(a, b), 3) yields the first three items; wrap it in list().
  • partial(power, exponent=2) returns a function that needs only the base. The cache is added with @lru_cache(maxsize=None).
Show solution
Python · solution
from functools import lru_cache, partial
from itertools import chain, islice

print(list(islice(chain([1, 2], [3, 4]), 3)))


def power(base, exponent):
    return base ** exponent


square = partial(power, exponent=2)
print(square(5))


@lru_cache(maxsize=None)
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)


print(fib(30))

Quick quiz

1. What must be true of the data before itertools.groupby groups it correctly?

  • A. It must be a list
  • B. It must contain no duplicates
  • C. It must be sorted by the grouping key
  • D. It must be numeric
Show answer

C. groupby only groups items that are next to each other.

2. What restriction does lru_cache place on the arguments of the function?

  • A. They must be numbers
  • B. They must be strings
  • C. There must be exactly one
  • D. They must be hashable
Show answer

D. The arguments are used as a dictionary key, so a list argument raises TypeError.

3. What does itertools.count(1) produce?

  • A. An endless sequence 1, 2, 3, ...
  • B. The number 1
  • C. A list of one item
  • D. The number of items in an iterable
Show answer

A. It must be limited with islice, takewhile or a break.

Interview questions

What does functools.lru_cache do?

Show answer

It wraps a function so that its results are stored by argument. A later call with the same arguments returns the stored result without running the function again. maxsize limits how many results are kept, discarding the least recently used. It suits functions that are pure, meaning the same input always gives the same output, and costly to compute.

What is functools.partial used for?

Show answer

It creates a new function from an existing one with some arguments already filled in. partial(int, base=2) is a function that parses binary strings. It is used to adapt a general function to a place that expects fewer arguments, such as a callback, and is often clearer than writing a lambda.

Key Points to Remember

  • itertools gives lazy building blocks: count, cycle, chain, islice, accumulate, pairwise, batched...
  • groupby groups consecutive items — sort by the key first.
  • product, permutations and combinations generate combinatorial sequences.
  • functools: partial, reduce, lru_cache/cache, wraps, total_ordering, cmp_to_key.

Practice in the playground

Write and run your own code for this topic, or start from the exercise or an example. Runs Python in your browser. Your code for each topic is kept in this browser.

Run the examples

Change an input, predict the result, then compare it with the output. Explain why the result changes.

WebNest Studio

Written by

WebNest Studio Engineering Team

Practical lessons from developers working with Java, Python, React, databases and production systems.

Led by Vansh Duggal, Founder, WebNest Studio · Full-stack & software engineering