By WebNest Studio Engineering Team
Python Tutorial
The itertools and functools Modules
itertools and functools are two standard-library modules that make functional-style Python concise and fast. itertools provides building blocks for efficient looping — infinite counters, chaining, grouping, slicing iterators, combinations and permutations — all lazily evaluated. functools provides tools for working with functions — caching, partial application, reduction, ordering and decorators.
This lesson walks through the most useful functions of both modules with practical examples.
Why it matters
These two modules are the standard toolkit for working with iterables and functions. lru_cache can turn a slow recursive function into a fast one with a single line, partial fixes some of a function's arguments, and the itertools functions replace many hand-written loops with tested, memory-efficient code.
Infinite and Terminating Iterators
count(start, step), cycle(iterable) and repeat(x, n) generate values endlessly (combine with islice or zip). chain() joins iterables, islice() slices any iterator, accumulate() produces running totals, takewhile()/dropwhile() cut sequences by a condition, compress() filters by a selector, pairwise() yields overlapping pairs, batched() (3.12+) splits into fixed-size chunks, and zip_longest() pads uneven iterables.
Grouping and Combinatorics
groupby(iterable, key) groups consecutive items with the same key, so sort by the key first. product() gives the Cartesian product (nested loops), permutations() ordered arrangements, combinations() unordered selections, and combinations_with_replacement().
functools
lru_cache and cache memoise function results; partial() pre-fills some arguments of a function; reduce() folds a sequence into one value; wraps() preserves metadata in decorators; total_ordering completes comparison methods; cached_property caches a computed attribute; singledispatch dispatches on type; cmp_to_key adapts old-style comparison functions for sorting.
Examples
Infinite iterators and iterator tools
from itertools import count, cycle, repeat, islice, chain, accumulate, takewhile, dropwhile, compress, pairwise, batched, zip_longest
print(list(islice(count(100, 5), 4)))
print(list(zip(["Mon", "Tue", "Wed", "Thu"], cycle(["on-call", "off"]))))
print(list(repeat("ab", 3)))
print(list(chain([1, 2], (3,), "ab")))
print(list(accumulate([100, 250, -50, 400])))
print(list(takewhile(lambda x: x < 5, [1, 3, 6, 2])), list(dropwhile(lambda x: x < 5, [1, 3, 6, 2])))
print(list(compress("ABCDEF", [1, 0, 1, 0, 1, 1])))
print(list(pairwise([10, 13, 11, 20])))
print(list(batched(range(1, 8), 3)))
print(list(zip_longest("abc", [1, 2], fillvalue="-")))
[100, 105, 110, 115]
[('Mon', 'on-call'), ('Tue', 'off'), ('Wed', 'on-call'), ('Thu', 'off')]
['ab', 'ab', 'ab']
[1, 2, 3, 'a', 'b']
[100, 350, 300, 700]
[1, 3] [6, 2]
['A', 'C', 'E', 'F']
[(10, 13), (13, 11), (11, 20)]
[(1, 2, 3), (4, 5, 6), (7,)]
[('a', 1), ('b', 2), ('c', '-')]
groupby and combinatorics
from itertools import groupby, product, permutations, combinations, combinations_with_replacement
sales = [("north", 100), ("south", 50), ("north", 70), ("south", 30), ("east", 90)]
sales.sort(key=lambda s: s[0]) # groupby needs sorted input
for region, rows in groupby(sales, key=lambda s: s[0]):
print(region, sum(amount for _, amount in rows))
print(list(product("AB", [1, 2])))
print(["".join(p) for p in permutations("abc", 2)])
print(list(combinations([1, 2, 3, 4], 2)))
print(len(list(combinations_with_replacement("xyz", 2))))
east 90
north 170
south 80
[('A', 1), ('A', 2), ('B', 1), ('B', 2)]
['ab', 'ac', 'ba', 'bc', 'ca', 'cb']
[(1, 2), (1, 3), (1, 4), (2, 3), (2, 4), (3, 4)]
6
functools: partial, reduce, lru_cache, cmp_to_key and wraps
from functools import partial, reduce, lru_cache, cmp_to_key, wraps
import operator
def power(base, exponent):
return base ** exponent
square = partial(power, exponent=2)
cube = partial(power, exponent=3)
print(square(7), cube(2))
print(reduce(operator.mul, range(1, 6)), reduce(lambda a, b: a + b, ["a", "b", "c"]))
@lru_cache(maxsize=128)
def slow_price(item):
print(f"looking up {item}...")
return len(item) * 100
print(slow_price("pen"), slow_price("pen"), slow_price.cache_info().hits)
def by_length_then_alpha(a, b):
return (len(a) - len(b)) or ((a > b) - (a < b))
print(sorted(["kiwi", "fig", "apple", "date"], key=cmp_to_key(by_length_then_alpha)))
def shout(func):
@wraps(func)
def wrapper(*args):
return func(*args).upper()
return wrapper
@shout
def greet(name):
"""Return a greeting."""
return f"hello {name}"
print(greet("asha"), greet.__name__, greet.__doc__)
49 8
120 abc
looking up pen...
300 300 1
['fig', 'date', 'kiwi', 'apple']
HELLO ASHA greet Return a greeting.
Common Mistakes
- Using groupby on unsorted data and getting several groups for the same key.
- Materialising huge combinatoric iterators with list() — they grow factorially.
- Caching functions with lru_cache whose arguments are unhashable (lists) or whose results change over time.
- Forgetting functools.wraps in decorators, losing the original name and docstring.
Exercise
Use chain and islice to print the first three items of two lists joined together. Use partial to create a square function from power. Add lru_cache to fib so that fib(30) is computed quickly.
Expected output: [1, 2, 3], 25, 832040 (one per line)
from functools import lru_cache, partial
from itertools import chain, islice
# TODO: print the first three items of [1, 2] followed by [3, 4]
def power(base, exponent):
return base ** exponent
# TODO: create square from power using partial, and print square(5)
# TODO: add a cache to this function
def fib(n):
return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(30))
Show hints
islice(chain(a, b), 3)yields the first three items; wrap it inlist().partial(power, exponent=2)returns a function that needs only the base. The cache is added with@lru_cache(maxsize=None).
Show solution
from functools import lru_cache, partial
from itertools import chain, islice
print(list(islice(chain([1, 2], [3, 4]), 3)))
def power(base, exponent):
return base ** exponent
square = partial(power, exponent=2)
print(square(5))
@lru_cache(maxsize=None)
def fib(n):
return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(30))
Quick quiz
1. What must be true of the data before itertools.groupby groups it correctly?
- A. It must be a list
- B. It must contain no duplicates
- C. It must be sorted by the grouping key
- D. It must be numeric
Show answer
C. groupby only groups items that are next to each other.
2. What restriction does lru_cache place on the arguments of the function?
- A. They must be numbers
- B. They must be strings
- C. There must be exactly one
- D. They must be hashable
Show answer
D. The arguments are used as a dictionary key, so a list argument raises TypeError.
3. What does itertools.count(1) produce?
- A. An endless sequence 1, 2, 3, ...
- B. The number 1
- C. A list of one item
- D. The number of items in an iterable
Show answer
A. It must be limited with islice, takewhile or a break.
Interview questions
What does functools.lru_cache do?
Show answer
It wraps a function so that its results are stored by argument. A later call with the same arguments returns the stored result without running the function again. maxsize limits how many results are kept, discarding the least recently used. It suits functions that are pure, meaning the same input always gives the same output, and costly to compute.
What is functools.partial used for?
Show answer
It creates a new function from an existing one with some arguments already filled in. partial(int, base=2) is a function that parses binary strings. It is used to adapt a general function to a place that expects fewer arguments, such as a callback, and is often clearer than writing a lambda.
Key Points to Remember
- itertools gives lazy building blocks: count, cycle, chain, islice, accumulate, pairwise, batched...
- groupby groups consecutive items — sort by the key first.
- product, permutations and combinations generate combinatorial sequences.
- functools: partial, reduce, lru_cache/cache, wraps, total_ordering, cmp_to_key.
Practice in the playground
Write and run your own code for this topic, or start from the exercise or an example. Runs Python in your browser. Your code for each topic is kept in this browser.
Run the examples
Change an input, predict the result, then compare it with the output. Explain why the result changes.
Written by
WebNest Studio Engineering Team
Practical lessons from developers working with Java, Python, React, databases and production systems.
Led by Vansh Duggal, Founder, WebNest Studio · Full-stack & software engineering