By WebNest Studio Engineering Team
Python Tutorial
Multithreading in Python
Many programs spend most of their time waiting: for web pages to download, for API responses, for database queries, for files to be read. Threads let one program wait on many things at once, so twenty slow downloads take about as long as the slowest one instead of the sum of all of them.
This lesson covers the threading module, the high-level concurrent.futures.ThreadPoolExecutor, race conditions and locks, thread-safe queues for producer/consumer designs, and the Global Interpreter Lock (GIL) — including Python's new free-threaded builds.
Why it matters
Threads let a program do several slow things at once, such as downloading many files or calling several APIs, without waiting for each in turn. They also introduce race conditions, bugs that appear only occasionally and are hard to reproduce. Interviewers ask about the GIL to see whether you understand what threads can and cannot speed up in Python.
Threads and the GIL
A thread is an independent flow of execution within a process; threads share memory. In standard CPython, the Global Interpreter Lock lets only one thread execute Python bytecode at a time, so threads do not speed up CPU-heavy pure-Python code — but the GIL is released while waiting for I/O, so threads are excellent for I/O-bound work. Python 3.13+ offers an optional free-threaded build without the GIL; for CPU-bound work today, use multiprocessing.
Creating Threads
threading.Thread(target=func, args=(...)) creates a thread, start() runs it and join() waits for it to finish. Daemon threads (daemon=True) are killed when the main program exits. In most code, prefer ThreadPoolExecutor: it reuses a pool of threads, returns results through futures, and propagates exceptions.
Race Conditions and Locks
When threads modify shared data, operations like count += 1 (read, add, write) can interleave and lose updates. Protect shared state with threading.Lock in a with block, or avoid sharing entirely by returning results and using thread-safe queue.Queue. Other primitives include RLock, Semaphore (limit concurrency), Event (signal between threads) and Condition.
Threads vs asyncio vs Processes
Use threads for I/O-bound work with blocking libraries (requests, database drivers). Use asyncio for very high numbers of concurrent I/O operations with async libraries. Use processes for CPU-bound work.
Examples
Starting and joining threads
import threading
import time
def download(name, seconds):
time.sleep(seconds) # simulated network wait
print(f"{name} done") # finishing order can vary between runs
start = time.perf_counter()
threads = [threading.Thread(target=download, args=(f"file{i}", 0.3)) for i in range(1, 4)]
for t in threads:
t.start()
for t in threads:
t.join()
elapsed = time.perf_counter() - start
print(f"3 downloads in ~{elapsed:.1f}s (sequential would take ~0.9s)")
file1 done
file2 done
file3 done
3 downloads in ~0.3s (sequential would take ~0.9s)
ThreadPoolExecutor with results and exceptions
from concurrent.futures import ThreadPoolExecutor, as_completed
import time
def fetch_price(product):
time.sleep(0.2)
if product == "broken":
raise ValueError("price service error")
return product, len(product) * 100
products = ["pen", "notebook", "broken", "bag"]
with ThreadPoolExecutor(max_workers=4) as pool:
futures = {pool.submit(fetch_price, p): p for p in products}
results = {}
for future in as_completed(futures):
name = futures[future]
try:
product, price = future.result()
results[product] = price
except ValueError as e:
results[name] = f"failed: {e}"
print(dict(sorted(results.items())))
with ThreadPoolExecutor() as pool:
print(list(pool.map(str.upper, ["a", "b", "c"])))
{'bag': 300, 'broken': 'failed: price service error', 'notebook': 800, 'pen': 300}
['A', 'B', 'C']
A race condition and fixing it with a Lock
import threading
import time
counter = 0
lock = threading.Lock()
def unsafe_increment(n):
global counter
for _ in range(n):
value = counter
time.sleep(0) # give another thread a chance to run mid-update
counter = value + 1
def safe_increment(n):
global counter
for _ in range(n):
with lock:
value = counter
time.sleep(0)
counter = value + 1
for worker in (unsafe_increment, safe_increment):
counter = 0
threads = [threading.Thread(target=worker, args=(10_000,)) for _ in range(4)]
[t.start() for t in threads]
[t.join() for t in threads]
print(f"{worker.__name__}: {counter} (expected 40000)")
# Without time.sleep(0), CPython's GIL rarely switches threads between the read
# and the write, so the bug can hide for a long time. It is still a bug: real
# code does I/O or other work there, and that is when updates get lost.
unsafe_increment: 10296 (expected 40000)
safe_increment: 40000 (expected 40000)
(The unsafe total is different on every run. The locked total is always 40000.)
Producer/consumer with queue.Queue
import queue
import threading
tasks = queue.Queue()
results = []
def worker(worker_id):
while True:
item = tasks.get()
if item is None: # sentinel: stop
tasks.task_done()
break
results.append((item, item ** 2))
tasks.task_done()
workers = [threading.Thread(target=worker, args=(i,)) for i in range(3)]
for w in workers:
w.start()
for n in range(1, 7):
tasks.put(n)
for _ in workers:
tasks.put(None)
tasks.join()
print(sorted(results))
[(1, 1), (2, 4), (3, 9), (4, 16), (5, 25), (6, 36)]
Common Mistakes
- Using threads to speed up CPU-bound pure-Python code (the GIL prevents it) — use processes.
- Modifying shared variables from several threads without a lock.
- Forgetting join(), so the main program continues before workers finish.
- Swallowing exceptions in threads; ThreadPoolExecutor futures re-raise them with result().
Exercise
Use a ThreadPoolExecutor to square three numbers and print the results. Then start four threads that each add 1 to a shared counter 1,000 times, protecting the counter with a Lock, and print the final count.
Expected output: [1, 4, 9] then 4000
import threading
from concurrent.futures import ThreadPoolExecutor
def square(number):
return number * number
# TODO: use a ThreadPoolExecutor and its map method to square [1, 2, 3], and print the list
counter = 0
lock = threading.Lock()
def add_many():
global counter
for _ in range(1000):
# TODO: increase counter while holding the lock
pass
# TODO: start four threads running add_many, wait for them all, and print counter
Show hints
executor.map(square, [1, 2, 3])returns the results in the order of the inputs.with lock:acquires the lock and releases it at the end of the block. Calljoin()on each thread before printing.
Show solution
import threading
from concurrent.futures import ThreadPoolExecutor
def square(number):
return number * number
with ThreadPoolExecutor(max_workers=3) as executor:
print(list(executor.map(square, [1, 2, 3])))
counter = 0
lock = threading.Lock()
def add_many():
global counter
for _ in range(1000):
with lock:
counter += 1
threads = [threading.Thread(target=add_many) for _ in range(4)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
print(counter)
Quick quiz
1. What does the GIL do in CPython?
- A. Makes threads run on separate CPU cores
- B. Prevents the use of threads
- C. Allows only one thread to execute Python bytecode at a time
- D. Encrypts memory
Show answer
C. Threads therefore do not speed up pure-Python calculations.
2. For which kind of task are threads useful in Python?
- A. CPU-bound calculations
- B. Only GUI programs
- C. Neither
- D. I/O-bound work such as network requests and file access
Show answer
D. A thread releases the GIL while it waits for I/O, letting other threads run.
3. What is a race condition?
- A. A bug where the result depends on the timing of threads that access shared data
- B. Two threads finishing at the same time
- C. A thread that runs too quickly
- D. A deadlock
Show answer
A. It is prevented by protecting the shared data with a lock.
Interview questions
What is the Global Interpreter Lock?
Show answer
The GIL is a lock in the standard CPython interpreter that lets only one thread run Python bytecode at any moment. It keeps the interpreter's memory management simple and safe, but it means threads cannot run Python code in parallel on several cores. Threads still help with I/O-bound work, because the GIL is released during blocking I/O. For CPU-bound work, multiple processes are used; recent CPython versions also offer an optional build without the GIL.
How do you prevent a race condition?
Show answer
Make sure that only one thread at a time can read and modify the shared data, by wrapping that code in a threading.Lock used with a with statement. Better still, avoid sharing mutable state: pass data between threads through a queue.Queue, which is thread-safe, or have each thread return its result to be combined afterwards.
Key Points to Remember
- Threads suit I/O-bound work; the GIL limits CPU-bound parallelism in standard CPython.
- ThreadPoolExecutor with submit/map/as_completed is the easiest API.
- Protect shared state with Lock, or avoid sharing and use queue.Queue.
- Choose threads, asyncio or processes based on the workload.
Practice in the playground
Write and run your own code for this topic, or start from the exercise or an example. Runs Python in your browser. Your code for each topic is kept in this browser.
Run the examples
Change an input, predict the result, then compare it with the output. Explain why the result changes.
These examples need a project environment on your computer, so they can be copied but not run here. Codelab runs Python, FastAPI backends with SQLite, plain Java 17 and HTML/CSS/JavaScript.
Written by
WebNest Studio Engineering Team
Practical lessons from developers working with Java, Python, React, databases and production systems.
Led by Vansh Duggal, Founder, WebNest Studio · Full-stack & software engineering