Introduction to Data Handling in Python

Beyond writing individual scripts, one of Python's most common real-world uses is working with structured data — reading it in, examining it, filtering it, and summarizing it. This is the foundation for everything from simple reporting scripts to full data-analysis pipelines built on libraries like pandas.

Even without any external libraries, Python's own core toolkit — lists of dictionaries, comprehensions, loops, and built-in functions like sum(), len(), and sorted() — is already enough to explore and summarize a small dataset, and understanding this "plain Python" foundation makes it much easier to learn data-focused libraries later.

Why it matters

Rows from a database, lines of a CSV file and JSON from an API all arrive in Python as a list of dictionaries. Filtering, totalling, sorting and finding the top record in that shape are the bread and butter of scripts and back-end code, and doing them well in plain Python is the foundation for moving on to pandas.

Representing Structured Data as a List of Dictionaries

A very common and natural way to represent a small table of records in plain Python is as a list of dictionaries, where each dictionary is one record (like one row) and its keys are the field names (like column headers). This mirrors exactly how data typically arrives from a JSON API or a parsed CSV file, which is why it is a natural starting point before reaching for a dedicated data library.

Iterating and Filtering Records

A plain for loop over a list of dictionaries lets you inspect, filter, or transform each record. Combined with an if condition, this is enough to answer questions like "which records match a certain criterion," and a list comprehension can express the same filtering operation in a single concise line.

Summarizing Data with Built-in Functions

Python's built-in functions sum(), len(), max(), min(), and sorted() cover many basic summary tasks without needing any external library: computing totals and averages, counting records, finding extremes, and ordering data by a chosen field (using the key= argument to sort by a dictionary value, often combined with a lambda expression).

Examples

sales = [
    {"product": "Widget", "units": 12, "price": 9.5},
    {"product": "Gadget", "units": 5, "price": 25.0},
    {"product": "Widget", "units": 8, "price": 9.5},
    {"product": "Gizmo", "units": 20, "price": 3.25},
]

total_units = sum(item["units"] for item in sales)
print("Total units sold:", total_units)

widget_sales = [item for item in sales if item["product"] == "Widget"]
print("Widget records:", widget_sales)

total_revenue = sum(item["units"] * item["price"] for item in sales)
print(f"Total revenue: {total_revenue:.2f}")

Total units sold: 45
Widget records: [{'product': 'Widget', 'units': 12, 'price': 9.5}, {'product': 'Widget', 'units': 8, 'price': 9.5}]
Total revenue: 380.00

sales = [
    {"product": "Widget", "units": 12},
    {"product": "Gadget", "units": 5},
    {"product": "Gizmo", "units": 20},
]

sorted_by_units = sorted(sales, key=lambda item: item["units"], reverse=True)
for record in sorted_by_units:
    print(record["product"], record["units"])

best_seller = max(sales, key=lambda item: item["units"])
print("Best seller:", best_seller["product"])

Gizmo 20
Widget 12
Gadget 5
Best seller: Gizmo

Exercise

From the list of sales records, print the total amount for the north region, the item with the largest amount, and the item names ordered from the largest amount to the smallest.

Expected output: 420, bag, ['bag', 'book', 'pen'] (one per line)

sales = [
    {"item": "pen", "region": "north", "amount": 120},
    {"item": "bag", "region": "south", "amount": 450},
    {"item": "book", "region": "north", "amount": 300},
]

# TODO: print the total amount for the north region
# TODO: print the item with the largest amount
# TODO: print the item names sorted by amount, largest first

sales = [
    {"item": "pen", "region": "north", "amount": 120},
    {"item": "bag", "region": "south", "amount": 450},
    {"item": "book", "region": "north", "amount": 300},
]

print(sum(record["amount"] for record in sales if record["region"] == "north"))

print(max(sales, key=lambda record: record["amount"])["item"])

ordered = sorted(sales, key=lambda record: record["amount"], reverse=True)
print([record["item"] for record in ordered])

Quick quiz

Interview questions