By WebNest Studio Engineering Team
Python Tutorial
Regression: Predicting Numbers with Linear Regression
Regression predicts a continuous number: a house price, tomorrow's demand, a delivery time. Linear regression is the classic starting point: it fits the line (or, with several features, the flat surface) y = w1·x1 + w2·x2 + … + b that minimises the squared prediction errors. It is fast, easy to interpret — each coefficient says how much the prediction changes per unit of a feature — and a strong baseline to beat.
This lesson predicts house prices from several features, evaluates the model with MAE, RMSE and R², interprets the coefficients, adds polynomial features for curved relationships, and compares with a tree-based model.
Why it matters
Linear regression is the starting point for predicting a number: a price, a demand, a delivery time. It is fast, and its coefficients can be explained to a manager, which is why it is still used widely. It is also the baseline against which any more complicated model has to prove itself.
Regression Metrics
- MAE (mean absolute error) — average size of the errors, in the target's units. Easy to explain: "off by Rs 2.1 lakh on average".
- RMSE (root mean squared error) — like MAE but punishes large errors more.
- R² (coefficient of determination) — share of the variation explained: 1.0 is perfect, 0 is no better than always predicting the mean, and it can be negative.
- Always compare against a baseline, such as predicting the average price for every house.
Beyond Straight Lines
If the relationship curves, PolynomialFeatures adds squared and interaction terms so a linear model can fit curves. Regularised versions — Ridge and Lasso — shrink coefficients to reduce overfitting when there are many features. Tree ensembles such as RandomForestRegressor and HistGradientBoostingRegressor capture non-linear patterns and interactions automatically and are often the most accurate choice on tabular data, at the cost of interpretability.
Examples
Predicting house prices with several features
import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(42)
n = 500
houses = pd.DataFrame({
"area_sqft": rng.integers(500, 3000, n),
"bedrooms": rng.integers(1, 5, n),
"age_years": rng.integers(0, 40, n),
"distance_km": rng.uniform(1, 25, n).round(1),
})
# The "true" price formula (in lakh Rs) plus noise — in real life this is unknown.
houses["price_lakh"] = (0.045 * houses["area_sqft"] + 4 * houses["bedrooms"]
- 0.6 * houses["age_years"] - 1.5 * houses["distance_km"]
+ 20 + rng.normal(0, 6, n)).round(1)
X = houses.drop(columns="price_lakh")
y = houses["price_lakh"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)
model = LinearRegression().fit(X_train, y_train)
predictions = model.predict(X_test)
print("MAE :", round(mean_absolute_error(y_test, predictions), 2), "lakh")
print("RMSE:", round(mean_squared_error(y_test, predictions) ** 0.5, 2), "lakh")
print("R^2 :", round(r2_score(y_test, predictions), 3))
print("baseline MAE (always predict the mean):", round((y_test - y_train.mean()).abs().mean(), 2))
for feature, weight in zip(X.columns, model.coef_):
print(f"{feature:<12} {weight:+.3f} lakh per unit")
new_house = pd.DataFrame([{"area_sqft": 1200, "bedrooms": 2, "age_years": 5, "distance_km": 8}])
print("estimate for a new house:", model.predict(new_house).round(1)[0], "lakh")
MAE : 4.47 lakh
RMSE: 5.65 lakh
R^2 : 0.972
baseline MAE (always predict the mean): 28.11
area_sqft +0.045 lakh per unit
bedrooms +3.797 lakh per unit
age_years -0.586 lakh per unit
distance_km -1.464 lakh per unit
estimate for a new house: 66.5 lakh
Curved relationships: polynomial features vs a straight line
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import PolynomialFeatures
rng = np.random.default_rng(0)
speed = np.linspace(10, 120, 60).reshape(-1, 1) # km/h
fuel = 0.002 * (speed.ravel() - 60) ** 2 + 5 + rng.normal(0, 0.3, 60) # U-shaped curve
straight = LinearRegression().fit(speed, fuel)
curve = make_pipeline(PolynomialFeatures(degree=2), LinearRegression()).fit(speed, fuel)
print("straight line R^2:", round(r2_score(fuel, straight.predict(speed)), 3))
print("degree-2 curve R^2:", round(r2_score(fuel, curve.predict(speed)), 3))
print("most efficient speed ~", int(speed[curve.predict(speed).argmin()][0]), "km/h")
straight line R^2: 0.121
degree-2 curve R^2: 0.982
most efficient speed ~ 60 km/h
Comparing linear models with a random forest on real data
from sklearn.datasets import load_diabetes
from sklearn.ensemble import RandomForestRegressor
from sklearn.linear_model import LinearRegression, Ridge
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import train_test_split
X, y = load_diabetes(return_X_y=True, as_frame=True) # disease progression after one year
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=7)
models = {
"LinearRegression": LinearRegression(),
"Ridge(alpha=1)": Ridge(alpha=1.0),
"RandomForest": RandomForestRegressor(n_estimators=300, random_state=7),
}
for name, model in models.items():
model.fit(X_train, y_train)
print(f"{name:<17} MAE={mean_absolute_error(y_test, model.predict(X_test)):.1f} R^2={model.score(X_test, y_test):.3f}")
forest = models["RandomForest"]
top = sorted(zip(forest.feature_importances_, X.columns), reverse=True)[:3]
print("most important features:", [(name, round(float(imp), 2)) for imp, name in top])
# Here the simple linear model wins: always compare against simple baselines.
LinearRegression MAE=41.9 R^2=0.503
Ridge(alpha=1) MAE=47.1 R^2=0.457
RandomForest MAE=44.0 R^2=0.484
most important features: [('s5', 0.28), ('bmi', 0.28), ('bp', 0.11)]
Common Mistakes
- Reporting R² alone without an error in real units (MAE/RMSE) and a baseline.
- Interpreting coefficients of features on very different scales as importance without scaling.
- Extrapolating far outside the range of the training data.
- Using a high polynomial degree and overfitting wildly.
- Assuming a linear model is wrong just because a complex one scores slightly higher — interpretability matters.
Exercise
Fit a linear regression to data generated by the rule y = 3x + 5. Print the coefficient and the intercept the model has learned, rounded to 2 decimals, and then the R² score of the model on the same data.
Expected output: 3.0 5.0 then 1.0
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score
x = [[0], [1], [2], [3]]
y = [5, 8, 11, 14]
# TODO: fit the model
# TODO: print the coefficient and the intercept, rounded to 2 decimals, on one line
# TODO: print the R² score of the predictions for x, rounded to 2 decimals
Show hints
- After fitting, the slope is
model.coef_[0]and the constant ismodel.intercept_. r2_score(y, model.predict(x))compares the true values with the predictions.
Show solution
from sklearn.linear_model import LinearRegression
from sklearn.metrics import r2_score
x = [[0], [1], [2], [3]]
y = [5, 8, 11, 14]
model = LinearRegression()
model.fit(x, y)
print(round(float(model.coef_[0]), 2), round(float(model.intercept_), 2))
print(round(float(r2_score(y, model.predict(x))), 2))
Quick quiz
1. What does an R² score of 1.0 mean?
- A. The model explains all the variation in the target
- B. The model is useless
- C. The model is overfitted by definition
- D. There is one feature
Show answer
A. A score of 0 means the model does no better than always predicting the mean.
2. In which units is RMSE expressed?
- A. It has no units
- B. The same units as the target
- C. Percent
- D. The square of the target's units
Show answer
B. This makes it easy to interpret: an RMSE of 5,000 on house prices is an error of about 5,000.
3. What does a coefficient of a linear regression tell you?
- A. The accuracy of the model
- B. The number of rows
- C. How much the prediction changes when that feature increases by one unit, the others staying fixed
- D. The intercept
Show answer
C. This interpretability is one of the main strengths of linear models.
Interview questions
Which metrics are used to evaluate a regression model?
Show answer
Mean absolute error, MAE, is the average size of the errors. Mean squared error, MSE, averages the squared errors, which penalises large mistakes more heavily; its square root, RMSE, is in the units of the target. R² is the share of the variation in the target that the model explains, with 1 being perfect. MAE and RMSE say how far off the predictions are; R² says how much better the model is than predicting the average.
What are the assumptions and limits of linear regression?
Show answer
It assumes that the target is roughly a straight-line combination of the features, that the errors are independent and of similar size across the range, and that the features are not strongly correlated with one another. It is sensitive to outliers and cannot capture curved relationships unless features such as squared terms are added. When these assumptions fail, tree-based models are a common alternative.
Key Points to Remember
- Regression predicts numbers; LinearRegression fits y = Σ wᵢxᵢ + b.
- Evaluate with MAE, RMSE and R² on a test set, compared to a baseline.
- Coefficients explain how each feature moves the prediction.
- PolynomialFeatures handles curves; Ridge/Lasso regularise.
- Tree ensembles are strong non-linear alternatives on tabular data.
Practice in the playground
Write and run your own code for this topic, or start from the exercise or an example. Runs Python in your browser. Your code for each topic is kept in this browser.
Run the examples
Change an input, predict the result, then compare it with the output. Explain why the result changes.
These examples need a project environment on your computer, so they can be copied but not run here. Codelab runs Python, FastAPI backends with SQLite, plain Java 17 and HTML/CSS/JavaScript.
Written by
WebNest Studio Engineering Team
Practical lessons from developers working with Java, Python, React, databases and production systems.
Led by Vansh Duggal, Founder, WebNest Studio · Full-stack & software engineering