Correlation, covariance and linear regression

Covariance and correlation

Covariance measures whether two variables move together: positive when above-average values of X go with above-average values of Y. Its size depends on units, so it is hard to interpret.

Pearson correlation r rescales covariance to −1 … +1:

r = cov(X, Y) ÷ (SD_X × SD_Y)

Worked example by hand

Marketing spend (k USD) and new leads over 5 months:

x̄ = 14, ȳ = 380. Σ products = 840 → sample covariance = 840 ÷ 4 = 210. SD_X = √(Σ(X−x̄)² ÷ 4) = √(40 ÷ 4) = √10 ≈ 3.162; SD_Y = √(Σ(Y−ȳ)² ÷ 4) = √(17,400 ÷ 4) = √4,350 ≈ 65.95. r = 210 ÷ (3.162 × 65.95) ≈ 0.995 — an almost perfectly linear relationship in this small example.

Simple linear regression

Regression fits the line ŷ = b₀ + b₁x that minimises the sum of squared residuals (ordinary least squares, OLS).

• Slope b₁ = Σ(X−x̄)(Y−ȳ) ÷ Σ(X−x̄)² = 840 ÷ 40 = 21 → each extra 1k of spend is associated with 21 more leads.
• Intercept b₀ = ȳ − b₁x̄ = 380 − 21 × 14 = 86 → the model's predicted leads at zero spend. Interpret intercepts cautiously: zero spend lies outside the observed data (10–18k), so 86 is an extrapolation.
• Prediction at 15k: 86 + 21 × 15 = 401 leads.
• Residual = actual − predicted. Month 3: 370 − (86 + 21 × 14) = 370 − 380 = −10.

R²

R² = share of the variation in Y explained by the model = 1 − SS_residual ÷ SS_total. With one predictor, R² = r² ≈ 0.995² ≈ 0.99. R² does not tell you whether the relationship is causal, or whether the line is the right shape.

Real data: does spend predict monthly revenue?

import pandas as pd, numpy as np
pnl = pd.read_csv("finance/pnl_monthly.csv")
wide = pnl.pivot_table(index="year_month", columns="line_item", values="actual")
x, y = wide["Marketing"] / 1000, wide["Revenue"] / 1000
r = np.corrcoef(x, y)[0, 1]
b1, b0 = np.polyfit(x, y, 1)
print(f"r = {r:.2f}, slope = {b1:.1f}k revenue per 1k marketing, intercept = {b0:.0f}k")
resid = y - (b0 + b1 * x)
print("residual SD (k):", round(resid.std(), 1))
print(wide.assign(resid=resid.round(0))[["Marketing", "Revenue"]].join(resid.round(0).rename("resid_k")).tail(3))

r = 0.73, slope = 11.8k revenue per 1k marketing, intercept = 719k
residual SD (k): 186.1
            Marketing     Revenue  resid_k
year_month                                
2025-10      73692.45  1558065.59    -28.0
2025-11      78774.32  1922058.27    276.0
2025-12      92780.31  2362393.42    551.0

There is a positive correlation — but in this dataset marketing spend is set as a percentage of revenue (it is budgeted from expected sales). The arrow runs from revenue to spend, not only the other way. The regression line describes an association; it cannot tell you what an extra 1k of marketing would cause. This is the single most important caveat in business regression and the topic of the causality lesson.

Multiple regression (preview)

With several predictors, ŷ = b₀ + b₁x₁ + b₂x₂ + …, each coefficient is the expected change in Y for a one-unit change in that X holding the other Xs constant. That "holding constant" is why adding a confounder can flip a coefficient's sign.

Model assumptions (for valid inference)

Monthly time series often violate independence (this month resembles last month) and contain seasonality — so standard errors from simple regression on time series are usually too optimistic.

Practice

Summary

• Covariance shows direction; correlation r (−1…+1) shows strength of a linear relationship.
• Regression slope = expected change in Y per unit of X; intercept is often an extrapolation; residual = actual − predicted.
• R² measures fit, not causality or correctness of shape.
• Check assumptions with residual plots; time series often break independence.