Descriptive statistics — centre, spread, shape and outliers

Why describe before you analyse

A single average is the most misused number in business. "Average order value is 90" tells you nothing about whether most orders are around 90, or whether a few huge orders hide many tiny ones. Descriptive statistics summarise a whole distribution.

Measures of centre — by hand first

Seven employees' monthly bonuses (in USD): 200, 250, 250, 300, 350, 400, 1,800.

The mean (507) is higher than six of the seven bonuses because one value (1,800) pulls it up. The median is robust to extreme values. For an even number of values, the median is the average of the two middle values.

Spread: variance and standard deviation

Variance is the average squared distance from the mean; standard deviation (SD) is its square root, back in the original units.

Sample variance: s² = Σ(xᵢ − x̄)² ÷ (n − 1). Using n − 1 (Bessel's correction) for a sample gives an unbiased estimate; divide by n for a full population. Excel: STDEV.S (sample) vs STDEV.P (population).

Worked example with daily orders at a small store over 5 days: 8, 10, 12, 9, 11.

1. Mean = 50 ÷ 5 = 10
2. Deviations: −2, 0, 2, −1, 1
3. Squared: 4, 0, 4, 1, 1 → sum = 10
4. Sample variance = 10 ÷ (5 − 1) = 2.5
5. SD = √2.5 ≈ 1.58 orders

Interpretation: on a typical day the store's orders are about 1.6 away from the average of 10.

Real data: order values at Orion Retail

import pandas as pd, numpy as np
orders = pd.read_csv("retail/orders.csv", parse_dates=["order_date"])
items = pd.read_csv("retail/order_items.csv")
items["rev"] = items["quantity"] * items["unit_price"] * (1 - items["discount_pct"])
order_value = (items.groupby("order_id")["rev"].sum()
               .reindex(orders.loc[(orders.status == "completed") & (orders.order_date.dt.year == 2025), "order_id"]))
s = order_value
print("n =", len(s))
print("mean   ", round(s.mean(), 2))
print("median ", round(s.median(), 2))
print("std    ", round(s.std(), 2))
print("min/max", round(s.min(), 2), round(s.max(), 2))
print("skew   ", round(s.skew(), 2))
print(s.quantile([0.1, 0.25, 0.5, 0.75, 0.9, 0.99]).round(2).to_string())

n = 12713
mean    90.03
median  35.96
std     149.55
min/max 1.92 1866.23
skew    3.99
0.10      5.50
0.25     12.96
0.50     35.96
0.75     94.96
0.90    244.35
0.99    753.39

Reading it: the mean is well above the median and skewness is strongly positive — a right-skewed distribution: most orders are small, a minority of large orders (electronics) stretch the tail. The SD is larger than the median, so "average ± 1 SD" would include negative order values — a sign that SD-based summaries are inappropriate for this variable. Percentiles describe it better: half of orders are below the median, 90% below the 90th percentile.

Distributions and the normal distribution

A distribution describes how often each value occurs (histogram). The normal distribution is symmetric and bell-shaped, fully described by its mean μ and SD σ:

• ≈ 68% of values lie within μ ± 1σ
• ≈ 95% within μ ± 2σ (precisely 1.96σ)
• ≈ 99.7% within μ ± 3σ

Many measurement errors and averages of many observations are approximately normal (central limit theorem — next lessons). Many business variables (revenue per customer, claim sizes, time-to-convert) are not normal — they are right-skewed and bounded at zero.

Skewness: positive (right tail longer, mean > median), negative (left tail longer, mean < median), ≈ 0 for symmetric data.

Percentiles and quartiles

The p-th percentile is the value below which p% of observations fall. Quartiles: Q1 = 25th, Q2 = median, Q3 = 75th percentile. Interquartile range (IQR) = Q3 − Q1 — the spread of the middle 50%, robust to outliers.

Business use: service levels ("90% of tickets resolved within X hours" = 90th percentile), salary bands, delivery promises.

Outliers

Two standard rules:

1. IQR rule (robust, works for skewed data): outlier if below Q1 − 1.5·IQR or above Q3 + 1.5·IQR.
2. Z-score (assumes roughly normal data): z = (x − mean) ÷ SD; |z| > 3 is unusual.

Back to the bonuses (200, 250, 250, 300, 350, 400, 1800): Q1 = 250, Q3 = 375 (using linear interpolation), IQR = 125; upper fence = 375 + 1.5 × 125 = 562.5 → 1,800 is an outlier.

bonus = pd.Series([200, 250, 250, 300, 350, 400, 1800])
q1, q3 = bonus.quantile([0.25, 0.75])
iqr = q3 - q1
print("Q1", q1, "Q3", q3, "IQR", iqr, "upper fence", q3 + 1.5 * iqr)
z = (bonus - bonus.mean()) / bonus.std()
print(z.round(2).tolist())

fence = order_value.quantile(0.75) + 1.5 * (order_value.quantile(0.75) - order_value.quantile(0.25))
print("order value IQR fence:", round(fence, 2), "| orders above:", int((order_value > fence).sum()), f"({(order_value > fence).mean():.1%})")

Q1 250.0 Q3 375.0 IQR 125.0 upper fence 562.5
[-0.54, -0.45, -0.45, -0.36, -0.27, -0.19, 2.25]
order value IQR fence: 217.96 | orders above: 1403 (11.0%)

Notice the 1,800 bonus has z ≈ 2.25 — not above 3 — because the outlier itself inflates the SD it is measured against. With small samples or skewed data, the IQR rule is more reliable.

Business application

Customer service promises "most tickets resolved within a day". The telecom data shows resolution hours are right-skewed: the right KPI is the median and 90th percentile resolution time, not the mean — the mean is driven by a few very long cases that need separate investigation.

SELECT category,
       COUNT(*) AS tickets,
       ROUND(AVG(resolution_hours), 1) AS mean_hours,
       MAX(resolution_hours) AS max_hours
FROM support_tickets
GROUP BY category
ORDER BY mean_hours DESC;

Practice

Summary

• Mean is sensitive to extremes; median is robust; mode is the most frequent value.
• Variance/SD measure spread; use n − 1 for samples; SD is in original units.
• Many business variables are right-skewed — describe them with medians and percentiles.
• Detect outliers with the IQR rule (robust) or z-scores (normal data) — investigate, don't auto-delete.