Descriptive Statistics: Data Have Shape

Statistics
Descriptive Statistics
Data Science
Build intuition for center, spread, skew, outliers and robust summaries—and see why EDA is the first statistical model check.
Author

MLStation

Published

October 1, 2026

Before fitting a model, ask a simpler question:

What kind of data do I actually have?

A mean without a distribution can be misleading. A standard deviation without a plot can hide outliers. Descriptive statistics are not bookkeeping—they are the first layer of model diagnostics.

TipWhat you will learn
  • Mean, median, quantiles, variance, standard deviation and IQR
  • Why the same mean can describe very different datasets
  • How skew and outliers change summaries
  • When robust statistics are preferable
  • How EDA connects directly to ML preprocessing and model choice

One number cannot describe a dataset

For observations x_1,\ldots,x_n,

\bar{x}=\frac{1}{n}\sum_{i=1}^n x_i

is the sample mean, while

s^2=\frac{1}{n-1}\sum_{i=1}^n(x_i-\bar{x})^2

measures sample variance.

But center and spread are only coordinates on a larger map:

DATA
 ├── Center      → mean, median
 ├── Spread      → SD, IQR, range
 ├── Shape       → skew, tails, modes
 └── Exceptions  → outliers, missingness

Interactive: let one outlier pull the mean

Move the outlier. Watch the mean chase it while the median resists.

NoteRobustness

The median and IQR are resistant to extreme observations. The mean and standard deviation are much more sensitive. That is not a defect—the statistics answer different questions.

Simulation: four datasets, similar summaries

Show the code 
import numpy as np
import plotly.express as px
import pandas as pd

rng = np.random.default_rng(7)
n = 500
df = pd.DataFrame({
    "Normal": rng.normal(0,1,n),
    "Skewed": rng.exponential(1,n)-1,
    "Heavy tail": rng.standard_t(3,n)/np.sqrt(3),
    "Mixture": np.r_[rng.normal(-1,.45,n//2), rng.normal(1,.45,n//2)]
}).melt(var_name="Distribution", value_name="x")

fig = px.histogram(df, x="x", facet_col="Distribution", nbins=45,
                   histnorm="probability density", animation_frame=None)
fig.update_layout(height=360, showlegend=False)
fig.for_each_annotation(lambda a: a.update(text=a.text.split("=")[-1]))
fig.show()

Different shapes can share similar first-order summaries.

DS/ML Bridge: why EDA changes modeling decisions

Statistical clue Data-science consequence
Strong skew Transform feature, robust model, or suitable distribution
Large outliers Investigate data quality; consider robust loss/scaling
Multiple modes Hidden subgroups or mixture behavior
Very different scales Scaling may matter for distance/gradient methods
Missingness pattern Missingness may itself contain information
WarningCommon mistake

Do not automatically delete an outlier because a boxplot marks it. An outlier can be an error, a rare valid case, or the most important observation in the dataset.

Save the analysis

summary = df.groupby("Distribution")["x"].agg(
    ["count","mean","median","std","min","max"]
)
summary.to_csv("eda_summary.csv")

Save both summary statistics and plots/configuration. EDA is part of the reproducible analysis, not disposable scratch work.

Summary

  • Center, spread and shape describe different properties.
  • Mean/SD are sensitive; median/IQR are robust.
  • Visualization prevents summaries from hiding structure.
  • EDA often determines preprocessing, feature engineering and model assumptions.

Next: Probability by Simulation — Learning Chance by Repetition

Back to top