Logistic Regression Explained

ML
Supervised
How logistic regression turns a linear score into a probability, how it is trained, and how to read its decision boundary.
Author

MLStation

Published

September 27, 2026

Logistic regression is the go-to model for yes/no questions: will a customer churn, is an email spam, will a loan default. It is simple, fast and easy to explain, which is why it remains a strong baseline.

TipWhat you will learn
  • How the sigmoid turns any score into a probability
  • How the model is trained with log-loss
  • How to read the decision boundary

Intuition

A linear model gives a score z that can be any number. We need a probability between 0 and 1. The sigmoid function squashes the score into that range.

The math

The model computes a linear score and passes it through the sigmoid:

z = \mathbf{w}^\top \mathbf{x} + b, \qquad \sigma(z) = \frac{1}{1 + e^{-z}} \tag{1}

Training minimises the log-loss (binary cross-entropy):

\mathcal{L}(\mathbf{w}, b) = -\frac{1}{n}\sum_{i=1}^{n}\Big[y_i \log \hat{p}_i + (1-y_i)\log(1-\hat{p}_i)\Big] \tag{2}

where \hat{p}_i = \sigma(z_i) from Equation 1.

Definition 1 (Decision boundary) The set of points where \hat{p} = 0.5, which is exactly where z = 0. For two features it is a straight line.

Try it yourself

Move the slider to change the weight w. A larger weight makes the model more confident.

The sigmoid, interactively

Hover over the curve to read exact probabilities. Drag to zoom and double-click to reset.

Show the code 
import numpy as np
import plotly.graph_objects as go
from ms_charts import COLORS, PLOTLY_CONFIG   # also turns on the site theme

z = np.linspace(-8, 8, 300)
fig = go.Figure()
fig.add_scatter(x=z, y=1 / (1 + np.exp(-z)), mode="lines", name="σ(z)",
                line=dict(color=COLORS["indigo"], width=3),
                hovertemplate="z = %{x:.2f}<br>p = %{y:.3f}<extra></extra>")
fig.add_hline(y=0.5, line_dash="dash", line_color="#8A93A3", opacity=0.6)
fig.update_layout(height=360, xaxis_title="Score z", yaxis_title="Probability",
                  yaxis_range=[-0.03, 1.03], showlegend=False)
fig.show(config=PLOTLY_CONFIG)
Figure 1: The sigmoid maps any score to a probability between 0 and 1.

As Figure 1 shows, large positive scores give probabilities near 1.

A real classifier

We train a model on two features and colour the plane by predicted probability. Hover any point to see its class.

Show the code 
import numpy as np
import plotly.graph_objects as go
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from ms_charts import COLORS, PLOTLY_CONFIG

X, y = make_classification(n_samples=300, n_features=2, n_redundant=0,
                           n_clusters_per_class=1, class_sep=1.2, random_state=4)
model = LogisticRegression().fit(X, y)

g1 = np.linspace(X[:, 0].min() - .5, X[:, 0].max() + .5, 150)
g2 = np.linspace(X[:, 1].min() - .5, X[:, 1].max() + .5, 150)
G1, G2 = np.meshgrid(g1, g2)
P = model.predict_proba(np.c_[G1.ravel(), G2.ravel()])[:, 1].reshape(G1.shape)

fig = go.Figure()
fig.add_contour(x=g1, y=g2, z=P, colorscale=[[0, COLORS["teal"]], [0.5, "rgba(138,147,163,0.08)"], [1, COLORS["indigo"]]],
                opacity=0.45, contours=dict(start=0, end=1, size=0.1), line_width=0,
                colorbar=dict(title="p(y=1)", thickness=12, len=0.8), hoverinfo="skip")
fig.add_contour(x=g1, y=g2, z=P, contours=dict(start=0.5, end=0.5, coloring="none"),
                line=dict(color="#8A93A3", dash="dash", width=2), showscale=False, hoverinfo="skip",
                name="Decision boundary", showlegend=True)
for cls, color, name in [(0, COLORS["teal"], "Class 0"), (1, COLORS["indigo"], "Class 1")]:
    m = y == cls
    fig.add_scatter(x=X[m, 0], y=X[m, 1], mode="markers", name=name,
                    marker=dict(color=color, size=7, line=dict(color="white", width=0.8)))
fig.update_layout(height=440, xaxis_title="Feature 1", yaxis_title="Feature 2")
fig.show(config=PLOTLY_CONFIG)
Figure 2: Predicted probability across the feature space. The dashed line is the 0.5 decision boundary.

The code

Training a model takes three lines with scikit-learn:

train.py
from sklearn.linear_model import LogisticRegression

model = LogisticRegression().fit(X_train, y_train)
probs = model.predict_proba(X_test)[:, 1]      # probability of class 1

The same idea in other languages:

from sklearn.linear_model import LogisticRegression
LogisticRegression().fit(X, y)
glm(churn ~ tenure + spend, data = df, family = binomial)
-- Scoring a trained model inside the warehouse
SELECT customer_id,
       1 / (1 + EXP(-(0.8 * tenure - 0.03 * spend + 1.2))) AS churn_probability
FROM customers;

Metrics to report

Table 1: Common classification metrics
Metric What it tells you
Accuracy Share of correct predictions, misleading when classes are imbalanced
Precision / Recall Quality of positive predictions and share of positives found
ROC AUC Ranking quality across all thresholds
Log-loss How well calibrated the probabilities are
WarningCommon mistake

Always scale features (for example with StandardScaler) before fitting with regularisation, otherwise features with large units dominate the penalty.

Summary

  • Logistic regression is a linear score passed through the sigmoid.
  • It is trained by minimising log-loss (Equation 2).
  • The decision boundary is where z = 0, a straight line in feature space.

For a rigorous treatment, see Hastie et al. (2009).

Back to top

References

Hastie, Trevor, Robert Tibshirani, and Jerome Friedman. 2009. The Elements of Statistical Learning. 2nd ed. Springer. https://hastie.su.domains/ElemStatLearn/.