CNNs Explained: From Pixels to ResNet and YOLO

ML
Deep Learning
Computer Vision
Why plain neural networks fail on images, how convolution fixes it, the classic architectures from AlexNet to ResNet, and how YOLO finds objects in real time.
Author

MLStation

Published

October 1, 2026

A factory camera photographs every metal part on a conveyor belt and has to sort it into one of four bins: washer, hex nut, plate or bracket. Parts arrive at random angles, in slightly different places, under changing light. A person does this without thinking. In this article we teach a computer to do it, first the naive way, then the way modern vision systems really work.

TipWhat you will learn
  • Why a plain neural network (ANN) on raw pixels breaks as soon as the object moves
  • What a convolution, a filter, ReLU and pooling do, with live demos
  • How AlexNet, VGG16, VGG19, GoogLeNet and ResNet changed computer vision
  • How transfer learning lets you reuse those models on your own images
  • How YOLO goes from “what is this?” to “what is where?” in one pass

Step 1: what the computer sees

To a computer, an image is just a grid of numbers. Our camera produces 32 × 32 grayscale images, so each picture is 1,024 numbers between 0 (black) and 1 (white). We simulate 3,200 of them.

Simulate the factory camera (click to see how) 
import numpy as np, pandas as pd, warnings
warnings.filterwarnings("ignore")
from scipy.ndimage import convolve
from sklearn.neural_network import MLPClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
import plotly.graph_objects as go
from plotly.subplots import make_subplots
from ms_charts import COLORS, PLOTLY_CONFIG

rng = np.random.default_rng(1)
S = 32
yy, xx = np.mgrid[0:S, 0:S]
CLASSES = ["Washer", "Hex nut", "Plate", "Bracket"]

def draw_part(c, cx, cy, r, angle):
    """Return a 32x32 mask of one part centred at (cx, cy), radius r, rotated by angle."""
    dx, dy = xx - cx, yy - cy
    a = np.deg2rad(angle)
    u, v = dx * np.cos(a) + dy * np.sin(a), -dx * np.sin(a) + dy * np.cos(a)
    d = np.hypot(dx, dy)
    if c == 0:                                   # washer: a ring
        m = (d < r) & (d > r * 0.55)
    elif c == 1:                                 # hex nut: hexagon with a hole
        th = np.arctan2(v, u)
        edge = r * np.cos(np.pi / 6) / np.cos((th % (np.pi / 3)) - np.pi / 6)
        m = (d < edge) & (d > r * 0.35)
    elif c == 2:                                 # plate: a square
        m = (np.abs(u) < r * 0.85) & (np.abs(v) < r * 0.85)
    else:                                        # bracket: an L shape
        m = ((np.abs(u) < r * .85) & (v > r * .35) & (v < r * .85)) | ((u > -r * .85) & (u < -r * .35) & (np.abs(v) < r * .85))
    return m.astype(float)

def camera(n, where="centre"):
    X, y = [], []
    for _ in range(n):
        c, r = rng.integers(4), rng.uniform(6, 9)
        if where == "centre":
            cx, cy = rng.uniform(13, 19, 2)                      # belt guide keeps parts near the middle
        else:
            cx, cy = [rng.choice([rng.uniform(9, 11), rng.uniform(21, 23)]) for _ in range(2)]  # guide broke
        img = draw_part(c, cx, cy, r, rng.uniform(0, 360)) * rng.uniform(.6, 1) + rng.uniform(0, .3)
        X.append(np.clip(img + rng.normal(0, .08, (S, S)), 0, 1)); y.append(c)
    return np.array(X), np.array(y)

X_train, y_train = camera(2000)                 # training images, parts near the centre
X_test, y_test = camera(600)                    # new images, same conditions
X_shift, y_shift = camera(600, where="corner")  # new images, parts pushed towards the corners
Show the code 
fig = make_subplots(rows=2, cols=4, subplot_titles=[CLASSES[c] for c in y_train[:8]],
                    horizontal_spacing=0.02, vertical_spacing=0.1)
for i in range(8):
    fig.add_trace(go.Heatmap(z=X_train[i], colorscale="gray", showscale=False, hoverinfo="skip"),
                  row=i // 4 + 1, col=i % 4 + 1)
fig.update_xaxes(visible=False); fig.update_yaxes(visible=False, autorange="reversed", scaleanchor="x")
fig.update_layout(height=380, margin=dict(t=30, b=0, l=0, r=0))
fig.show(config=PLOTLY_CONFIG)
Figure 1: Eight simulated camera images. Same four parts, different angles, sizes and lighting.

Step 2: the naive way, an ANN on raw pixels

The obvious idea: flatten the 32 × 32 grid into a list of 1,024 numbers and feed it to a normal neural network. We train one with two hidden layers.

ann = MLPClassifier(hidden_layer_sizes=(128, 64), max_iter=300,
                    early_stopping=True, random_state=0)
# flatten each 32x32 image into 1,024 numbers
ann.fit(X_train.reshape(len(X_train), -1), y_train)

acc_ann_same = ann.score(X_test.reshape(len(X_test), -1), y_test)
acc_ann_shift = ann.score(X_shift.reshape(len(X_shift), -1), y_shift)

On new images taken under the same conditions it scores 97%. Impressive. Then the belt guide breaks and parts start arriving near the corners. Accuracy drops to 22%, which is close to random guessing among four classes (25%).

WarningWhy the ANN breaks

When you flatten an image, pixel 300 is just “input number 300”. The network learns that a washer means these particular inputs are bright. Move the washer a few pixels and completely different inputs light up, so the network sees something it has never met. It also has no idea that pixel 300 and pixel 301 are neighbours. An ANN is blind to space.

It is also wasteful. A colour photo of 224 × 224 pixels has 150,528 inputs. Connecting those to just 256 hidden neurons already needs 38,535,168 weights.

Step 3: convolution, a small window that slides

A convolutional neural network (CNN) looks at the image through a small window, typically 3 × 3 pixels, called a filter or kernel. The filter holds 9 weights. It slides across the whole image and, at every position, multiplies the 9 pixels under it by its 9 weights and adds them up:

\text{out}(i,j) = \sum_{m=-1}^{1}\sum_{n=-1}^{1} w_{m,n}\; x_{i+m,\,j+n} + b \tag{1}

The result is a new image called a feature map, which is bright wherever the pattern the filter is looking for appears. The key ideas:

  • The same 9 weights are used everywhere (weight sharing). A filter that detects a vertical edge in the top-left corner detects it in the bottom-right too.
  • Neighbours stay neighbours. The filter always looks at pixels that touch each other.
  • In a CNN the filter weights are learned, exactly like the weights in our earlier neural network article. Below we use classic hand-made filters so you can see what they do.

Pick a filter and move the window with the sliders. The left panel is the camera image, the right panel is the feature map.

Notice how the vertical-edge filter lights up the left and right sides of the bracket in opposite colours, and stays near zero on flat areas. Turn on ReLU (\max(0, z)) and only the positive responses survive, exactly like in a normal neural network.

Step 4: pooling, and why it gives tolerance to movement

After convolution and ReLU, a CNN usually pools: it divides the feature map into small blocks (often 2 × 2) and keeps only the largest value in each. This:

  • shrinks the map, so later layers have less to process,
  • keeps the strongest evidence (“there is an edge somewhere here”),
  • makes the network care less about the exact position of a pattern.

Repeat convolution → ReLU → pooling a few times, and early layers find edges, middle layers combine edges into corners and curves, and deep layers respond to whole parts like “ring” or “L shape”.

flowchart LR
  A["Image<br/>32×32"] --> B["Conv + ReLU<br/>edges"] --> C["Pool<br/>16×16"] --> D["Conv + ReLU<br/>corners, curves"] --> E["Pool<br/>8×8"] --> F["Dense layer"] --> G["Washer / Nut /<br/>Plate / Bracket"]

Step 5: does it help? A mini-CNN test

Real CNNs learn their filters with frameworks like Keras or PyTorch. To keep this page light, we build a mini-CNN with eleven fixed filters (the edge, outline and blur filters above, in both polarities), apply ReLU and max pooling, summarise each feature map, and train a simple classifier on top. Then we run the broken-belt test again.

Mini-CNN: fixed filters + ReLU + pooling + classifier 
BASE = [np.array(k, float) for k in (
    [[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]],      # vertical edge
    [[-1, -2, -1], [0, 0, 0], [1, 2, 1]],      # horizontal edge
    [[0, 1, 2], [-1, 0, 1], [-2, -1, 0]],      # diagonal edge
    [[2, 1, 0], [1, 0, -1], [0, -1, -2]],      # other diagonal
    [[0, 1, 0], [1, -4, 1], [0, 1, 0]])]       # outline (Laplacian)
FILTERS = BASE + [-k for k in BASE] + [np.ones((3, 3)) / 9]   # both polarities + blur = 11 filters

def mini_cnn_features(X):
    out = []
    for img in X:
        f = []
        for k in FILTERS:
            fmap = np.maximum(convolve(img, k, mode="nearest"), 0)        # convolution + ReLU
            pooled = fmap.reshape(8, 4, 8, 4).max(axis=(1, 3))            # 4x4 max pooling -> 8x8
            hist = np.histogram(fmap, bins=6, range=(0, fmap.max() + 1e-6))[0] / fmap.size
            f += [pooled.mean(), pooled.max(), (pooled > pooled.mean()).mean(), pooled.std(), *hist]
        out.append(f)
    return np.array(out)

F_train, F_test, F_shift = map(mini_cnn_features, (X_train, X_test, X_shift))
scaler = StandardScaler().fit(F_train)
head = LogisticRegression(max_iter=3000).fit(scaler.transform(F_train), y_train)
acc_cnn_same = head.score(scaler.transform(F_test), y_test)
acc_cnn_shift = head.score(scaler.transform(F_shift), y_shift)
Show the code 
fig = go.Figure()
for name, vals, c in [("ANN on raw pixels", [acc_ann_same, acc_ann_shift], COLORS["slate"]),
                      ("Mini-CNN", [acc_cnn_same, acc_cnn_shift], COLORS["indigo"])]:
    fig.add_bar(x=["Same conditions", "Parts moved to corners"], y=vals, name=name, marker_color=c,
                text=[f"{v:.0%}" for v in vals], textposition="outside")
fig.add_hline(y=0.25, line_dash="dot", line_color="#8A93A3")
fig.update_layout(barmode="group", height=360, yaxis=dict(tickformat=".0%", range=[0, 1.12], title="Accuracy"))
fig.show(config=PLOTLY_CONFIG)
Figure 2: The ANN memorises pixel positions and collapses when parts move. Convolution and pooling look for patterns wherever they are. Dotted line: random guessing (25%).

The mini-CNN scores 95% in normal conditions and 97% after the parts move. The ANN fell to 22%. That robustness to position, with far fewer weights, is the whole reason CNNs took over computer vision.

A real, trainable CNN in Keras

This is how you would build a CNN that learns its own filters. The code is shown, not run, so the page builds without TensorFlow installed.

cnn_keras.py
from tensorflow import keras
from tensorflow.keras import layers

model = keras.Sequential([
    keras.Input(shape=(32, 32, 1)),
    layers.Conv2D(16, 3, activation="relu", padding="same"),  # 16 filters
    layers.MaxPooling2D(2),                                     # -> 16x16
    layers.Conv2D(32, 3, activation="relu", padding="same"),  # 32 filters
    layers.MaxPooling2D(2),                                     # -> 8x8
    layers.GlobalAveragePooling2D(),                            # 1 per filter
    layers.Dense(4, activation="softmax"),                      # 4 parts
])
model.compile(optimizer="adam",
              loss="sparse_categorical_crossentropy",
              metrics=["accuracy"])
model.fit(X_train[..., None], y_train, validation_split=0.2, epochs=15)

A convolution layer has (k \times k \times C_{in} + 1) \times C_{out} parameters, where k is the filter size and C the number of channels:

Model Parameters
ANN on flattened pixels (1024 → 128 → 64 → 4) 139,716
Small CNN above (Conv 16, Conv 32, Dense 4) 4,932

About 28× fewer weights, because each filter’s few weights are reused at every position.

Step 6: the famous architectures

Between 2012 and 2015, the yearly ImageNet challenge (1.2 million photos, 1,000 classes) turned CNNs from a research idea into the standard. The headline metric is top-5 error: how often the right answer is not among the model’s five best guesses.

Table 1: Milestones in image classification
Year Model Big idea ImageNet top-5 error
2012 AlexNet First deep CNN trained on GPUs; ReLU and dropout at scale 15.3%, won by over 10 points (Wikipedia)
2014 VGG16 / VGG19 Only 3 × 3 filters, stacked deep (16 or 19 weight layers) 7.3%, 2nd place (AI Wiki)
2014 GoogLeNet (Inception) Parallel filters of several sizes; about 12× fewer parameters than AlexNet 6.67%, 1st place (Szegedy et al.)
2015 ResNet Skip connections make 152-layer networks trainable 3.57%, 1st place (He et al.)
2017 to 2019 MobileNet, EfficientNet Small and fast enough for phones and edge devices
2020 onward Vision Transformer, ConvNeXt Attention-based and modernised CNN designs

VGG16 and VGG19: simple and deep

VGG’s recipe is easy to remember: blocks of 3 × 3 convolutions, then a 2 × 2 max pool, with the number of filters doubling each block (64, 128, 256, 512, 512), and three large dense layers at the end. VGG16 has 13 convolution layers + 3 dense; VGG19 has 16 + 3. Let us count the parameters from the recipe:

Show the code 
def vgg_params(convs_per_block):
    rows, c_in = [], 3
    for b, (n, c_out) in enumerate(zip(convs_per_block, [64, 128, 256, 512, 512]), 1):
        p = 0
        for _ in range(n):
            p += (3 * 3 * c_in + 1) * c_out; c_in = c_out
        rows.append((f"Conv block {b}", p))
    rows += [("Dense 4096 (fc6)", (7 * 7 * 512 + 1) * 4096), ("Dense 4096 (fc7)", (4096 + 1) * 4096),
             ("Dense 1000 (fc8)", (4096 + 1) * 1000)]
    return pd.DataFrame(rows, columns=["layer", "params"])

vgg16, vgg19 = vgg_params([2, 2, 3, 3, 3]), vgg_params([2, 2, 4, 4, 4])
fig = go.Figure(go.Bar(x=vgg16.params / 1e6, y=vgg16.layer, orientation="h",
                       marker_color=[COLORS["teal"]] * 5 + [COLORS["indigo"]] * 3,
                       text=[f"{p / 1e6:.1f}M" for p in vgg16.params], textposition="outside"))
fig.update_layout(height=360, xaxis_title="Parameters (millions)", yaxis=dict(autorange="reversed"), margin=dict(l=130))
fig.show(config=PLOTLY_CONFIG)
Figure 3: Where VGG16’s parameters live. The convolution blocks do the seeing; the dense layers at the end hold most of the weights.

Total: 138,357,544 parameters for VGG16 and 143,667,240 for VGG19, matching the published 138.4M and 143.7M (Keras Applications). The first dense layer alone holds 74% of VGG16. Later designs replaced those giant dense layers with global pooling.

ResNet: the shortcut that made depth work

Stacking more layers should help, but before 2015 very deep plain networks trained worse. The learning signal (the gradient) is multiplied layer after layer on its way back to the start, and it fades away. ResNet adds a skip connection: each block learns only a correction \mathcal{F}(x) and adds the input back (He et al.):

y = \mathcal{F}(x) + x \tag{2}

Because of the “+ x”, the signal always has a direct path backwards. This toy model shows the effect. Each plain layer passes back a fraction of the signal; each residual block passes back the same fraction plus the shortcut.

With 50 layers at 85%, the plain network keeps about of its learning signal; the early layers barely learn. The residual network keeps most of it. This is why ResNet-50 and ResNet-152 are still standard backbones a decade later.

Accuracy vs size of ready-made models

All of these come pre-trained on ImageNet in Keras. Hover a bubble for details.

Show the code 
zoo = pd.DataFrame([
    ("VGG16", 138.4, 71.3, 528), ("VGG19", 143.7, 71.3, 549), ("ResNet50", 25.6, 74.9, 98),
    ("ResNet152", 60.4, 76.6, 232), ("InceptionV3", 23.9, 77.9, 92), ("MobileNetV2", 3.5, 71.3, 14),
    ("EfficientNetB0", 5.3, 77.1, 29), ("ConvNeXtTiny", 28.6, 81.3, 109)],
    columns=["model", "params_m", "top1", "size_mb"])
fig = go.Figure(go.Scatter(
    x=zoo.params_m, y=zoo.top1, mode="markers+text", text=zoo.model,
    textposition=["bottom center", "top center", "bottom center", "top center", "top center", "top center", "top center", "top center"],
    textfont=dict(size=12),
    marker=dict(size=np.sqrt(zoo.params_m) * 4 + 8, color=COLORS["indigo"], opacity=.7, line=dict(color="white", width=1)),
    customdata=zoo[["size_mb"]], hovertemplate="%{text}<br>%{x}M parameters<br>Top-1 %{y}%<br>%{customdata[0]} MB<extra></extra>"))
fig.update_layout(height=420, xaxis=dict(type="log", title="Parameters (millions, log scale)", tickvals=[3, 5, 10, 25, 50, 100, 150], range=[0.35, 2.35]),
                  yaxis=dict(title="Top-1 accuracy (%)", range=[69, 83]))
fig.show(config=PLOTLY_CONFIG)
Figure 4: Top-1 ImageNet accuracy against size for pre-trained Keras models. Bubble size shows parameter count. Data from Keras Applications (keras.io/api/applications).

Bigger is not better: VGG19 has 40 times the parameters of MobileNetV2 for the same accuracy, and newer designs like EfficientNet and ConvNeXt beat both.

Step 7: transfer learning, the practical shortcut

You rarely train a large CNN from scratch. Instead you take a network pre-trained on ImageNet, keep its convolution layers (which already know edges, textures and shapes), and train only a small new head for your classes. A few hundred images per class are often enough.

transfer_resnet.py
from tensorflow import keras
base = keras.applications.ResNet50(
    weights="imagenet", include_top=False,
    input_shape=(224, 224, 3), pooling="avg")
base.trainable = False                      # keep the learned filters

model = keras.Sequential([
    base,
    keras.layers.Dense(4, activation="softmax"),   # new 4-class head
])
model.compile("adam", "sparse_categorical_crossentropy",
              metrics=["accuracy"])
transfer_vgg.py
from tensorflow import keras
base = keras.applications.VGG16(
    weights="imagenet", include_top=False,
    input_shape=(224, 224, 3), pooling="avg")
base.trainable = False

model = keras.Sequential([
    base,
    keras.layers.Dense(4, activation="softmax"),
])
transfer_torch.py
import torch, torchvision
model = torchvision.models.resnet50(weights="IMAGENET1K_V2")
for p in model.parameters():
    p.requires_grad = False                             # freeze the backbone
model.fc = torch.nn.Linear(model.fc.in_features, 4)     # new 4-class head

Step 8: from “what” to “what and where”: YOLO

Classification answers what is in the picture. A real factory belt carries several parts at once, so we need object detection: a box around every part, with a label and a confidence score.

Older detectors worked in two stages: propose thousands of candidate regions, then classify each one. YOLO (You Only Look Once) does it in a single pass of one CNN, which made it the first truly real-time detector at 45 frames per second (Redmon et al., 2016). The idea:

  1. Divide the image into a grid.
  2. Each grid cell predicts a few boxes (centre, width, height), a confidence and a class.
  3. Many boxes overlap the same object, so keep the most confident one and delete boxes that overlap it too much. This clean-up is non-maximum suppression (NMS).

Overlap is measured with Intersection over Union:

\text{IoU} = \frac{\text{area of overlap}}{\text{area of union}} \tag{3}

Try it. The detector has proposed many boxes on a belt with three parts. Raise the confidence threshold to drop weak guesses, and lower the NMS IoU threshold to remove duplicates more aggressively.

With sensible settings you end with exactly three boxes, one per part. Set the confidence threshold too low and false alarms appear; set the NMS threshold near 1 and duplicates survive.

Where detection is today

YOLO has been refined many times. The current Ultralytics release, YOLO26 (January 2026), is designed for edge devices, offers NMS-free end-to-end prediction by default, and covers detection, segmentation, classification, pose, depth and oriented boxes in one family (Ultralytics docs). Other notable approaches include two-stage detectors such as Faster R-CNN (accurate, slower) and transformer-based detectors such as DETR and RT-DETR, which treat detection as a set-prediction problem without hand-made anchors.

detect_parts.py
from ultralytics import YOLO

model = YOLO("yolo26n.pt")              # small pre-trained model
results = model("conveyor_belt.jpg")    # boxes, classes, confidences

# fine-tune on your own labelled images
model.train(data="factory_parts.yaml", epochs=50, imgsz=640)

The big picture

flowchart LR
  A["ANN on pixels<br/>blind to space"] --> B["CNN<br/>filters + pooling"] --> C["Deep CNNs<br/>AlexNet, VGG, ResNet"] --> D["Transfer learning<br/>reuse backbones"] --> E["Detection<br/>YOLO, DETR"]

Common pitfalls

Pitfall Fix
Flattening images into an ANN Use convolution layers, or a pre-trained CNN
Training only on tidy, centred images Augment: random shifts, rotations, flips, brightness
Training a huge model from scratch on little data Transfer learning from an ImageNet backbone
Forgetting to scale pixels Divide by 255, or use the backbone’s own preprocessing
Judging a detector by accuracy alone Use mean average precision (mAP) and check IoU thresholds

Summary

  • An image is a grid of numbers. An ANN on flattened pixels memorises positions and breaks when objects move.
  • A CNN slides small learned filters over the image (Equation 1), reuses the same weights everywhere, and pools to tolerate movement.
  • AlexNet started the deep learning era, VGG16/19 showed that stacked 3 × 3 filters work, and ResNet’s skip connections (Equation 2) made very deep networks trainable.
  • Transfer learning lets you reuse these networks on your own images with little data.
  • YOLO detects many objects in one pass using a grid, confidence scores and IoU (Equation 3); modern versions such as YOLO26 drop the NMS step entirely.

For the theory behind convolutional networks, see Goodfellow et al. (2016).

Back to top

References

Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. https://www.deeplearningbook.org.