M44: Convolutions, pooling, and CNN architectures
CSCI 3151 — Foundations of Machine Learning
Convolution: big picture 🖼️
Convolution layer:
- Slides a small kernel (filter) across the image.
- At each location, computes a weighted sum of pixel values in a local patch.
- Reuses the same weights at all spatial locations (weight sharing).
- Produces a new image-like array: a feature map.
From here
Intuition
- A kernel can act as:
- Edge detector
- Blur / smoothing filter
- Texture detector
- Different kernels detect different patterns; stacking them gives rich feature hierarchies.
Discrete 1D convolution (warm-up)
For 1D signals \(x\) (input) and \(w\) (kernel), the discrete convolution is:
\[ (y = x * w)[i] = \sum_{k} x[i - k] w[k] \]
From here
In deep learning libraries, we often use cross-correlation instead:
\[ (y = x \star w)[i] = \sum_{k} x[i + k] w[k] \]
- Same flavour, just flipped indexing; many libraries call this “convolution.”
2D convolution — definition
For single-channel 2D input \(X \in \mathbb{R}^{H \times W}\) and kernel \(K \in \mathbb{R}^{k_H \times k_W}\), ignoring stride & padding:
\[ Y[i, j] = (X \star K)[i, j] = \sum_{u=0}^{k_H-1} \sum_{v=0}^{k_W-1} K[u, v] \, X[i + u, j + v] \]
- Output \(Y\) has shape \((H - k_H + 1) \times (W - k_W + 1)\).
- With padding and stride, we adjust indices and shapes.
Conv hyperparameters: stride & padding
- Kernel size: \(k_H \times k_W\)
- Stride: how far we move the kernel each step
- E.g., stride = 2 → skip every other pixel
- Padding: how many pixels we add around the border (usually zeros)
For input \(H \times W\), kernel \(k_H \times k_W\), padding \(p_H, p_W\), stride \(s_H, s_W\):
\[ H_{ \text{out}} = \left\lfloor \frac{H + 2p_H - k_H}{s_H} \right\rfloor + 1,\quad W_{ \text{out}} = \left\lfloor \frac{W + 2p_W - k_W}{s_W} \right\rfloor + 1 \]
Channels & feature maps
For multi-channel input (e.g., RGB):
- Input: \(X \in \mathbb{R}^{C_\text{in} \times H \times W}\)
- One conv filter has weights \(K \in \mathbb{R}^{C_\text{in} \times k_H \times k_W}\)
- The output of this single filter (one feature map):
\[ Y[i, j] = \sum_{c=1}^{C_\text{in}} \sum_{u,v} K[c, u, v] \, X[c, i+u, j+v] \]
To produce \(C_\text{out}\) feature maps, we use \(C_\text{out}\) filters:
- Weight tensor: \(W \in \mathbb{R}^{C_\text{out} \times C_\text{in} \times k_H \times k_W}\)
- Output: \(Y \in \mathbb{R}^{C_\text{out}\times H_\text{out} \times W_\text{out}}\)
From here
Applying the kernel
Code
def conv2d_valid(x, k):
H, W = x.shape
kH, kW = k.shape
out = np.zeros((H - kH + 1, W - kW + 1))
for i in range(out.shape[0]):
for j in range(out.shape[1]):
patch = x[i:i+kH, j:j+kW]
out[i, j] = np.sum(patch * k)
return out
edge = conv2d_valid(img, kernel)
edge_df = pd.DataFrame(edge)
print("Our edge-detecting kernel:")
edge_dfOur edge-detecting kernel:| 0 | 1 | 2 | |
|---|---|---|---|
| 0 | 3.0 | 4.0 | 3.0 |
| 1 | 0.0 | 0.0 | 0.0 |
| 2 | -3.0 | -4.0 | -3.0 |
Adding pooling
Code
def max_pool2d(x, pool_size=2, stride=2):
H, W = x.shape
out_H = (H - pool_size) // stride + 1
out_W = (W - pool_size) // stride + 1
out = np.zeros((out_H, out_W))
for i in range(out_H):
for j in range(out_W):
patch = x[i*stride:i*stride+pool_size,
j*stride:j*stride+pool_size]
out[i, j] = patch.max()
return out
pooled = max_pool2d(edge, pool_size=2, stride=1)
pd.DataFrame(pooled)| 0 | 1 | |
|---|---|---|
| 0 | 4.0 | 4.0 |
| 1 | 0.0 | 0.0 |
Code
Note
- Pooling keeps where the edge is in a coarse sense, while reducing resolution.
- ✏️ In your usual tweaks to the code, what happens if you set
stride=2?
Receptive fields
Receptive field of a unit = region of the input it “sees.”
- In a single 3×3 conv layer (stride 1, no pooling):
- Each output pixel depends on a 3×3 patch in the input.
- Stack another 3×3 conv (stride 1):
- Receptive field becomes 5×5.
- Add pooling with stride 2:
- Receptive field grows and the effective spacing between receptive fields increases.
Mini-table example
| Layer | Kernel / stride | Effective RF size |
|---|---|---|
| Conv1 | 3×3 / 1 | 3×3 |
| Conv2 | 3×3 / 1 | 5×5 |
| MaxPool (2×2) | 2×2 / 2 | 6×6 (coarser grid) |
Receptive fields help explain how deep layers can “see” entire objects.
From here
Training curves
Code
epochs = range(1, len(mlp_hist["train_loss"]) + 1)
plt.figure(figsize=(6, 4))
plt.plot(epochs, mlp_hist["train_loss"], label="MLP train")
plt.plot(epochs, mlp_hist["test_loss"], label="MLP test")
plt.plot(epochs, cnn_hist["train_loss"], label="CNN train", linestyle="--")
plt.plot(epochs, cnn_hist["test_loss"], label="CNN test", linestyle="--")
plt.xlabel("Epoch")
plt.ylabel("Cross-entropy loss")
plt.title("MLP vs CNN on CIFAR-10 subset")
plt.legend()
plt.tight_layout()
