--- config: theme: 'neutral' look: 'handDrawn' --- flowchart LR W["<b>Noisy world</b>"] --> D["<b>Data</b><br>recorded values"] D --> I["<b>Information</b><br>organized patterns"] I --> E["<b>Evidence</b><br>claim + comparison +<br>uncertainty"] E --> A["<b>Action</b><br>a decision someone<br> can inspect"]
AI Readiness Arcade
Rules vs learning
Gate-machine example
- Rule-based: “If the bronze casing is hot and the pressure needle trembles, mark the gate as unsafe.”
- ML-based: “Here are 2112 gate-machine records: temperature, pressure, vibration, age, and whether the gate later failed. Learn a pattern.”
The archive mostly contains calm summer readings from well-maintained inner gates. What might go wrong when we use the model on weathered outer gates in winter?
Vector = one measurement bundle
A vector is an ordered list of measurements. Its arithmetic only makes sense if each entry has a meaning.
Code
v = np.array([3, 2])
fig, ax = plt.subplots(figsize=(5, 5))
ax.set_aspect("equal", adjustable="box")
ax.set_xlim(-1, 5)
ax.set_ylim(-1, 5)
ax.axhline(0)
ax.axvline(0)
ax.arrow(0, 0, v[0], v[1], length_includes_head=True, head_width=0.2)
ax.scatter([v[0]], [v[1]])
ax.text(v[0] + 0.1, v[1] + 0.1, r"$\mathbf{v}=[3,2]$", fontsize=12)
ax.set_xlabel("feature 1")
ax.set_ylabel("feature 2")
plt.show()
Vector differences
If two vectors use the same entries in the same order, we can perform arithmetic. E.g., the difference vector tells us how one measurement bundle must move to become the other.
Code
x = np.array([3, 2])
z = np.array([1, 4])
d = x - z
fig, ax = plt.subplots(figsize=(5, 5))
ax.set_aspect("equal", adjustable="box")
ax.set_xlim(-1, 5)
ax.set_ylim(-1, 5)
ax.axhline(0)
ax.axvline(0)
# x and z as arrows from the origin
ax.arrow(0, 0, x[0], x[1], length_includes_head=True, head_width=0.2)
ax.arrow(0, 0, z[0], z[1], length_includes_head=True, head_width=0.2, linestyle="--")
# x - z as the arrow from z to x
ax.arrow(z[0], z[1], d[0], d[1], length_includes_head=True, head_width=0.2)
ax.scatter([x[0], z[0]], [x[1], z[1]])
ax.text(x[0] + 0.1, x[1] + 0.1, r"$\mathbf{x}=[3,2]$", fontsize=12)
ax.text(z[0] + 0.1, z[1] + 0.1, r"$\mathbf{z}=[1,4]$", fontsize=12)
ax.text(2.0, 3.2, r"$\mathbf{x}-\mathbf{z}=[2,-2]$", fontsize=12)
ax.set_xlabel("feature 1")
ax.set_ylabel("feature 2")
plt.show()
If feature 1 is “temperature” and feature 2 is “pressure,” what does
\(\mathbf{x}-\mathbf{z}=[2,-2]\) mean in words?
Matrix = many measurement bundles
Rows are examples. Columns are features.
Code
X_demo = np.array([
[2, 0, 1, 3],
[1, 2, 0, 1],
[0, 1, 3, 2],
])
fig, ax = plt.subplots(figsize=(6, 2.8))
im = ax.imshow(X_demo, aspect="auto", cmap="Blues", vmin=0, vmax=3)
#im = ax.imshow(X_demo, aspect="auto")
for i in range(X_demo.shape[0]):
for j in range(X_demo.shape[1]):
ax.text(j, i, str(X_demo[i, j]), ha="center", va="center")
ax.set_title("Matrix = many vectors together")
ax.set_xlabel("columns = features")
ax.set_ylabel("rows = observations")
ax.set_xticks(range(X_demo.shape[1]))
ax.set_yticks(range(X_demo.shape[0]))
plt.colorbar(im, ax=ax, fraction=0.03, pad=0.04)
plt.show()
Matrix-matrix multiplication
When both operands are matrices,
\[ (AB)_{ik} = \sum_{j} A_{ij} B_{jk}. \]
For \(A \in \mathbb{R}^{m \times n}\) and \(B \in \mathbb{R}^{n \times p}\), the result is in \(\mathbb{R}^{m \times p}\).
Dot product as an explainable score
Consider that we can decompose the final score in terms of its components. That can help us explain why a particular outcome occurred.
contributions: {'hours': 8.0, 'sleep': 6.0, 'stress': -12.0, 'bias': 5.0}
score = 7.0Which feature helped most? Which feature hurt? Why does the bias count as a contribution?
Projection picture
Local change: zoom in
A rate of change asks:
\[ \frac{\text{change in output}}{\text{change in input}} \]
For a big step, this is an average rate of change.
For a tiny step, it becomes a local rate of change.
A derivative is what the slope looks like when you zoom in until the curve is almost straight.
Corners and cliffs:
derivatives can fail to exist
The derivative exists only when the limiting slope settles to one value. Corners are a useful warning for future ML.
For ReLU (a popular function in ML), the derivative is 0 for negative inputs and 1 for positive inputs. At exactly 0, the classical derivative is undefined. ML libraries typically choose a convenient convention there.
Why is “undefined at one point” not the same as “useless everywhere”?
Gradient as compass 🧭
For a loss (e.g., error) surface, the gradient is an uphill compass. Training methods often move roughly the opposite way.
Code
x_grid = np.linspace(-3, 3, 100)
y_grid = np.linspace(-3, 3, 100)
Xg, Yg = np.meshgrid(x_grid, y_grid)
Z = Xg**2 + 2*Yg**2
point = np.array([1.5, 1.0])
grad = np.array([2*point[0], 4*point[1]])
neg_grad = -grad / np.linalg.norm(grad)
fig, ax = plt.subplots(figsize=(6,4))
ax.contour(Xg, Yg, Z, levels=12)
ax.scatter([point[0]], [point[1]], zorder=5)
ax.arrow(point[0], point[1], grad[0]/3, grad[1]/3, head_width=0.12, length_includes_head=True, label="gradient")
ax.arrow(point[0], point[1], neg_grad[0], neg_grad[1], head_width=0.12, length_includes_head=True, linestyle="--")
ax.text(point[0] + grad[0]/3, point[1] + grad[1]/3, "uphill")
ax.text(point[0] + neg_grad[0], point[1] + neg_grad[1], "downhill")
ax.set_xlabel("parameter 1")
ax.set_ylabel("parameter 2")
ax.set_title("Gradient points uphill; -gradient points downhill")
plt.show()
If the gradient is [3, 4], what unit direction increases fastest? What unit direction decreases fastest?
Sigmoid
One common S-shaped function is the sigmoid:
\[ \sigma(z)=\frac{1}{1+e^{-z}}. \]
Its derivative has a convenient form:
\[ \sigma'(z)=\sigma(z)(1-\sigma(z)). \]
The derivative is largest near \(z=0\) and small when the curve has saturated near 0 or 1.
Why does \(\sigma'(0)=0.25\)?
Probability grammar:
the language of uncertainty
Probability is a way to think carefully about uncertain experiments.
Our running example:
One play of an arcade claw machine.
🐉
dragon
dragon
⛵️
sailboat
sailboat
⏱️
clock
clock
○
empty
empty
| Word | Meaning | Claw-machine example |
|---|---|---|
| Experiment | a process with an uncertain outcome | one claw-machine play |
| Outcome | one possible result | dragon |
| Sample space | all possible outcomes | {dragon, sailboat, clock, empty} |
| Event | a set of outcomes | wins any prize |
| Complement | the event does not happen | empty grab |
| Intersection | both events happen | wins prize and prize is dragon |
| Union | at least one event happens | wins dragon or sailboat |
Run it: frequency estimates probability
Frequency and probability are not quite the same thing but the former becomes a better estimate of the latter when we see many more examples.
Code
rng_freq = np.random.default_rng(315103)
p_win = 0.50
n_plays = 300
wins = rng_freq.binomial(1, p_win, size=n_plays)
running_win_rate = np.cumsum(wins) / np.arange(1, n_plays + 1)
fig, ax = plt.subplots(figsize=(7, 3.5))
ax.plot(np.arange(1, n_plays + 1), running_win_rate, linewidth=1.5)
ax.axhline(p_win, linestyle="--", linewidth=2, label="model probability")
ax.set_ylim(0, 1)
ax.set_xlabel("number of plays observed")
ax.set_ylabel("observed win frequency")
ax.set_title("A short-run frequency can wander before settling")
ax.legend()
plt.show()
print("first 20-play win rate:", wins[:20].mean())
print("final 300-play win rate:", wins.mean())
first 20-play win rate: 0.65
final 300-play win rate: 0.49If the first 20 plays looked unlucky, has the true probability necessarily changed?
Venn diagrams: probability as regions
A Venn diagram helps visualize probability space. Using the same claw table:
\[ A = \text{wins a prize}, \qquad B = \text{claw closes around a toy}. \]
Which region is counted twice by \(P(A)+P(B)\)? Which region is the complement of \(A\cup B\)?
Base-rate demo: positives are not all true positives
This is a reasoning exercise, not medical advice.
Code
N = 1000
prevalence = 0.01
sensitivity = 0.90
specificity = 0.91
n_condition = int(round(N * prevalence))
n_no = N - n_condition
tp = int(round(n_condition * sensitivity))
fn = n_condition - tp
fp = int(round(n_no * (1 - specificity)))
tn = n_no - fp
rng_demo = np.random.default_rng(1109)
all_idx = np.arange(N)
condition_idx = rng_demo.choice(all_idx, size=n_condition, replace=False)
tp_idx = rng_demo.choice(condition_idx, size=tp, replace=False)
noncondition_idx = np.setdiff1d(all_idx, condition_idx)
fp_idx = rng_demo.choice(noncondition_idx, size=fp, replace=False)
positive_idx = np.sort(np.concatenate([tp_idx, fp_idx]))
cols = 40
x = all_idx % cols
y = all_idx // cols
fig, ax = plt.subplots(figsize=(10, 5))
ax.scatter(x, y, marker="s", s=55, facecolors="none", edgecolors=(0, 0, 0, 0.18), linewidths=0.5)
ax.scatter(x[positive_idx], y[positive_idx], marker="s", s=55, alpha=0.30, linewidths=0.5, label="positive test")
ax.scatter(x[tp_idx], y[tp_idx], marker="s", s=55, alpha=0.75, linewidths=0.5, label="true positive")
ax.scatter(x[condition_idx], y[condition_idx], marker="x", s=45, linewidths=1.1, label="condition")
ax.set_aspect("equal")
ax.set_xticks([]); ax.set_yticks([])
ax.set_ylim(N//cols, -1)
ax.set_title(f"Among {tp+fp} positive tests, {tp} are true positives: {tp/(tp+fp):.1%}")
#ax.legend(loc="lower right")
ax.legend(
loc="center left",
bbox_to_anchor=(1.03, 0.5),
frameon=True,
facecolor=paper,
edgecolor=ink,
fontsize=10,
title="Prediction"
)
fig.subplots_adjust(right=0.76)
plt.show()
Run it: thresholds change denominators
A model score becomes a yes/no decision only after we choose a threshold. Moving the threshold changes TP, FP, FN, TN, and therefore the conditional probabilities.
Code
y_true = np.array([1, 1, 1, 1, 0, 0, 0, 0, 1, 0])
scores = np.array([0.95, 0.85, 0.70, 0.40, 0.65, 0.55, 0.30, 0.20, 0.45, 0.10])
thresholds = np.arange(0.20, 0.91, 0.10)
rows = []
for th in thresholds:
y_pred = (scores >= th).astype(int)
m = metric_summary(y_true, y_pred)
rows.append({
"threshold": round(float(th), 2),
"TP": m["TP"], "FP": m["FP"], "FN": m["FN"], "TN": m["TN"],
"precision": m["precision"],
"recall": m["recall"],
})
if pd is not None:
threshold_table = pd.DataFrame(rows)
print(threshold_table.to_string(index=False, float_format=lambda z: f"{z:.2f}"))
else:
for row in rows:
print(row)
precision = np.array([row["precision"] for row in rows], dtype=float)
recall = np.array([row["recall"] for row in rows], dtype=float)
fig, ax = plt.subplots(figsize=(7, 3.5))
ax.plot(thresholds, precision, marker="o", label="precision = P(true + | predicted +)")
ax.plot(thresholds, recall, marker="o", label="recall = P(predicted + | true +)")
ax.set_ylim(0, 1.05)
ax.set_xlabel("decision threshold")
ax.set_ylabel("metric value")
ax.set_title("Same scores, different thresholds, different denominators")
ax.legend(fontsize=8)
plt.show() threshold TP FP FN TN precision recall
0.20 5 4 0 1 0.56 1.00
0.30 5 2 0 3 0.71 1.00
0.40 4 2 1 3 0.67 0.80
0.50 3 2 2 3 0.60 0.60
0.60 3 1 2 4 0.75 0.60
0.70 2 0 3 5 1.00 0.40
0.80 2 0 3 5 1.00 0.40
0.90 1 0 4 5 1.00 0.20
When the threshold goes up, which denominator usually shrinks: predicted positives, true positives, or both?
Expected value appears gradually
Expected value is a long-run centre under repeated draws. One short run can still look strange.
Code
rng_ev = np.random.default_rng(2026)
values = np.array([1, 10, 100])
probs = np.array([0.70, 0.25, 0.05])
draws = rng_ev.choice(values, size=400, p=probs)
running_mean = np.cumsum(draws) / np.arange(1, len(draws) + 1)
true_ev = np.sum(values * probs)
fig, ax = plt.subplots(figsize=(6, 3.5))
ax.plot(running_mean)
ax.axhline(true_ev, linestyle="--", linewidth=2)
ax.set_xlabel("number of plays")
ax.set_ylabel("running average value")
ax.set_title("One run wanders; the long-run average becomes visible")
plt.show()
print("theoretical expected value:", true_ev)
print("final running average:", running_mean[-1])
theoretical expected value: 8.2
final running average: 7.6375Why can the running average move sharply upward even late in the run?
Same mean, different distributions
A mean is not a distribution.
Code
class_A = np.array([69, 70, 71, 68, 72, 70, 69, 71])
class_B = np.array([20, 30, 40, 70, 100, 100, 100, 100])
fig, axes = plt.subplots(1, 2, figsize=(8, 4), sharey=True)
axes[0].hist(class_A, bins=np.arange(15, 106, 10), edgecolor="black")
axes[0].set_title(f"Class A: mean={class_A.mean():.0f}")
axes[1].hist(class_B, bins=np.arange(15, 106, 10), edgecolor="black")
axes[1].set_title(f"Class B: mean={class_B.mean():.0f}")
for ax in axes:
ax.set_xlabel("score")
ax.set_ylabel("count")
plt.tight_layout()
plt.show()
Distribution checklist: centre, spread, shape, tails
Quantiles and IQR: the middle 50%
Quantiles mark positions in sorted data.
- \(Q_1\): ~25% of observations are at or below this value.
- Median: ~50% are at or below this value.
- \(Q_3\): ~75% are at or below this value.
\[ \operatorname{IQR}=Q_3-Q_1. \]
Q1, median, Q3, IQR: 12.75 17.5 22.0 9.25Why might IQR be a safer summary of spread than range?
Bootstrap intuition
Bootstrapping asks: how much might a statistic change if sampling were repeated? Or, how much do our statistics depend on the small sample we happened to have obtained?
For many subsamples of your data, compute your statistic (e.g., mean), and then resample again and again.
- you can see how your statistic changes over the many samples, which can give you a more correct idea of the expected statistic, and its spread.
Code
rng_boot = np.random.default_rng(42)
sample = np.array([72, 75, 78, 80, 95])
boot_means = []
for _ in range(2000):
resample = rng_boot.choice(sample, size=len(sample), replace=True)
boot_means.append(resample.mean())
boot_means = np.array(boot_means)
fig, ax = plt.subplots(figsize=(6,3.5))
ax.hist(boot_means, bins=25, edgecolor="black")
ax.axvline(sample.mean(), linestyle="--", linewidth=2)
ax.set_xlabel("bootstrap sample mean")
ax.set_ylabel("count")
plt.show()
print("sample mean:", sample.mean())
print("middle 90% of bootstrap means:", np.quantile(boot_means, [0.05, 0.95]))
sample mean: 80.0
middle 90% of bootstrap means: [74.8 86. ]Remember that statistics depend greatly on the data from which they were obtained.
Run it: a test score is a noisy statistic
Even if a model’s true accuracy were fixed (e.g., at 80%), the measured test accuracy would vary from one finite test sample to another.
Code
rng_score = np.random.default_rng(315104)
true_accuracy = 0.80
test_sizes = [50, 200, 1000]
repeats = 5000
bins = np.linspace(0.50, 1.00, 26)
fig, axes = plt.subplots(1, 3, figsize=(8, 3.2), sharex=True, sharey=True)
for ax, n_test in zip(axes, test_sizes):
observed_accuracy = rng_score.binomial(n_test, true_accuracy, size=repeats) / n_test
lo, hi = np.quantile(observed_accuracy, [0.05, 0.95])
ax.hist(observed_accuracy, bins=bins, edgecolor="black", alpha=0.75)
ax.axvline(true_accuracy, linestyle="--", linewidth=2)
ax.set_title(f"test n={n_test}\n90% data range: {lo:.2f}–{hi:.2f}")
ax.set_xlabel("observed accuracy")
axes[0].set_ylabel("simulation count")
plt.suptitle("Reported test accuracy depends on the test sample")
plt.tight_layout()
plt.show()
A bigger held-out test set usually gives a more certain estimate. A peeked-at test set gives a dishonest estimate.
Why might a jump from 0.80 to 0.86 mean something different when the test set has 50 rows versus 1000 rows?
Boxplots: compact distribution comparison
A boxplot is a compact view of median, IQR, and potential tail values.
Code
class_A = np.array([69, 70, 71, 68, 72, 70, 69, 71])
class_B = np.array([20, 30, 40, 70, 100, 100, 100, 100])
fig, ax = plt.subplots(figsize=(5.5, 3.5))
ax.boxplot([class_A, class_B], labels=["Class A", "Class B"], showmeans=True)
ax.set_ylabel("score")
ax.set_title("Same mean, different spread and tails")
plt.show()
Boxplots are useful for comparing groups quickly. Histograms show shape better. Use both when stakes are high.
What does the boxplot reveal faster than the mean? What does it hide compared with the histogram?
Plots can tell the truth badly
Same values on different axes conveys meaning differently.
Code
groups = ["old", "new"]
values = [82, 86]
fig, axes = plt.subplots(1, 2, figsize=(8, 3.5))
axes[0].bar(groups, values)
axes[0].set_ylim(0, 100)
axes[0].set_title("Full scale")
axes[0].set_ylabel("score")
axes[1].bar(groups, values)
axes[1].set_ylim(80, 88)
axes[1].set_title("Truncated scale")
axes[1].set_ylabel("score")
plt.tight_layout()
plt.show()
When would someone be incentivized to over-emphasize? To under-emphasize?
