M27: Nonlinear dimensionality reduction: t-SNE, UMAP & manifolds

CSCI 3151 — Foundations of Machine Learning

Frank Rudzicz

Outcomes for M27

By the end of this module, you should be able to:

  1. Explain the goal of nonlinear dimensionality reduction and the informal idea of a data manifold.
  2. Implement and tune t-SNE and UMAP on small datasets, adjusting key hyperparameters and interpreting their effects.
  3. Compare PCA, t-SNE, and UMAP qualitatively, relating each method to different analysis goals.
  4. Diagnose common misuses and failure modes of t-SNE/UMAP plots and articulate simple good practices.
  5. Connect nonlinear DR to broader themes in representation learning and visualization in modern ML.

Conceptual scaffold:
manifolds & neighbourhoods

Dimensionality reduction (DR) (recap)

  • Map \(x \in \mathbb{R}^d\) to \(z \in \mathbb{R}^k\) with \(k \ll d\),
  • Trying to keep the “important structure” of the data.

Manifold intuition

  • Data often concentrate near a lower-dimensional surface inside \(\mathbb{R}^d\).
  • Locally, the surface looks flat (like a plane), but globally it can curve and twist.
  • We say the data lie on (or near) a \(k\)-dimensional manifold.

Local vs global structure

  • Local: who are my nearest neighbours? (preserve small distances)
  • Global: where are clusters relative to each other? (preserve longer-range geometry)

Linear methods like PCA are good at preserving global variance, but can badly distort local neighbourhoods when the manifold is curved.

This motivates nonlinear DR: methods that try to unfold curved manifolds while keeping nearby points nearby.

A classic example is t-SNE, which directly focuses on preserving local neighbourhoods rather than global variance.

Code 
import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D  # noqa: F401
from sklearn.datasets import make_swiss_roll
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE

# Generate a colourful Swiss roll
X, t = make_swiss_roll(n_samples=1000, noise=0.05, random_state=3151)

# Linear 2D projection via PCA
pca = PCA(n_components=2, random_state=3151)
X_pca = pca.fit_transform(X)

# Nonlinear 2D embedding via t-SNE
tsne = TSNE(
    n_components=2,
    perplexity=30,
    learning_rate="auto",
    init="random",
    random_state=3151,
)
X_tsne = tsne.fit_transform(X)

# Build the figure
fig = plt.figure(figsize=(10, 3))
fig.patch.set_facecolor("white")

cmap = "viridis"
point_size = 6
alpha = 0.85

# 1. High-dimensional manifold (Swiss roll in 3D)
ax1 = fig.add_subplot(1, 3, 1, projection="3d")
ax1.scatter(
    X[:, 0],
    X[:, 1],
    X[:, 2],
    c=t,
    cmap=cmap,
    s=point_size,
    alpha=alpha,
)
ax1.set_title("High-dimensional manifold", pad=10)
ax1.set_xticks([])
ax1.set_yticks([])
ax1.set_zticks([])
ax1.view_init(elev=10, azim=130)

# 2. Linear projection (PCA → 2D)
ax2 = fig.add_subplot(1, 3, 2)
ax2.scatter(
    X_pca[:, 0],
    X_pca[:, 1],
    c=t,
    cmap=cmap,
    s=point_size,
    alpha=alpha,
)
ax2.set_title("Linear projection (PCA → 2D)")
ax2.set_xticks([])
ax2.set_yticks([])

# 3. Nonlinear DR (t-SNE → 2D)
ax3 = fig.add_subplot(1, 3, 3)
ax3.scatter(
    X_tsne[:, 0],
    X_tsne[:, 1],
    c=t,
    cmap=cmap,
    s=point_size,
    alpha=alpha,
)
ax3.set_title("Nonlinear DR (t-SNE → 2D)")
ax3.set_xticks([])
ax3.set_yticks([])

plt.tight_layout()
Figure 1: Manifold intuition: a 3D Swiss roll (left), linear PCA projection to 2D (middle), and a nonlinear t-SNE embedding (right) that better respects the curved manifold.

Linear vs nonlinear DR

Linear DR (e.g., PCA)

  • Uses linear maps \(z = W^\top x\).
  • Good when:
    • The main structure is roughly linear or gently curved.
    • We care about variance explained and global trends.
  • Limitations:
    • Can’t “unroll” strongly nonlinear manifolds.
    • Often mixes distinct clusters if they overlap in linear projections.

Nonlinear DR

  • Learns a nonlinear mapping \(x \mapsto z\) (often implicitly).
  • Tries to preserve:
    • Local neighbourhoods (who’s close to whom),
    • Sometimes approximate global geometry.
  • t-SNE & UMAP:
    • Designed mainly as visualization tools for high-dimensional data.
    • Produce low-dimensional embeddings where clusters are easier to see (but easy to over-interpret!).

t-SNE, informally

t-SNE (t-distributed Stochastic Neighbor Embedding) (Maaten and Hinton 2008) starts from a simple idea:

If two points are close in the original space, they should be close in the embedding.

Steps (informal):

  1. In high dimensions, turn distances into probabilities:

    • For each point \(x_i\), define neighbours \(x_j\) with conditional probabilities \[p_{j\mid i} \propto \exp\Big(-\frac{\|x_i - x_j\|^2}{2\beta_i^2}\Big).\]
    • The bandwidth \(\beta_i\) is chosen so that each point has a fixed perplexity (an “effective # neighbours”).

  2. Symmetrize to get joint probabilities \(P_{ij}\) for pairs \((i,j)\).

  3. In low dimensions, we place points \(y_i\) and define \[q_{ij} \propto \Big(1 + \|y_i - y_j\|^2\Big)^{-1},\] a 🔗 Student t-distribution with heavy tails.

  4. Choose the \(y_i\) to make \(Q\) (low-dim similarities) match \(P\) (high-dim similarities) by minimizing 🔗 KL divergence \[C = \mathrm{KL}(P \,\|\, Q) = \sum_{i,j} P_{ij} \log \frac{P_{ij}}{Q_{ij}}.\]

Optimization is done with gradient descent, from a random (or PCA) initialization.

Visual comparison: PCA vs t-SNE

Same data, four embeddings:

  • Left: 2D PCA (linear, variance-focused).
  • Right three: 2D t-SNE with different perplexities (how many neighbours each point “cares about”).
    • Small perplexity → very local, can fragment clusters.
    • Medium perplexity → often a nice compromise.
    • Large perplexity → emphasizes broader structure.

Notice how t-SNE tends to produce tighter, more separated digit clusters than PCA,
while different perplexities trade off how fine-grained those clusters look.

🔬 Why t-SNE loves local structure

Key property:

  • The KL divergence \(C = \sum_{i,j} P_{ij} \log \tfrac{P_{ij}}{Q_{ij}}\) weights pairs with large \(P_{ij}\) much more than pairs with tiny \(P_{ij}\).

Consequences:

  • If \(P_{ij}\) is big (points are close in high-dim), but \(Q_{ij}\) is small (far apart in 2D),
    • The term \(P_{ij} \log \tfrac{P_{ij}}{Q_{ij}}\) is large → strong gradient push to bring \(y_i\) and \(y_j\) closer.
  • If \(P_{ij}\) is tiny (far in high-dim), then even large distortions in \(Q_{ij}\) barely affect \(C\).

Short reasoning sketch

  • For a fixed \(P_{ij}\), \(C\) as a function of \(Q_{ij}\) behaves like \(P_{ij}(-\log Q_{ij})\) plus constants.
  • The gradient magnitude \(\left|\frac{\partial C}{\partial Q_{ij}}\right| \approx \frac{P_{ij}}{Q_{ij}}\):
    • Big when \(P_{ij}\) is large and \(Q_{ij}\) is small.
    • Tiny when \(P_{ij}\) is small.

So t-SNE prioritizes keeping neighbours together (high \(P_{ij}\)), and is much more relaxed about global distances between far-away points.

Note

This is why t-SNE plots often show nice separated clusters, but relative distances between those clusters are not trustworthy (Maaten 2014).

t-SNE knobs you must know

Important hyperparameters:

  • Perplexity
    • Roughly, the effective number of neighbours each point “cares about” (10–50 is common).
    • Smaller → focus on very local structure (can fragment clusters).
    • Larger → consider broader neighbourhoods (can merge small clusters).
  • Learning rate
    • Too small → very slow convergence.
    • Too large → points can overshoot and form weird artifacts.
  • Initialization
    • init="random" vs init="pca":
      • PCA init can help stabilize global layout but doesn’t fix all issues.
  • Random seed
    • Different seeds → different-looking embeddings(!)
    • Good practice: run t-SNE multiple times and check qualitative stability.

t-SNE is mainly for visual exploration, not as a generic “feature extractor” for downstream models (Wattenberg, Viégas, and Johnson 2016).

UMAP: idea in words

UMAP (“Uniform Manifold Approximation and Projection”) [McInnes, Healy, and Melville (2018);mcinnes2018umap-software] starts from:

Assume data lie on a manifold with a certain local dimension and metric.
Approximate its structure with a weighted graph, then find a low-dim embedding that preserves this graph as much as possible.

Key steps (informal):

  1. High-dimensional graph
    • For each point, find its \(k\) nearest neighbours.
    • Turn distances into edge weights (fuzzy connection strengths).
    • Combine these into a weighted, undirected graph.
  2. Low-dimensional graph
    • Place points in 2D (or 3D) and define a similar fuzzy graph in the embedding.
  3. Optimization
    • Minimize a cross-entropy between the high- and low-dimensional fuzzy graphs.
    • This pushes strongly-connected pairs together and weakly-connected pairs apart.

Important hyperparameters:

  • n_neighbors — trade-off between local vs global structure
    • Small: emphasizes very local neighbourhoods.
    • Large: more global structure, clusters less separated.
  • min_dist — how tightly points can pack within a cluster.
    • Small: tight, dense clusters.
    • Larger: more spread-out clusters.

UMAP is typically faster than t-SNE and scales better to very large datasets (Becht et al. 2019).

❌ Anti-example: over-interpreting a t-SNE/UMAP plot

Suppose you run t-SNE on patient embeddings from an ICU dataset and see:

  • A big blue cluster (patients who survived),
  • A smaller red cluster (patients who died),
  • The red cluster appears far away from the blue one.

A tempting (but wrong) story:

“Red is far from blue, so these patients are fundamentally different,
and the model has discovered a clearly separable high-risk subgroup.”

Issues:

  • Distances between clusters in t-SNE / UMAP are not metric distances:
    • Layout can change dramatically with perplexity / n_neighbors, min_dist, or random seed.
    • Rotation, reflection, and non-linear warping of the embedding don’t change the cost.
  • The embedding may hide confounders:
    • If you didn’t account for hospital, ward, or time, clusters may reflect data-collection quirks.
  • The plot mixes train and test sets, so you can’t tell if the pattern generalizes.

Better practice:

  • Report classification metrics (e.g., ROC–AUC, calibration) on a held-out test set.
  • Use t-SNE/UMAP as supporting visual evidence, not the main result.
  • Check robustness:
    • Change hyperparameters and seeds.
    • Compare to PCA or a simple baseline embedding.

Example 1 — t-SNE vs PCA on handwritten digits

Goal: compare PCA and t-SNE as visualization tools on the digits dataset.

  • Dataset: sklearn.datasets.load_digits() (64-dimensional, 0–9 digits).
  • We’ll:
    1. Subsample for speed.
    2. Standardize features.
    3. Compute a 2D PCA embedding.
    4. Compute a 2D t-SNE embedding.
    5. Look at both plots side-by-side.
Code 
import numpy as np
import pandas as pd
from sklearn.datasets import load_digits
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA

rng = np.random.default_rng(3151)

digits = load_digits()
X_full, y_full = digits.data, digits.target

# Subsample for speed
idx = rng.choice(len(X_full), size=1000, replace=False)
X = X_full[idx]
y = y_full[idx]

X_scaled = StandardScaler().fit_transform(X)

pca2 = PCA(n_components=2, random_state=3151)
X_pca = pca2.fit_transform(X_scaled)

var_table = pd.DataFrame(
    {
        "component": [1, 2],
        "explained_variance_ratio": pca2.explained_variance_ratio_,
    }
)
var_table
Table 1: Explained variance ratio for the first two PCA components on a digits subset.
component explained_variance_ratio
0 1 0.122386
1 2 0.096081
Code 
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt

tsne = TSNE(
    n_components=2,
    perplexity=30,
    learning_rate="auto",
    init="random",
    random_state=3151,
)
X_tsne = tsne.fit_transform(X_scaled)

fig, axes = plt.subplots(1, 2, figsize=(8, 4),constrained_layout=True)

scatter0 = axes[0].scatter(X_pca[:, 0], X_pca[:, 1], c=y, s=5, cmap="tab10")
axes[0].set_title("PCA (2D)")
axes[0].set_xticks([])
axes[0].set_yticks([])

axes[1].scatter(X_tsne[:, 0], X_tsne[:, 1], c=y, s=5, cmap="tab10")
axes[1].set_title("t-SNE (2D)")
axes[1].set_xticks([])
axes[1].set_yticks([])

fig.colorbar(scatter0, ax=axes, label="digit label", fraction=0.046)
#plt.tight_layout()
Figure 3: Digits embedded into 2D using PCA (left) and t-SNE (right). Colours indicate digit labels.

Example 1 — what to notice

From the table (PCA):

  • The first two components often explain a modest fraction of total variance.
  • Using only 2D PCA can collapse different digits together if they vary in less important directions.

From the plots:

  • PCA:
    • Clusters are somewhat separated but often overlap.
    • Directions are linear combinations of all pixels; structure is global.
  • t-SNE:
    • Different digits tend to form compact clusters.
    • Cluster shape and relative position can change with hyperparameters and random seed.

Note

  • t-SNE is great for visual clustering and inspection.
  • PCA remains useful as a linear baseline and for explaining global variance.
  • Neither plot alone tells you which classifier is best — you still need evaluation metrics.

Example 2 — Swiss roll with PCA, t-SNE, and UMAP

Goal: show how nonlinear DR can unroll a curved manifold.

  • Dataset: synthetic Swiss roll from sklearn.datasets.make_swiss_roll.
  • We’ll compare 2D embeddings from:
    • PCA (linear),
    • t-SNE,
    • UMAP.
Code 
import pandas as pd

dr_summary = pd.DataFrame(
    [
        ("PCA", "linear", None, None),
        ("t-SNE", "nonlinear", "perplexity=30", None),
        ("UMAP", "nonlinear", "n_neighbors=15", "min_dist=0.1"),
    ],
    columns=["method", "type", "locality_hyperparam", "density_hyperparam"],
)
dr_summary
Table 2: Summary of DR methods and hyperparameters for the Swiss roll example.
method type locality_hyperparam density_hyperparam
0 PCA linear None None
1 t-SNE nonlinear perplexity=30 None
2 UMAP nonlinear n_neighbors=15 min_dist=0.1
Code 
from sklearn.datasets import make_swiss_roll
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
import matplotlib.pyplot as plt

try:
    import umap
except ImportError:  # graceful message if umap-learn is missing
    umap = None

# Generate synthetic manifold data
X, t = make_swiss_roll(n_samples=2000, noise=0.05, random_state=3151)

pca = PCA(n_components=2, random_state=3151)
X_pca = pca.fit_transform(X)

tsne2 = TSNE(
    n_components=2,
    perplexity=30,
    learning_rate="auto",
    init="random",
    random_state=3151,
)
X_tsne = tsne2.fit_transform(X)

if umap is not None:
    umap_model = umap.UMAP(
        n_neighbors=15,
        min_dist=0.1,
        random_state=3151,
    )
    X_umap = umap_model.fit_transform(X)
else:
    X_umap = None

fig, axes = plt.subplots(1, 3, figsize=(12, 4),constrained_layout=True)

sc0 = axes[0].scatter(X_pca[:, 0], X_pca[:, 1], c=t, s=3, cmap="viridis")
axes[0].set_title("PCA (2D)")
axes[0].set_xticks([])
axes[0].set_yticks([])

sc1 = axes[1].scatter(X_tsne[:, 0], X_tsne[:, 1], c=t, s=3, cmap="viridis")
axes[1].set_title("t-SNE (2D)")
axes[1].set_xticks([])
axes[1].set_yticks([])

if X_umap is not None:
    axes[2].scatter(X_umap[:, 0], X_umap[:, 1], c=t, s=3, cmap="viridis")
    axes[2].set_title("UMAP (2D)")
else:
    axes[2].text(
        0.5,
        0.5,
        "Install `umap-learn`\nto see UMAP here.",
        ha="center",
        va="center",
    )
    axes[2].set_title("UMAP (not installed)")

axes[2].set_xticks([])
axes[2].set_yticks([])

fig.colorbar(sc0, ax=axes, label="position along the roll", fraction=0.046)
#plt.tight_layout()
Figure 4: Swiss roll embedded into 2D using PCA, t-SNE, and UMAP. Colour encodes position along the roll.

Example 2 — what to notice

  • PCA
    • Flattens the roll but still leaves overlaps.
    • Nearby colours can be folded over each other — global linear projection.
  • t-SNE
    • Often produces a nicely unrolled band, but:
      • Global shape can be distorted.
      • Ends of the roll might curl or separate depending on perplexity.
  • UMAP
    • Also tends to unroll the manifold.
    • n_neighbors controls how much global structure is enforced.
    • min_dist affects how tight the band becomes.

Note

  • Nonlinear DR can reveal latent low-dimensional structure that linear methods miss.
  • But the exact appearance depends heavily on hyperparameters.
  • Always compare multiple runs and multiple methods.

Evaluation & quality checks for DR

Decision context: here DR is used as an exploratory visualization tool.

Useful checks:

  • Multiple seeds & hyperparameters
    • Change perplexity / n_neighbors / min_dist.
    • Check if the qualitative story (“there are 3-ish clusters”) is stable.
  • Compare to baselines
    • PCA, random projections, or simple feature pairs.
    • If t-SNE/UMAP shows patterns that never appear in simpler plots, be suspicious.
  • Quantitative sanity checks (optional, more advanced)
    • Nearest-neighbour classification accuracy using the 2D embeddings vs original space.
    • Trustworthiness / continuity metrics (how well local neighbourhoods are preserved).
  • Reproducibility
    • Log hyperparameters and random seeds.
    • Keep code + environment so figures can be regenerated.

Remember: pretty pictures do not replace proper model evaluation on held-out data.

Ethics & risks

Where can nonlinear DR go wrong ethically?

  • Overclaiming structure
    • t-SNE/UMAP plots of patients, students, or communities can make spurious clusters look real.
    • This can influence resource allocation, triage policies, or “risk tier” labels.
  • Hiding minorities
    • Rare subgroups can be collapsed or scattered, making them harder to see.
    • Visual summaries can thus hide harms or disparities.
  • Cherry-picking hyperparameters
    • It’s easy to tune perplexity / n_neighbors until you see the story you want.
    • Without transparency, this is a form of p-hacking for plots.

Simple mitigations (appropriate for this course):

  • Always show hyperparameters and seeds alongside plots.
  • Compare embeddings to quantitative metrics and domain knowledge.
  • Be explicit about what the plot does not say (e.g., no calibrated probabilities, no causal claims).

Curiosity: nonlinear DR in practice & research

Nonlinear DR methods show up in many real applications:

  • Single-cell genomics
    • t-SNE/UMAP plots of thousands of cells help biologists discover new cell types and lineages.
  • Representation learning
    • Visualizing neural network embeddings (images, text, audio) to understand structure and failure modes.
  • Information retrieval & recommendation
    • Inspecting user/item embeddings to debug weird neighbourhoods or biases.
  • ML research
    • Comparing learned representations across models or layers, e.g. how different networks cluster inputs.

Open questions (beyond this course):

  • How can we make embeddings that are faithful, interpretable, and robust?
  • How should we summarize uncertainty and instability in DR plots?
  • Can we design methods that give formal guarantees about neighbourhood preservation?

Quick check — M27

Q1 (MCQ).

Which statement about distances in t-SNE / UMAP plots is most accurate?

A. Euclidean distance in the plot is always proportional to the true distance in the original space.
B. Distances within clusters are somewhat meaningful, but distances between clusters are hard to interpret.
C. Distances in the plot are meaningless; only colours matter.
D. Distances are only trustworthy if perplexity is very small.

Q2 (MCQ).

Increasing t-SNE perplexity from 10 to 50 (with everything else fixed) tends to:

A. Focus more on very small local neighbourhoods, splitting clusters.
B. Emphasize a wider neighbourhood, sometimes merging small clusters.
C. Have no effect on the embedding.
D. Only change the runtime, not the layout.

Q3 (short).

Name one reason why using a 2D t-SNE embedding as features for a classifier (instead of the original data) can be problematic.

Q4 (short numeric / conceptual).

You run UMAP with n_neighbors = 5 and n_neighbors = 50 on the same dataset.

  • Which setting will usually emphasize more global structure, and why?

References

Becht, Etienne, Leland McInnes, John Healy, Charles-Antoine Dutertre, Immanuel WH Kwok, Lai Guan Ng, Florent Ginhoux, and Evan W Newell. 2019. “Dimensionality Reduction for Visualizing Single-Cell Data Using UMAP.” Nature Biotechnology 37 (1): 38–44. https://doi.org/10.1038/nbt.4314.
Maaten, Laurens van der. 2014. “Accelerating t-SNE Using Tree-Based Algorithms.” Journal of Machine Learning Research 15: 3221–45.
Maaten, Laurens van der, and Geoffrey Hinton. 2008. “Visualizing Data Using t-SNE.” Journal of Machine Learning Research 9: 2579–2605.
McInnes, Leland, John Healy, and James Melville. 2018. “UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction.” arXiv Preprint arXiv:1802.03426. https://arxiv.org/abs/1802.03426.
Wattenberg, Martin, Fernanda Viégas, and Ian Johnson. 2016. “How to Use t-SNE Effectively.” Distill 1 (10): e2. https://doi.org/10.23915/distill.00002.