Most self-supervised learning methods begin by deciding which changes should leave meaning intact: crop an image, perturb a signal, or corrupt a table. That decision quietly embeds domain knowledge into the learning pipeline. Learning from Randomness (LFR) asks whether useful representations can be learned without designing those transformations at all—by training an encoder to reconstruct the outputs of several frozen, randomly initialized networks.

The surprising hypothesis

If a compact representation can support many diverse and unpredictable projection tasks through simple prediction heads, then it must preserve rich information about the original input.

When an augmentation changes the meaning

Augmentation-based self-supervision assumes that two transformed views describe the same underlying object. That works when invariances are known. In specialized domains, however, common transformations can produce samples that are unrealistic or physically impossible.

Histopathology image patches before and after standard color jitter. The original top row has characteristic blue nuclei and pink cytoplasm; the augmented bottom row contains unnaturally altered colors and contrast.
Figure 1 from the paper. Original H&E-stained tissue appears in the top row. Standard color jitter produces the bottom row, where clinically meaningful staining patterns can become unrealistic. Designing safe views requires domain knowledge.

The same problem appears elsewhere. Shifting a highly periodic time series may create uninformative duplicate views. Swapping tabular features can violate dependencies such as conservation laws. A generic method should not need a new augmentation theory for every modality and application.

Random networks become pretext tasks

LFR uses three sets of functions. An encoder fθ maps each input x to a compact representation z. Frozen random projectors g(k) produce pseudo-targets directly from the input. Small trainable predictors h(k)φ try to reproduce each pseudo-target using only z.

Learning from Randomness architecture. An input is encoded into representation z and separately passed through K frozen random projectors. K simple predictors reconstruct the resulting pseudo-targets from z, with stopped gradients on the random-projector branch.
Figure 2 from the paper. The random projectors define multiple fixed tasks. To solve all of them through low-capacity predictors, the shared representation must retain information that transfers beyond any single projection.

The joint objective sums reconstruction error across examples and projectors:

Learning-from-randomness objective
arg minθ,Φ ∑xᵢ∈X ∑k D[ g(k)(xi), h(k)φ(fθ(xi)) ] + λ1Ω1(θ) + λ2Ω2(Φ)

The projectors are random and frozen. Only the representation model and simple prediction heads are optimized. Limiting predictor capacity prevents each head from solving its task without relying on a useful shared representation.

Learning instance structure, not merely matching vectors

A simple mean-squared error can underweight subtle distinctions. LFR instead introduces Batch-wise Barlow Twins, which forms a cosine-similarity matrix between random targets and predicted targets across examples in a minibatch:

Batch-wise Barlow Twins
ℒBBT = ∑k ∑i [(1 − c(k)ii)² + λ∑j≠i(c(k)ij)²]

The diagonal term matches every example to its own pseudo-target. The off-diagonal term prevents different examples from becoming interchangeable, preserving instance-level structure in the representation.

Training alternates between updating the encoder and updating the predictor heads, following an expectation-maximization interpretation. Keeping one side fixed while optimizing the other stabilizes learning; the predictors receive several update steps so they remain useful evaluators of the current representation.

EUpdate the encoder

Hold the predictors fixed and improve the representation for all random tasks.

MUpdate the predictors

Hold the encoder fixed and fit each simple head to its frozen projector.

↻Alternate

Repeat so neither side outruns the other and destabilizes the objective.

Diversity makes randomness informative

Several random projectors may accidentally respond to the same aspects of the data. LFR generates a larger candidate pool and selects a diverse subset by comparing the pairwise similarity patterns they induce over a representative batch. It also encourages different initial filters through Beta-distributed weights and frozen weight dropout.

This diversity is essential: each projector should ask a meaningfully different question of the representation. In the Kvasir ablation, applying diversity encouragement during both initialization and selection produces the strongest downstream accuracy.

One method across three modalities

The experiments use time-series, tabular, and image data from healthcare, banking, activity recognition, particle physics, and other scientific settings. The same basic LFR framework is used throughout; only the network architecture needed to consume each modality changes.

Time seriesHAR · Epilepsy · MIMIC-III

Activity, neurological, and clinical signals.

TabularIncome · Theorem · HEPMASS

Social, logical, and particle-physics data.

ImagesKvasir

Medical endoscopy imagery.

Performance table comparing supervised, randomly initialized, autoencoder, augmentation-based, modality-specific, and LFR representations across three time-series datasets, three tabular datasets, and the Kvasir medical image dataset.
Table 2 from the paper. LFR is the strongest self-supervised method on HAR, Epilepsy, Income, Theorem, and Kvasir, and remains competitive on MIMIC-III and HEPMASS—all without designing modality-specific augmentations.

LFR reaches 93.1% on HAR, 97.9% on Epilepsy, 85.2% on Income, 51.6% on Theorem, and 74.9% on Kvasir. On the Income dataset, its linear evaluation even exceeds the paper’s supervised baseline. On the two medical benchmarks, it leads the self-supervised comparison on Kvasir and remains close to the best methods on MIMIC-III.

A different source of supervision

LFR does not claim that each random projector captures a meaningful human concept. Meaning emerges from the shared constraint: one compact representation must preserve enough structure for simple heads to solve many diverse tasks simultaneously. The randomness supplies breadth; the bottleneck and prediction heads force the encoder to organize that breadth into reusable features.

The result broadens where self-supervised learning can be applied. When trustworthy augmentations are obvious, invariance remains powerful. When the data’s valid transformations are unknown, expensive to design, or scientifically constrained, random projectors offer another route to useful representation learning.

Read the research