Modern AI systems often produce a single answer even when the evidence is ambiguous. Conformal prediction changes the interface: instead of asking a model to sound certain, it wraps the model with a calibrated set of possibilities and a clear, user-chosen error tolerance.
Use past examples to calibrate a threshold, then construct a prediction set for a new example. Under exchangeability, the set contains the correct answer with probability of at least 1 − α over repeated examples.
A guarantee about sets, not confidence scores
Suppose we choose an error rate of α = 0.10. Conformal prediction constructs sets with at least 90% marginal coverage: across exchangeable test examples, the correct label is included at least 90% of the time. This is different from saying that every individual set has a 90% chance of being correct. The guarantee is finite-sample and distribution-free, but it relies on the calibration examples and future examples being exchangeable.
The calibrated threshold q̂ is the ⌈(n + 1)(1 − α)⌉ / n empirical quantile of the held-out nonconformity scores. Exchangeability makes the rank of the next score uniform, producing the coverage bound.
Illustration only: coverage is a long-run property, not a promise about exactly nine of every ten consecutive cases.
Project 1: keeping RAG answers factual
Retrieval-augmented generation grounds an answer in retrieved documents, but a fluent response can still mix well-supported and weak claims. In Conditional Conformal Factuality, Naihe Feng and collaborators split a response into sub-claims and score each claim using both its similarity to retrieved documents and those documents’ relevance to the original query.
Calibration turns those relevance scores into a factuality threshold. At inference time, low-scoring claims are removed and the retained claims are merged into a coherent response. A Mondrian, or group-conditional, version calibrates separate thresholds for different domains, so one broad category does not hide weaker performance in another.
Moderate alcohol consumption is generally considered compatible with taking Tylenol.
The UK National Health Service indicates that a small amount of alcohol with acetaminophen is usually safe.
After heavier consumption - more than three drinks in the paper's example - one should wait before taking Tylenol.
No exact interval is given; the generated answer proposes waiting a few hours for alcohol to metabolize.
The manufacturer recommends consulting a doctor after significant alcohol consumption.
A healthcare professional should be consulted when there is uncertainty.
High Tylenol doses increase the risk of liver damage, particularly with alcohol.
Therefore, high doses of Tylenol should not be combined with alcohol.
Adapted from Figure 1 of the Conformal-RAG paper. At 75%, the answer preserves seven sub-claims. Raising the target to 90% retains only the three claims supported strongly enough to cross the stricter calibrated threshold.
Across FActScore, PopQA, HotpotQA, and MedLFQA experiments, Conformal-RAG maintained the intended reliability guarantee while retaining up to 60% more high-quality sub-claims than applying conformal factuality directly to the language model. The important advance is not simply another factuality score; it is a decision rule with a statistical guarantee.
Project 2: finding where an agent went wrong
A multi-agent system leaves a trajectory: a sequence of decisions, messages, tool calls, and observations. When the task fails, selecting one supposedly responsible step hides uncertainty and can make recovery brittle. In Conformal Agent Error Attribution, the prediction set becomes a contiguous interval of that trajectory, calibrated to contain the decisive error.
Define the population, intervention, comparator, and outcome.
Retrieve the trial and its reported relative risk of 0.82.
Pass “relative risk = 0.82” into the shared workspace.
Incorrectly reads 0.82 as an 82% reduction, rather than an 18% reduction.
decisive errorDescribes the treatment as producing a dramatic reduction in risk.
Confirms the source exists but misses the numerical interpretation.
Delivers a well-cited but substantively incorrect conclusion.
Re-run the highlighted interval with the instruction to distinguish relative risk from relative risk reduction. The corrected interpretation is 18% lower relative risk.
The paper introduces left, right, and two-way filtration. Left filtration removes early steps and returns a suffix; right filtration returns a prefix; two-way filtration intersects both to isolate a shorter interval anywhere in the trace. The methods are model-agnostic: a prompted or fine-tuned evaluator can supply scores, while conformal calibration supplies the uncertainty layer.
The decisive step must survive filtration from both directions. Therefore, the two-way conformal score is the larger of the left- and right-filtration scores; calibration then converts this score into a contiguous prediction interval.
Trim from the beginning; retain a calibrated suffix.
Trim from the end; retain a calibrated prefix.
Intersect both views; isolate a compact interval.
Experiments verify that empirical coverage meets the theoretical lower bound across the studied methods. The work then uses conformal sets operationally: roll the system back far enough to cover the decisive error, let it restart, and measure recovery success against the cost of redoing steps.
One principle, two kinds of reliability
Both projects treat uncertainty as something a system should act on. In RAG, the action is to withhold unsupported content. In agentic systems, it is to revisit a calibrated region of the execution trace. The shared design pattern is simple: score, calibrate, form a useful set, and connect that set to a concrete recovery action.
Read the research
- Naihe Feng, Yi Sui, Shiyi Hou, Jesse C. Cresswell, and Ga Wu. Response Quality Assessment for Retrieval-Augmented Generation via Conditional Conformal Factuality, 2025.
- Naihe Feng, Yi Sui, Shiyi Hou, Ga Wu, and Jesse C. Cresswell. Conformal Agent Error Attribution, 2026. Code ↗