[Library](https://stochastly.com/library) / [Machine learning in finance](https://stochastly.com/library/machine-learning)

# Conformal Prediction: Intervals With a Guarantee and No Distributional Assumption

When does a conformal interval retain its promised coverage after market data change?

The finite-sample marginal coverage statement requires exchangeable observations, a condition financial series often violate through dependence and regime shifts. Calibrate on an untouched period, then report realized coverage by later period and regime. Treat any adapted time-series guarantee according to that method's stated assumptions.

The coverage guarantee is marginal under exchangeability; it is not conditional coverage in every regime.

An unconnected conformal uncertainty node that builds split-conformal intervals and needs at least three configurations.

## Evidence map

| Aspect | Finding
| What it is | A method for turning any prediction model into one that outputs a set or interval with a guaranteed coverage rate, requiring no more than that the data be exchangeable.
| Key result / formula | The construction uses a nonconformity measure, a score saying how unusual a candidate outcome looks given the model and the past data.
| Why it matters for backtesting | The attraction for backtesting is that the guarantee does not require the return distribution to be anything in particular, which is welcome given how badly financial returns satisfy the usual assumptions.

## What it is

It is unusual in offering a finite-sample guarantee where most methods give an asymptotic one, and it wraps around an existing model and leaves it in place.

## Key result / formula

For a new observation, the candidate values whose nonconformity score is not extreme relative to the scores observed on held-out data are collected into the prediction set. The theorem is that this set contains the true outcome with at least the stated probability, in finite samples, for any underlying model and any data distribution, provided the observations are exchangeable. The price is that the guarantee is marginal rather than conditional: it holds on average over the data and need not hold within each region of the feature space, so intervals can be uninformatively wide where the model is uncertain and the coverage can be uneven across regimes. The split variant makes the computation cheap by using a single held-out calibration set.

## Why it matters for backtesting

The obstacle is the exchangeability condition, which financial time series violate: returns are dependent and their distribution shifts, so the guarantee does not hold as stated. Two positions are defensible. Use conformal intervals as an uncertainty diagnostic, comparing realised coverage against nominal coverage on held-out periods, where a large shortfall is itself a measurement of non-stationarity. Or use the variants developed for shifted and dependent data, stating that the guarantee is then approximate. What an agent should not do is present the guarantee as intact on a financial series, since the assumption that carries it is the one the data breaks.

## Source

Shafer & Vovk, "A Tutorial on Conformal Prediction", Journal of Machine Learning Research 9, 2008, 371-421. [Primary source](https://jmlr.csail.mit.edu/papers/v9/shafer08a.html)

## Related in the library

[The Virtue of Complexity Debate: More Parameters Than Observations](https://stochastly.com/library/machine-learning/the-virtue-of-complexity-debate)

[Granger Causality Measures Prediction, Which Is Weaker Than Cause (Granger, Pearl)](https://stochastly.com/library/statistics/granger-causality-vs-cause-pearl)

[The Leverage Effect and the Retarded Volatility Model](https://stochastly.com/library/econophysics/leverage-effect-retarded-volatility)

[How the assistant cites the library](https://stochastly.com/product/assistant) · [The checks behind a verdict](https://stochastly.com/product/guards)

---
Source: https://stochastly.com/library/machine-learning/conformal-prediction-intervals-without-assumptions
