Survivorship bias in backtesting
What changes when a backtest excludes assets that later disappeared?
Keeping assets that survive until today changes the historical investment universe. Brown and coauthors document this selection issue in mutual-fund data; the six-stock example below illustrates the arithmetic, not their dataset.
The six-stock example is synthetic and does not reproduce the cited mutual-fund dataset.
Survivorship bias enters a historical test when the asset list was selected using knowledge of which assets still exist at the end of the sample.
The missing observations
A current ticker list omits delisted names that were eligible at the historical decision time. A backtest on that list conditions its sample on later survival. Brown, Goetzmann, Ibbotson and Ross showed that such truncation can create apparent return predictability.
A six-stock universe with two delistings
Suppose six equally weighted stocks were eligible at the start of a year. Four remain listed and return 12%, 8%, 5% and -1%. A fifth is delisted after a bankruptcy with a terminal total return of -90%. The sixth is acquired for cash at a price that gives +25%.
A list built from the names still listed at year end holds four stocks: (12% + 8% + 5% - 1%) / 4 = 24% / 4 = 6.0%. The universe as it stood at the start holds six: (12% + 8% + 5% - 1% - 90% + 25%) / 6 = -41% / 6 = -6.83%. The survivor filter raises the displayed average by 12.83 percentage points in this example. The cash acquisition shows that a delisting can end with a positive terminal return, so the size of the bias depends on the mix of delisting reasons.
Universe return = sum(weight at decision time × subsequent total return)What the data must retain
Record historical membership, symbol changes, corporate actions and the terminal return on a delisting. Shumway documented missing negative delisting returns in a historical equity database. A point-in-time universe also needs dates for when each security became investable.
Scope of the product
Stochastly can backtest data supplied to the desktop application. The backtest cannot restore delisted securities absent from an imported universe. Inspect the provider's membership and delisting fields before treating a cross-sectional result as investable.
Frequently asked questions
Is survivorship bias the same as look-ahead bias?
Survivorship is selection of the asset universe using a later outcome. It is one route by which future information enters a historical test.
Can delisting returns be set to zero?
A zero terminal return is an assumption, not a recorded delisting return. Preserve the provider's terminal observation and document gaps.
Can a backtest engine repair a survivor-filtered feed?
The missing assets and their historical returns must come from the data source.
Sources
Brown, Goetzmann, Ibbotson and Ross (1992). Survivorship Bias in Performance Studies. Review of Financial Studies 5(4), 553-580.
Shumway (1997). The Delisting Bias in CRSP Data. Journal of Finance 52(1), 327-340.
Arnott, Harvey and Markowitz (2019). A Backtesting Protocol in the Era of Machine Learning. Journal of Financial Data Science 1(1), 64-74.
In the library
Survivorship, Look-Ahead and Timestamp Conventions
Learning From Survivors Distorts What We Learn
Nassim Taleb (the sample you see is the survivors)
Related
Look-ahead bias
How to backtest a trading strategy
Survivorship, look-ahead and timestamps