Library / Backtest overfitting
Does Publication Destroy Stock Return Predictability?
Does publishing a return predictor weaken its later performance?
Compare the same frozen predictor in its original sample, an untouched prepublication interval and a later postpublication interval. McLean and Pontiff report average declines in their studied predictor cohort; those comparisons suggest selection and market learning while leaving other changes as possible causes.
Prepublication and postpublication declines are cohort associations; the comparison alone does not isolate a causal mechanism.
Evidence map
| Aspect | Finding |
|---|---|
| What it is | McLean and Pontiff study a collection of published cross-sectional stock-return predictors. |
| Key result / formula | In their sample, average predictor returns decline out of sample and decline further after publication. |
| Why it matters for backtesting | A paper strategy should be evaluated after the discovery sample and after the date when its method became public, using only data and instruments actually available at each decision. |
What it is
They compare returns in the paper's original sample, an out-of-sample interval before publication, and a later interval after publication. This comparison helps separate several possible sources of decay, but it is not a causal decomposition for every predictor.
Key result / formula
The paper reports an out-of-sample decline of about 26 percent before publication and an additional post-publication decline of about 58 percent relative to the in-sample benchmark. The pre-publication drop gives an upper bound on the contribution of statistical selection in their interpretation; it is not a direct measurement saying that all of that decline was caused by data mining. The additional decline is consistent with investors learning from publication: trading activity and short interest change for relevant stocks, and easier-to-trade predictors tend to decay more. Those patterns support the learning account without proving that publication alone caused every change. The reported percentages are features of the studied predictor cohort, period and return convention; they are not a universal haircut.
Why it matters for backtesting
A manual test can compare the same frozen strategy in the original sample, the pre-publication holdout and the post-publication period, with costs and survivorship handled consistently. That comparison can reveal a change; it cannot by itself name its cause. Trading rules, market structure, funding and data vendors may also have changed. Some predictors may show little decay, and a result without decay is not automatically a code error. Report the cohort used, publication dates, definitions of returns, and how much of the result remains after costs.
Worked decision
If a predictor earns 1.0 unit per period in a paper's discovery sample, a later value of 0.7 would show a 30 percent decline for that predictor. It would not prove that 0.3 was selected noise. A subsequent value of 0.4 after publication would raise a learning hypothesis, alongside other changes that occurred between periods. Preserve the rule and measurement convention across all three windows before interpreting the difference.
Source
McLean & Pontiff, "Does Academic Research Destroy Stock Return Predictability?", Journal of Finance 71(1), 2016, 5-32. Primary source