Library / Backtest overfitting
The Sharpe Ratio Can Be Manufactured by Selling Tails
Can a performance score be raised by changing the payoff shape without adding skill?
A manager can improve a conventional ratio by reshaping returns, including selling rare large losses, without adding underlying skill. Inspect tail losses, skewness and payoff concentration alongside the ratio. Goetzmann and coauthors derive a manipulation-proof measure only under its stated utility and market conditions.
MPPM has assumptions; skewness alone cannot distinguish skill from manipulation.
Evidence map
| Aspect | Finding |
|---|---|
| What it is | The demonstration that common performance statistics can be raised deliberately, without any skill, by reshaping the distribution of returns, and the construction of a measure that cannot be gamed in that way. |
| Key result / formula | Goetzmann, Ingersoll, Spiegel and Welch show that a manager with no forecasting ability can raise a reported Sharpe ratio by taking positions whose payoff is concave: a strategy that collects a small premium most of the time and loses heavily but rarely produces a high mean relative to its measured standard deviation, precisely because the large losses are too infrequent to appear in the sample. |
| Why it matters for backtesting | A user optimising a strategy on its Sharpe ratio is running exactly the search this paper describes, and the search will find the concave payoffs whether or not the user intended it. |
What it is
It matters because the statistic a backtest reports is also the statistic a strategy can be optimised to produce.
Key result / formula
Selling options is the transparent version, but the same effect is produced by any rule that cuts gains and lets losses run, by leverage that increases after gains, and by any strategy whose exposure is negatively related to volatility. They also show the manipulation can be dynamic, adjusting exposure through the measurement period in response to accumulated performance. They then derive a performance measure, based on expected utility with a specified risk aversion, that cannot be increased by such manipulation, and which reduces to familiar quantities when returns are well behaved.
Why it matters for backtesting
The practical checks can be performed on a documented return series; individual in-app metrics require separate capability verification. Report the shape of the distribution alongside the ratio: skewness, the size of the largest losses, and the fraction of total return contributed by the largest few periods. Judge against a tail measure as well as a ratio (see [Tail Risk: CVaR / Expected Shortfall]). And treat a strategy whose return series looks like a short option position as one whose sample has probably not yet contained its defining event. The diagnostic that separates skill from manipulation is not in the mean, it is in the shape.
Source
Goetzmann, Ingersoll, Spiegel & Welch, "Portfolio Performance Manipulation and Manipulation-proof Performance Measures", Review of Financial Studies 20(5), 2007, 1503-1546. Primary source
Related in the library
The Fundamental Law of Active Management (and why breadth is hard to buy)
False Strategy Theorem (Expected Maximum Sharpe under N trials)
The Standard Error of a Sharpe Ratio, and What Autocorrelation Does to It
Guide on this topic: Sharpe vs Sortino ratio in backtests
How the assistant cites the library · The checks behind a verdict