stochastly
Start free trial

Library / Backtest overfitting

The Efficient-Market Null, Stated by Its Defenders

What evidence would reject an efficient-market null for a proposed trading rule?

Declare the return benchmark, information set, trading costs and test period before checking the rule?s incremental result. A rejection concerns the joint market-efficiency and benchmark assumptions, so identify which model could fail before attributing profit to inefficiency.

Market-efficiency tests involve a joint hypothesis and a declared benchmark model.

Shared intervals are counted by strategy and equal-weight universe direction.
Shared intervals are counted by strategy and equal-weight universe direction.

Evidence map

AspectFinding
What it isThe hypothesis each backtest is implicitly testing against, in the form its proponents gave it.
Key result / formulaFama's review defines the hypothesis in its three forms, according to the information set prices are said to reflect: past prices, all public information, or all information including private.
Why it matters for backtestingStochastly's verdict is, in substance, a test against this null, and two consequences follow for how an agent should speak.

What it is

An agent that cannot state the null precisely cannot explain what a result would mean, and these three sources state it precisely and in different registers.

Key result / formula

The formulation that matters for testing is his insistence that market efficiency is not testable on its own: any test is a joint test of efficiency and of a model of expected returns, so any rejection can be attributed to a wrong model of expected returns as readily as to inefficiency. Jensen supplies the empirical companion, evaluating mutual fund performance against a risk-adjusted benchmark over two decades and finding that funds on average did not earn their fees, with little evidence of persistent skill. Malkiel's book carries the argument to a wide audience and adds the practical conclusion, that a passive holding of the market is the appropriate default and the burden of proof lies on anything else. Together they define both the statistical null and the economic alternative a strategy must beat.

Why it matters for backtesting

First, the joint hypothesis problem means a positive result does not demonstrate inefficiency by itself: it demonstrates that returns differ from what the chosen benchmark predicts, and the benchmark is a choice. Naming it is part of reporting the result. Second, the economic alternative, a passive position at comparable risk, is the correct comparison for any strategy, which is the rule that two arms are compared at equal risk and equal span (see [Two arms are compared at equal risk, equal exposure and equal span]). An agent asked to defend the null should be able to put the strongest version of it, not a caricature, since that is what the strategy has to beat.

Source

Fama, "Efficient Capital Markets: A Review of Theory and Empirical Work", Journal of Finance 25(2), 1970, 383-417; Jensen, "The Performance of Mutual Funds in the Period 1945-1964", Journal of Finance 23(2), 1968, 389-416; Malkiel, A Random Walk Down Wall Street, W. W. Norton, 1973. Primary source

Related in the library