[Library](https://stochastly.com/library) / [Machine learning in finance](https://stochastly.com/library/machine-learning)

# Two Cultures, and No Free Lunch

How do statistical and algorithmic models fail differently on market data?

Choose a model for the decision at hand, then test prediction on future data and inspect whether its explanation is adequate for the intended use. Breiman contrasts data models with algorithmic prediction; Wolpert's no-free-lunch result averages across possible problems and supplies no universal ranking for market datasets.

Breiman does not imply black-box methods always fail; no-free-lunch does not force a formal prior for every method.

## Evidence map

| Aspect | Finding
| What it is | Three statements about what modelling is for, which together set the terms an agent should use when a user asks whether to prefer a statistical model or a machine learning one.
| Key result / formula | Breiman contrasts two cultures. One assumes a data-generating model, fits it, and interprets its parameters; its virtue is interpretability and its risk is that conclusions depend on an assumption rarely tested.
| Why it matters for backtesting | In Stochastly the two cultures correspond to two different questions, and users conflate them.

## What it is

The answer depends on the goal, and no algorithm is best in general.

## Key result / formula

The other treats the mechanism as unknown and optimises predictive accuracy on held-out data; its virtue is that accuracy is measurable and its cost is that its predictions may require additional explanation for a causal question. Breiman's argument is that the first culture's dominance led statisticians to irrelevant problems, and that predictive accuracy is the proper arbiter. Wolpert's result constrains both: averaged over all possible problems, no learning algorithm outperforms any other, so a useful method must be evaluated against the distribution of problems actually faced. This theorem does not require a formal prior for every model choice. Domingos assembles the working knowledge that follows: generalisation is the goal and training error is not a proxy for it, data usually beats cleverness, feature engineering is where effort pays, and simplicity does not by itself imply accuracy.

## Why it matters for backtesting

If the goal is to understand why a strategy works, so that its failure can be anticipated, an interpretable model or an explicit explanation of a predictive model is needed for that decision. If the goal is the best prediction on the next period, accuracy on properly held-out data is the criterion that counts and interpretability is optional. An agent should ask which question is being asked before recommending anything. No free lunch supplies the accompanying caution: a model class that works on the user's instruments does so because its assumptions match those series, so a method that succeeded elsewhere carries no guarantee, and the comparison must be run on the data at hand. Compare candidates on a chronologically held-out sample, record the target and error measure, and check whether the simpler model remains adequate at the same decision horizon.

## Source

Breiman, "Statistical Modeling: The Two Cultures", Statistical Science 16(3), 2001, 199-231; Wolpert, "The Lack of A Priori Distinctions Between Learning Algorithms", Neural Computation 8(7), 1996, 1341-1390; Domingos, "A Few Useful Things to Know About Machine Learning", Communications of the ACM 55(10), 2012, 78-87. [Primary source](https://doi.org/10.1214/ss/1009213726)

## Related in the library

[Machine Learning in Finance: Which Textbook Answers Which Question](https://stochastly.com/library/machine-learning/machine-learning-in-finance-textbooks)

[Granger Causality Measures Prediction, Which Is Weaker Than Cause (Granger, Pearl)](https://stochastly.com/library/statistics/granger-causality-vs-cause-pearl)

[The Virtue of Complexity Debate: More Parameters Than Observations](https://stochastly.com/library/machine-learning/the-virtue-of-complexity-debate)

[How the assistant cites the library](https://stochastly.com/product/assistant) · [The checks behind a verdict](https://stochastly.com/product/guards)

---
Source: https://stochastly.com/library/machine-learning/two-cultures-and-no-free-lunch
