Library / Machine learning in finance
Can Machines Learn Finance? The Sober Answer
Can a machine learn a stable financial pattern beyond a simple baseline?
Compare the proposed model with a simple baseline on a chronological untouched period, record the size of the improvement and check its stability across subperiods. Israel, Kelly and Moskowitz describe limited samples, weak signals and shifting relationships; model capacity should be set by effective data and validation evidence.
Complex regularized models can be useful; product enforceability requires a runnable graph test.
Evidence map
| Aspect | Finding |
|---|---|
| What it is | The essay that states plainly why finance is a hostile environment for machine learning and what follows from that. |
| Key result / formula | Three features of financial data separate it from the domains where machine learning succeeded. |
| Why it matters for backtesting | This is the reference to give a user arriving with expectations formed elsewhere, and it converts disappointment into design. |
What it is
Israel, Kelly and Moskowitz are practitioners and researchers who use these methods, which makes the list of obstacles more useful than a sceptic's would be.
Key result / formula
The data is small: decades of monthly returns amount to a few hundred observations per series, against the millions of labelled examples behind image recognition, and the sample cannot be enlarged by collecting more, since history arrives at one year per year. The signal-to-noise ratio is extremely low: the predictable component of returns is a tiny fraction of their variation, so a method that would find a weak signal in a clean domain is overwhelmed here. And the data-generating process is not stationary: markets adapt, and a relationship that held is competed away, so the assumption of a fixed mapping from features to outcomes fails in the direction that matters. The authors describe constraints that shape the useful application of these methods, favouring heavy regularisation, economic structure imposed in advance, modest model complexity, and expectations of small effects.
Why it matters for backtesting
Small data means the number of parameters must be small and the validation must be economical with observations. Low signal means the realistic target is a small improvement, and a model that appears to predict strongly is far more likely to be leaking than to be good. Non-stationarity means the test must be chronological and the result must be checked for decay across sub-periods. Each check requires a separately verified workflow and enough independent data, and an agent should enforce them rather than argue about model choice, since model choice is the least important decision in this setting (see [Financial data as low signal-to-noise (why ML overfits here)]).
Source
Israel, Kelly & Moskowitz, "Can Machines 'Learn' Finance?", Journal of Investment Management 18(2), 2020, 23-36. Primary source