Tested strategies Verdict: No edge

Do AI and machine-learning EAs actually work?

By Otto Research 2 min read Tested across 17 years, with real costs

AI EAs claim to use machine learning, neural networks or similar to predict price or to filter trades. The marketing rarely says what the model is, what it was trained on, or how it was validated.

The claim is that a model can find patterns in price that rules cannot.

We tested machine-learning classifiers trained to predict the outcome of trades, across a range of models and features, on 17 years of data. The classifiers performed at chance: they could not distinguish trades that would win from trades that would lose any better than a coin. Models that appeared to do better on the data they were trained on lost that ability entirely on data they had not seen.

The reason is that price series contain very little predictable structure at the horizons retail strategies trade, and a flexible model fits the noise with great confidence.

If you buy one anyway, ask for out-of-sample results, ask how often the model is retrained, and treat a model that is retrained monthly as a curve-fitting procedure with a marketing department.

The ones that survived

See what Otto trades instead.

Most strategies do not survive an honest test. Otto is built on the few that did, and every trade is recorded live on Karnek.