A backtest is limited by its data, its assumptions and the person who ran it. Each limit pushes the result in the same direction, and it is the favourable one. Knowing where the gaps are is the difference between a backtest that informs and one that misleads.
The data
Historical data is not what a trader would have seen. Spreads in the data are averages or estimates; the widening at news and rollover is absent. Gaps are smoothed. Ticks are often generated rather than real. Some instruments' history is short, patched from other sources, or adjusted after the fact. A strategy that looks good on clean data may have been trading prices that never existed.
The fills
The tester assumes every stop and limit order fills at its level, every market order fills at the price shown, and there is no slippage. Live, stops slip in fast markets, limit orders are missed, and market orders in thin conditions fill worse. The more often a strategy trades and the tighter its stops, the larger this gap.
The costs
Commission and swap are applied only if the tester is told to, and the spread is whatever was entered. A strategy holding positions for weeks with swap omitted, or trading forty times a day at a fixed half-pip spread, has been tested under conditions that do not exist.
The choices
The strategy's parameters were almost always chosen on the same data the backtest shows. A strategy fitted to a period will look good on that period; that is what fitting means. The strategy tested is also the survivor of every version the developer tried and discarded, so the backtest shows the best of many attempts, not a fair draw. The page on curve fitting covers this in more detail.
The regime
A backtest covers the conditions that happened, in the proportions they happened. A five-year test through a rising market tells you about rising markets. It says nothing about the next fall, and it cannot, because the future contains conditions the data does not.
A backtest of a gold strategy over four years shows 280 trades, a 61 per cent win rate and a smooth curve. It was run on one-minute bars with a fixed 20-cent spread, no swap and no slippage, on parameters chosen the previous week. Live, the spread at the hours the strategy trades averages 32 cents and widens to a dollar at US data releases, the swap on the multi-day holds costs a third of the average winner, and the win rate in the first six months is 48 per cent. The backtest was not dishonest. It described a market that had been tidied up.
Treat any backtest as an upper bound. Ask what it assumed about spread, slippage, commission and swap; what tick model it used; whether the parameters were chosen on the tested period; and what happened in the years it does not cover. Then look at the live record, which has none of these problems, and judge the gap between the two. A vendor whose live results run modestly below their backtest is showing you something real. A vendor with no live results is showing you a backtest, and this page is a list of the reasons that is not enough.