Why You Can't Trust a Backtest With Few Trades

When people look at an EA's backtest, they read the profit factor or the annual return first. But what decides whether you can trust those numbers is the trade count. With too few trades, even a spectacular profit factor may be nothing more than luck.

PF 5.56 with 28 trades might just be chance

A concrete example. A Bitcoin daily (D1) trend EA showed this over 8 years of real-tick testing:

• Profit factor 5.56
• Maximum drawdown only 3.2%
• Trade count 28

PF 5.56 is a beautiful number, and many people jump at it. But the moment you see only 28 trades in 8 years, the picture changes. If it trades three or four times a year, then a couple of large wins can make the whole result look dramatically good. What if 3 of those 28 trades happened to be lucky big winners? Remove those three and the edge might vanish.

By contrast, a USDJPY trend EA over 9.4 years: 631 trades, profit factor 1.50, win rate 34.5%. The PF is modest — but 631 trades strongly suggests that 1.50 is not an accident. Remove a handful of those 631 and the result barely moves.

Why count matters — the coin-flip

Flip a coin three times and get three heads. Can you conclude the coin is biased toward heads? No — three in a row happens one time in eight with a perfectly fair coin.

The same thing happens with EAs. With few trades, you cannot distinguish "it happened to catch a good market" from "it has a real edge." The more trades, the harder it becomes to explain the result by luck, and the more likely the numbers are real. Statisticians call this sample size.

A rough guide: how many is enough

There is no exact threshold, but a practical guide:

More important still: read the trade count together with the test period. If 500 trades are all crammed into one year, you have seen exactly one market regime. Only when 500+ trades span multiple regimes — the 2008 crisis, the 2020 crash, the 2022 hiking cycle — can you call it robust.

Tricks that hide a thin sample

Sales pages use a few tactics to hide a low trade count:

• Inflating net profit with compounding. A "+5,000%" figure comes from the multiplication of compounding, not from the number of trades. Net profit can look enormous even with few trades.
• Showing only the profit factor. Like PF 5.56 (28 trades) — the PF is emphasized and the count is omitted.
• Quoting only the annual return. Annual return is convenient for hiding trade count.

So when you read a backtest, always look for the trade count. If it is not stated, that omission is itself a warning.

Check it yourself

Our free EA (a closed-bar Donchian breakout) is open-source, real-tick verified, with 631 trades on USDJPY H1. Run the backtest and you can see the trade count, profit factor and year-by-year P&L with your own eyes — a working example of what a high-count, multi-regime EA looks like statistically.

Donchian Trend Engine (free EA)

Our paid products are held to the same standard, with trade count, test period and real data published for every one. A product with only a few dozen trades is never a flagship, however high its PF.

Summary

• A backtest's reliability is decided by its trade count (sample size).
• 631 trades at PF 1.50 is more trustworthy than 28 at PF 5.56.
• Aim for 500+ trades spanning multiple market regimes.
• Compounding, PF-alone and annual-return-alone are used to hide a thin sample.
• Distrust any backtest that does not state its trade count.

Related: How to read an MT5 backtest report · Why the same EA gives opposite results on real ticks vs OHLC · How to spot curve fitting