Same Trade Count, Different Reliability: Independence

Conclusion: Recount Trades as "Independent Trials"

The more trades a backtest contains, the more you can trust its results. But that holds only when each trade is independent of the others. An EA that opens several positions at the same time in the same direction, or several EAs on different symbols that react to the same market move, can have an effective sample size far smaller than the trade count in the report.

When we evaluate an EA, we look at "how many independent decisions were made" before "how many trades were taken." This article covers that way of thinking and the steps to check it.

Why Correlation Shrinks the Sample Size

You May Be Counting the Same Decision Five Times

Suppose an EA opens 5 positions at once every time a signal appears, and all 5 close on the same price move. The report shows 100 trades, but the EA made only 20 decisions about the market. Flipping a coin 100 times is very different from flipping it 20 times and copying each result 5 times. The second rules out chance far less well.

In statistics, the stronger the correlation between trades, the smaller the effective sample size. In a simple model, with n trades and an average correlation ρ between them, the effective sample size is roughly n ÷ (1 + (n − 1) × ρ). If ρ is 0, you get the trade count as-is, but even a slightly positive ρ cuts the effective sample size sharply. Real correlations are never neatly constant, so treat this formula as a rule of thumb for building intuition.

A "Different EA" Is Not Necessarily Independent

The problem doesn't stop inside a single EA. Take our MEGAMAX DONCHIAN and DONCHIAN ENGINE. In backtests on USDJPY H1 over the same test period (2017.01.01 - 2026.06.01), they made 629 and 631 trades, with average holding times of 32:58:21 and 32:42:37. With numbers this close, it is natural to assume their entry timing overlaps heavily. (We haven't matched trades one by one, so we can't say how closely they coincide.) Running both together should not be treated as diversification backed by 1,260 independent trades.

Main Patterns That Undermine Independence

For multi-symbol portfolio EAs such as ATLAS, the symbol combination itself determines the quality of diversification. The trade count alone doesn't tell you how many of the backtest's 1193 trades came from low-correlation conditions.

Concrete Steps to Check

1. Export the deal history as CSV. You can export it from the "Deals" tab of the MT5 backtest report.
2. Group by entry time. Treat trades that entered in the same direction on the same bar (the same hour on H1) as one group. The number of groups is a rough measure of independent decisions.
3. Check for overlapping holding periods. An EA whose average holding time is longer than the interval between trades may be holding multiple positions at all times.
4. Recount using daily P/L. Recalculate win rate and PF using each day's P/L as one sample instead of each trade. This removes much of the padding from overlapping positions.
5. Compare the EAs you run together. Line up the daily P/L of the two EAs and check whether they win and lose on the same days.

We also cover the basics of validation in EA & MT5 Know-How.

Limits and Caveats

This approach has its own limits.

• Correlation is not constant. Symbols that are usually weakly correlated tend to move in the same direction together during sudden shocks. Even if they look "independent" in normal conditions, the assumption can break down exactly when it matters most.
• Aggregating to daily data also loses information. You get closer to independence, but the sample gets smaller, which makes judgment difficult in a different way. There is no fixed answer to which unit of aggregation is correct.
• We haven't recounted all of our own EAs. We haven't yet been able to publish the results of applying these steps to every EA. The trade counts we display are the number of fills in the report.
• Confirming independence does not guarantee profit. It only gives a rough measure of how much of the results chance could explain. It says nothing about future performance.

Summary

• Even with the same trade count, correlation between trades lowers statistical reliability
• Overlapping positions, overlapping holding periods, highly correlated symbols, and EAs with similar logic all reduce the effective sample size
• Grouping by entry time or recounting with daily P/L gives a rough count of "independent decisions"
• Trades that look independent in normal conditions can become strongly correlated in sudden shocks
• When combining multiple EAs, judge by whether their trade timing overlaps, not by symbol or name