Backtest Not Reproducing? 5 Things to Check First

"The published results don't match the backtest I ran myself." Among the inquiries we receive about the EAs we distribute, this is one of the most common.

Here is the conclusion first: even if you test the same EA over the same period, the results will differ if even one part of the environment is different. In most cases the cause is not the EA itself but a difference in the testing conditions. The five places to suspect, in rough order, are:

1. Server time and time zone
2. Symbol specifications
3. Swap
4. History data
5. Model type (tick generation method)

That said, aligning all five does not guarantee an exact match. We are upfront about this in the "Limits" section later on.

Why the Same EA Can Produce Different Results

An EA is a machine that makes decisions from the prices and symbol information it is given. If the input differs even slightly, the moment of entry shifts. Once the first trade shifts, the account balance, lot size and position状況 that follow all change in a chain, and over several years of testing the final result can look very different.

It helps to think of reproducibility problems as cases where a small difference in one place gets amplified over time.

The 5 Places to Check

1. Server Time and Time Zone

The time in MT5 is the broker's server time, not your computer's clock. Server time differs by broker: some switch with daylight saving time and some do not.

The EAs most affected are:

• EAs with time-of-day filters, such as "Tokyo session only" or "exclude the New York session"
• EAs that use the daily open, high and low as references (the daily bar cutoff time differs by broker)
• EAs with special handling at the weekly open or before the Friday close

With a broker that creates a short daily bar on Sunday, the number of daily bars itself changes. For a daily-bar-based breakout EA, this alone can produce an entirely different result.

2. Symbol Specifications

Even under the same name, "XAUUSD", the contents are not necessarily the same. Check the following in the specification window:

• Number of digits (decimal places) and the size of a point
• Contract size
• Minimum lot and lot step
• Stop level (the minimum distance at which pending orders can be placed)
• Margin calculation method

If the digits differ, a stop loss specified in "points" can differ by a factor of 10 in actual distance. With a broker that has a wide stop level, an order with a tight stop loss may not be accepted at all.

3. Swap

The MT5 tester calculates by applying the swap values from the symbol settings at the time you run the test to the entire period. It does not use the actual historical swap changes. For this reason, even with the same broker, the result can change depending on the day you run the test.

• The longer an EA holds positions, the greater the impact
• The day on which three days' worth of swap is charged differs by broker
• On account types without swap (swap-free), other fees may apply

For EAs that hold positions for several days, a difference in swap can show up clearly as a difference in profit and loss.

4. History Data

The price data the tester uses is downloaded from the server of the broker you are connected to. The start of the data, gaps and tick density differ from broker to broker.

• The data is incomplete at the beginning of the test period
• The download is unfinished, and the result changes on a second run
• Even with the same broker, results change later because of data updates on the server side

Be sure to check the Journal tab for any entries about data synchronization or missing history.

5. Model Type

Whether you choose "Open prices only", "1 minute OHLC", "Every tick" or "Every tick based on real ticks", the same EA can produce completely different results. The closer the stop loss and take profit are, the more sensitive the EA is to how ticks are generated.

Also check these two:

• Spread setting: Choosing "Current spread" fixes the spread at the value from the time you ran the test. If you run it on a weekend or early in the morning, you may end up testing with a wide spread
• Execution delay setting: For short-term trading EAs, results diverge depending on whether a delay is applied

Reproducibility Checklist

When comparing against published results, filling in the table below from the top makes it easier to narrow down the cause.

The last item, "Input parameters", is surprisingly easy to overlook. Check every time that values you changed in a previous test have not been left in place.

If you are unsure about your basic MT5 settings, the quickest fix is to go back through the steps in MT5 Setup.

Limits and Caveats

This part is the inconvenient one.

• Even if you align all five, you may not get an exact match on another broker's data. The ticks themselves differ by broker, and this cannot be resolved on the tester's side
• The backtest figures we publish were also measured with a specific broker's data at a specific point in time. We do not guarantee that you will get the same numbers in your own environment
• In some cases you cannot precisely isolate which difference is having how much effect. When a gap appears, the only way is to change one item at a time and retest
• Even when your backtests line up, they are only verification on past data. They do not promise future results

Conversely, an EA whose results do not fall apart when you change brokers is easier to trust. Testing across several brokers and comparing is time-consuming, but it is an effective check. Differences in specifications between brokers are summarized in Broker Comparison.

Summary

• Even with the same EA and period, results differ because of environment differences. The cause is often the testing conditions, not the EA itself
• The order to check is: server time, symbol specs, swap, history, model type
• Note that swap uses the value from the time of the test across the entire period
• Spread settings, delay and leftover input parameters are also easy to miss
• Even with everything aligned, results may not match exactly on a different broker
• Whether an EA holds up when you change brokers is a useful criterion when choosing one

Other testing know-how is collected in EA & MT5 Know-How.