Python Backtest vs. MT5 Live Test: A 92-98% Gap
Contents
- Strategy Specification — Asian Range → NY Session Breakout
- Python Walk-Forward Results — Impressive Numbers
- But Then — The 14-Year MT5 Live-Environment Backtest (USDJPY_NY_BREAKOUT_EA.mq5 v2.0)
- Why Is the Gap So Large? — Five Hypotheses
- Industry-Level Lessons
- FXEA365's Decision
- Recommendation for the Industry — To Python BT Users
- Related Links
How Python Testing and MT5 Live Testing Diverged by 92-98%
Note (2026-10-11): The EAs introduced in this article are no longer distributed. The figures are the verification records from that time. For current EAs, see the EA list & ranking.
This is the operator of FXEA365.
In this article, I disclose in full how, on 2026-05-29, I compared an NY Session Breakout strategy using a Python walk-forward test and a 14-year MT5 live-environment backtest, and found a 92-98% gap in PF.
This is important data that proves once again, at an industry level, that "impressive numbers in a Python backtest do not guarantee the strategy will work in MT5."
Strategy Specification — Asian Range → NY Session Breakout
| Item | Setting |
|---|---|
| Timeframe | H1 |
| Asia range | 0-4h UTC (Asian session range formation) |
| Trading hours | 15-20h UTC (during the NY session) |
| Entry | Breakout of the Asian session High/Low |
| Exit | SL = ATR × 0.5 / TP1 = ATR × 0.5 (50% close) / TP2 = ATR × 7.0 |
| Lot | Risk 2% (CASCADE semi-compounding) |
In theory, the strategy has an edge:
- The NY session (UTC 15:00-20:00 = JST 24:00-05:00 the next day) is when institutional investors are active and liquidity concentrates
- The Asian range is a natural reference level, and a break above or below it is a signal with strong directional intent
- Two-stage exits (TP1 = ATR×0.5 / TP2 = ATR×7) defend against false breakouts while capturing trends
Python Walk-Forward Results — Impressive Numbers
From 2026-05-27 to 28, I ran a walk-forward test on 9 currency pairs (Train 60% / Test 40%) with scripts/gen_h1_ny_walkforward.py:
| Pair | train PF | test PF | test DD | test WR | Consistency | OOS survival rate |
|---|---|---|---|---|---|---|
| EURUSD | 7.42 | 9.03 | 5.47% | 79.6% | 0.82 | 500/500 |
| GBPUSD | 11.78 | 13.00 | 31.85% | 84.2% | 0.91 | 500/500 |
| XAUUSD | 7.76 | 11.96 | 13.18% | 80.9% | 0.65 | (previous test) |
| EURJPY | 9.41 | 7.93 | 7.46% | 79.2% | 0.84 | 500/500 |
| GBPJPY | 9.74 | 8.24 | 12.30% | 79.3% | 0.85 | 500/500 |
| USDJPY | 6.12 | 11.62 | 36.24% | 80.7% | 0.53 | (previous test) |
| CADJPY | 7.59 | 5.76 | 32.81% | 75.9% | 0.76 | 499/500 |
| AUDJPY | 8.49 | 6.32 | 27.33% | 74.4% | 0.74 | 351/500 |
| NZDJPY | 9.22 | 6.51 | 15.05% | 70.6% | 0.71 | 500/500 |
All 9 pairs showed PF ≥ 5 × win rate ≥ 70% × OOS survival rate of 500/500.
By industry convention, this is the kind of result you would publish on your site with headlines like "A dream universal edge discovered!" or "Expanding the lineup with an all-pairs roadmap!"
But Then — The 14-Year MT5 Live-Environment Backtest (USDJPY_NY_BREAKOUT_EA.mq5 v2.0)
Following the CLAUDE.md "live testing is absolutely mandatory" rule, I retested the 4 strongest pairs from the Python walk-forward in a 14-year MT5 live-environment backtest:
| Pair | Python WF (test) | MT5 live 14-year BT | Gap | Account result |
|---|---|---|---|---|
| USDJPY | PF 11.62 / DD 36% | PF 0.77 / DD 99.72% | −93% | -$9,966 (near-total loss) |
| EURUSD | PF 9.03 / DD 5.47% | PF 0.73 / DD 99.84% | −92% | -$9,984 (near-total loss) |
| GBPUSD | PF 13.00 / DD 31.85% | PF 0.21 / DD 101.12% | −98% | -$10,112 (account wiped out) |
| EURJPY | PF 7.93 / DD 7.46% | PF 0.49 / DD 100.01% | −94% | -$10,001 (account wiped out) |
All 4 pairs failed, with a PF gap of 92-98% versus the Python walk-forward.
Why Is the Gap So Large? — Five Hypotheses
1. Spread and slippage not accounted for
The Python backtest calculates on a close-to-close basis. In live execution, every fill is degraded by spread (0.5-2 pips) + slippage. A loss of 1-3 pips per trade accumulates.
2. Data quality
Python uses Dukascopy M5 data aggregated into H1. The MT5 live environment uses broker tick data (more granular price movement). Even for the "same" H1 candle, the price movement inside it differs.
3. Divergent execution models
Python assumes ideal fills (always filled at the desired price). In the MT5 live environment, volume/spread constraints cause the actual fill price to deviate from the desired price. This is especially pronounced during high-volatility hours (such as right after the NY open).
4. The live-environment fragility of the ATR × two-stage TP
The design is elegant in theory, but TP2 = ATR×7 is not reached in live conditions.
- Python BT: calculates the optimal SL/TP "with hindsight" on historical data
- Live: SL = ATR×0.5 is hit frequently, trends do not last until TP2 = ATR×7, and stop-losses pile up
5. The live-environment downside of CASCADE semi-compounding
A design that dynamically adjusts the lot size according to the balance accelerates the damage during losing phases:
- Balance $10K → Risk 2% → Lot 0.2
- After a losing streak, balance $8K → Risk 2% → Lot 0.16 (slight decrease)
- After a losing streak, balance $5K → Risk 2% → Lot 0.10 (loss suppression does not work)
The theoretical "magic of compounding" works in reverse in live conditions.
Industry-Level Lessons
1. Python BT / walk-forward is not a reliable guide to MT5 live results
- Even "PF ≥ 5 × WR ≥ 70% × OOS survival rate 500/500" is meaningless in live conditions
- Walk-forward "robust" or "OOS-robust" verdicts assess robustness within the Python data, not a proxy for live performance
2. Even "economically rational hypotheses" require live testing
- "Institutional investors are active during NY hours" and "a break above or below the Asian range is a directional signal" are theoretically correct
- Yet they all failed over 14 years in the MT5 live environment = the gap between theory and live execution is large
3. Stronger grounds for the CLAUDE.md "live testing is absolutely mandatory" rule
- 25 EAs + 4 NY Session pairs = a total of 29 tests conclusively established that Python BT cannot be trusted
- This is the data that supports, at an industry level, FXEA365's brand pillar ("live-environment proof, not flashy numbers")
FXEA365's Decision
The NY Session strategy cannot be published on the site, as all 9 pairs failed MT5 live testing.
The published lineup at the time was limited to the four main pillars: EURJPY ASIA / GBPJPY ASIA / BLAZE GOLD / AUDCAD PEACE-CLONE (all four are now discontinued).
A combined annual return of +35.5% (Risk 5%) is by no means flashy, but these are reliable numbers that genuinely held up in a 14-year live-environment backtest.
Recommendation for the Industry — To Python BT Users
This is not a rejection of Python backtesting. However, I recommend using it as follows:
| Proper use of Python BT | Recommendation |
|---|---|
| Initial validation of strategy ideas (is this direction promising?) | ✅ Recommended |
| Screening parameter ranges (where to focus precise testing) | ✅ Recommended |
| Walk-forward robustness checks (overfitting checks with Train/Test separation) | ✅ Recommended |
| Basis for figures published on a site | ❌ Prohibited (diverges from live results) |
| Using "OOS survival rate" as a selling point | ❌ Prohibited (not a proxy for live results) |
| Decisions on publishing an EA | ❌ Prohibited (MT5 live testing is mandatory) |
Python BT figures are only "initial screening indicators". MT5 live testing is required before a production release.
Related Links
- Currently published EAs: EA list & ranking (the four main EAs of that time are discontinued)
- Related blog posts:
FXEA365 — We publish for free only strategies that genuinely work in a 14-year MT5 live-environment backtest, not flashy advertised figures.
Related
📧 Free 8-email course + subscriber-only pack of 13 MT5 indicators
Right after you sign up, you get a download link for a subscriber-only pack of our 13 MT5 indicators: compiled files (.ex5), source code and a setup guide. Then 8 emails on algo-trading essentials, reading backtests correctly and choosing a broker (one a day for the first 5 days, then 7, 14 and 30 days after signing up). You also get notified before each price rise of the versions sold on the MQL5 Market (launch pricing that steps up as copies sell). Our EAs are free on this site with a live account at a partner broker.
* Privacy strictly protected. You can unsubscribe at any time.
Have a question about EAs or FX?
You can post it on the Nattzy trading forum (free sign-up). Questions beyond this article are welcome too — EA settings, choosing a broker and more.
Ask on Nattzy ↗