How we test a strategy
Most trading strategies online come with a screenshot and a promise. We take the rules, write them down exactly, and run them through the same process every time. Most of them fail, and we publish those results as prominently as the ones that pass. This page is that process, step by step.
1. Write the rules down before testing
Every test starts from the strategy as it is sold or taught: a book, a course, a video, a forum post or an academic paper. We turn it into exact rules for entry, exit, stop and position size, and note where the source was vague. Those rules are fixed before the first test runs.
2. Measure the effect before writing any code
Many strategies rest on a simple claim, such as "price tends to reverse after the London fix". Before building anything, we measure that effect straight from the data and compare it with the full cost of trading it: spread, commission, overnight financing and the hours you can actually trade. If the effect is smaller than its costs, the strategy is rejected at this stage, and we write up why.
3. The data
Market data comes from Dukascopy. Most tests run on real tick data in MetaTrader 5, using its "every tick based on real ticks" mode, with the spread as it actually was minute by minute. Some tests use one-minute data instead, for example on stock indices or for early measurements. Each write-up states which data it used.
4. The costs
- Commission: $3.50 per lot per side, IC Markets' Raw account rate.
- Spread: the real, changing spread from the tick data, not a fixed average.
- Overnight financing (swap): charged on every position held past the daily rollover.
- Index CFDs: spread plus overnight financing, priced at IC Markets.
- Bitcoin: re-priced to IC Markets' real cost (about a $5 spread and long financing of about -20% a year), because the data vendor's Bitcoin spread is many times wider than a broker's.
5. In-sample and out-of-sample
The data is split in two. In-sample (2019 to 2021) is where we are allowed to try settings. Out-of-sample (2022 to 2025) is held back and used once. Slower strategies that trade only a few times a year start their in-sample period in 2017 instead, with the same out-of-sample years and the same gates.
Some edges depend on something that did not exist in 2019 to 2021, such as a new product or a change in central bank policy. Only for those, and only when the reason and the date it began are written down before any test, we use a later split: in-sample 2022 to 2023, out-of-sample 2024 to 2025, with the same 14 gates. We still run the older years and report them, and we lean more on the forward test because the history is shorter. A recently published strategy on a market that has not changed does not qualify. If it fails on 2019 to 2021, that is a real result, not bad luck with the dates.
We do not pick the single best setting from the in-sample search, because that is usually luck. We pick a setting whose neighbours also work: nearby values have to average at least 70% of its performance, and at least 70% of them have to be profitable.
The out-of-sample test then runs once on the frozen setting. Our testing code refuses to run on the out-of-sample years at all unless the run is approved on purpose, and we count every out-of-sample run a strategy uses.
6. Checking the code and the trades
A backtest is only as honest as the code behind it. Before any test runs, the strategy's code is reviewed against a checklist of mistakes we have learned to look for. After each test, the trade list is checked against the rules: did it enter when it should have, exit where it should have, and size the trade correctly? A test does not count until the trades match the rules.
7. The 14 gates
To be called validated, a strategy has to pass every one of these. They are the same for every strategy.
| Gate | Pass rule |
|---|---|
| Enough in-sample trades | At least 150 |
| Enough out-of-sample trades | At least 100 |
| Trades often enough | At least 30 a year |
| In-sample profit factor | At least 1.30 |
| Out-of-sample profit factor | At least 1.15 |
| Out-of-sample drawdown | No more than 20% |
| Return per trade vs its swings | Trade Sharpe at least 0.05 |
| Edge kept out-of-sample | At least 70% of the in-sample profit factor |
| Return kept out-of-sample | At least 50% of the in-sample annual return |
| Drawdown did not blow up | Out-of-sample drawdown at most 1.5x in-sample |
| Profitable most years | At least half of out-of-sample years positive |
| Survives higher costs | Still profitable with $5 more commission per lot per side |
| Not just a lucky order of trades | Monte Carlo chance of a loss at most 15% |
| Beats doing nothing clever | Risk-adjusted return at least as good as buy-and-hold and 12-1 momentum |
Three of them need a little more explanation:
- Higher costs: we re-run the out-of-sample test with an extra $5 commission per lot per side. An edge that disappears when costs rise a little would not survive a worse broker or a busy market.
- Monte Carlo: we redraw the out-of-sample trades at random 10,000 times. If more than 15% of those reshuffled histories lose money, the result may have been a lucky run. Try it on your own trades with our Monte Carlo simulator.
- Benchmark: over the same out-of-sample years, the strategy's return relative to its drawdown has to beat simply buying and holding the market, and a basic 12-month momentum rule. If something that simple did better, the strategy is not worth the effort.
8. The verdicts
- Validated: passed all 14 gates.
- Mixed: a real edge that held out-of-sample but failed at least one gate. These are the near-misses, and the best of them go to the forward test.
- Rejected: failed the gates, or failed in-sample before an out-of-sample run was worth spending. Rejected strategies are written up in full, with the numbers.
Every verdict is on the results page, losers included, and each strategy has a longer write-up on the blog.
9. Every published number is checked twice
Any number that decides a verdict or appears in a write-up has to be reproduced a second way, for example by a separate script or an independent calculation, before it is published. If the two disagree, nothing goes out until we know why.
10. After the backtest: the forward test
A backtest can only look backwards. The strategies with a real edge then trade in real time on IC Markets demo accounts, one account per strategy, and every closed trade is published on the forward test page next to what the backtest expected.
What this cannot tell you
- Past results do not guarantee future results. Markets change, and edges fade.
- Demo fills are not live fills. Real slippage and spreads at busy moments can be worse than a backtest or a demo account shows.
- Our risk numbers assume trades are independent. Real losing streaks tend to bunch up when conditions change.
- We test the rules as written. A trader who adds judgement may do better or worse than the mechanical version.
- This is research, not financial advice. See our risk disclaimer.