We Finally Tested Our First Strategy the Hard Way. The Edge Held.

A few days ago I wrote about the strategy that started this whole project: a simple Asian-session range breakout on USD/JPY. It made $35,000 on paper.

But at the end of that post I had to admit a problem. I chose the trading hours using the same years I then tested on. There was no separate data to check it. I called the result "encouraging, not conclusive."

I did not like writing that. So I ran the strategy through our full testing process. This is the same process that has rejected most of our ideas. Here is what happened.

What the strategy does

The rules did not change. They are still simple:

• Find the high and the low of the Asian session (the quiet hours before London opens). The space between them is the range.
• If a candle closes above the range high, buy. If it closes below the range low, sell.
• Put the stop loss (the price where we close a losing trade) at the other end of the range.
• If the trade is still open at a fixed hour in the afternoon, close it.
• One trade per day, USD/JPY only.

Why only USD/JPY? When I first tested the same rules on EUR/USD, the account lost half its value. The profit factor (total wins divided by total losses) was 1.00. In other words, every dollar it made went to the spread (the cost to trade).

The Asian range means something during the yen's home hours. On EUR/USD it means nothing. So the fair test is the pair where there is a real reason for the edge to exist.

What is different this time: the data is split in two

The first demo found its best settings and tested them on the same years, 2021 to 2025. Our whole testing process exists to stop this mistake. If you give a computer the answers, it will always find settings that look great when you look back.

This time we split the data with a hard wall:

• In-sample (the data used to find settings): 2019 to 2022. The computer chose the session hours and the closing hour here.
• Then I froze those settings.
• Out-of-sample (new data the strategy never saw): 2022 to 2026. I ran it exactly once.

If the edge was only curve fitting (settings that fit past luck), it would fail here.

In-sample it made $9,295 with a profit factor of 1.21. That is fine, but in-sample results are supposed to look fine. The real question is the next chart.

Out-of-sample equity on USD/JPY, 2022 to 2026, net of modeled commission. $10,000 grew to $20,230 with a real drawdown in 2023.

On the new data, $10,000 grew to $20,230. That is:

• $10,230 net profit
• 1,012 trades
• profit factor 1.16
• roughly 19% a year

The profit factor kept 96% of its in-sample value. This almost never happens with an overfit strategy. Usually the edge drops sharply as soon as it sees new data. This one hardly changed.

It was not a smooth ride. In 2023 the account fell from $14,000 back toward $11,000 and stayed there for months. The worst drawdown (the biggest drop from a high point) was 23.7%.

The win rate (share of winning trades) is under half, at 45.75%. It works because the average win ($157) is bigger than the average loss ($114). This is normal for a breakout: often wrong, but right when it counts.

One good backtest is not proof

One equity curve, even on new data, is only one path through history. It could still be luck. So every strategy that passes now gets three more checks. These do not need a new backtest. This was the first strategy to face all three.

Check 1: double the trading costs

I re-ran the same out-of-sample trades with commission at $8.50 per side instead of our normal $3.50. That is more than any retail broker charges. The strategy still made $5,445 with a profit factor of 1.08. The profit is smaller, of course. But the edge does not need very cheap trading to work.

Check 2: replay the trades 10,000 times

This is the check I care about most. It is called a Monte Carlo test. We take the 1,012 trades and pick them at random, allowing repeats. Then we rebuild the account from them. We do this ten thousand times.

• Only 4% of these made-up histories lost money.
• The middle result (the median) made $10,111, almost the same as the real result.
• In the unluckiest 2.5% of runs, the profit factor fell just under 1.00. So the edge is real, but it is not big.
• When I shuffle the order of the real trades, the drawdown can reach 41% in a bad order. The real test only hit 24%. If you trade this, plan for 41%, not 24%.

The good part: the real drawdown landed at the 50th percentile of the shuffles, right in the middle. Be careful with a strategy whose reported drawdown is the luckiest result it could have had. That one hurts you later. This one is honest about its own risk.

Check 3: beat the simple options

A strategy is only worth the effort if it beats doing nothing clever. So I compared its MAR (yearly return divided by the biggest drawdown) with two simple options:

• just buying and holding USD/JPY
• a simple trend rule that buys when the price went up over the past year

Risk-adjusted return (MAR) versus naive baselines over the same out-of-sample window. The breakout scores 0.81 against 0.54 for buy-and-hold and 0.48 for 12-month momentum.

The breakout scored 0.81. Buy-and-hold scored 0.54 and the trend rule scored 0.48. So it beat the best simple option by about half again.

This matters to me. The last strategy I checked this way was a Bitcoin trend system. It lost to the simple trend rule two to one. Without this check, it would have looked good enough to use. The breakout passes this check with no help.

So why is it not trading live?

Passing these honesty checks does not mean it is good enough. It misses two of our limits:

• Our rule says a live strategy can have at most a 20% drawdown. This one has 24%.
• Our minimum in-sample profit factor is 1.30. This one had 1.21.

It is real, it is solid, and it is honest about its risk. But it is still a step below where I would put real money.

Before, we had only two choices: use it live or reject it. Neither fit. So this is the first strategy we are moving to forward testing. It goes into a tracked demo account and trades on prices that do not exist yet. No clever backtest can fake that test.

If the edge is real, it keeps working on next quarter's prices. If it is not, forward testing is where it will show. Either way, we will find out in public.

That is an unusual happy ending for the strategy that started the project. It did not come back as a winner or as a failure. It came back as "maybe, let's watch." To be honest, that is the most a backtest should ever earn.

The full numbers, equity curve, and results for each year are on the results page.

Past performance does not mean future results. These are backtests with realistic trading costs. They are not live trading results.