We test trading ideas the way we'd want ours tested: rules written down before the first calculation, real costs, a placebo and a single shot at the hold-out sample. This is where we describe the ones that didn't survive — along with the mistakes we made on the way.
We don't name vendors or products: we describe the result of our own reconstruction of the rules on our data, not a review of anyone's offer.
The most dangerous bug in a backtest is the one that improves the result, because nobody goes looking for it. In our channel-breakout test, a filter taken from the daily chart “knew” the day’s close before it happened. It turned a near-zero result into a strategy with a PF of 1.29, survived our first fix, and only came to light when we compared the backtest with signals from a demo account, value by value.
Read →The time next to a candle on the chart is the broker’s server time, not universal time and not Polish time. In three months we mixed up these clocks at least nine times: in backtests, in automated systems running live, and even in our own rule that was supposed to protect us from exactly this. In a New York session breakout test, the error shifted every session by 2–3 hours and gave itself away with an odd count: the strategy produced 3 trades, the placebo 127. The cheapest check turned out to be a single question: at what time of day does gold move the most?
Read →The indicator draws an arrow, and the chart history looks like a textbook: the arrows sit just before the move. We measured what a trader actually gets by entering at the moment the arrow appears. The same set of signals scored on the saved history gave a PF of 1.33; on real ticks it gave a PF of 0.54, or −0.30 R per trade. The difference comes from arrows that live for about a minute and then vanish from the history.
Read →We tested 1,344 variants of simple rules built from popular indicators and picked the 13 best in 2018–2022. On average they made +0.20 R per trade. In 2023–2025, years we did not look at while choosing, the same rules made +0.05 R, and none of them passed validation. About three quarters of the “edge” was the cost of picking the best out of many.
Read →We tested the opening range breakout (ORB) on gold, separately for the London and New York sessions, on about 930 sessions of each from 2018–2023. The best variant gave a PF of 1.06, and 1.03 without the three best trades, with a drawdown that in simulation reached 20–32% of capital. The 5 R target was hit 29 times in 926 sessions in London and not once in New York, so the “rarely, but big” strategy in practice closed positions at a fixed time. Before we got to the result, we caught our own clock error, which had shifted every session by 2–3 hours.
Read →Traders like round numbers: they place their take-profit orders on them more often, and their stops just beyond them. Studies of real orders at a currency-dealing bank from around the turn of the century showed that this is not a myth. We checked whether you can make money from it today, on gold, six indices and USD/JPY in 2018–2023. A limit order at the nearest round number in the direction of the trend lost money after costs on all eight instruments (PF 0.83–0.93), and after the correction for eight tests, round numbers turned out no better than random levels at the same distance.
Read →Three times we built the same strategy: a limit order in the direction of the trend, waiting for a pullback to an “important level”. We changed only the source of the levels (the previous day’s levels, round numbers and a profile of real exchange volume), and each time we compared the result with a market entry, with the same stop and the same position management. On eight instruments this gave 11 comparisons, and the limit did worse in 9 of them. The phenomenon has a name: adverse selection of fills.
Read →We assembled four popular elements of Smart Money Concepts into a single mechanical rule and tested it twice, on six instruments and with two different ways of exiting. With a tight target, the signals were slightly better than random entries but too weak to cover costs: PF from 0.73 to 1.13, and none of the 12 combinations survived the correction for multiple testing. With a wide trend-following exit, they did worse than random entries on all six instruments.
Read →The textbook says: if US data come in better than forecast, the dollar strengthens and gold gets cheaper. We tested this with criteria written down before the first calculation, and found no direction. Then we found two of our own mistakes in that test. A correct measurement showed where the direction really is: in the first two minutes, out of reach for an ordinary trader. This chapter tells both stories, because the second teaches more than the first.
Read →Carry is the oldest premium on the currency market: you buy a currency with a high interest rate against one with a low rate and collect the difference every night. On 28 pairs in 2005–2026, our rule picked the sides of pairs clearly better than chance (placebo p = 0.004), so the premium exists even after costs at a CFD broker. It earned, however, an average of 2.9% a year with a drawdown of 37.6%. After cutting the risk to a level an account can bear, 0.2% a year was left. This chapter is about the fact that a real signal and a profitable strategy are two different things.
Read →On a platform with signals to copy, we went through 1,920 public signals and, before seeing a single equity curve, mechanically selected those with a long history on gold. Two candidates were left. The first was eliminated by the test for increasing the lot size after a loss. While writing this piece, we found that the test had read the history backwards; after the fix, the candidate still does not meet the condition, but the word “martingale” would be too strong. The second had a real, positive result even after adding our own cost, but no simple rule reproduced its entries: to catch more than 90% of them, a rule would have had to flag more than 20 times as many candles as the criterion allowed.
Read →We scored over 1,200 gold signals from three public channels the way an automated copy-trading system would: entering shortly after publication, with the stop and targets from the message and real costs. The channel with the longest history, over five years, averaged −0.012 R per signal — that is, zero. In none of the channels did the direction of the signals differ from random, and in one of them more than 90% of the message numbers were missing.
Read →We built 40 artificial markets on which, by construction, no one can make money, and on each of them we searched 100,000 variants of a simple breakout strategy. In the training years the best variant looked like a finished system: on average +0.24 R per trade and a PF of 1.47. In the validation years: −0.02 R and a PF of 0.97. This is not the bad luck of one simulation but the arithmetic of picking the best out of many. That is why our strategy tester deliberately has no search function.
Read →