The one line of code that made our backtest lie by 11 points
We built a test of the 200-day moving average. It reported 18.6% a year, beating buy-and-hold by eight points with a third of the drawdown. It was wrong. The real answer was 7.2% — and the bug was a single array index.
The result that was too good
A strategy that beats the S&P by eight points a year while cutting drawdown by two-thirds would be one of the great discoveries in finance. Published research on this exact rule finds roughly equity-like returns with lower volatility — never free alpha of that size.
So the first useful instinct is not excitement. It is suspicion. When a backtest reports something extraordinary, the overwhelmingly likely explanation is a bug in the backtest, not an edge nobody else noticed.
The bug
Here is the loop, simplified. See if you can spot it:
for i in range(1, n):
want = px[i] > sma(i, 200) # decide using TODAY's close
if want != inpos:
inpos = want
if inpos:
val *= px[i] / px[i-1] # then collect TODAY's moveThe strategy decides whether to be in the market using day i's closing price — then captures day i's return. It buys knowing how the day already ended. In reality you can only act on a close after it happens.
The fix is one index:
for i in range(1, n):
want = px[i-1] > sma(i-1, 200) # decide with YESTERDAY's close
if want != inpos:
inpos = want
if inpos:
val *= px[i] / px[i-1] # collect today's moveBefore the fix: 18.64% a year, −11.2% worst drawdown.
After the fix: 7.18% a year, −27.7% worst drawdown.
Eleven and a half points of annual return, invented by one character.
Why this matters more than it sounds
Compounded over 30 years, that difference turns $10,000 into $1.7 million instead of $80,000. Every strategy sold to you on the strength of a backtest is one index away from this. And the bias is seductive precisely because it produces plausible results — not a 900% monthly return that anyone would question, but a smooth equity curve that looks like skill.
How to check your own
1. Shift your signal by one bar and re-run. If the returns collapse, your strategy was living on information it couldn't have had. A genuine edge degrades slightly; a lookahead bug falls apart.
2. Be suspicious of any drawdown that's far below the asset's own.Sidestepping crashes almost perfectly is the signature of peeking.
3. Check what the benchmark includes. Comparing a timing strategy against a price-only index while your strategy gets dividends quietly gifts it ~2% a year.
4. Ask where the money comes from. Every dollar of edge is somebody else's loss. If you can't name who is on the other side and why they keep showing up, the edge is probably an artifact.
We published this because we got caught by it
Not in a hypothetical. In our own research, an hour before publishing. The number was beautiful and it was fake, and the only reason it didn't become a headline is that it was too beautiful.
If a test we wrote, reviewed, and intended to publish could get this wrong, assume the screenshot in the ad you're looking at has the same bug — or a deliberate version of it.
We test one strategy people swear by and publish whatever the data says — including when it says the strategy doesn't work. Free, no pitch.