4,096 trading strategies, 20 years of the S&P 500: none beat luck

Out of thousands of rule combinations run on twenty years of S&P 500 prices, only a handful beat buy and hold in the past and kept beating it later—but after correcting for luck and the sheer number of tries, not a single one remained. Here is what the test found, why the best past performer failed to last, and what this means for anyone searching for a trading edge.

How many strategies "worked" before and after 2018?

The test covered all 4,096 combinations of twelve simple trading rules—like "price above the 200-day average" or "skip Mondays"—on daily prices of the S&P 500 ETF (SPY) from 2006 through 2026. The strategies held SPY when all their chosen rules agreed, otherwise they sat in cash. Every in-or-out switch paid a 0.05% cost, and sitting in cash earned nothing [1].

In the first half ("in sample": 2006–2017), 439 of the 4,095 rule combinations beat buy and hold's Sharpe ratio, a measure of risk-adjusted return [1]. When followed into the next nine years ("out of sample": 2018–2026), just 31 of those still managed to beat buy and hold [1]. This is less than 1% of all tries.

But this is before accounting for luck. With so many combinations, some rules will always look good on past data by pure chance.

Why almost every edge disappears: the luck correction

When you try thousands of strategies, the best in the backtest is almost certain to be overfit—tuned to noise, not skill. To account for this, the test applied the deflated Sharpe ratio (Bailey & López de Prado, 2014): it asks, "Given 4,096 tries, is the best result better than what luck would likely deliver?" [1,2].

For this test, the deflated Sharpe said that none of the 4,096 strategies had even a 95% chance of true skill after correcting for the number of tries [1]. In other words, every winner could just as easily be luck. Even the strongest, which combined "golden cross," "winter," and "momentum12m" rules, shrank from a Sharpe of 0.89 in the first twelve years to just 0.18 after 2018—falling from first to 1,077th place out of 4,096 [1].

What happened to the best rules?

The best in-sample strategy—"golden cross" (50-day average above 200), "winter" (only November–April), and "momentum12m" (up over the last 12 months)—spent just 37% of days in the market, switching 28 times [1]. In the past, its Sharpe was 0.89, far above buy and hold's 0.52. But in the nine years after, its Sharpe dropped to 0.18—barely better than cash and much worse than buy and hold's 0.81 [1].

The table below shows this decay:

This pattern held broadly. Of the top 10% of strategies in the backtest, only 30 of 409 beat buy and hold out of sample, and the median future rank of the top in-sample decile was 1,183 of 4,096 [1]. The correlation between past and future ranking was just 0.15: knowing what worked in the past told you almost nothing about the future [1].

A worked example: Why correcting for luck matters

Suppose a reader tries 50 different trading rules and picks the best Sharpe ratio. The deflated Sharpe ratio asks: if you flipped 50 fair coins and called heads a "win," how often would the best coin look much better than chance? With 4,096 tries, the bar is even higher—a Sharpe ratio of about 1.20 is what the luckiest strategy would hit just by chance, even if all were random [1,2]. In our test, the best real strategy reached only 0.89 in sample, well below this threshold [1].

The more strategies you test, the more your top result is likely to be a mirage.

What does this not tell us?

This test covers one market (the S&P 500), one set of rules, and one split between past and future [1]. Cash earned nothing, which means strategies that sit out are understated for years when cash paid 4–5% [1]. Transaction costs were a flat 0.05% per switch; taxes and slippage were not included [1]. Many strategies share rules, so the correction for multiple tries is strict—real skill would have to be very strong to survive it [1]. A rule that failed here can still work elsewhere, and past survival never guarantees the future.

Questions readers ask

How many strategies would have survived with a looser test?

If the bar were set lower than a 95% chance of true skill, a few more strategies would pass, but the risk of taking luck for skill rises sharply. The strict bar is what keeps the winners honest [1,2].

Did any single rule help more than others?

Only "calm" (20-day volatility below its one-year average) helped in both periods, adding +0.09 to the average Sharpe in sample and +0.14 out of sample. "Dip" (buy the dip) helped most in the backtest, +0.31, and hurt after 2018, -0.08. No single rule made a strategy survive the luck correction [1].

If buy and hold did so well, is there any reason to use rules?

In this run, buy and hold was hard to beat. But rules can help in markets, time periods, or for goals that this test did not cover (like risk reduction or tax timing) [1]. The method here shows how hard it is to find a real edge, not that none can exist.

How the numbers work

These figures were computed for this article, not estimated by its writer. Inputs come from the numbered sources or are assumptions, labelled as such; change an assumption and the results change with it.

Sources

1. The site owner's own test data (2026-10-29): Strategy Lab: 4,096 strategies, 2006-2026
2. Bailey, D. H. and López de Prado, M. (2014), "The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting and Non-Normality", Journal of Portfolio Management