How lucky can the best of N useless trading variants look? Three ways to compute it
Our own calculation, 8 November 2026 (UTC), for the article "Why Backtests Overfit". The question: if you try N strategy variants that have no edge and keep the best, how high is its Sharpe ratio expected to be by luck alone? Each variant's measured Sharpe ratio is treated as a standard normal draw (no edge), in units of its standard error.
What we found
- Way 1, the approximation in Bailey and López de Prado (2014): the expected best of N is (1 − γ)·Φ⁻¹(1 − 1/N) + γ·Φ⁻¹(1 − 1/(N·e)) standard errors, with γ ≈ 0.5772. It gives 1.57 for 10 variants, 1.90 for 20 and 2.53 for 100.
- Way 2, the exact expectation of the maximum of N standard normals (numerical integration): 1.54, 1.87 and 2.51.
- Way 3, a simulation of 20,000 searches each: 1.54, 1.87 and 2.51 (simulation error under 0.005). It agrees with the exact values, so the approximation is slightly generous, by about 0.03.
- With five years of data, one standard error of an annual Sharpe ratio near zero is about 1/√5 ≈ 0.45 (0.4472).
- In annual Sharpe terms with five years of data, the approximation gives 0.70, 0.85 and 1.13; the exact values give 0.69, 0.83 and 1.12. After trying 100 useless variants, the best one is expected to show an annual Sharpe ratio of about 1.1 by luck alone.
- A single annual Sharpe ratio of 0.85 over five years is 0.85 × √5 ≈ 1.9 standard errors from zero.
- With 10 independent tests at the 5% level, the chance that at least one looks significant by luck is 1 − 0.95^10 = 40% (0.4013); with 20 tests it is 64% (0.6415). The Bonferroni level for 20 tests is 0.05/20 = 0.0025.
What this cannot tell you: real variants are correlated (similar look-back windows), so the effective N is smaller than the number tried; the figures are for independent variants.