How many trades does a backtest need before it means anything?
The morning plan's one job for today: the reader question "How do I determine my sample size for backtesting?". Every figure below is our own calculation (scipy/numpy, scripts in _work/handoff/presents/sample-size/compute.py and exact.py, seed 20261030), each checked a second way (exact binomial and a simulation, or a closed form and a simulation). Nothing here was measured on market data: it is arithmetic about luck and noise. Write "our calculation", never "our test" or "our data".
The answer in one line
A strategy that wins 55% of 1:1 trades needs about 800 trades before a backtest can tell it from a coin flip (783 by the usual formula, 820 for the exact test to reach 80% power); at 52% it needs about 4,900, and if you picked the best of 20 variants the 55% case needs about 1,500.
Sources
- Our own calculation, 2026-10-30 (scripts above). All tables.
- Andrew W. Lo, "The Statistics of Sharpe Ratios", Financial Analysts Journal 58(4), 2002, https://doi.org/10.2469/faj.v58.n4.2453 : the Sharpe ratio's standard error; with independent returns, t = SR × √(years) for an annualised Sharpe ratio.
- David H. Bailey and Marcos López de Prado, "The Sharpe Ratio Efficient Frontier", Journal of Risk 15(2), 2012, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1821643 : the minimum track record length, MinTRL = 1 + [1 − skew·SR + (kurtosis − 1)/4 · SR²] · (z / SR)², with SR per period and kurtosis 3 for normal returns.
- Campbell R. Harvey and Yan Liu, "Backtesting", Journal of Portfolio Management 42(1), 2015, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2345489 : a strategy found by trying many must clear a higher bar (multiple-testing "haircut").
- Our article "Is a win due? What a losing streak does and does not tell you", https://hypothesess.com/blog/is-a-win-due-what-a-losing-streak-does-and-does-not-tell-you (same site; streaks at 55% over 200 and 1,000 trades).
1. Win rate against a coin flip (wins and losses the same size), two-sided test at 5%, 80% power (source 1)
Formula: n = ((1.96 × 0.5 + 0.8416 × √(p(1−p))) / (p − 0.5))². Check: exact binomial power at that n, and 200,000 simulated backtests.
| True win rate | Trades (formula) | Exact power at that n | Simulated power | Trades for exact power ≥ 80% |
|---|---|---|---|---|
| 51% | 19,620 | 79.91% | 79.80% | (not computed) |
| 52% | 4,904 | 79.30% | 79.41% | (not computed) |
| 53% | 2,178 | 79.07% | 78.97% | 2,228 |
| 55% | 783 | 78.85% | 78.92% | 820 |
| 60% | 194 | 76.43% | 76.48% | 210 |
The formula is the normal approximation; the exact test needs a few percent more trades because wins come in whole numbers.
2. Picked the best of K variants? The bar rises (source 1; Bonferroni, source 4)
Each test at 5%/K two-sided. True win rate 55%, 80% power.
| Variants tried (K) | z needed | Trades needed | Times the single-test number |
|---|---|---|---|
| 1 | 1.9600 | 783 | 1.00 |
| 10 | 2.8070 | 1,329 | 1.70 |
| 20 | 3.0233 | 1,491 | 1.90 |
| 100 | 3.4808 | 1,865 | 2.38 |
3. What 100, 300 and 1,000 trades can tell (source 1)
A backtest showing a 55% win rate (exact 95% confidence interval, Clopper-Pearson), and how often a strategy with no edge (true 50%) shows 55% or better by luck.
| Trades | 95% interval around an observed 55% | Share of no-edge strategies showing ≥ 55% |
|---|---|---|
| 100 | 44.7% to 65.0% | 18.41% (about 1 in 5) |
| 300 | 49.2% to 60.7% | 4.70% (about 1 in 21) |
| 1,000 | 51.9% to 58.1% | 0.087% (about 1 in 1,156) |
So 100 trades at 55% cannot rule out a losing strategy: the interval reaches 44.7%.
4. When returns vary: edge per trade in standard deviations (source 1)
For trades of different sizes, measure the edge as mean return per trade ÷ standard deviation per trade. Trades for 80% power, two-sided 5%: n = ((1.96 + 0.8416) / edge)².
| Edge per trade (mean ÷ sd) | Trades needed |
|---|---|
| 0.02 | 19,623 |
| 0.05 | 3,140 |
| 0.10 | 785 |
| 0.20 | 197 |
(A 55% win rate with 1:1 payoffs is an edge of 0.1005 sd per trade, which is why rows 55% and 0.10 agree: 783 and 785.)
5. Measured in years: an annualised Sharpe ratio (sources 2 and 1)
With independent daily returns, t ≈ SR × √(years) (source 2). Years of data:
| Annual Sharpe | Years for an observed SR to be just significant (two-sided 5%) | Years for 80% power |
|---|---|---|
| 0.5 | 15.37 | 31.40 |
| 1.0 | 3.84 | 7.85 |
| 1.5 | 1.71 | 3.49 |
| 2.0 | 0.96 | 1.96 |
Simulation check: 20,000 strategies with a true Sharpe of 1.0 over 7.85 years (1,978 trading days): 80.25% reach t > 1.96.
6. The minimum track record length (source 3), one-sided 95%, benchmark Sharpe 0
| Annual Sharpe | Return shape | Minimum track record |
|---|---|---|
| 0.5 | normal (skew 0, kurtosis 3) | 2,730 days = 10.83 years |
| 1.0 | normal | 684 days = 2.71 years |
| 2.0 | normal | 173 days = 0.69 years |
| 1.0 | skew −1, kurtosis 6 | 729 days = 2.89 years |
| 1.0 | skew −2, kurtosis 10 | 775 days = 3.07 years |
Why 2.71 years here and 3.84 in section 5: MinTRL uses a one-sided test (z = 1.645, "is it above zero?"); section 5 is two-sided (z = 1.96). Fat tails and negative skew (strategies that sell options, martingales) need longer.
How to use it (the reader's steps)
- Write down the edge you think you have before looking: a win rate, or mean ÷ sd per trade.
- Read the trades needed from table 1 or 4; multiply by table 2's factor for every variant you tried.
- If your backtest has fewer trades, the honest result is "not yet known", not "it works".
- In years: a Sharpe of 1 needs about 3 to 4 years of daily data just to be significant, about 8 for a fair chance of showing it.
What I am unsure of / limits
- Assumes independent trades. Trades that overlap in time or share a market regime count as fewer independent trades; the numbers are then floors.
- Bonferroni is conservative for correlated variants (parameter grids): the true factor in table 2 is lower when variants are similar.
- 80% power and 5% significance are conventions, not laws; the tables move with them.
- Source 3's formula is quoted from the paper (SSRN version); kurtosis is the non-excess kurtosis (3 for normal).
Suggested devices
- GUESS: "A strategy wins 55% of its trades, wins and losses the same size. How many trades before a backtest can tell it from a coin flip?" choices: about 100; about 300; about 800. Answer: about 800 (783 by formula, 820 exact).
- Simulation inputs: true win rate (default 0.55), variants tried (default 1), significance two-sided (0.05), power (0.80). Results: trades needed (formula of section 1 with z = norm quantile of 1 − 0.025/K).
- REMEMBER: "Under about 800 trades, a 55% backtest is still a coin you have not finished flipping."
- CHECK YOURSELF: (1) Your backtest: 120 trades at 58%. Enough? — No: a true 58% needs about 300 trades (304 by the formula) even before any variant search. (2) You tried 20 parameter sets; the best wins 55% over 900 trades. Enough? — No: the best of 20 needs about 1,500. (3) Sharpe 1.5 over 2 years: significant? — Just: 1.71 years is the bar for significance, 3.49 for 80% power.