Does moving-average cross (n over 200) survive out of sample on TLT?
Verdict: DID NOT SURVIVE (out-of-sample Sharpe above 0; edge not significant after correction (p 0.274; the correction needed at most 0.0056 at its rank 5 of 90)). Rule family: moving-average cross (n over 200) (trend), setting chosen in sample only: 20. Market: TLT (Bonds). Costs: 5 basis points per switch. Data to 2026-10-30. Run 2026-11-02-panel.
The verdict rule
SURVIVES if the rule's out-of-sample Sharpe ratio is above 0 AND its edge over the benchmark (buy and hold; zero for a currency traded both ways) is still significant after the best-of-N correction: Benjamini-Hochberg at 10% false discoveries across every test in the run. Otherwise it DID NOT SURVIVE. The rule is the same for every case.
Honesty about the order of events: the panel run (all 90 tests, its correction and its settings) was fixed in code and run on 2026-11-02; this verdict rule was written on 2026-11-04 after that run's table was on disk and had been read, so it is a labelled rule applied to a finished run, not a pre-registration. It is the same for every case and is not changed after reading a case.
The answer in numbers
| moving-average cross (n over 200) on TLT | Benchmark: buy and hold | |
|---|---|---|
| Sharpe ratio, unseen years 2016-01-04 to 2026-10-30 | 0.21 | 0.02 |
| Yearly growth | 1.7% | -0.8% |
| Worst fall | -25.9% | -48.4% |
Edge over the benchmark (information ratio): 0.18. t-statistic: 0.60 over 10.8 unseen years. One-sided p-value: 0.274. Position changes in the unseen years: 23, each charged 5 basis points.
Best-of-N: what the correction counts
The rule was not tried alone. In sample (2003-08-11 to 2015-12-31, 12.4 years) it was tried at 3 settings and the best was kept:
| Setting (days) | In-sample Sharpe | |
|---|---|---|
| 20 | 0.36 | chosen (best in sample) |
| 50 | 0.32 | |
| 100 | 0.30 |
Across the run, 6 rule families were tried on 15 markets: 90 tests, each with its own search over settings. The correction counts all of them, so a result must beat what the best of 90 would show by luck.
| All 90 tests of the run | Count |
|---|---|
| Out-of-sample Sharpe not above 0 | 27 |
| Sharpe above 0 but edge not significant after correction | 63 |
| Sharpe above 0 and edge significant after correction (survive) | 0 |
| Total | 90 |
Without the correction, 0 of 90 tests have a one-sided p-value of 0.05 or lower and 1 have 0.10 or lower. By chance alone, 90 independent tests would be expected to give about 4.5 at 0.05 and 9.0 at 0.10. This case ranks 5 of 90 by p-value (rank 1 is the smallest, 0.0655, bollinger on AUDUSD=X); to pass, the smallest p-value of all needs to be at most 0.0011, and 0 of 90 passed.
How it was tested
- Data: daily closes from Yahoo Finance's chart data (https://query1.finance.yahoo.com/v8/finance/chart/TLT), adjusted for dividends where the market has them, fetched 2026-11-02, last close 2026-10-30. Stored in the run folder as prices-TLT.json.
- Split: setting chosen on data before 2016-01-04; unseen years from then on. A signal is read at a day's close and acts on the next day.
- Benchmark: buy and hold. Long or in cash only; cash earns nothing in this test.
- Correction: Benjamini-Hochberg at 10% over all 90 tests (the t-statistic is the edge's Sharpe times the square root of the unseen years; one-sided normal p-value).
- Rerun:
python -I scripts/strategy-lab/cases.py --run <the run folder> --rule cross --market TLT --dry-runin the AutomationSolution repository recomputes this case from the stored prices and stops if its out-of-sample Sharpe does not equal the run's summary.json. The panel itself isscripts/strategy-lab/panel.py.
What this cannot tell
- 10.8 unseen years are one path of history; a rule that failed here can work in another regime, and a rule that had passed could fail in the next one.
- The p-values treat tests as independent; rules of the same family on correlated markets are not, which makes the correction rougher than it looks.
- Taxes, financing on shorts and currency carry are not counted.
- The verdict rule was written after the run was read (see above).