Does Bollinger band reversion survive out of sample on AUDUSD=X?
Verdict: DID NOT SURVIVE (out-of-sample Sharpe above 0; edge not significant after correction (p 0.065; the correction needed at most 0.0011 at its rank 1 of 90)). Rule family: Bollinger band reversion (mean reversion), setting chosen in sample only: 10. Market: AUDUSD=X (Currencies). Costs: 1.5 basis points per switch. Data to 2026-11-01. Run 2026-11-02-panel.
The verdict rule
SURVIVES if the rule's out-of-sample Sharpe ratio is above 0 AND its edge over the benchmark (buy and hold; zero for a currency traded both ways) is still significant after the best-of-N correction: Benjamini-Hochberg at 10% false discoveries across every test in the run. Otherwise it DID NOT SURVIVE. The rule is the same for every case.
Honesty about the order of events: the panel run (all 90 tests, its correction and its settings) was fixed in code and run on 2026-11-02; this verdict rule was written on 2026-11-04 after that run's table was on disk and had been read, so it is a labelled rule applied to a finished run, not a pre-registration. It is the same for every case and is not changed after reading a case.
The answer in numbers
| Bollinger band reversion on AUDUSD=X | Benchmark: zero (traded both ways) | |
|---|---|---|
| Sharpe ratio, unseen years 2016-01-01 to 2026-11-01 | 0.45 | 0.00 |
| Yearly growth | 2.6% | 0.0% |
| Worst fall | -12.1% | n/a |
Edge over the benchmark (information ratio): 0.45. t-statistic: 1.51 over 11.2 unseen years. One-sided p-value: 0.065. Position changes in the unseen years: 292, each charged 1.5 basis points.
Best-of-N: what the correction counts
The rule was not tried alone. In sample (2007-05-14 to 2015-12-31, 8.9 years) it was tried at 3 settings and the best was kept:
| Setting (days) | In-sample Sharpe | |
|---|---|---|
| 10 | 0.43 | chosen (best in sample) |
| 20 | -0.15 | |
| 50 | -0.32 |
Across the run, 6 rule families were tried on 15 markets: 90 tests, each with its own search over settings. The correction counts all of them, so a result must beat what the best of 90 would show by luck.
| All 90 tests of the run | Count |
|---|---|
| Out-of-sample Sharpe not above 0 | 27 |
| Sharpe above 0 but edge not significant after correction | 63 |
| Sharpe above 0 and edge significant after correction (survive) | 0 |
| Total | 90 |
Without the correction, 0 of 90 tests have a one-sided p-value of 0.05 or lower and 1 have 0.10 or lower. By chance alone, 90 independent tests would be expected to give about 4.5 at 0.05 and 9.0 at 0.10. This case ranks 1 of 90 by p-value (rank 1 is the smallest, 0.0655, bollinger on AUDUSD=X); to pass, the smallest p-value of all needs to be at most 0.0011, and 0 of 90 passed.
How it was tested
- Data: daily closes from Yahoo Finance's chart data (https://query1.finance.yahoo.com/v8/finance/chart/AUDUSD=X), adjusted for dividends where the market has them, fetched 2026-11-02, last close 2026-11-01. Stored in the run folder as prices-AUDUSD_X.json.
- Split: setting chosen on data before 2016-01-01; unseen years from then on. A signal is read at a day's close and acts on the next day.
- Benchmark: zero (traded both ways). This market is traded both ways (long and short), so the edge is measured against zero.
- Correction: Benjamini-Hochberg at 10% over all 90 tests (the t-statistic is the edge's Sharpe times the square root of the unseen years; one-sided normal p-value).
- Rerun:
python -I scripts/strategy-lab/cases.py --run <the run folder> --rule bollinger --market AUDUSD=X --dry-runin the AutomationSolution repository recomputes this case from the stored prices and stops if its out-of-sample Sharpe does not equal the run's summary.json. The panel itself isscripts/strategy-lab/panel.py.
What this cannot tell
- 11.2 unseen years are one path of history; a rule that failed here can work in another regime, and a rule that had passed could fail in the next one.
- The p-values treat tests as independent; rules of the same family on correlated markets are not, which makes the correction rougher than it looks.
- Taxes, financing on shorts and currency carry are not counted.
- The verdict rule was written after the run was read (see above).