Does price vs its moving average survive out of sample on EURUSD=X?
Verdict: DID NOT SURVIVE (out-of-sample Sharpe above 0; edge not significant after correction (p 0.204; the correction needed at most 0.0033 at its rank 3 of 90)). Rule family: price vs its moving average (trend), setting chosen in sample only: 200. Market: EURUSD=X (Currencies). Costs: 1.5 basis points per switch. Data to 2026-11-01. Run 2026-11-02-panel.
The verdict rule
SURVIVES if the rule's out-of-sample Sharpe ratio is above 0 AND its edge over the benchmark (buy and hold; zero for a currency traded both ways) is still significant after the best-of-N correction: Benjamini-Hochberg at 10% false discoveries across every test in the run. Otherwise it DID NOT SURVIVE. The rule is the same for every case.
Honesty about the order of events: the panel run (all 90 tests, its correction and its settings) was fixed in code and run on 2026-11-02; this verdict rule was written on 2026-11-04 after that run's table was on disk and had been read, so it is a labelled rule applied to a finished run, not a pre-registration. It is the same for every case and is not changed after reading a case.
The answer in numbers
| price vs its moving average on EURUSD=X | Benchmark: zero (traded both ways) | |
|---|---|---|
| Sharpe ratio, unseen years 2016-01-01 to 2026-11-01 | 0.25 | 0.00 |
| Yearly growth | 1.5% | 0.0% |
| Worst fall | -19.7% | n/a |
Edge over the benchmark (information ratio): 0.25. t-statistic: 0.83 over 11.2 unseen years. One-sided p-value: 0.204. Position changes in the unseen years: 72, each charged 1.5 basis points.
Best-of-N: what the correction counts
The rule was not tried alone. In sample (2004-11-30 to 2015-12-31, 11.4 years) it was tried at 5 settings and the best was kept:
| Setting (days) | In-sample Sharpe | |
|---|---|---|
| 50 | -0.32 | |
| 100 | -0.35 | |
| 150 | 0.05 | |
| 200 | 0.12 | chosen (best in sample) |
| 250 | -0.10 |
Across the run, 6 rule families were tried on 15 markets: 90 tests, each with its own search over settings. The correction counts all of them, so a result must beat what the best of 90 would show by luck.
| All 90 tests of the run | Count |
|---|---|
| Out-of-sample Sharpe not above 0 | 27 |
| Sharpe above 0 but edge not significant after correction | 63 |
| Sharpe above 0 and edge significant after correction (survive) | 0 |
| Total | 90 |
Without the correction, 0 of 90 tests have a one-sided p-value of 0.05 or lower and 1 have 0.10 or lower. By chance alone, 90 independent tests would be expected to give about 4.5 at 0.05 and 9.0 at 0.10. This case ranks 3 of 90 by p-value (rank 1 is the smallest, 0.0655, bollinger on AUDUSD=X); to pass, the smallest p-value of all needs to be at most 0.0011, and 0 of 90 passed.
How it was tested
- Data: daily closes from Yahoo Finance's chart data (https://query1.finance.yahoo.com/v8/finance/chart/EURUSD=X), adjusted for dividends where the market has them, fetched 2026-11-02, last close 2026-11-01. Stored in the run folder as prices-EURUSD_X.json.
- Split: setting chosen on data before 2016-01-01; unseen years from then on. A signal is read at a day's close and acts on the next day.
- Benchmark: zero (traded both ways). This market is traded both ways (long and short), so the edge is measured against zero.
- Correction: Benjamini-Hochberg at 10% over all 90 tests (the t-statistic is the edge's Sharpe times the square root of the unseen years; one-sided normal p-value).
- Rerun:
python -I scripts/strategy-lab/cases.py --run <the run folder> --rule ma --market EURUSD=X --dry-runin the AutomationSolution repository recomputes this case from the stored prices and stops if its out-of-sample Sharpe does not equal the run's summary.json. The panel itself isscripts/strategy-lab/panel.py.
What this cannot tell
- 11.2 unseen years are one path of history; a rule that failed here can work in another regime, and a rule that had passed could fail in the next one.
- The p-values treat tests as independent; rules of the same family on correlated markets are not, which makes the correction rougher than it looks.
- Taxes, financing on shorts and currency carry are not counted.
- The verdict rule was written after the run was read (see above).