MP9 | Starlight Protector (Field-Tested)
7-day forecast validation for the SMG category, 19 items (walk-forward, 47 folds)
| Model | Mean abs. error | vs. no change | Folds won | Range coverage (calibrated) | Interval score | Direction accuracy |
|---|---|---|---|---|---|---|
| No change (baseline) | 0.0733 | baseline | — | 79.9% | 0.3922 | — |
| Average drift | 0.0740 | -1.0% | 16/47 | 79.6% | 0.4006 | 49.3% |
| AR(1) momentum/reversion | 0.0725 | +1.1% | 33/47 | 79.1% | 0.3914 | 54.9% |
| Market-aware boosted trees (pooled over all items) | 0.0710 | +3.1% | 43/47 | 79.1% | 0.3774 | 59.8% |
Market-aware boosted trees (pooled over all items) has 3.1% smaller errors than “no change” and won 43/47 folds. Errors are on log-price changes; significance is tested on per-date averages with Newey-West standard errors. Range widths are calibrated on earlier folds only; the closer coverage is to 80%, the more trustworthy the range. The interval score rewards ranges that are both narrow and right (lower is better).
30-day forecast validation for the SMG category, 19 items (walk-forward, 47 folds)
| Model | Mean abs. error | vs. no change | Folds won | Range coverage (calibrated) | Interval score | Direction accuracy |
|---|---|---|---|---|---|---|
| No change (baseline) | 0.1254 | baseline | — | 79.8% | 0.7268 | — |
| Average drift | 0.1311 | -4.5% | 18/47 | 78.5% | 0.7009 | 52.9% |
| AR(1) momentum/reversion | 0.1315 | -4.8% | 19/47 | 79.0% | 0.7021 | 50.6% |
| Market-aware boosted trees (pooled over all items) | 0.1233 | +1.7% | 31/47 | 79.7% | 0.6790 | 54.8% |
No model beats “no change” significantly and consistently (best: Market-aware boosted trees (pooled over all items), t=-2.19, 31/47 folds won). Errors are on log-price changes; significance is tested on per-date averages with Newey-West standard errors. Range widths are calibrated on earlier folds only; the closer coverage is to 80%, the more trustworthy the range. The interval score rewards ranges that are both narrow and right (lower is better).