B Squadron Officer | SAS
7-day forecast validation for the Agent category, 10 items (walk-forward, 24 folds)
| Model | Mean abs. error | vs. no change | Folds won | Range coverage (calibrated) | Interval score | Direction accuracy |
|---|---|---|---|---|---|---|
| No change (baseline) | 0.0623 | baseline | — | 78.6% | 0.3567 | — |
| Average drift | 0.0629 | -1.1% | 12/24 | 78.9% | 0.3551 | 52.4% |
| AR(1) momentum/reversion | 0.0630 | -1.2% | 11/24 | 78.8% | 0.3546 | 51.0% |
| Market-aware boosted trees (pooled over all items) | 0.0615 | +1.3% | 15/24 | 76.7% | 0.3440 | 56.3% |
Market-aware boosted trees (pooled over all items) has 1.3% smaller errors than “no change” and won 15/24 folds. Errors are on log-price changes; significance is tested on per-date averages with Newey-West standard errors. Range widths are calibrated on earlier folds only; the closer coverage is to 80%, the more trustworthy the range. The interval score rewards ranges that are both narrow and right (lower is better).
30-day forecast validation for the Agent category, 10 items (walk-forward, 24 folds)
| Model | Mean abs. error | vs. no change | Folds won | Range coverage (calibrated) | Interval score | Direction accuracy |
|---|---|---|---|---|---|---|
| No change (baseline) | 0.1384 | baseline | — | 77.7% | 0.7600 | — |
| Average drift | 0.1452 | -4.9% | 10/24 | 78.3% | 0.7489 | 51.6% |
| AR(1) momentum/reversion | 0.1458 | -5.3% | 10/24 | 78.5% | 0.7490 | 51.3% |
| Market-aware boosted trees (pooled over all items) | 0.1373 | +0.8% | 14/24 | 78.3% | 0.7034 | 55.4% |
No model beats “no change” significantly and consistently (best: Market-aware boosted trees (pooled over all items), t=-1.01, 14/24 folds won). Errors are on log-price changes; significance is tested on per-date averages with Newey-West standard errors. Range widths are calibrated on earlier folds only; the closer coverage is to 80%, the more trustworthy the range. The interval score rewards ranges that are both narrow and right (lower is better).