Operation Riptide Case
7-day forecast validation for the Container category, 40 items (walk-forward, 50 folds)
| Model | Mean abs. error | vs. no change | Folds won | Range coverage (calibrated) | Interval score | Direction accuracy |
|---|---|---|---|---|---|---|
| No change (baseline) | 0.0823 | baseline | — | 79.6% | 0.4316 | — |
| Average drift | 0.0829 | -0.7% | 24/49 | 79.4% | 0.4369 | 52.3% |
| AR(1) momentum/reversion | 0.0823 | +0.1% | 25/49 | 79.3% | 0.4317 | 52.8% |
| Market-aware boosted trees (pooled over all items) | 0.0811 | +1.4% | 38/49 | 82.5% | 0.4419 | 55.7% |
Market-aware boosted trees (pooled over all items) has 1.4% smaller errors than “no change” and won 38/49 folds. Errors are on log-price changes; significance is tested on per-date averages with Newey-West standard errors. Range widths are calibrated on earlier folds only; the closer coverage is to 80%, the more trustworthy the range. The interval score rewards ranges that are both narrow and right (lower is better).
30-day forecast validation for the Container category, 40 items (walk-forward, 50 folds)
| Model | Mean abs. error | vs. no change | Folds won | Range coverage (calibrated) | Interval score | Direction accuracy |
|---|---|---|---|---|---|---|
| No change (baseline) | 0.1764 | baseline | — | 80.4% | 0.9603 | — |
| Average drift | 0.1793 | -1.6% | 21/49 | 79.2% | 0.8997 | 54.9% |
| AR(1) momentum/reversion | 0.1797 | -1.8% | 23/49 | 79.3% | 0.8990 | 56.2% |
| Market-aware boosted trees (pooled over all items) | 0.1746 | +1.0% | 30/49 | 83.5% | 0.9681 | 53.0% |
No model beats “no change” significantly and consistently (best: Market-aware boosted trees (pooled over all items), t=-2.40, 30/49 folds won). Errors are on log-price changes; significance is tested on per-date averages with Newey-West standard errors. Range widths are calibrated on earlier folds only; the closer coverage is to 80%, the more trustworthy the range. The interval score rewards ranges that are both narrow and right (lower is better).