USP-S | Bleeding Edge (Field-Tested)
7-day forecast validation for the Pistol category, 48 items (walk-forward, 49 folds)
| Model | Mean abs. error | vs. no change | Folds won | Range coverage (calibrated) | Interval score | Direction accuracy |
|---|---|---|---|---|---|---|
| No change (baseline) | 0.0699 | baseline | — | 80.1% | 0.3810 | — |
| Average drift | 0.0703 | -0.5% | 24/48 | 79.7% | 0.3934 | 50.6% |
| AR(1) momentum/reversion | 0.0695 | +0.7% | 33/48 | 79.6% | 0.3871 | 52.3% |
| Market-aware boosted trees (pooled over all items) | 0.0684 | +2.3% | 45/48 | 79.7% | 0.3686 | 58.8% |
Market-aware boosted trees (pooled over all items) has 2.3% smaller errors than “no change” and won 45/48 folds. Errors are on log-price changes; significance is tested on per-date averages with Newey-West standard errors. Range widths are calibrated on earlier folds only; the closer coverage is to 80%, the more trustworthy the range. The interval score rewards ranges that are both narrow and right (lower is better).
30-day forecast validation for the Pistol category, 48 items (walk-forward, 49 folds)
| Model | Mean abs. error | vs. no change | Folds won | Range coverage (calibrated) | Interval score | Direction accuracy |
|---|---|---|---|---|---|---|
| No change (baseline) | 0.1305 | baseline | — | 80.5% | 0.7534 | — |
| Average drift | 0.1331 | -2.0% | 24/48 | 79.5% | 0.6985 | 50.7% |
| AR(1) momentum/reversion | 0.1359 | -4.1% | 17/48 | 79.4% | 0.7193 | 50.0% |
| Market-aware boosted trees (pooled over all items) | 0.1283 | +1.7% | 38/48 | 79.9% | 0.6763 | 56.8% |
No model beats “no change” significantly and consistently (best: Market-aware boosted trees (pooled over all items), t=-2.30, 38/48 folds won). Errors are on log-price changes; significance is tested on per-date averages with Newey-West standard errors. Range widths are calibrated on earlier folds only; the closer coverage is to 80%, the more trustworthy the range. The interval score rewards ranges that are both narrow and right (lower is better).