Model risk · inventory and monitoring
The structure of a model-risk review (an inventory, then outcome checks against stated limits) applied to a personal research system. The limits are this page's working thresholds, stated so a reader can disagree with them.
Monitoring checks · exceptions first
| Model | Check | Value | Status | History | Artifact | Note |
|---|---|---|---|---|---|---|
| M4 Joint Heston calibrator | Heston median |model − mid|, out of sample (half-spreads) | 14.8 (3.1% inside)OK ≤ 2 · watch ≤ 5 · breach above | BREACH | 4.4–17.2 · rank today: 6th of 8 | data_ | 3,290 quotes of 2026-10-28; Yahoo Finance download, fetched 2026-10-29 07:32 UTC; Yahoo reports no quote times, T measured from the fetch. |
| M5 VIX index and VX futures under Heston/Bates | VX futures table lag behind the market DB (NYSE sessions) | 13OK ≤ 0 · watch ≤ 1 · breach above | BREACH | single observation | data_ | vix_futures_daily through 2026-10-09; market DB 2026-10-28. Only the fits' out-of-sample VX score reads them, so a fit after 2026-10-09 has none. |
| M11 XGBoost regime classifier | Classifier accuracy minus the persistence baseline (percentage pts) | −12.1OK ≥ 0 · breach below | BREACH | single observation | data_ | Research only since C17: below the no-skill baseline, so it never trades. |
| M4 Joint Heston calibrator | Worst short-dated expiry OOS RMSE (≤ 30 days) | 3.06 (2026-11-12)OK ≤ 2 · watch ≤ 5 · breach above | WATCH | 2.10–7.46 · rank today: 3rd of 8 | data_ | One-factor Heston misses the short end; see the calibration page. |
| V1 Yahoo Finance (yfinance) | Quotes failing static arbitrage at bid/ask (% of scored quotes) | 3.2% (106 of 3,343)OK ≤ 2 · watch ≤ 5 · breach above | WATCH | 2.7–9.0 · rank today: 3rd of 8 | data_ | Bid/ask-level butterfly, vertical and calendar screen of the stored chain. Single-quote outliers: 25 of its 25 largest mid violations (bad quote 2027-03-31 P6265 and 6 more legs), counted in the share. |
| M4 Joint Heston calibrator | Headline fit age (NYSE sessions) | 0OK ≤ 1 · watch ≤ 4 · breach above | OK | single observation | data_ | Newest gate-passing fit against the newest session with data. |
| M4 Joint Heston calibrator | Parameters on a box bound | noneOK ≤ 0 · watch ≤ 1 · breach above | OK | 0–1 · rank today: tied best of 8 | data_ | A parameter on its bound is set by the box, not the market. σ ≤ 10 refit: OOS 1.40, VVIX 188.4 vs 96.6 (2026-10-28). |
| M4 Joint Heston calibrator | Out-of-sample SPX IV RMSE (vol pts) | 1.40OK ≤ 1.5 · watch ≤ 3 · breach above | OK | 1.28–2.96 · rank today: 2nd of 8 | data_ | 3,290 quotes not used in the fit. |
| M4 Joint Heston calibrator | VIX index RMSE (index pts) | 0.42OK ≤ 0.5 · watch ≤ 1 · breach above | OK | 0.42–0.89 · rank today: best of 8 | data_ | ^VIX9D, ^VIX, ^VIX3M, ^VIX6M against the model. |
| M6 Greeks surface | Greeks surface built from the headline fit | yesOK if yes · watch if unknown · breach if no | OK | single observation | data_ | Surface 2026-10-28. |
| M7 β = ATM vol / VIX proxy | β drift: newest chain's ATM vol / VIX against the frozen β | 0.785 on 2026-10-28 vs β = 0.82OK ≤ 0.05 · watch ≤ 0.1 · breach above | OK | 0.004–0.118 · rank today: 6th of 9 | data_data_ | The backtest marks every straddle at β·VIX: β as the run recorded it (atm_to_vix_ratio) against the market DB's newest chain. |
| M10 Rule regime labels | Market database lag behind the rule labels (sessions) | 0OK ≤ 0 · watch ≤ 1 · breach above | OK | single observation | data_data_ | Labels 2026-10-28; market DB 2026-10-28. |
| V1 Yahoo Finance (yfinance) | VIX replicated from the stored chain − Cboe close (index pts) | +0.28OK ≤ 0.5 · watch ≤ 1.5 · breach above | OK | −0.78 to +0.28 · rank today: 2nd of 7 | data_ | Cboe VIX formula on the stored two-sided chain, with 13 listed departures (Surface); a gap points at a thin or stale strip. |
| M4 Joint Heston calibrator | Feller condition (diagnostic, not imposed) | VIOLATED, margin −2.25informational | INFO | single observation | data_ | Not a no-arbitrage condition; reported, not limited. |
Value: links to the panel that shows the evidence; the bar under it runs from 0 through the OK, watch and breach bands of the limits printed beneath it, with a tick at the value. History: one dot per stored session, oldest first, coloured by today’s limits; stored sessions, irregular spacing; today’s limits applied retroactively. A dot is a measurement on that session, not a point of a control chart.
How to read the status
OK, WATCH and BREACH compare the value with the limits in the same row (green ≤ the first limit, watch ≤ the second, breach above); every status also carries a word and a glyph, so colour is never the only cue. INFO rows are reported, not limited: the Feller condition is a diagnostic here, not a no-arbitrage requirement. A BREACH is not an outage: it marks where the model is weakest, which the linked page explains. The rows run worst first (breach, watch, OK, informational), and within a status in inventory order.
Model inventory
| ID | Model | Tier | Materiality | Use | Inputs → outputs | Known limitations |
|---|---|---|---|---|---|---|
| M1 | Heston / Bates pricing enginejoint_ · page | production | high | Lewis (2001) contour integral with a Black–Scholes control variate on an adaptive trapezoid; prices every option the calibrator, the Greeks surface and the scorecard need. | Heston / Bates parameters, forward, strike, maturity → data_ | Quadrature target 1e-10 of DF·F; 5e-9 of F at the slowest-decaying corners of the parameter box. |
| M2 | Implied-vol solversjoint_ · page | production | high | Safeguarded Newton (implied_vol_black) and bisection (black76_implied_vol): chain cleaning, calibration targets, Greeks. | Option prices, forwards, discount factors → data_ | An in-the-money price whose time value is lost to rounding has no implied vol: the solver returns a missing value by design. |
| M3 | Option-chain hygienejoint_ · page | production | high | Two-sided quotes, the parity forward and discount per expiry, the quote-state gate and session dating, before every fit. | Raw SPX chain (Yahoo, Cboe) → data_ | The calibrator does not drop quotes in static arbitrage (M14 screens and reports them); Yahoo closing books carry no quote time. |
| M4 | Joint Heston calibratorjoint_ · page | production | high | Daily fit to the SPX smile and the VIX index term structure (multi-start trust-region least squares, analytic Jacobian) and its quality gate; headline of the Calibration page and source of the Greeks surface. | SPX chain (two-sided OTM quotes, parity forwards), ^VIX9D/^VIX/^VIX3M/^VIX6M → data_ | One-factor diffusion: misses the short end; σ sits on its box ceiling in 2 of 5 fits, and κ moves with that cap along a κ–σ ridge (refitted with σ ≤ 10 and κ ≤ 60 on 2026-10-16: σ 6.9, κ 27, out-of-sample 2.96 → 1.19 vol pts); the boxed fits price VVIX far above the market; the VIX-options leg has weight 0. |
| M5 | VIX index and VX futures under Heston/Batesjoint_ · page | production | medium | The model VIX indices of the calibrator's VIX leg (exact CIR Laplace transform) and VX futures out of sample. | Heston parameters, VIX tenors, VX settlements → data_ | Heston misses real VX futures by 1.85–3.18 index pts RMSE out of sample (4 sessions; none stored for 2026-10-16, 2026-10-26, 2026-10-27, 2026-10-28; settlements from vix_futures_daily in data_store/vol_system.db, through 2026-10-09). |
| M6 | Greeks surfacejoint_ · page | production | medium | Black–Scholes Greeks at the model IV (vomma, vanna, vega, OTM delta) and QV convexity by maturity and strike; the unstable-node flag. | Headline Heston parameters, spot, parity carry → data_ | Black–Scholes Greeks at the model IV, carry per maturity (Treasury r(T), q(T) from the fit's parity forwards); the vomma shock is sized from the fitted σ, on its 2.0 cap in the headline fit. |
| M7 | β = ATM vol / VIX proxyjoint_ · page | backtest | high | Prices and marks every backtest straddle at ATM vol = β × VIX with Black–Scholes; it decides the sign of S1. | VIX term structure, a constant β estimated from stored SPX chains → data_ (the β the run used, recorded as atm_to_vix_ratio; its marks are in the NAV of full_results.parquet) | Mark-to-model: no traded prices in the window; β is held constant over 2018–2025, and the first later chain (2026-10-16) gives 0.70. |
| M8 | Backtest engine and signalsjoint_ · page | backtest | high | Signals S1, S1C, S2, S3, S4 as delta-hedged 30-day straddles, marked to model; the Research page's negative result. | SPX closes, VIX family, rule labels, PDV forecast, β → data_ + data_ (BacktestEngine.save: the daily NAV per sleeve, then the trades and metrics) | All P&L is mark-to-model at β·VIX; no option prices inside the window. |
| M9 | PDV volatility forecasterjoint_ · page | production | medium | 21-day RMS-vol forecast refitted daily (purged walk-forward): the Overview's VIX − forecast spread and the S1/S1C input. | SPX log returns → data_ | EWMA approximation of a path-dependent-vol kernel; backward-looking, so it cannot anticipate jumps; 6th of 7 on QLIKE among the forecasts compared. |
| M10 | Rule regime labelsjoint_ · page | production | medium | Deterministic same-day label (R0 / R1 / R2) from rv₂₀, VIX and an adaptive VVIX gate; the backtest trades it lagged one session. | SPX closes, VIX, VVIX → data_ | Thresholds are fixed rules, not estimated; VVIX gaps are carried one session; 5.9% of labels flip within measurement noise (C22). |
| M11 | XGBoost regime classifierjoint_ · page | research | low | Next-session regime probabilities, shown on the Research page only (demoted in C17). | Market features (vvix excluded) → data_ | Loses to the no-skill persistence baseline; never trades. |
| M12 | NN surrogate pricerjoint_ | research | low | Fast approximation of the Fourier pricer; on no production path. | Heston parameters, strike, maturity → data_ | Trained domain narrower than the calibrator's box; out-of-domain inputs are refused; no committed benchmark backs a speed-up. |
| M13 | Two-factor Quintic OUjoint_ · page | research | low | Joint SPX/VIX research model compared with Heston on one session. | SPX chain, VIX options, VIX indices → data_ | Not converged (iteration limit; ε on its −0.99 bound); Monte Carlo SPX pricing. |
| M14 | Quote-grade surfacejoint_ · page | production | medium | Static-arbitrage screen and l1 repair, eSSVI and calendar-ordered SVI, one scorecard for every model in half-spreads, the VIX replicated from the chain; written for each accepted fit; the Surface page and the Overview's first result. | The session's stored SPX chain and the session's own fit (rate reference, quote universe) → data_ | Fitted to alternate strikes, so its out-of-sample set is an interpolation test; Yahoo quotes carry no quote time; the butterfly and calendar certificates are on dense grids, not in closed form. |
| M15 | Rough Hestonjoint_ · page | research | low | Lifted rough Heston (20 factors), compared with Heston on the same anchors, VIX leg and quotes in each session's calibration diagnostics; feeds no production number. | As M4, plus H in [0.02, 1/2] → data_ | H is weakly identified (on its 0.02 floor on 2026-10-16; 1/2, i.e. Heston, on 2 of 5 sessions); the 20-factor lift approximates the Volterra kernel (at most 0.085 vol pts from QuantLib's Adams scheme at H = 0.02, 14 days to 1 year). |
| M16 | Spot × vol scenario grid of a reference straddlejoint_ · page | production | low | Full-revaluation P&L of a delta-hedged 30-day ATM straddle under spot and parallel vol shocks, sticky-strike and sticky-moneyness; rebuilt from each new headline fit. | The headline fit's 30-day smile and per-maturity carry → data_ | Instantaneous shocks (no theta); parallel vol shifts; one reference book, not a position book. |
| M17 | Statistical inference of the backtestscripts/ · page | backtest | medium | Bootstrap intervals, PSR/DSR, SPA/StepM, CSCV PBO, exposure-matched timing nulls and the forecast comparison for every sleeve; the Research page and the Overview's third result. | The canonical backtest run, the β grid, the C22 bridge, docs/trials_ledger.yaml, Cboe strategy indices → data_ | Inference about the β·VIX proxy, not executable P&L; one window, on which the rules were revised; the closed forms assume IID returns. |
| M18 | Forward ledgerjoint_ · page | production | medium | Each session's signals as listed SPX straddles at the captured bid/ask, hash-chained per session and marked on later captured books: the only executable-price evidence. | The captured option book, the signals of an accepted fit → data_ (no entry committed yet: the first is written on the next session with a live captured book, whatever the fit does) | No entry committed yet; marks are the captured book. |
| M19 | Treasury zero curvejoint_ · page | production | medium | Bills and par yields to a zero curve, look-ahead gated and refused when older than 7 days: the calibrator's rate reference, the chain cleaner, C7 and the Greeks carry. | U.S. Treasury bill and par yields, NY Fed SOFR → data_ | Daily; falls back to ^IRX and records it when stale. |
| M20 | Delta-hedged straddle simulator (C7)joint_ · page | research | low | Daily Greek P&L explain of a hedged 1-year ATM straddle in 2020 (the Risk page's P&L explain). | SPX closes, VIX term structure, the 1-year Treasury zero rate at entry → data_ | One path; financing at one rate for the year (the 1-year Treasury zero rate at entry); Black–Scholes Greeks at the VIX term structure. |
The canonical model inventory: every model the system runs or keeps, with its use, inputs, outputs and known limitations. The IDs and names are the ones the validation pack uses (docs/model_validation/README.md §1); test_dashboard.py checks that every pack ID is here under the same name. Tier: production = feeds a published number every session; backtest = feeds the backtest; research = kept for comparison or as a negative result, feeds nothing. Materiality: how much a published number depends on the model (high: a headline number or its sign; medium: a secondary published number; low: diagnostics only). test_dashboard.py checks that every listed module and artifact path exists.
| ID | Source | What it supplies | Note |
|---|---|---|---|
| V1 | Yahoo Finance (yfinance) | ^GSPC, ^IRX, the SPX chain the bot fits, provisional VIX-family rows | Delayed, end of day; no quote time on closing books; personal-use terms. |
| V2 | Cboe | The VIX index family (primary from C23), VX settlements, the SPX/VIX book capture, strategy indices | 15-minute delay; personal, non-commercial terms; each capture records its fetch time, quote time and SHA-256. |
| V3 | U.S. Treasury, NY Fed | Bill rates, par curve, SOFR | Daily; used only after publication (look-ahead gate). |
| V4 | Library numerics (NumPy, SciPy, pandas) | Every computation | Exact lock (requirements-lock.txt), CPU-tolerant artifact comparison. |
Conventions and definitions · every page
Units and signs
- Vol levels
- in %: e.g. an implied or realised vol of 20.00% (σ = 0.2000).
- Vol differences and errors
- in vol pts, percentage points of vol: 21.00% − 20.00% = 1.00 vol pts.
- VIX-family levels
- VIX, VIX9D, VIX3M, VIX6M and VVIX in index pts, never '%': e.g. VIX 20.00.
- Quote residuals
- in half-spreads (|z|): 1 is the edge of the quoted bid–ask.
- Money
- $ per SPX contract (×100 multiplier) where a row says so; $ with k / M suffixes; backtest P&L is marked to model at β·VIX on collateral assumed to earn T-bills separately, so it is an excess return.
- Signs
- model − market (a positive bias is a model above the mid), implied − realised (a positive premium is implied above realised), replicated − Cboe close; each table's caption restates its sign.
- Moneyness and maturity
- ln(K/F) against the expiry's parity forward F; days are calendar days to expiry unless a column says NYSE sessions; 1M = 30 days.
- Probabilities
- risk-neutral where the Surface prints them (they price claims; they are not forecasts).
Terms
- half-spread
- Half a quote's bid–ask in implied-vol terms, (ask IV − bid IV)/2. Residuals are measured in it; the surface fits divide each residual by it, floored at 0.05 vol pts.
- |z| absolute residual in half-spreads
- |model IV − mid IV| / half-spread of one quote; |z| ≤ 1 is inside the quoted bid–ask. Scorecards report its median over a model's out-of-sample quotes.
- vol pts volatility points
- A difference of two vols in percentage points: 21.00% against 20.00% is 1.00 vol pts. RMSE, bias, RR, BF and premia are in vol pts; vol levels are in %.
- index pts index points
- The unit of a VIX-family level and its errors (VIX 20.00 is 20.00 index pts), and of SPX option prices (×100 for $ per contract).
- ATMF at-the-money-forward vol
- The implied vol of the SVI-cal slice at the forward strike, ln(K/F) = 0.
- RR25 25-delta risk reversal
- Vol of the 25Δ call − vol of the 25Δ put, at undiscounted forward delta on the slice's own vol (not premium-adjusted); negative: puts over calls.
- ATM skew
- ∂σ/∂ln(K/F) at the forward, k = ln(K/F) = 0; for an SVI-cal slice w′(0)/(2·√(w(0)·T)) in decimal vol. Calibration and the Overview quote it per unit k; the Surface grid scales it to vol pts per 0.1 of k, ×10: −1.50 per unit k is −15.0 vol pts per 0.1 of k. Negative: puts over calls.
- SVI-cal calendar-ordered SVI surface
- The calendar-ordered SVI surface: one raw-SVI slice (a, b, ρ, m, σ) per listed expiry, each with a butterfly certificate (Durrleman's g ≥ 0, so its density is non-negative).
- eSSVI extended surface SVI
- One extended-SSVI surface fitted jointly to every expiry, its parameters constrained against static arbitrage.
- out of sample
- Quotes a model was not fitted to: the surfaces hold out alternate strikes of each expiry; Heston is fitted to its anchors and scored on every other quote.
- PSR probabilistic Sharpe ratio
- Bailey & López de Prado (2012): P(true Sharpe > 0) given the sample Sharpe, its length, skew and kurtosis, assuming IID returns (the bootstrap intervals are the primary inference).
- DSR deflated Sharpe ratio
- Bailey & López de Prado (2014): the PSR against the expected maximum Sharpe of N̂ trials, N̂ = ρ̄ + (1 − ρ̄)·M for M trials with average pairwise correlation ρ̄.
- SPA superior predictive ability test
- Hansen (2005), the studentised and recentred Reality Check (arch.bootstrap.SPA). H0: no strategy has a positive expected excess return; the 'consistent' p-value is the headline.
- StepM stepwise multiple testing
- Romano & Wolf (2005) via arch.bootstrap.StepM: which strategies, if any, beat cash with family-wise error rate ≤ the test size.
- PBO probability of backtest overfitting
- Bailey, Borwein, López de Prado & Zhu (2017), by CSCV: the share of splits in which the strategy with the best in-sample Sharpe ranks at or below the median out of sample (logit of its relative rank ≤ 0).
- CSCV combinatorially symmetric cross-validation
- The PBO's resampling: the sample is cut into S blocks and every combination of S/2 blocks serves once as the in-sample set, its complement as the out-of-sample set.
- MCS model confidence set
- Hansen, Lunde & Nason (2011) via arch: the forecasts not rejected as worse than the best at the test size, with a p-value per forecast.
- QLIKE quasi-likelihood loss
- QLIKE(y², f) = y²/f − log(y²/f) − 1 on annualised variance (Patton 2011: robust to the noise in a realised-variance proxy); lower is better.
- Holm Holm step-down adjustment
- Holm (1979) step-down adjusted p-values: family-wise error control across the tests named beside it.
- z_vrp VRP z-score
- S4's z-score of VRP = VIX − 20-day realised vol (decimal vol, rebuilt as vix × (1 − 1/fear_premium)): (VRP − its 252-session mean) / its 252-session sd.
- RV21 21-session realised vol
- 100·√(252/21 · Σ r²) over 21 daily SPX close-to-close log returns: trailing (known at the close) or forward (the next 21 sessions).
- rv₂₀ the rule's 20-session realised vol
- 100·√(252 · mean r²) over the last 20 daily SPX log returns (at least 15 present): the realised vol the regime rule compares with VIX. It differs from RV21 by its window (20 against 21 sessions) and its minimum count, not by method.
- R0–R2 rule regime labels
- The rule's regime labels: R0 long gamma, R1 short gamma, R2 vomma active (VVIX above its threshold at the close); the backtest reads them lagged one session.
- vomma P&L ratio
- ½·vomma·Δσ² / (vega·|Δσ|) = ½·d₁·d₂·|Δσ|/σ for a vol move Δσ: the second-order vol P&L against the first; a node is flagged convexity-dominated above the monitor's threshold.
The first use of each term on every page links here. A definition quotes the formula the code computes (the file named after it), not a textbook variant.