Checklist and unproven

Live status of the master checklist (state/CHECKLIST.md) and an honest list of what is not proven yet.
Validated
39
83%
Implemented / testing
0
0%
Unproven
7
15%
Failed
0
0%
Not started
1
2%

Overall

■ validated 39 ■ unproven 7 ■ not started 1
Nothing is marked validated merely because code exists; a box turns green only with evidence written next to it.

Progress by area

AreaItemsValidatedProgress
Phase 044
■ validated 4
Algorithm1514
■ validated 14 ■ unproven 1
Find volatility63
■ validated 3 ■ unproven 3
Test1714
■ validated 14 ■ unproven 3
Live54
■ validated 4 ■ not started 1

UNPROVEN: what we do not yet know

Every item that is not validated, failed first.
IDAreaStateWhat is unproven and why
A12AlgorithmunprovenAdditional data: DONE - FRED
19 series, lags audited), sector ETFs (validated, inception respected), minute bars (A6), EDGAR delisted-exit registry (9,250 exits 1996-2026; data/cache/delisted_*). UNPROVEN - prices for dead US names before ~2015 are not obtainable from free sources (Yahoo 404s them, Stooq now requires JavaScript; only ~250 mostly post-2020 names recovered), so survivorship cannot be removed from free data; it is measured and quarantined (T20: analytic drag -4 bps/week vs +32 bps/week model edge; pre-1983 windows THIN). Needs a paid/CRSP-type source to close.
T19TestunprovenLearning delta
canon C54/C55): harness VALID (self-check: generaliser/memoriser/non-learner/leak all classified correctly). FIRST REAL RESULT (ld1, 12 windows, memory-bank learner): same-year disguised effect +0.23%/wk (CI +0.01..+0.54) but TRANSFER -0.03%/wk (CI -0.09..+0.02) -> verdict MEMORISATION (the memory recognises a year by its market-context fingerprint); in-band share HARMED -1.4 pt. Under the C56 past-only rule the loop's memory cannot see a year's own first run, so production is protected; the honest headline (transfer) is: the learning system does NOT yet generalise. Making transfer > 0 is now the main research objective.
T3TestunprovenTiered 7% objective
>= 50% of weeks in the 5-10% band): UNPROVEN - target not reached after controlled testing; research continues (Bible Phase 45). Exact reason: the real system lands 22-23% of weeks in band (39 revealed windows, 2,049 weeks); no 1-25 stock volatility basket reaches 30% (proxy); the timeline dial (14%), volatility targeting (no gain), the basis search (nothing adopted) and six learners (band_cfg buys ~2 pt only with significantly worse risk) all fail to raise it without extra risk.
T6Testunproven13D correctness: UNPROVEN before 2014. The subject-company fix
engine/edgar.fix_activist, resumable) finished 2026-09-29: kept 31,417 of 53,865 activist events; 2014+ verified (about 78% kept - the rest were filer-side misattributions), but verification found almost no matches for 2001-2013 (about 1-3 per year kept vs ~1,100-1,700 before), which points to a verification failure on older SEC filing pages, not to those events being wrong. The fixed file keeps only verified events (pre-2014 activist data effectively absent; backup data/cache/events_before_13dfix.parquet). Impact: nil today - ev_activist is excluded from the miner and the evidence weights; it must stay excluded until pre-2014 verification is fixed.
V1Find volatilityunproven95% mover target: UNPROVEN - target not reached after controlled testing; research continues
Bible Phase 45). Exact reason: movement IS predictable - picks touch +-10% within the week 57.9% end-to-end (CI 55.8-60.2%) vs 14.6% base, 72.6% among top picks (83-92% since 2020; a naive ATR ranking matches it); weeks with 10 candidates above a 95% threshold: 0.1%. With free daily data the 95% bar is not reachable at 10 picks/week.
V2Find volatilityunprovenDirection at >=80% calibrated confidence: UNPROVEN - target not reached after controlled testing; research continues
Bible Phase 45). Exact reason: no direction signal on movers exists in the available data - B07 (stand-in inputs) 51.8%, Brier = base rate; V5 run: gate open 0/716 weeks; B25 (real literature-backed inputs on PREDICTED movers: post-earnings drift, insider clusters, reversal/momentum, 52-week high, miner score, event type, regime; ~970 accuracy/coverage cells, 2018-2026): 49-53% at full coverage vs 51.7% always-up, no cell reaches 80%, best lower bounds 0.55-0.65 (adjusted 0.45-0.59); planted-signal control recovered at 92-95% and shuffled labels read ~50%, so the pipeline could see a signal if one existed.
V5Find volatilityunproven8 of 10 at +10% across blind eras: UNPROVEN - target not reached after controlled testing; research continues
Bible Phase 45). Exact reason: it requires direction (V2), which has no measurable signal; the full pipeline therefore takes no bets (share10 lower bound 0.00). Movers-only: 15% of picks finish >= +10%.
L1Livenot startedUpgrade only after a validated research edge
owner decision). Also for the owner: live decides at 15:42 ET and fills before the close, while the simulations decide at the close and fill at the next open (execution parity).

Rollbacks and voided results

Changes that were adopted and then undone, or results thrown away because the check failed.
WhenEventIdReason
2026-09-28T20:29:59rolled_backv1.3full 2017-2026: $3,579 vs SPY $3,987, max drawdown -73%; round-1 guards only checked single months
2026-09-28T21:21:20voided_adjustmentre-tester mismatch vs live trader; fixed bookkeeping; reset to v1

Full checklist

IDStateItemEvidence
P0.1validatedCanon integrity`canon/build_canon.py --verify` (56 directives, C1-C56); `engine/provenance.verify_integrity()` checks the canon and the Bible and fails closed (tests/test_provenance.py).
P0.2validatedExperiment registry with provenance: every ACTIVE writer passes cfg+seed and all Phase 0.2 fields - Test loop rounds + basis search
scripts/livesim_loop2.py), Phase 25, self-tuning, Live retrain (engine/train.py), Live promotion path (engine/improve.py: challenger/promoted/rejected/rolled_back/confirmed), heavy tests, antimemo, exits, stops; log_experiment stamps code_hash/code_files/code_mixed, git, canon/Bible hashes, config hash, data snapshot, sha256 ids, and warns on any missing field. 8 superseded scripts archived in scripts/legacy (documented as not provenance-complete). Historic records stay as written (append-only).
none recorded
P0.3validatedCheckpoint bundle per major run: engine/checkpoint.checkpoint_run() wired into the Test loop
one bundle per round), Phase 25 and self-tuning runners. First real bundle: state/checkpoints/loop2_round003_20260929T081401 - verify() ok (2026-09-29).
none recorded
P0.4validatedImmutable baseline snapshot. Frozen 2026-09-28 23:25: state/baseline/B_2026-09-28_5a62d137
Live champion + Test basis v1, 19 registry metrics, whole-engine code hash, data snapshot); verify() ok, no drift. Unmeasured at freeze (no registry source): mover/pattern/analog/missed-winner/direction metrics.
none recorded
A1validatedCandles and micro-signals. tests/test_candles.py 12 pass
point-in-time truncation with a NaN-aware control); real-cache parity 60 tickers x 12 dates, candles family strict-vs-batch + future-invariance + gappy data: 0 failures (state/research/parity/latest.json, 2026-09-29).
none recorded
A2validatedPattern miner: validated by Phase 25 on original and hold-out seeds
A13); bit-reproducible across runs and hash seeds (B11); ticker-name invariant (tests/integration disguise test); 154 miner tests.
none recorded
A2bvalidatedWeek-clustered/HAC statistics, full-search permutation null, BH P(real), redundancy pruning, validation gate: engine/pattern_stats.py matches the miner on 100+ real patterns
B14); P(real)>0.9 bin 96-97% truly real in Phase 25 (exact truth).
none recorded
A3validatedHeavy tests across eras: engine/heavy_tests.py real run
B04; 677k rows 1973-2026, 2 seeds, rolling origin): verdict PASS on all 10 gates - rank IC 0.0317, Newey-West t 12.95, positive in every era (worst era 0.0035), beats the shuffled-outcome control, net spread > 0 after costs, seed agreement 1.0. Earlier single-cut results kept in the history (movement IC 0.38; direction ~0).
state/research/heavy_algo/H9b8e5c64e6b5a404.md.
A4validatedLong-term pattern bank: engine/pattern_bank.py
B03, 53 tests: name-keyed, hash-chained versions, earlier-windows-only reads, integrity verify). Real run (1,500 tickers, 458k weekly rows, six eras): verify ok, 0 audit problems; active patterns +0.10%/wk out of sample (t 2.31, optimistic), discarded ~0.
state/research/pattern_bank/report_v3.md.
A5validatedPattern scores -> Find volatility: EVALUATED - NOT ADOPTED. Six-arm ablation on real data
B04; state/research/pattern_movers/ablation_real_370892cce0386506.json): pattern inputs add mover AUC +0.0010..+0.0017 per era, but random/shuffled-pattern controls gain up to +0.0015, so deploy = False in every era; movers grid _ipat 85.8% vs _i 84.7% top-10 is within that control band. PatternMoverModel returns zeros unless its ablation passes.
none recorded
A6validatedMinute collector: .github/workflows/intraday.yml
after every weekday close + Saturday 60-day backfill) verified end to end 2026-09-29: GitHub run 36527933768 succeeded and uploaded 14 day files to release intraday-2026-09; scripts/sync_intraday.py pulled them back (7 new day files). Only completed sessions are written (6 tests; caught a time-comparison bug).
none recorded
A7validatedSelf-tuning Algorithm: engine/miner_tuning.py
walk-forward coordinate search over the miner's own settings, fold-share + week-bootstrap adoption rule, untouched holdout; 7 tests incl. noise adopts nothing and a regime switch prefers short memory). Real run (movement target, 600 tickers, 2008+): EVALUATED - no change adopted (holdout IC 0.352 unchanged); fdr_q inert; shrink_k removed (inert under empirical Bayes).
state/research/algorithm/tuning/tune_move_0.json, data/tune_move0.log.
A8validatedMarket analog engine: built, point-in-time
macro lags fixed), heavy-tested on real data (B05: no-analog baseline beats every analog mode, skill -0.17/-0.18, p~1). EVALUATED - NOT ADOPTED: no edge.
state/research/analogs_ext/.
A9validatedTimeline dial and yearly pacing: engine/timeline.py
+ pacing, admission, gaming guards; B09, 106 tests). Real replays (15 test windows): in-band 22.1% -> 14.3% with the dial, risk better; fails the tiered gate. EVALUATED - NOT ADOPTED; hook stays off (meta dial_on).
state/research/timeline_basis/dial_offline_seed0.json.
A10validatedLesson memory: engine/lessons.py
mistake kinds, hash-chained LessonStore; B06, 56 tests). Real data: lessons from other windows -0.19%/wk (90% CI -0.34..-0.02); 7 era triples beat the shuffled null in 2/7. EVALUATED - NOT ADOPTED.
state/research/lessons/.
A10bvalidatedRerun anti-memorisation: engine/antimemo.py on the real archive
45 windows): memorisation (same window vs renamed+shifted) = 0.000; positive control (RecallControl memoriser) +4.9%/wk on its window, 0 on the disguised rerun - flagged in 45/45, so the harness can see memorisation.
state/research/lessons/archive_v2.md.
A11validatedImprove-or-discard lifecycle: decaying pattern 0% held on both seed sets
decay death rule + Bonferroni-corrected rescue at in-scope size, C43); regime patterns correctly rescoped; engine/pattern_lifecycle.py + bank integrity on real data (B03, 53 tests).
none recorded
A12unprovenAdditional data: DONE - FRED
19 series, lags audited), sector ETFs (validated, inception respected), minute bars (A6), EDGAR delisted-exit registry (9,250 exits 1996-2026; data/cache/delisted_*). UNPROVEN - prices for dead US names before ~2015 are not obtainable from free sources (Yahoo 404s them, Stooq now requires JavaScript; only ~250 mostly post-2020 names recovered), so survivorship cannot be removed from free data; it is measured and quarantined (T20: analytic drag -4 bps/week vs +32 bps/week model edge; pre-1983 windows THIN). Needs a paid/CRSP-type source to close.
none recorded
A13validatedPlanted-pattern calibration: VALIDATED on BOTH seed sets with the final code
2026-09-29) - original seeds 100+ (FDR 6.7%, decaying 0% held, P(real)>0.9 bin 97% real) and fresh hold-out seeds 300+ (FDR 9.9%, decaying 0%, 96%); all 10 criteria pass.
state/research/algorithm/planted/report_final_seeds100.md, report.md (hold-out), data/planted_cal9_holdout.log, data/planted_cal10_origseeds.log. Fixes: exact-truth discovery scoring, decay death rule, Bonferroni-corrected rescue scored at in-scope size. Still open (separate, stricter): tests/test_patterns_integration.py strict xfail - small fast-setting noise panels admit ~0.67 false patterns/run. History:
V1unproven95% mover target: UNPROVEN - target not reached after controlled testing; research continues
Bible Phase 45). Exact reason: movement IS predictable - picks touch +-10% within the week 57.9% end-to-end (CI 55.8-60.2%) vs 14.6% base, 72.6% among top picks (83-92% since 2020; a naive ATR ranking matches it); weeks with 10 candidates above a 95% threshold: 0.1%. With free daily data the 95% bar is not reachable at 10 picks/week.
state/research/fv/run1/report.md, state/research/direction2/run1/report.md.
V2unprovenDirection at >=80% calibrated confidence: UNPROVEN - target not reached after controlled testing; research continues
Bible Phase 45). Exact reason: no direction signal on movers exists in the available data - B07 (stand-in inputs) 51.8%, Brier = base rate; V5 run: gate open 0/716 weeks; B25 (real literature-backed inputs on PREDICTED movers: post-earnings drift, insider clusters, reversal/momentum, 52-week high, miner score, event type, regime; ~970 accuracy/coverage cells, 2018-2026): 49-53% at full coverage vs 51.7% always-up, no cell reaches 80%, best lower bounds 0.55-0.65 (adjusted 0.45-0.59); planted-signal control recovered at 92-95% and shuffled labels read ~50%, so the pipeline could see a signal if one existed.
state/research/direction2/run1/report.md.
V2bvalidatedPer-stock-type trust tables: engine/trust.py + trust_store.py
versioned, drift monitor; B07, 44 tests). Real walk-forward: no type carries direction skill. EVALUATED - NOT ADOPTED (neutral weights).
state/research/direction/, docs/trust_report_direction.html.
V3validatedExit learner: engine/exits.py
B08). Real walk-forward (19,304 positions, 2005-2022): out-of-sample gain CIs straddle 0, chosen rule flips between folds; pattern-failure exits carry no information (z~1). EVALUATED - NOT ADOPTED: exits stay week-end.
state/research/exits_stops/, state/research/pattern_fail/.
V4validatedStop/risk learner: engine/stops.py + engine/gaprisk.py
B08, 51 tests). Stops cut P(loss>20%) 3.3% -> 1.2% but gaps leave a -73% worst loss; the conditional gap model is calibrated (5.2%/1.06% exceedance vs 5%/1%); holding the -20% cap at 1% needs dropping ~41% of positions (loss-cap filter).
state/research/gaprisk/. The -20% GOAL itself is tracked under V5.
V5unproven8 of 10 at +10% across blind eras: UNPROVEN - target not reached after controlled testing; research continues
Bible Phase 45). Exact reason: it requires direction (V2), which has no measurable signal; the full pipeline therefore takes no bets (share10 lower bound 0.00). Movers-only: 15% of picks finish >= +10%.
state/research/fv/run1/report.md.
T1validatedArchive
39 windows rebuilt with the move signal, insider data and opening prices; regen logs).
none recorded
T2validatedBlind loop: running on the C56-fixed code
PID 26844, data/loop2_run5.log). Round 3 (2026-09-29): all gates OK on 3/3 windows, hardened feed, past-only basis lineage (3/3 on the untrained neutral basis - nothing trained precedes them yet), THIN window reported apart, reveal after the basis decision (1965, 1997, 1991), checkpoint bundle written. Performance targets are T3/T19.
none recorded
T3unprovenTiered 7% objective
>= 50% of weeks in the 5-10% band): UNPROVEN - target not reached after controlled testing; research continues (Bible Phase 45). Exact reason: the real system lands 22-23% of weeks in band (39 revealed windows, 2,049 weeks); no 1-25 stock volatility basket reaches 30% (proxy); the timeline dial (14%), volatility targeting (no gain), the basis search (nothing adopted) and six learners (band_cfg buys ~2 pt only with significantly worse risk) all fail to raise it without extra risk.
state/research/timeline_basis/, state/research/voltarget/, state/research/learners/.
T4validatedWeekly self-adjustment: fires 0-6 times per window
engine/adaptive.Adapter + memory; B15, 171 tests; deterministic across hash seeds). EVALUATED through the learning-delta harness: memory-bank learner transfer -0.03%/wk (memorisation), basis learner transfer -0.002%/wk - no demonstrated value on unseen years; B24 is searching for learners that transfer.
state/research/learning_delta/ld1, ld2_basis.
T5validatedInsider leak protection
parity 0.0 on 4 windows × 8 days with insider data on).
none recorded
T6unproven13D correctness: UNPROVEN before 2014. The subject-company fix
engine/edgar.fix_activist, resumable) finished 2026-09-29: kept 31,417 of 53,865 activist events; 2014+ verified (about 78% kept - the rest were filer-side misattributions), but verification found almost no matches for 2001-2013 (about 1-3 per year kept vs ~1,100-1,700 before), which points to a verification failure on older SEC filing pages, not to those events being wrong. The fixed file keeps only verified events (pre-2014 activist data effectively absent; backup data/cache/events_before_13dfix.parquet). Impact: nil today - ev_activist is excluded from the miner and the evidence weights; it must stay excluded until pre-2014 verification is fixed.
none recorded
T7validatedPattern Explorer: docs/explorer.html generated from state artefacts
B16, 29 tests); viewed live on GitHub Pages at phone width 390px on 2026-09-29 - renders, no horizontal page overflow (375<=390), no console errors except a missing favicon; now shows the long-term bank (11 hash-chained versions from the newest research bank until a production bank exists). Content refreshes each rebuild.
none recorded
T8validatedSensitivity page: docs/sensitivity2.html
28 settings, 106 values; 2 pass |t|>=3.5; losing values shown); viewed live at phone width 390px on 2026-09-29 - renders, no horizontal overflow. Also dashboard.html (safety-audited: it refused to publish while planted test secrets leaked in via pytest's cache - fixed).
none recorded
T9validatedRe-tester parity. Round 2
2026-09-28): re-tester OK on 3/3 windows under the provenance/stale-code gate; engine/retester.py (0.5% relative, code hash first).
data/loop2_run3.log.
T10validatedFuture scramble: OK on every window incl. round 2
3/3, 2026-09-28); tests/test_session.py peeking-rule control fails it as required; blind_gates look-ahead probe 6/6 honest pass + 6/6 peeker caught (state/research/blind_gates/results.json).
none recorded
T11validatedTime fence: Session raises on future prices
tested); pit.Guard with hash-chained audit log (55 tests); features.build future-invariant on real data (5 cuts, 107k rows) and real-cache parity 60x12 with future-invariance per family: 0 failures. (The WIDER future-knowledge audit of C56 is T20.
none recorded
T12validatedWorker health: engine/health.py wired into loop2
supervise, heartbeat, exclusion report); on real data it correctly EXCLUDED 3 windows whose results held NaN (cause fixed, 8ef3835).
data/loop2.log, state/livesim/w02*/health2.jsonl.
T13validatedReproducibility: PatternMiner identical across 2 runs and 2 PYTHONHASHSEEDs; reference evaluator reproducible; re-tester faithful replay PASS on 819 items; Session tie-order made hash-seed independent
B15); final report reproducibility audit PASS (state/reports/final_report.json).
none recorded
T17validatedPlanted-pattern calibration: same as A13
VALIDATED on original and hold-out seeds).
none recorded
T18validatedFill audit
C33): round 2 proved all 490 fills at the next open (164+142+184) via engine/fill_audit.gate -> pit.audit_fills; 5 planted defects caught in tests.
none recorded
T19unprovenLearning delta
canon C54/C55): harness VALID (self-check: generaliser/memoriser/non-learner/leak all classified correctly). FIRST REAL RESULT (ld1, 12 windows, memory-bank learner): same-year disguised effect +0.23%/wk (CI +0.01..+0.54) but TRANSFER -0.03%/wk (CI -0.09..+0.02) -> verdict MEMORISATION (the memory recognises a year by its market-context fingerprint); in-band share HARMED -1.4 pt. Under the C56 past-only rule the loop's memory cannot see a year's own first run, so production is protected; the honest headline (transfer) is: the learning system does NOT yet generalise. Making transfer > 0 is now the main research objective.
state/research/learning_delta/ld1/report.md. LEARNER SEARCH (B24, ls1): band_cfg transfers - in-band +1.1 pt on unseen years (CI +0.2..+2.2), no memorisation; 5 other learners idle; Replication (ls2_replicate, seed 7, 16 pairs): NOT CONFIRMED - the learner's planted-world self-check was INVALID on that seed, and band_cfg's in-band gain (+1.07 pt) came with significantly worse risk on unseen years (worst-5% -0.37 pt, max DD -3.2 pt). Not adopted. Gate then made robust (family-wise max-t threshold; self-check VALID on seeds 1-12) and judged on all tiers (C39): on two fresh seeds (ls3_s11, ls3_s12; 16 past-only pairs each) NO learner passes - band_cfg raises in-band ~+2 pt but significantly worsens worst week and max drawdown on unseen years; the other five never adopt. Current truth: no learning mechanism yet produces gains that transfer to unseen years without extra risk. STATE: UNPROVEN - target not reached after controlled testing; research continues (Bible Phase 45). The harness itself is VALID and running; the owner's C54-vs-C56 ruling on same-year reruns is pending.
T20validatedFuture-leak audit
canon C56): 12 channels audited on real caches (engine/leak_audit.py, 52 tests); fixes APPLIED and live: hardened feed, NetworkGuard, crypto removed before disguise, past-only BasisLineage from neutral defaults, split-invariant tradable filter (Test path), THIN windows apart, learning headline = transfer delta; 210 affected tests pass; round 3 ran clean on it. Quarantined (documented, cannot be removed from free data): survivorship (pre-1983 thin), year fingerprints from levels, macro revisions unmeasurable. Owner question pending: C54 same-year reruns vs C56.
state/research/leak_audit/report.md, data/loop2_run5.log.
L1not startedUpgrade only after a validated research edge
owner decision). Also for the owner: live decides at 15:42 ET and fills before the close, while the simulations decide at the close and fill at the next open (execution parity).
none recorded
L2validatedPaper-only safeguards: paper=True in the single broker client
engine/broker.py) and tick; tests/test_live_safety.py + audit: paper-only clients broker.py True, tick.py True; 0 credential reads outside the live side (state/research/data_live_safety/report.md).
none recorded
L3validatedTrading-hours firewall: broker.regular_hours uses the strict NYSE calendar
holidays + half days) since 2026-09-29 - it had allowed 1,090 closed half-hour slots; tests/test_broker_hours.py (5) + tests/test_live_safety.py (the audit catches a planted weekday-only rule; real broker 0 unsafe slots); scheduler (engine/tick.py) screens non-sessions too.
none recorded
L4validatedBroker safety audit: scripts/data_live_audit.py
state/research/data_live_safety/report.md): the one defect found (holidays/half days allowed) FIXED 2026-09-29 with tests/test_broker_hours.py; the one research->Live write path (improve.test_and_promote, Bonferroni-gated) recorded as a reviewed exception, audit fails on any second writer.
none recorded
L5validatedResearch/live isolation: engine/isolation.py + quality gate import_boundaries PASS
research never imports the live order path; engine/explain.py split out); tests/test_isolation.py pass; CI runs the gate on every push.
none recorded
2 checklist line(s) did not match the expected format and are not shown.

Built 2026-09-29 12:40 UTC. Sources: checklist: state/CHECKLIST.md; registry: state/experiments.jsonl. Generated by scripts/site_build.py; sealed test windows and credentials are never read. Paper trading only; nothing here is investment advice.