Checklist and unproven
Live status of the master checklist (state/CHECKLIST.md) and an honest list of what is not proven yet.
Validated
39
83%
Implemented / testing
0
0%
Unproven
7
15%
Failed
0
0%
Not started
1
2%
Overall
■ validated 39 ■ unproven 7 ■ not started 1
Nothing is marked validated merely because code exists; a box turns green only with evidence written next to it.
Progress by area
| Area | Items | Validated | Progress |
|---|---|---|---|
| Phase 0 | 4 | 4 | ■ validated 4 |
| Algorithm | 15 | 14 | ■ validated 14 ■ unproven 1 |
| Find volatility | 6 | 3 | ■ validated 3 ■ unproven 3 |
| Test | 17 | 14 | ■ validated 14 ■ unproven 3 |
| Live | 5 | 4 | ■ validated 4 ■ not started 1 |
UNPROVEN: what we do not yet know
Every item that is not validated, failed first.
| ID | Area | State | What is unproven and why |
|---|---|---|---|
| A12 | Algorithm | unproven | Additional data: DONE - FRED 19 series, lags audited), sector ETFs (validated, inception respected), minute bars (A6), EDGAR delisted-exit registry (9,250 exits 1996-2026; data/cache/delisted_*). UNPROVEN - prices for dead US names before ~2015 are not obtainable from free sources (Yahoo 404s them, Stooq now requires JavaScript; only ~250 mostly post-2020 names recovered), so survivorship cannot be removed from free data; it is measured and quarantined (T20: analytic drag -4 bps/week vs +32 bps/week model edge; pre-1983 windows THIN). Needs a paid/CRSP-type source to close. |
| T19 | Test | unproven | Learning delta canon C54/C55): harness VALID (self-check: generaliser/memoriser/non-learner/leak all classified correctly). FIRST REAL RESULT (ld1, 12 windows, memory-bank learner): same-year disguised effect +0.23%/wk (CI +0.01..+0.54) but TRANSFER -0.03%/wk (CI -0.09..+0.02) -> verdict MEMORISATION (the memory recognises a year by its market-context fingerprint); in-band share HARMED -1.4 pt. Under the C56 past-only rule the loop's memory cannot see a year's own first run, so production is protected; the honest headline (transfer) is: the learning system does NOT yet generalise. Making transfer > 0 is now the main research objective. |
| T3 | Test | unproven | Tiered 7% objective >= 50% of weeks in the 5-10% band): UNPROVEN - target not reached after controlled testing; research continues (Bible Phase 45). Exact reason: the real system lands 22-23% of weeks in band (39 revealed windows, 2,049 weeks); no 1-25 stock volatility basket reaches 30% (proxy); the timeline dial (14%), volatility targeting (no gain), the basis search (nothing adopted) and six learners (band_cfg buys ~2 pt only with significantly worse risk) all fail to raise it without extra risk. |
| T6 | Test | unproven | 13D correctness: UNPROVEN before 2014. The subject-company fix engine/edgar.fix_activist, resumable) finished 2026-09-29: kept 31,417 of 53,865 activist events; 2014+ verified (about 78% kept - the rest were filer-side misattributions), but verification found almost no matches for 2001-2013 (about 1-3 per year kept vs ~1,100-1,700 before), which points to a verification failure on older SEC filing pages, not to those events being wrong. The fixed file keeps only verified events (pre-2014 activist data effectively absent; backup data/cache/events_before_13dfix.parquet). Impact: nil today - ev_activist is excluded from the miner and the evidence weights; it must stay excluded until pre-2014 verification is fixed. |
| V1 | Find volatility | unproven | 95% mover target: UNPROVEN - target not reached after controlled testing; research continues Bible Phase 45). Exact reason: movement IS predictable - picks touch +-10% within the week 57.9% end-to-end (CI 55.8-60.2%) vs 14.6% base, 72.6% among top picks (83-92% since 2020; a naive ATR ranking matches it); weeks with 10 candidates above a 95% threshold: 0.1%. With free daily data the 95% bar is not reachable at 10 picks/week. |
| V2 | Find volatility | unproven | Direction at >=80% calibrated confidence: UNPROVEN - target not reached after controlled testing; research continues Bible Phase 45). Exact reason: no direction signal on movers exists in the available data - B07 (stand-in inputs) 51.8%, Brier = base rate; V5 run: gate open 0/716 weeks; B25 (real literature-backed inputs on PREDICTED movers: post-earnings drift, insider clusters, reversal/momentum, 52-week high, miner score, event type, regime; ~970 accuracy/coverage cells, 2018-2026): 49-53% at full coverage vs 51.7% always-up, no cell reaches 80%, best lower bounds 0.55-0.65 (adjusted 0.45-0.59); planted-signal control recovered at 92-95% and shuffled labels read ~50%, so the pipeline could see a signal if one existed. |
| V5 | Find volatility | unproven | 8 of 10 at +10% across blind eras: UNPROVEN - target not reached after controlled testing; research continues Bible Phase 45). Exact reason: it requires direction (V2), which has no measurable signal; the full pipeline therefore takes no bets (share10 lower bound 0.00). Movers-only: 15% of picks finish >= +10%. |
| L1 | Live | not started | Upgrade only after a validated research edge owner decision). Also for the owner: live decides at 15:42 ET and fills before the close, while the simulations decide at the close and fill at the next open (execution parity). |
Rollbacks and voided results
Changes that were adopted and then undone, or results thrown away because the check failed.
| When | Event | Id | Reason |
|---|---|---|---|
| 2026-09-28T20:29:59 | rolled_back | v1.3 | full 2017-2026: $3,579 vs SPY $3,987, max drawdown -73%; round-1 guards only checked single months |
| 2026-09-28T21:21:20 | voided_adjustment | re-tester mismatch vs live trader; fixed bookkeeping; reset to v1 |
Full checklist
| ID | State | Item | Evidence |
|---|---|---|---|
| P0.1 | validated | Canon integrity | `canon/build_canon.py --verify` (56 directives, C1-C56); `engine/provenance.verify_integrity()` checks the canon and the Bible and fails closed (tests/test_provenance.py). |
| P0.2 | validated | Experiment registry with provenance: every ACTIVE writer passes cfg+seed and all Phase 0.2 fields - Test loop rounds + basis search scripts/livesim_loop2.py), Phase 25, self-tuning, Live retrain (engine/train.py), Live promotion path (engine/improve.py: challenger/promoted/rejected/rolled_back/confirmed), heavy tests, antimemo, exits, stops; log_experiment stamps code_hash/code_files/code_mixed, git, canon/Bible hashes, config hash, data snapshot, sha256 ids, and warns on any missing field. 8 superseded scripts archived in scripts/legacy (documented as not provenance-complete). Historic records stay as written (append-only). | none recorded |
| P0.3 | validated | Checkpoint bundle per major run: engine/checkpoint.checkpoint_run() wired into the Test loop one bundle per round), Phase 25 and self-tuning runners. First real bundle: state/checkpoints/loop2_round003_20260929T081401 - verify() ok (2026-09-29). | none recorded |
| P0.4 | validated | Immutable baseline snapshot. Frozen 2026-09-28 23:25: state/baseline/B_2026-09-28_5a62d137 Live champion + Test basis v1, 19 registry metrics, whole-engine code hash, data snapshot); verify() ok, no drift. Unmeasured at freeze (no registry source): mover/pattern/analog/missed-winner/direction metrics. | none recorded |
| A1 | validated | Candles and micro-signals. tests/test_candles.py 12 pass point-in-time truncation with a NaN-aware control); real-cache parity 60 tickers x 12 dates, candles family strict-vs-batch + future-invariance + gappy data: 0 failures (state/research/parity/latest.json, 2026-09-29). | none recorded |
| A2 | validated | Pattern miner: validated by Phase 25 on original and hold-out seeds A13); bit-reproducible across runs and hash seeds (B11); ticker-name invariant (tests/integration disguise test); 154 miner tests. | none recorded |
| A2b | validated | Week-clustered/HAC statistics, full-search permutation null, BH P(real), redundancy pruning, validation gate: engine/pattern_stats.py matches the miner on 100+ real patterns B14); P(real)>0.9 bin 96-97% truly real in Phase 25 (exact truth). | none recorded |
| A3 | validated | Heavy tests across eras: engine/heavy_tests.py real run B04; 677k rows 1973-2026, 2 seeds, rolling origin): verdict PASS on all 10 gates - rank IC 0.0317, Newey-West t 12.95, positive in every era (worst era 0.0035), beats the shuffled-outcome control, net spread > 0 after costs, seed agreement 1.0. Earlier single-cut results kept in the history (movement IC 0.38; direction ~0). | state/research/heavy_algo/H9b8e5c64e6b5a404.md. |
| A4 | validated | Long-term pattern bank: engine/pattern_bank.py B03, 53 tests: name-keyed, hash-chained versions, earlier-windows-only reads, integrity verify). Real run (1,500 tickers, 458k weekly rows, six eras): verify ok, 0 audit problems; active patterns +0.10%/wk out of sample (t 2.31, optimistic), discarded ~0. | state/research/pattern_bank/report_v3.md. |
| A5 | validated | Pattern scores -> Find volatility: EVALUATED - NOT ADOPTED. Six-arm ablation on real data B04; state/research/pattern_movers/ablation_real_370892cce0386506.json): pattern inputs add mover AUC +0.0010..+0.0017 per era, but random/shuffled-pattern controls gain up to +0.0015, so deploy = False in every era; movers grid _ipat 85.8% vs _i 84.7% top-10 is within that control band. PatternMoverModel returns zeros unless its ablation passes. | none recorded |
| A6 | validated | Minute collector: .github/workflows/intraday.yml after every weekday close + Saturday 60-day backfill) verified end to end 2026-09-29: GitHub run 36527933768 succeeded and uploaded 14 day files to release intraday-2026-09; scripts/sync_intraday.py pulled them back (7 new day files). Only completed sessions are written (6 tests; caught a time-comparison bug). | none recorded |
| A7 | validated | Self-tuning Algorithm: engine/miner_tuning.py walk-forward coordinate search over the miner's own settings, fold-share + week-bootstrap adoption rule, untouched holdout; 7 tests incl. noise adopts nothing and a regime switch prefers short memory). Real run (movement target, 600 tickers, 2008+): EVALUATED - no change adopted (holdout IC 0.352 unchanged); fdr_q inert; shrink_k removed (inert under empirical Bayes). | state/research/algorithm/tuning/tune_move_0.json, data/tune_move0.log. |
| A8 | validated | Market analog engine: built, point-in-time macro lags fixed), heavy-tested on real data (B05: no-analog baseline beats every analog mode, skill -0.17/-0.18, p~1). EVALUATED - NOT ADOPTED: no edge. | state/research/analogs_ext/. |
| A9 | validated | Timeline dial and yearly pacing: engine/timeline.py + pacing, admission, gaming guards; B09, 106 tests). Real replays (15 test windows): in-band 22.1% -> 14.3% with the dial, risk better; fails the tiered gate. EVALUATED - NOT ADOPTED; hook stays off (meta dial_on). | state/research/timeline_basis/dial_offline_seed0.json. |
| A10 | validated | Lesson memory: engine/lessons.py mistake kinds, hash-chained LessonStore; B06, 56 tests). Real data: lessons from other windows -0.19%/wk (90% CI -0.34..-0.02); 7 era triples beat the shuffled null in 2/7. EVALUATED - NOT ADOPTED. | state/research/lessons/. |
| A10b | validated | Rerun anti-memorisation: engine/antimemo.py on the real archive 45 windows): memorisation (same window vs renamed+shifted) = 0.000; positive control (RecallControl memoriser) +4.9%/wk on its window, 0 on the disguised rerun - flagged in 45/45, so the harness can see memorisation. | state/research/lessons/archive_v2.md. |
| A11 | validated | Improve-or-discard lifecycle: decaying pattern 0% held on both seed sets decay death rule + Bonferroni-corrected rescue at in-scope size, C43); regime patterns correctly rescoped; engine/pattern_lifecycle.py + bank integrity on real data (B03, 53 tests). | none recorded |
| A12 | unproven | Additional data: DONE - FRED 19 series, lags audited), sector ETFs (validated, inception respected), minute bars (A6), EDGAR delisted-exit registry (9,250 exits 1996-2026; data/cache/delisted_*). UNPROVEN - prices for dead US names before ~2015 are not obtainable from free sources (Yahoo 404s them, Stooq now requires JavaScript; only ~250 mostly post-2020 names recovered), so survivorship cannot be removed from free data; it is measured and quarantined (T20: analytic drag -4 bps/week vs +32 bps/week model edge; pre-1983 windows THIN). Needs a paid/CRSP-type source to close. | none recorded |
| A13 | validated | Planted-pattern calibration: VALIDATED on BOTH seed sets with the final code 2026-09-29) - original seeds 100+ (FDR 6.7%, decaying 0% held, P(real)>0.9 bin 97% real) and fresh hold-out seeds 300+ (FDR 9.9%, decaying 0%, 96%); all 10 criteria pass. | state/research/algorithm/planted/report_final_seeds100.md, report.md (hold-out), data/planted_cal9_holdout.log, data/planted_cal10_origseeds.log. Fixes: exact-truth discovery scoring, decay death rule, Bonferroni-corrected rescue scored at in-scope size. Still open (separate, stricter): tests/test_patterns_integration.py strict xfail - small fast-setting noise panels admit ~0.67 false patterns/run. History: |
| V1 | unproven | 95% mover target: UNPROVEN - target not reached after controlled testing; research continues Bible Phase 45). Exact reason: movement IS predictable - picks touch +-10% within the week 57.9% end-to-end (CI 55.8-60.2%) vs 14.6% base, 72.6% among top picks (83-92% since 2020; a naive ATR ranking matches it); weeks with 10 candidates above a 95% threshold: 0.1%. With free daily data the 95% bar is not reachable at 10 picks/week. | state/research/fv/run1/report.md, state/research/direction2/run1/report.md. |
| V2 | unproven | Direction at >=80% calibrated confidence: UNPROVEN - target not reached after controlled testing; research continues Bible Phase 45). Exact reason: no direction signal on movers exists in the available data - B07 (stand-in inputs) 51.8%, Brier = base rate; V5 run: gate open 0/716 weeks; B25 (real literature-backed inputs on PREDICTED movers: post-earnings drift, insider clusters, reversal/momentum, 52-week high, miner score, event type, regime; ~970 accuracy/coverage cells, 2018-2026): 49-53% at full coverage vs 51.7% always-up, no cell reaches 80%, best lower bounds 0.55-0.65 (adjusted 0.45-0.59); planted-signal control recovered at 92-95% and shuffled labels read ~50%, so the pipeline could see a signal if one existed. | state/research/direction2/run1/report.md. |
| V2b | validated | Per-stock-type trust tables: engine/trust.py + trust_store.py versioned, drift monitor; B07, 44 tests). Real walk-forward: no type carries direction skill. EVALUATED - NOT ADOPTED (neutral weights). | state/research/direction/, docs/trust_report_direction.html. |
| V3 | validated | Exit learner: engine/exits.py B08). Real walk-forward (19,304 positions, 2005-2022): out-of-sample gain CIs straddle 0, chosen rule flips between folds; pattern-failure exits carry no information (z~1). EVALUATED - NOT ADOPTED: exits stay week-end. | state/research/exits_stops/, state/research/pattern_fail/. |
| V4 | validated | Stop/risk learner: engine/stops.py + engine/gaprisk.py B08, 51 tests). Stops cut P(loss>20%) 3.3% -> 1.2% but gaps leave a -73% worst loss; the conditional gap model is calibrated (5.2%/1.06% exceedance vs 5%/1%); holding the -20% cap at 1% needs dropping ~41% of positions (loss-cap filter). | state/research/gaprisk/. The -20% GOAL itself is tracked under V5. |
| V5 | unproven | 8 of 10 at +10% across blind eras: UNPROVEN - target not reached after controlled testing; research continues Bible Phase 45). Exact reason: it requires direction (V2), which has no measurable signal; the full pipeline therefore takes no bets (share10 lower bound 0.00). Movers-only: 15% of picks finish >= +10%. | state/research/fv/run1/report.md. |
| T1 | validated | Archive 39 windows rebuilt with the move signal, insider data and opening prices; regen logs). | none recorded |
| T2 | validated | Blind loop: running on the C56-fixed code PID 26844, data/loop2_run5.log). Round 3 (2026-09-29): all gates OK on 3/3 windows, hardened feed, past-only basis lineage (3/3 on the untrained neutral basis - nothing trained precedes them yet), THIN window reported apart, reveal after the basis decision (1965, 1997, 1991), checkpoint bundle written. Performance targets are T3/T19. | none recorded |
| T3 | unproven | Tiered 7% objective >= 50% of weeks in the 5-10% band): UNPROVEN - target not reached after controlled testing; research continues (Bible Phase 45). Exact reason: the real system lands 22-23% of weeks in band (39 revealed windows, 2,049 weeks); no 1-25 stock volatility basket reaches 30% (proxy); the timeline dial (14%), volatility targeting (no gain), the basis search (nothing adopted) and six learners (band_cfg buys ~2 pt only with significantly worse risk) all fail to raise it without extra risk. | state/research/timeline_basis/, state/research/voltarget/, state/research/learners/. |
| T4 | validated | Weekly self-adjustment: fires 0-6 times per window engine/adaptive.Adapter + memory; B15, 171 tests; deterministic across hash seeds). EVALUATED through the learning-delta harness: memory-bank learner transfer -0.03%/wk (memorisation), basis learner transfer -0.002%/wk - no demonstrated value on unseen years; B24 is searching for learners that transfer. | state/research/learning_delta/ld1, ld2_basis. |
| T5 | validated | Insider leak protection parity 0.0 on 4 windows × 8 days with insider data on). | none recorded |
| T6 | unproven | 13D correctness: UNPROVEN before 2014. The subject-company fix engine/edgar.fix_activist, resumable) finished 2026-09-29: kept 31,417 of 53,865 activist events; 2014+ verified (about 78% kept - the rest were filer-side misattributions), but verification found almost no matches for 2001-2013 (about 1-3 per year kept vs ~1,100-1,700 before), which points to a verification failure on older SEC filing pages, not to those events being wrong. The fixed file keeps only verified events (pre-2014 activist data effectively absent; backup data/cache/events_before_13dfix.parquet). Impact: nil today - ev_activist is excluded from the miner and the evidence weights; it must stay excluded until pre-2014 verification is fixed. | none recorded |
| T7 | validated | Pattern Explorer: docs/explorer.html generated from state artefacts B16, 29 tests); viewed live on GitHub Pages at phone width 390px on 2026-09-29 - renders, no horizontal page overflow (375<=390), no console errors except a missing favicon; now shows the long-term bank (11 hash-chained versions from the newest research bank until a production bank exists). Content refreshes each rebuild. | none recorded |
| T8 | validated | Sensitivity page: docs/sensitivity2.html 28 settings, 106 values; 2 pass |t|>=3.5; losing values shown); viewed live at phone width 390px on 2026-09-29 - renders, no horizontal overflow. Also dashboard.html (safety-audited: it refused to publish while planted test secrets leaked in via pytest's cache - fixed). | none recorded |
| T9 | validated | Re-tester parity. Round 2 2026-09-28): re-tester OK on 3/3 windows under the provenance/stale-code gate; engine/retester.py (0.5% relative, code hash first). | data/loop2_run3.log. |
| T10 | validated | Future scramble: OK on every window incl. round 2 3/3, 2026-09-28); tests/test_session.py peeking-rule control fails it as required; blind_gates look-ahead probe 6/6 honest pass + 6/6 peeker caught (state/research/blind_gates/results.json). | none recorded |
| T11 | validated | Time fence: Session raises on future prices tested); pit.Guard with hash-chained audit log (55 tests); features.build future-invariant on real data (5 cuts, 107k rows) and real-cache parity 60x12 with future-invariance per family: 0 failures. (The WIDER future-knowledge audit of C56 is T20. | none recorded |
| T12 | validated | Worker health: engine/health.py wired into loop2 supervise, heartbeat, exclusion report); on real data it correctly EXCLUDED 3 windows whose results held NaN (cause fixed, 8ef3835). | data/loop2.log, state/livesim/w02*/health2.jsonl. |
| T13 | validated | Reproducibility: PatternMiner identical across 2 runs and 2 PYTHONHASHSEEDs; reference evaluator reproducible; re-tester faithful replay PASS on 819 items; Session tie-order made hash-seed independent B15); final report reproducibility audit PASS (state/reports/final_report.json). | none recorded |
| T17 | validated | Planted-pattern calibration: same as A13 VALIDATED on original and hold-out seeds). | none recorded |
| T18 | validated | Fill audit C33): round 2 proved all 490 fills at the next open (164+142+184) via engine/fill_audit.gate -> pit.audit_fills; 5 planted defects caught in tests. | none recorded |
| T19 | unproven | Learning delta canon C54/C55): harness VALID (self-check: generaliser/memoriser/non-learner/leak all classified correctly). FIRST REAL RESULT (ld1, 12 windows, memory-bank learner): same-year disguised effect +0.23%/wk (CI +0.01..+0.54) but TRANSFER -0.03%/wk (CI -0.09..+0.02) -> verdict MEMORISATION (the memory recognises a year by its market-context fingerprint); in-band share HARMED -1.4 pt. Under the C56 past-only rule the loop's memory cannot see a year's own first run, so production is protected; the honest headline (transfer) is: the learning system does NOT yet generalise. Making transfer > 0 is now the main research objective. | state/research/learning_delta/ld1/report.md. LEARNER SEARCH (B24, ls1): band_cfg transfers - in-band +1.1 pt on unseen years (CI +0.2..+2.2), no memorisation; 5 other learners idle; Replication (ls2_replicate, seed 7, 16 pairs): NOT CONFIRMED - the learner's planted-world self-check was INVALID on that seed, and band_cfg's in-band gain (+1.07 pt) came with significantly worse risk on unseen years (worst-5% -0.37 pt, max DD -3.2 pt). Not adopted. Gate then made robust (family-wise max-t threshold; self-check VALID on seeds 1-12) and judged on all tiers (C39): on two fresh seeds (ls3_s11, ls3_s12; 16 past-only pairs each) NO learner passes - band_cfg raises in-band ~+2 pt but significantly worsens worst week and max drawdown on unseen years; the other five never adopt. Current truth: no learning mechanism yet produces gains that transfer to unseen years without extra risk. STATE: UNPROVEN - target not reached after controlled testing; research continues (Bible Phase 45). The harness itself is VALID and running; the owner's C54-vs-C56 ruling on same-year reruns is pending. |
| T20 | validated | Future-leak audit canon C56): 12 channels audited on real caches (engine/leak_audit.py, 52 tests); fixes APPLIED and live: hardened feed, NetworkGuard, crypto removed before disguise, past-only BasisLineage from neutral defaults, split-invariant tradable filter (Test path), THIN windows apart, learning headline = transfer delta; 210 affected tests pass; round 3 ran clean on it. Quarantined (documented, cannot be removed from free data): survivorship (pre-1983 thin), year fingerprints from levels, macro revisions unmeasurable. Owner question pending: C54 same-year reruns vs C56. | state/research/leak_audit/report.md, data/loop2_run5.log. |
| L1 | not started | Upgrade only after a validated research edge owner decision). Also for the owner: live decides at 15:42 ET and fills before the close, while the simulations decide at the close and fill at the next open (execution parity). | none recorded |
| L2 | validated | Paper-only safeguards: paper=True in the single broker client engine/broker.py) and tick; tests/test_live_safety.py + audit: paper-only clients broker.py True, tick.py True; 0 credential reads outside the live side (state/research/data_live_safety/report.md). | none recorded |
| L3 | validated | Trading-hours firewall: broker.regular_hours uses the strict NYSE calendar holidays + half days) since 2026-09-29 - it had allowed 1,090 closed half-hour slots; tests/test_broker_hours.py (5) + tests/test_live_safety.py (the audit catches a planted weekday-only rule; real broker 0 unsafe slots); scheduler (engine/tick.py) screens non-sessions too. | none recorded |
| L4 | validated | Broker safety audit: scripts/data_live_audit.py state/research/data_live_safety/report.md): the one defect found (holidays/half days allowed) FIXED 2026-09-29 with tests/test_broker_hours.py; the one research->Live write path (improve.test_and_promote, Bonferroni-gated) recorded as a reviewed exception, audit fails on any second writer. | none recorded |
| L5 | validated | Research/live isolation: engine/isolation.py + quality gate import_boundaries PASS research never imports the live order path; engine/explain.py split out); tests/test_isolation.py pass; CI runs the gate on every push. | none recorded |
2 checklist line(s) did not match the expected format and are not shown.
Built 2026-09-29 12:40 UTC. Sources: checklist: state/CHECKLIST.md; registry: state/experiments.jsonl. Generated by scripts/site_build.py; sealed test windows and credentials are never read. Paper trading only; nothing here is investment advice.