Here is the compressed summary: Core thesis Discovery is c...

Prompt

Here is the compressed summary: Core thesis Discovery is creative and can be opaque; acceptance must be white‑box. You or an auto‑generator set direction via seeds, then agents industrialize everything: a seed becomes a structured hypothesis card, is expanded into a bounded family of variants, smoke‑tested, validated on honest tick fills, passed through explicit anti‑overfit gates and a locked hold‑out vault, and only then graduates to a live‑ready strategy folder. Each run writes back what worked into an idea bank and evolution loop that injects new seeds. 1. Three lanes mapped to your flow You decompose the pipeline into three lanes that align with your earlier doc and current vision: Intake (seed intake): LLM back‑and‑forth, market‑context auto‑generation, or idea‑bank retrieval feeding hypothesis cards. Discovery (expansion): Parallel mutation (thresholds, lookbacks, regimes, inversions, analogies) into families plus dedup and novelty scoring. Validation (control): Smoke tests, full tick backtest, overfit gates, post‑mortem, reporting, and explicit authorization into strategies/. Each stage has a folder in the scaffold (intake/, discovery/, gauntlet/, qtickengine/, backtesting/, validation/, evolution/, ideabank/, strategies/, governance/, config/, orchestrator/). 2. Anti‑overfit as a first‑class subsystem Because broad mutation plus large‑scale testing is a selection‑bias machine, you turn anti‑overfitting into a dedicated subsystem with locked guardrails. Key components: Trials‑aware metrics: Every variant that reaches full backtest counts as a trial; family headline stats are deflated via Deflated Sharpe Ratio so “the luckiest Sharpe” cannot represent the whole family. PBO via CSCV: Probability of Backtest Overfitting estimated with combinatorially symmetric cross‑validation; families above a configured pbo_max are rejected outright. Hold‑out vault: A most‑recent slice of data (default 15%, hard floor 10%) never touched by selection or tuning; a candidate touches it exactly once at authorization, and failure archives the strategy. Bounded breadth: Family width capped (e.g., max 64 variants) plus dedup and novelty floors so cosmetic rewrites do not inflate trial counts. These controls live in config/overfit.toml and validation/overfit.py, with non‑removable locks in config/guardrails.toml. Agents and the research‑lead cannot disable deflated metrics, PBO, or the vault; only you can change these locks. Smoke tests remain useful not for compute savings but to limit how many trials reach sacred OOS and vault data, enforced by configurable rules like skip_if_full_seconds_under. 3. Lane details Lane 1 — Intake (grey‑box) Seeds arrive via: LLM dialogue with you. Single hypotheses generated from current market context. Auto‑generator drawing from the idea bank. A normalizer converts any seed into a falsifiable hypothesis card (thesis, context, inputs, trigger, exit, allowed mutations, kill criteria, priority), then scores priority using novelty, falsifiability, relevance, and implementation cost. The card becomes the durable object the rest of the system operates on, acting as a compiler between intuition and testing. Lane 2 — Discovery (grey‑box) The system expands a card into a family of variants: Mutations: thresholds, lookbacks, regime‑specific versions, inversions, analogies. Deduplication and novelty scoring against the idea bank to prevent re‑testing the same idea with different wording. This lane proposes; it never decides. It maximizes exploration while keeping trial counts trackable for honest statistics. Lane 3 — Validation (white‑box) The lane is strictly controlled and transparent: Smoke pre‑filter: Cheap, PnL‑masked single pass to kill obvious failures and cap trials admitted to deeper stages. Honest tick validation: qtickengine runs frozen walk‑forward OOS backtests. Anti‑overfit gates: Deflated metrics, PBO via CSCV, and the hold‑out vault as hard guards. Post‑mortem: Regime‑by‑regime analysis into findings.yaml (strengths, weaknesses, regime notes, next ideas). Report and metrics: Human‑readable validation_report.md and metrics.json. Adversarial authorization: The research‑lead attempts to falsify candidates before any promotion into strategies/. Acceptance is reproducible and triggered by a single command. 4. Output: live‑ready strategy folders Every accepted strategy gets its own folder (strategies/<id>/) that contains: card.yaml — original hypothesis card. lineage.json — mutation ancestry and sibling count (trial context). metrics.json — OOS, overfit metrics, vault results. findings.yaml — structured strengths/weaknesses and regime notes. validation_report.md — human‑readable write‑up. strategy.py — frozen, inspectable strategy referencing the atom library and exact cleared parameters. Changing parameters requires a new hypothesis card and full re‑validation; live artifacts are immutable snapshots. 5. Idea bank and embedded memory The ideabank/ uses an embedded DuckDB (plus a vector store/VSS) to track hypotheses and outcomes: Vector embeddings of every hypothesis card for novelty/similarity search. A typed connection graph with weighted edges like mutated_from, shares_feature, co_winner, corr_pnl, works_in_regime. Edges are written from realized evidence only (not LLM assertions), so the memory cannot hallucinate relationships. The evolution agent traverses strong edges to propose new seeds and balances exploitation with an exploration budget. Regarding the DuckDB vs LanceDB decision for vector search: DuckDB recently added a Lance extension that allows you to attach a LanceDB directory directly into DuckDB. This gives you the robust, multimodal vector management of LanceDB queried seamlessly using standard DuckDB SQL. It serves as an excellent hybrid approach. 6. Agents and their scopes You define seven agents plus you, each tied to a folder and a slice of config/weights.toml: intake‑agent — controls source mix and seed priority scoring. discovery‑agent — manages nucleus/ atoms, operators, mutation mix, breadth. gauntlet‑agent — tunes smoke sub‑score weights and thresholds. qtickengine‑agent — maintains qtickengine/, backtesting/, validation stats, latency/queues, walk‑forward configs, and computes deflated Sharpe/PBO/vault metrics. evolution‑agent — tunes selection and injection rates via ideabank/. data‑agent — owns data/KDB+/q schemas and data quality gates. research‑lead — controls governance, policy, and authorization; approves structural features and escalates changes to you. Each role‑agent remains dormant until it reaches a maturity threshold (enough generations and in‑scope samples) defined in config/cadence.toml. Once mature, agents re‑evaluate on a cadence, rewrite their own weights within guardrails (logged in weight_history), and escalate structural changes up. Structural changes and new features route to you; autonomy is limited to numeric weights inside hard bounds. 7. Governance and authorization Governance principles: Agents have full autonomy over their own numeric weights within guardrails, with all changes logged. Structural or high‑impact changes go to the research‑lead and then to you; you brief changes back down. The research‑lead acts as the white‑box authorization gate: no candidate becomes live‑ready without clearing vault and overfit gates and surviving adversarial review. Anti‑overfit locks (deflated metrics, PBO, vault) are controlled only by you. 8. Changes from V1 V1 focused on genetic feature discovery (atoms -> candidates -> gauntlet -> backtest -> evolve). V2 keeps that engine but reframes everything around hypothesis cards: Genetic mutation operates on hypotheses, not raw features. The single gauntlet pass becomes a smoke stage inside the validation lane. New elements: a structured intake lane, explicit anti‑overfit subsystem, and idea bank for closing the loop. Agents and invariants from V1 carry over with extended contexts. 9. What you have now (scaffold) You now have a rebuilt scaffold for the entire three‑lane flow: intake/ (SPEC + card schema + sources/ + normalizer/) nucleus/ (atoms + operators; your NUCLEUS core lands here) discovery/ (mutate, analogy, dedup, novelty) data/ (KDB+/q adapters and synth) gauntlet/ (smoke, PnL‑masked passthrough) qtickengine/ (oracle core) backtesting/ (walk‑forward + frozen OOS) validation/ (smoke -> overfit -> postmortem -> report -> authorize) evolution/ (selection + injection) ideabank/ (DuckDB + VSS, embeddings + connection graph) warehouse/ (hypotheses, findings, validation results, promotions) strategies/ (_TEMPLATE for final strategy folder) orchestrator/ (conductor loop) governance/ (briefs, ledger) config/ (routing, weights, guardrails, cadence, overfit) .claude/agents/ (all seven agents) Configs parse, schemas build, PnL‑hidden views work, overfit locks exist, and agent ownership matches cadence config; all .py files are interfaces with no “invented edge.” 10. Next steps after your approval Implementation plan: Port the cores: Bring NUCLEUS atoms, QTICKENGINE bridge, walk-forward harness into folders. Implement Lane 3 first: Setup the safety rail (smoke, honest backtest, overfit gates, untouched vault). Wire intake and discovery: Link card schemas, parallel mutation, and novelty against the idea bank, followed by conductor loops. Dry-run: Run on synthetic data to ensure edge-finding and correct metric-based pruning, then apply real data. Activate agents: Trigger agents only after enough warehouse data is accumulated. 11. Open decisions for you Embedded DB engine: Stick with DuckDB + VSS, or move to LanceDB for vectors? (The DuckDB Lance extension provides a hybrid option to use both). Live strategy language: Python is the spec. Do you want a q export for live execution? Smoke vs straight-to-full: Maintain trial-count hygiene, or bypass for fast-engine runs? Hold-out vault fraction: Retain 15% (floor 10%)? Trial budget: Keep the 24 default (64 max) cap? v3 addendum: live, lifecycle, and four new layers Strategy lifecycle: Discovered, backtest_validated, forward_testing, forward_validated, live, retired. Strategy state lives in warehouse.strategy_state. Capital only attached by you at live. Forward-test gate: Paper runs on live ticks. Drift archives it. Kills overfit-in-hindsight strategies. Four new layers: regime/ (live classification), costs/ (locked model), portfolio/ (book sizing and correlation), live/ (paper executor and decay monitor). Live feed: Databento API fetching GLBX.MDP3 NQ 1-second bars. SQLite ledger. Dashboard: Local FastAPI server on localhost:8787 for monitoring. What changes should be made?