bilevel autoresearch

┌──────────────────────────────────────────────────┐ │ Level 2 — Mechanism Research (outer) │ │ 4-round LLM dialogue → Python code → inject │ │ Triggers: every 2 outer cycles (~10 inner iters)│ ├──────────────────────────────────────────────────┤ │ Level 1.5 — Search Strategy (mid) │ │ Reads trace, freezes/unfreezes params, │ │ injects guidance string │ │ Triggers: every 5 inner iterations │ ├──────────────────────────────────────────────────┤ │ Level 1 — Inner Loop (autoresearch) │ │ Propose → train (300s budget) → evaluate │ │ → keep/discard, fixed 30-iteration budget │ └──────────────────────────────────────────────────┘

1. Explore — read runner.py and the trace, survey mechanisms from adjacent fields (combinatorial optimization, online learning, DOE, Bayesian optimization), propose candidates. 2. Critique — evaluate candidates against the observed failure mode (repetitive proposals, parameter fixation), pick the most promising. 3. Specify — write a precise interface: class name, constructor args, methods with signatures, integration points in runner.py. 4. Generate — produce complete runnable Python implementing the mechanism, plus the runner.py modifications needed to call it.

The paper explicitly does NOT claim this, but gestures at it: if Level 2's mechanisms improve Level 1's search, the same discovery principle could be applied to the meta-level loop itself — Level 3 reads Level 2's mechanism-generation code and writes better mechanism-generation code. The paper notes this is a "natural extension rather than an empirical result established in this paper."