Recursive Language Models — L2 #

1. TL;DR #

RLM is an inference-time scaffold that stops feeding a long prompt $P$ into the model's context window and instead loads $P$ as a variable inside a persistent Python REPL; the root LM sees only constant-size metadata and writes code to peek, decompose, and recursively call itself (llm_query / rlm_query) over slices of $P$. It processes inputs >10M tokens (10× past the window), beats GPT-5 + compaction/CodeAct/Claude-Code scaffolds by a median of 13–130% at comparable cost, and a 1,000-sample fine-tune (RLM-Qwen3-8B) lifts the base model +28%.

2. Q1 / Q2 / Q3 #

Q1 — 痛点. Frontier LMs have hard context limits (e.g. GPT-5 at 272K tokens) and, even inside those limits, suffer context rot: quality degrades steeply as prompts lengthen, and faster for more semantically demanding tasks. The dominant mitigation — context compaction/summarization once a threshold is hit — is lossy: it presumes early details can be forgotten, so it fails on tasks needing dense access throughout the prompt. Prior coding/retrieval agents still funnel fetched snippets back into the same bounded window; prior self-delegation verbalizes sub-calls autoregressively and is capped by the model's output length.

Q2 — 方法. Treat the prompt as part of the environment, not as neural input. Given $P$, initialize a REPL environment $\mathcal{E}$ with $P$ bound to a variable plus a sub-call function. The root model $\mathcal{M}$ is shown only constant-size metadata (length, short prefix, access API) and repeatedly emits code; each turn updates REPL state and appends only constant-size stdout metadata to history. The model builds intermediate results into variables and may launch $\Omega(|P|)$ or $\Omega(|P|^2)$ programmatic sub-calls (llm_query for a single cheap call, rlm_query for a full recursive sub-loop), terminating when it sets a FINAL/FINAL_VAR answer.

核心技术壁垒: the load-bearing trick is symbolic recursion paired with metadata-only history — code inside $\mathcal{E}$ can invoke $\mathcal{M}$ over arbitrarily many programmatically-constructed transformations of $P$ while only constant-size stdout metadata is fed back, so the root window never fills and outputs can exceed $\mathcal{M}$'s output limit by being stitched from sub-call results in a variable.

Q3 — 结果. Across four complexity-graded long-context tasks plus a long-reasoning benchmark, RLM(depth≥1) maintains accuracy where base models collapse (OOLONG-Pairs F1 0.1 → 76.0 for GPT-5 at depth=3), scales to 6–11M-token corpora at ~$0.99/answer (cheaper than extrapolated direct ingest), and a tiny off-domain fine-tune transfers across all tasks while running >3× faster.

3. 架构 / 方法图 #

Paper's Figure 2 (caption: "A Recursive Language Model (RLM) treats prompts as part of the environment. It loads the input prompt as a variable inside a REPL environment $\mathcal{E}$ and writes code to peek into, decompose, and invoke itself recursively over programmatic snippets of the variable.").

The figure shows the two boundaries that define the method: the root LM (depth=0) never receives the prompt text — only the green REPL box holds prompt as a variable. The root emits code cells (In[1] does print(prompt[:100]), In[2] slices with prompt.split("Chapter 2") and fans out to llm_query calls), and each Out[k] is truncated. The right-hand RLM(depth=1) boxes are sub-calls, each a fresh LM that returns a short Sub-Response (e.g. "The silver flask...") back into a REPL variable. The final answer is assembled inside the REPL and returned, not generated directly by the root in one window.

The agent loop is a single tight cycle. Each turn: (observe constant-size metadata) → (root emits code) → (REPL executes, mutating state and emitting stdout) → (append code + stdout-metadata to history) → repeat until a FINAL/FINAL_VAR variable is set.

The defining contrast is Algorithm 1 (RLM) vs Algorithm 2 (a "deceptively similar" weak scaffold). Both have sub-calls, external objects, and code execution, but they differ on where the prompt and intermediate values live and where recursion happens:

4. 作者证明 #

无形式化作者证明 — 仅实证（the paper proves no convergence or success guarantee; it offers an informal complexity/budget argument plus expressivity contrast, and otherwise relies on benchmarks).

Notation table

Physical meaning of the budget bound. If each root turn is trimmed to $c$ tokens of history, then the root can run at most $K/c$ iterations before its own window fills:

$$\text{root iterations} \le K/c, \quad \text{each launching arbitrarily many sub-calls.}$$

The division by $c$ is the key lever: it converts "fill the window with prompt text" into "fill the window with bounded per-turn metadata," decoupling the number of root reasoning steps from $|P|$. Each iteration can fan out $\Omega(|P|)$ (or $\Omega(|P|^2)$ for pairwise tasks) sub-calls, so total semantic work is unbounded even though root steps are bounded.

6 checks

1. Window invariance — does the root window grow with $|P|$? No: only Metadata(state) and Metadata(stdout) (constant-size prefix + length) enter hist, so the root window is independent of prompt length. ✔ (Algorithm 1, Footnote 1)
2. Output unbounding — can $|Y|$ exceed $\mathcal{M}$'s output limit? Yes: Final is read from a REPL variable assembled across many sub-calls, not generated in one autoregressive pass. Algorithm 2's Finish action cannot. ✔
3. Horizon claim — is $\Omega(|P|^2)$ work actually reachable? Reachable in principle (loops over pairs of slices) and demonstrated empirically on OOLONG-Pairs where pairwise predicates force quadratic processing. ✔
4. Termination — does the loop halt? Only when Final/FINAL_VAR is set; otherwise capped by a max-iteration limit at each recursion level (a deliberate cap, "not a fundamental limitation"). ✔ (with caveat: §B.5 reports FINAL-tag detection is brittle)
5. Expressivity boundary — is the Algorithm-2 foil genuinely weaker, or a strawman? Each of its three flaws maps to a concrete failure row in Table 1 (CodeAct + sub-calls hits context limits ∗ on CodeQA/BrowseComp; compaction collapses on OOLONG-Pairs at 0.1). ✔
6. Cost monotonicity — does cost scale gracefully with horizon? Authors claim cost scales proportionally to task complexity while staying within an order of magnitude of base GPT-5; supported empirically (App F, Fig 11/16) but only the median is cheaper — the tail is heavy. ⚠ partial (high-variance, long-tail trajectories).

5. 实验与数据 #

Degradation vs length and complexity.



Paper's Figure 1 (caption: GPT-5 vs RLM(depth=1) on three tasks of increasing complexity, input length $2^{13}$–$2^{20}$; inputs beyond the red region exceed GPT-5's 272K window).

Left panel: GPT-5 holds 100% on constant-complexity S-NIAH but OOLONG (linear) and OOLONG-Pairs (quadratic) collapse — OOLONG-Pairs hits ~0 by 33K tokens, far short of the window. Right panel: the RLM keeps S-NIAH at 100% and OOLONG/OOLONG-Pairs in the 45–60% band even at 1M tokens (the green region past the window). The crossover is at $2^{14}$: beyond that the RLM consistently wins. This is the visual core of the paper's thesis — degradation is task-complexity-dependent, and offloading flattens it.

Main results across four tasks.

Model	CodeQA	BrowseComp+ (1K)	OOLONG	OOLONG-Pairs
GPT-5 Base	24.0∗	0.0∗	44.0	0.1
GPT-5 Compaction agent	58.0	70.5	46.0	0.1
GPT-5 CodeAct (+sub-calls)	24.0∗	0.0∗	40.0	28.4
GPT-5 RLM (depth=0)	58.0	88.0	36.0	43.9
GPT-5 RLM (depth=1)	62.0	91.3	56.0	58.0
GPT-5 RLM (depth=3)	58.0	92.0	58.0	76.0
Qwen3-Coder RLM (depth=0)	66.0	46.0	43.5	17.3
Qwen3-Coder RLM (depth=3)	44.0	68.7	32.0	21.1
Claude Code (+offload)	62.0	84.0	48.0	6.5

This table carries Observations 1–4. Note the two regimes: (a) the REPL alone (depth=0) already rescues long-input tasks — GPT-5 BrowseComp 0.0 → 88.0; (b) recursion is what unlocks information-dense tasks — OOLONG-Pairs 43.9 (depth=0) → 76.0 (depth=3). The loss cases are also visible and instructive: Qwen3-Coder degrades with depth (CodeQA 66.0 → 44.0), and on CodeQA GPT-5 RLM trails OpenCode(+offload)=64.0 — recursion is not universally beneficial.

Trajectory behaviors that drive the numbers.



Paper's Figure 8 (caption: (a) RLMs filter context via regex code; (b) decompose context through recursive sub-calls; (c) stitch recursive LM outputs into a longer composite output).

These three panels are the qualitative mechanism behind Table 1: (a) find_snippets regex scanning lets the model navigate without reading $P$ into context; (b) batched llm_query over chunked questions is how linear/quadratic tasks get done line-by-line; (c) FINAL_VAR(final_result) over a stitched list of 10,731 pairs is how output exceeds the window — exactly the OOLONG-Pairs win condition.

Decomposition sensitivity and error analysis.



Paper's Figure 4 (caption: (a) RLM(GPT-5) on OOLONG by varying in-context decomposition examples in the system prompt; (b) fraction of RLM(depth=1) trajectories with ≥1 syntax error, bucketed by correctness).

Figure 4(a) shows the first decomposition attempt is decisive and is steerable by even unrelated in-context examples; 4(b) shows RLM(Qwen3-Coder) carries far more syntax errors than RLM(GPT-5) — the mechanistic explanation for why higher recursion depth hurts the weaker coder model (errors propagate into sub-calls).

Training transfers and length-generalizes.



Paper's Figure 3 (caption: (a) rejection fine-tuning Qwen3-8B on distilled RLM(Qwen3-Coder) trajectories improves all benchmarks; (b) RL training RLM(Qwen3-4B) on 64k/2-needle MRCRv2 generalizes to 1M/8-needle, compared to a 1M-context frontier model).

Observation 6: 1,000 filtered off-domain trajectories (LongBenchPro) lift Qwen3-8B as an RLM across all four eval tasks and make it >3× faster (better decisions, fewer mistakes). Panel (b) is the stronger claim — pure RLVR on a short split generalizes to 16× longer contexts with 4× more needles, hinting RLM-style behavior is a trainable axis of scale.

A separate long-reasoning result (Table 2): RLM(GPT-5.2, depth=1) + decomposition hints reaches 65.6 overall vs base 38.7 (+69.5%), but the same hints given to base GPT-5.2 without a REPL make it worse (38.7 → 28.6, Table 3) — the REPL, not the hints, is doing the work.

6. 论证链 #

#	命题	论据（paper-internal）
1	Long prompts are not just a window-size problem; quality rots, and rot is task-complexity-dependent	Figure 1 left: GPT-5 holds S-NIAH but collapses on OOLONG-Pairs well before its window fills; §3 complexity taxonomy ($O(1)$/linear/quadratic)
2	Compaction cannot fix this because it is lossy for dense-access tasks	§1 argument + Table 1: GPT-5 Compaction = 0.1 on OOLONG-Pairs despite scaling on simpler tasks
3	The fix is to remove $P$ from the window: bind it as a REPL variable, feed root only constant-size metadata	Algorithm 1 + Footnote 1: hist gets Metadata(stdout) only → at most $K/c$ root iterations independent of $	P	$
4	This alone (depth=0) already lets the model beat window limits on long-input tasks	Table 1: GPT-5 RLM depth=0 BrowseComp 0.0→88.0, CodeQA 24→58; Obs 2
5	But information-dense tasks additionally require programmatic recursion (loop sub-calls over slices)	Table 1 OOLONG-Pairs depth=0=43.9 → depth=3=76.0; Figure 8(b,c) shows per-line sub-calls + stitched output; Obs 2
6	The scaffold also extends output length and reasoning horizon, not just input	Output stitched from variable (Fig 8c); Table 2 +69.5% on LongCoT-mini with decomposition
7	The behavior is cheap to instill by training, and transfers/length-generalizes	Figure 3(a) +28% from 1,000 off-domain samples; Figure 3(b) 64k→1M RL generalization; Obs 6
8	Therefore RLM is a general task-agnostic inference paradigm and a new axis of scale	§8 conclusion synthesizing 1–7; comparable cost (Obs 4, Fig 11)

7. 实现 cross-reference #

Reference implementation is public: https://github.com/alexzhang13/rlm (cited in §0/§8 of the source; specific file:line not enumerated in L1, so internal anchors are [实现未公开] at line granularity). The paper specifies the instantiation precisely enough to rebuild:

• Core loop — Algorithm 1: InitREPL(prompt=P) → AddFunction(state, sub_RLM_M) → while True { code ← LLM_M(hist); (state, stdout) ← REPL(state, code); hist ← hist ‖ code ‖ Metadata(stdout); if state[Final] return }. Python REPL; sub-LM/sub-RLM exposed as importable modules; stdout truncated to bound the window. (§2 Para 9)
• Tool catalog (§C.1): context variable; llm_query(prompt) — single sub-LM call, sub-model fits ~500K chars; rlm_query(context, query) at depth>1 — spawns a full nested RLM loop, auto-falls-back to llm_query at max depth; print() truncated. Final-answer contract: FINAL(text) or FINAL_VAR(var). Code emitted inside `repl fences.
• Backbone config (§3.2): GPT-5 (medium reasoning) as root + GPT-5-mini as recursive LM; Qwen3-Coder-480B-A35B (Fireworks costs); depths 0–3 (0 = no sub-calls, 1 = sub-LLMs, >1 = sub-RLMs).

关键实现细节 (easy-to-miss).

1. Prompt portability is not free (§B.1, §C.1): the same RLM system prompt across models misbehaves — Qwen3-Coder needed an extra line capping llm_query use (else thousands of sub-calls per trivial task; batch ~200K chars/call), and the Qwen3-8B variant re-tunes all char budgets for its ~32K window.
2. The FINAL tag is the dominant brittleness (§A.4, §B.5): ~16% of distillation turns misused FINAL() and ~13% misused FINAL_VAR(); a programmatic patch to fix these template mistakes was required for RLM-Qwen3-8B to train well, and a model can build the correct answer in a REPL variable then discard it and emit a wrong root answer (Example E.2).

核心技术壁垒 (replication-hardest insight). The single insight hardest to copy is not the REPL but the discipline of metadata-only history coupled with programmatic recursion: the root must be willing to never see $P$, to trust variables it cannot read in full, and to express recursion as code that loops sub-calls — a behavior that depends on strong coding ability (small/weak coders fail, §B.2) and on a robust final-answer protocol. Naive scaffolds that put $P$ or the Finish output in the window (Algorithm 2) inherit the window limit and silently collapse to compaction; reproducing RLM means reproducing this whole contract, not just adding a code tool.