Skip to content

Spectral Rewiring for Exploration, Purification, and Model Merging

🕒 Published (v1): 2026-07-03 00:00 UTC · Source: HuggingFace · link

Why this paper was selected

Spectral rewiring addresses RL post-training saturation and model merging for agent capability preservation

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Subspace-Aligned Rewiring (SAR) is a post-hoc, training-free editing method that projects RL post-training weight updates onto the pretrained model's SVD spectral subspace, discarding off-manifold residual directions. Using as little as ~0.58% of total parameters, SAR preserves over 99% of post-training reasoning performance, improves high-\(k\) exploration, reduces cross-domain interference, and enables expert model merging that surpasses individual domain experts.

Problem

Dense full-parameter RL post-training of LLMs creates two deployment bottlenecks: (1) suppressed reasoning diversity, manifesting as premature saturation of test-time scaling (Pass@k curves plateau early); and (2) cross-domain interference when consolidating multiple capabilities via mixed-domain RL or model merging (updates from math, coding, and instruction-following occupy incompatible directions in parameter space). No principled coordinate system existed to separate reasoning-effective update components from residual directions.

Method

SAR treats the base model's SVD coordinates as a functional basis for reasoning. For each weight matrix \(W_0 = U\Sigma V^\top\), the RL update \(\Delta W = W_{\text{RL}} - W_0\) is decomposed as:

\[\Delta W^* = P_U \Delta W P_V = U(U^\top \Delta W V)V^\top = UMV^\top\]

where \(M = U^\top \Delta W V \in \mathbb{R}^{r \times r}\) is the Rewiring Matrix. In practice: (1) compute top-\(k\) SVD of \(W_0\); (2) extract the top-\(k\) low-rank component of \(\Delta W\); (3) project it onto the pretrained spectral subspace via \(M\); (4) reconstruct \(\Delta W^* = UMV^\top\) and add back to \(W_0\). The off-diagonal elements \(M_{ij}\) mechanistically implement multi-conditional reasoning by routing evidence from input direction \(v_j\) to output direction \(u_i\), enabling cross-component integration beyond simple associative recall. The method is applied post-hoc with no retraining.

Key Contributions

  • Compact extraction: SAR preserves full-RL math performance across 1.5B–32B models retaining only 1%–30% spectral rank (storage as low as ~0.58% of total parameters for 1.5B and ~0.64% for 32B models).
  • Improved high-\(k\) exploration: SAR-projected models overtake full-RL baselines on Pass@\(k\) at \(k \geq 2\)–\(4\) and cover more AIME problems (26/30 vs. 25/30 at \(k=256\)) without retraining.
  • Mix-RL purification: On OLMo-3.1-32B-Think, SAR improves LiveCodeBench v5 (+1.48%), v6 (+0.95%), and AIME 25 Pass@16 (88.33% → 91.71%) while preserving instruction-following (IFEval: 92.42% → 92.05%).
  • Expert merging: SAR-merged math+code models at 1.5B and 14B simultaneously surpass the best single-domain math and code experts on AIME AVG@32 and LiveCodeBench AVG@8.
  • Agentic coding generalization: Top-1% spectral projection improves 6/7 open agentic coding benchmarks on an in-house model (+2.52% average, up to +25.10%).

Results

  • DeepScaleR 1.5B (AIME 24): Full RL AVG@32 = 40.31%; SAR (1% rank) = 40.21% — near-identical with ~0.58% parameter budget.
  • OLMo-3.1-32B-Think (AIME 24): Full RL AVG@32 = 79.56%; SAR (1% rank) = 78.88%; Pass@32 both 93.33%.
  • Pass@k exploration: SAR crosses the full-RL curve at \(k=2\) (1.5B) and \(k=4\) (32B), with continued gains to \(k=128\).
  • Mix-RL (32B): Coding improves LCB v5 67.61% → 69.09%, LCB v6 64.30% → 65.25%; math AIME 25 Pass@16 88.33% → 91.71%; IFEval 92.42% → 92.05%.
  • Expert merging (1.5B): SAR merged model achieves AIME AVG@32 = 43.44% vs. best math expert 40.31%; LCB AVG@8 = 32.25% vs. best code expert 31.80%. Outperforms Task Arithmetic, TIES, and DARE+TIES baselines.
  • Agentic coding (in-house): SWE-Bench-Pro +10.21%, MSWE-Bench (CodeAct) +22.86%, TerminalBench 1.0 +6.73%, TerminalBench 2.0 +25.10%.

Limitations

  • The rewiring matrix projection requires storing and computing SVD of all base model weight matrices, which is non-trivial at 32B+ scale.
  • The required retained spectral rank varies substantially by model starting point (1%–30%), requiring a rank sweep per model, with no a priori rule for selecting \(k\).
  • Agentic coding gains are demonstrated only on one in-house model; the method is not independently validated on public open-source agentic coding models.
  • Purification and merging results are demonstrated primarily at the 32B and 1.5B/14B scales respectively; generalization to other model families or tasks is unverified.
  • The mechanistic interpretation of off-diagonal \(M_{ij}\) as "reasoning circuits" is an analogy, not a formal proof.
  • One agentic coding benchmark (MSWE-Bench with ClaudeCode agent) showed a slight regression (−0.33%).

Relevance to Agentic AI / LLM Agents

SAR directly demonstrates improved performance on agentic coding benchmarks (SWE-Bench, MSWE-Bench, TerminalBench) by post-hoc editing of RL-trained models — a training-free alternative to re-running costly RL pipelines for agentic tasks. The finding that the reasoning-effective component of RL post-training is concentrated in the base model's spectral geometry suggests that agentic capabilities elicited by RL are largely latent in the pretrained model, not introduced by training, which has implications for how practitioners scope RL training for agents. The expert merging capability is particularly relevant for multi-domain agentic systems that must simultaneously handle math reasoning, code execution, and instruction following — a central challenge in capable agent deployment. SAR also provides a principled mechanism to improve test-time scaling (Pass@k) without retraining, directly improving best-of-\(k\) and repeated-sampling strategies commonly used in agentic inference.