Skip to content

Numerical Instability and Chaos: Quantifying the Unpredictability of Large Language Models

🕒 Published (v1): 2026-04-14 18:26 UTC · Source: Arxiv · link

Why this paper was selected

Numerical instability in LLMs undermines agentic harness reliability and determinism

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper provides a rigorous mathematical characterization of how floating-point rounding errors propagate through Transformer layers, producing chaotic and unpredictable LLM outputs even under identical inputs. The authors identify three distinct operating regimes (Constant, Chaotic, Signal-Dominated) and validate them across architectures and precision formats. A simple noise-averaging mitigation recovers stable condition number estimates with as few as \(n = 10\)–\(100\) forward passes.

Problem

Multi-agent LLM deployments exhibit non-reproducible outputs even with fixed seeds — AutoGen fails 23% of collaborative tasks, MetaGPT produces non-reproducible outputs in 31% of planning runs — but prior work treats numerical instability as an engineering nuisance rather than characterizing the underlying dynamics. No principled, layer-wise analysis existed of how machine-epsilon perturbations (\(\sim 10^{-14}\)) interact with Transformer computation to produce divergent outputs.

Method

The authors formalize sensitivity via the absolute directional condition number: $\(\kappa_\text{abs}(f, x, v) := \|J(x)v\|_2 \approx \frac{\|f(x + \epsilon v) - f(x)\|_2}{\epsilon}, \quad \epsilon \to 0\)$ where \(f: \mathbb{R}^n \to \mathbb{R}^{d_\text{model}}\) maps input embeddings to the last hidden state (the "last pseudo token," LPT, before the unembedding head). They sweep perturbation magnitude \(\epsilon\) across directions \(v\) drawn from the SVD spectrum of the Jacobian (singular vectors \(v_1, v_{50}, v_{500}, v_{2000}, v_{4096}\)), coordinate directions, and random directions, tracing \(D(\epsilon, v) = \|f(x+\epsilon v)-f(x)\|_2 / \epsilon\) layer-by-layer. Near-tie decision boundary geometry is probed in 2D singular-vector planes at \(\epsilon_{1,2} \in [-10^{-8}, 10^{-8}]\) with ULP-precise binary search for maximum stable perturbation \(s_\text{max}(\theta)\). A noise-averaging mitigation computes \(\kappa_\text{smooth} = \|\frac{1}{n}\sum_j f(x+\epsilon s_i + v_j) - \frac{1}{n}\sum_j f(x+v_j)\|/\epsilon\) where \(v_j\) are random perturbations of magnitude \(10^{-9}\).

Key Contributions

  • Identification of an avalanche effect in early Transformer layers: perturbations at floating-point epsilon (\(\sim 10^{-14}\)) either amplify exponentially (directional condition numbers exceeding \(10^6\)) or attenuate completely, independent of Jacobian singular value.
  • Characterization of three operating regimes: (1) Constant Region — perturbations below an input-dependent ULP threshold vanish; (2) Chaotic Region — rounding errors dominate and diverge outputs; (3) Signal-Dominated Region — true input variations override numerical noise.
  • Demonstration that \(s_\text{max}\) varies only \(3\times\) across singular values spanning six orders of magnitude (\(\sigma_1 = 615.3\) to \(\sigma_{4096} \approx 0\)), proving instability is determined by floating-point quantization, not spectral structure.
  • Chaotic decision boundaries with 15–16% flip frequency, 779–866 fragmented regions, and \(\sim\)50 crossing density per line scan across all tested perturbation planes.
  • Noise-averaging mitigation: \(n = 100\) samples reduces spurious \(\kappa_\text{abs}\) from \(>900\) to \(\sim 600\) (converging to the true top singular value of 615.31).

Results

  • Mean instability (finite-difference spikes): \(3.78 \times 10^6\) (Llama-3.1-8B/AdvBench), \(8.84 \times 10^6\) (Llama-3.1-8B/TruthfulQA), \(6.76 \times 10^5\) (GPT-OSS-20B/AdvBench); median instability = 0.0 across all model/dataset combinations, confirming prevalent constant-like plateaus punctuated by rare discrete jumps.
  • Max drift in LPT representation: \(1.02 \times 10^{-4}\) to \(1.46 \times 10^{-2}\) across configurations.
  • Angular stability in the \((v_1, v_2)\) plane: \(s_\text{max}\) ranges from \(1.80 \times 10^{-11}\) to \(7.03 \times 10^{-11}\) (4× variation), with mean \(3.39 \times 10^{-11}\).
  • Universal instability: across all 4096 singular vectors, mean \(s_\text{max} = 8.96 \times 10^{-11}\), SD \(= 4.12 \times 10^{-11}\) — essentially flat despite singular values spanning \(>10^5\) range.
  • Three-regime structure persists under BFloat16, Float32, and Float64; precision shifts transition scale but does not remove scale dependence.
  • Baselines: Llama-3.1-8B-Instruct on dual NVIDIA RTX A5000; GPT-OSS-20B on Intel i9-10900X CPU (Float32 enforced).

Limitations

  • Decision boundary chaos analysis is constructed at artificially near-tie logit points (\(L_1 \approx L_2\)), which may not represent typical inference.
  • Mitigation (noise averaging) is evaluated only for condition number estimation, not for downstream task accuracy or token prediction consistency.
  • GPT-OSS-20B experiments run on CPU due to VRAM constraints (10 prompts per dataset vs. 100 for Llama), limiting statistical power for the larger model.
  • No formal Lyapunov exponent computation; chaos claim is supported empirically but not via dynamical systems theory.
  • The proposed mitigation increases inference cost proportionally to \(n\); cost-accuracy tradeoffs for production agentic systems are not analyzed.

Relevance to Harnesses / Meta-Harnesses

For builders of multi-agent harnesses and meta-harnesses, this paper identifies a root cause of the non-determinism that makes orchestrated LLM pipelines brittle: floating-point rounding errors produce chaotic regime behavior that persists even with fixed seeds and identical hardware. The three-regime characterization directly informs harness design decisions — specifically, which inference conditions (perturbation scale, embedding exchange) push the system into the chaotic vs. stable regime. The noise-averaging mitigation is harness-implementable (multiple forward passes with injected noise, aggregated before token selection), offering a drop-in reliability improvement without model changes. The documented 23% AutoGen and 31% MetaGPT failure rates attributed to numerical instability motivate adding precision-stability checks or averaging wrappers as standard harness components.