Self-Healing Agentic Orchestrators for Reliable Tool-Augmented Large Language Model Systems¶

🕒 Published (v1): 2026-05-31 19:27 UTC · Source: Arxiv · link

Why this paper was selected

Self-healing orchestrators; failure taxonomy and recovery strategies for tool-augmented LLMs

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

This paper proposes a self-healing agentic orchestrator that treats LLM tool-use reliability as a bounded runtime control problem, implementing a monitor–detect–diagnose–recover–verify loop. Rather than applying uniform retry or replanning to every failure, the orchestrator maps observable failure signals to inferred failure classes and selects targeted, budgeted recovery actions. On a 100-task fault-injection benchmark it achieves 98.8% task success versus 94.5% for retry-only and 93.8% for full replanning, and reduces semantic silent failures to 0.0% with verifier-guided recovery.

Problem¶

Tool-augmented LLM agents fail not only due to model errors but due to orchestration-level faults: tool timeouts, malformed arguments, stale retrieval context, contradictory evidence, retry storms, and wrong-but-plausible ("silent") outputs. Existing recovery mechanisms—static workflows, retry, ReAct loops, full replanning—are failure-agnostic and create a recovery-mismatch problem: the selected action does not address the underlying cause. No prior work explicitly connects failure signals to failure classes, targeted recovery actions, bounded budgets, post-recovery verification, and traceable observability in a unified orchestration layer.

Method¶

The orchestrator separates a data plane (tool invocation, retrieval, model reasoning) from a reliability control plane with seven modular components: Planner, Execution Monitor, Failure Detector, Root-Cause Classifier, Recovery Policy Engine, Verifier, and Observability Layer.

Execution is formalized as a trajectory \(\tau = \{(S_t, a_t, o_t)\}\), where each state \(S_t\) carries task, plan, action history, tool outputs, verifier feedback, retry/recovery history, and budget state. When the detector identifies a failure signal \(x_t\), the classifier infers a failure class \(C_t\) (tool timeout, schema/argument failure, malformed output, incorrect tool selection, stale context, contradictory evidence, planning failure, verification failure, control-loop failure). The recovery policy then selects:

\[R_t = \pi(S_t, x_t, C_t, B_t)\]

from \(\mathcal{R} = \{\text{retry, repair, substitute, replan, refresh, escalate, degrade, terminate}\}\), subject to a multi-dimensional budget \(B_t = (b^r_t, b^p_t, b^s_t, b^m_t, b^l_t, b^k_t)\) covering retry count, replanning depth, substitution count, model escalation, latency, and cost.

The verifier is a first-class control-plane component: its rejection is fed back as a recoverable signal, enabling recovery from semantic silent failures that pass ordinary runtime checks. All decisions are recorded in observability traces used for evaluation, policy tuning, and regression testing.

Evaluation uses two settings: (1) a 100-task controlled fault-injection benchmark with deterministic tools, matched fault schedules, and five orchestration baselines; (2) a model-in-the-loop validation where a live tool-calling LLM performs tool selection and synthesis while tools remain local and fault-injected.

Key Contributions¶

Formal problem statement for self-healing orchestration: execution states, failure signals, failure classes, recovery actions, recovery budgets (\(B_t\)), and reliability objectives (TSR, RSR, SFR).
Modular orchestrator architecture separating detection, root-cause classification, targeted recovery, post-recovery verification, and observability into distinct, replaceable components.
Explicit failure-to-recovery mapping policy (Table 3) that selects the lowest-cost feasible recovery action per inferred failure class rather than applying uniform retry or replanning.
Multi-dimensional recovery budget formalism preventing unbounded retry storms and excessive replanning.
100-task controlled fault-injection benchmark with four baselines (static workflow, retry-only, ReAct-style, full replanning) across runtime, tool-output, context, and semantic fault conditions.
Recovery-budget sensitivity sweep demonstrating targeted recovery outperforms alternatives at every matched budget level, with largest gap under a single recovery attempt.
Semantic silent-failure analysis showing verifier-guided recovery eliminates wrong-but-plausible outputs.

Results¶

Main benchmark (100 tasks, full budget): Self-healing 98.8% TSR vs. retry-only 94.5%, full replanning 93.8%, ReAct-style and static workflow lower.
Budget-constrained (1 recovery attempt): Self-healing 94.0% vs. retry-only 85.3% vs. full replanning 88.2% — largest observed gap.
Recovery-budget sweep: Self-healing outperforms retry-only and full replanning at every tested budget level.
Semantic silent-failure setting: Verifier-guided self-healing reduces SFR to 0.0%; non-verifying baselines return wrong-but-plausible outputs at measurably higher rates.
Model-in-the-loop validation: Same orchestration control loop remains functional when a live tool-calling LLM drives tool selection, argument generation, and answer synthesis over local fault-injected tools.

Limitations¶

Benchmark is controlled (100 tasks, deterministic tools, synthetic fault injection) — not a production-API evaluation with real tool endpoints, real latency distributions, or stochastic model behavior at scale.
Fault models are predefined; the classifier and recovery mapping may not generalize to failure classes outside those enumerated in Table 3.
Cost model uses a simple call-count proxy (\(K = \alpha M + \beta U + \gamma V + \delta R\)) rather than real monetary or wall-clock latency costs.
Model-in-the-loop validation is described as "compact" and is not a full production benchmark — it provides a bridge but not production-strength evidence.
Recovery policy is rule-based and interpretable; more complex deployments may require learned or adaptive policies, which are not addressed.
No evaluation of orchestrator overhead on latency in real deployments.

Relevance to Harnesses / Meta-Harnesses¶

This paper is directly a meta-harness design paper: it defines an orchestration control plane that wraps arbitrary tool-augmented LLM agents and imposes structured failure detection, classification, recovery, verification, and observability on top of the base execution layer — exactly the concern of a harness that governs other agent components. The modular component interfaces (Table 2) and formal budget abstractions are directly applicable to harness architectures that must coordinate multiple sub-agents, tool-calling loops, and retrieval pipelines with bounded resource consumption. The observability layer — traces capturing failure signals, inferred classes, recovery actions, and budget state — is a concrete blueprint for the diagnostic instrumentation any production meta-harness needs. The recovery-budget sensitivity analysis is particularly valuable for harness designers choosing how aggressively to recover versus terminate gracefully under constrained token/call budgets.