Skip to content

Goal-Aware Identification and Rectification of Misinformation in Multi-Agent Systems

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

ICLR 2026; goal-aware detection and rectification of misinformation in MAS

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ARGUS is a training-free, two-stage defense framework that identifies and rectifies misinformation injections in LLM-based Multi-Agent Systems (MASs). It combines graph-topology-aware channel localization with goal-aware Chain-of-Thought persuasive rectification. The authors also introduce MisinfoTask, a 108-task red-teaming dataset for MAS misinformation evaluation.

Problem

MASs are uniquely vulnerable to misinformation—factually incorrect but semantically benign content—injected via agent prompts, RAG memory, or tool modules. Unlike overt malicious inputs, misinformation evades conventional safety filters and cascades through the MAS graph, causing task failure. Prior defenses focus on overtly malicious content and test on simplistic QA tasks insufficient to reflect real-world MAS complexity.

Method

ARGUS operates in two stages without any model fine-tuning:

Stage 1 — Adaptive Localization. At round \(r=1\), ARGUS ranks edges by normalized Edge Betweenness Centrality (Eq. 2) to identify structurally critical communication channels and deploys a corrective agent \(a_\text{cor}\) on the top-\(k\) edges (Eq. 4). For \(r>1\), it re-localizes dynamically by scoring each edge with a weighted combination of topological importance \(\text{Score}_\text{topo}(e)\), semantic relevance to the inferred misinformation goal \(\text{Score}_\text{rel}(e)\) (cosine similarity between message embeddings and inferred goal embeddings, Eq. 5–7), and channel usage frequency \(\text{Score}_\text{freq}(e)\) (Eq. 8–9).

Stage 2 — Goal-aware Persuasive Rectification. When a message traverses a monitored channel, \(a_\text{cor}\) applies a multi-stage CoT process: (i) sentence-level identification of suspicious factual claims, logical inconsistencies, and common-sense deviations; (ii) internal knowledge resonance—comparing intercepted content against the LLM's parametric knowledge; (iii) heuristic persuasive reconstruction generating corrective statements via root-cause analysis and cognitive reframing. Concurrently, \(a_\text{cor}\) infers the attacker's intent-driven goal \(g'_\text{mis}\) to feed the next round's re-localization.

Evaluation uses two metrics: Misinformation Toxicity (MT, LLM-judged semantic consistency with misinformation goal, \(\in [0,10]\)) and Task Success Rate (TSR, fraction of tasks where output meets the ground-truth threshold \(\theta_m\)).

Key Contributions

  • MisinfoTask dataset: 108 complex, multi-topic real-world tasks for MAS red-teaming, each with 4–8 plausible-but-fallacious misinformation arguments and ground truths, covering five categories (Conceptual Reasoning, Factual Verification, Procedural Application, Formal Language Interpretation, Logic Analysis).
  • ARGUS framework: Training-free, modular defense combining spatial (graph topology) and temporal (multi-round CoT) reasoning to localize and rectify misinformation.
  • Adaptive re-localization: Dynamic edge monitoring that updates every round using inferred attacker goals, enabling response to persistent coordinated attacks.
  • Empirical vulnerability characterization: Demonstrates that undefended MASs suffer a 20.04% TSR drop and MT rising from 1.28 to 4.71 under three distinct injection vectors.

Results

  • Undefended MAS: TSR drops from 87.47% → 67.70% (−20.04%); MT rises from 1.28 to ~4.71 across all attack methods (Prompt Injection, RAG Poisoning, Tool Injection).
  • ARGUS vs. attack: ~28.17% average reduction in MT and ~10.33% improvement in TSR under attack, evaluated across GPT-4o-mini, GPT-4o, DeepSeek-V3, Gemini-2.0-flash.
  • ARGUS reduces MT by approximately 38.24% across various core LLMs (broader metric reported in the abstract).
  • Baselines: Self-Check and G-Safeguard (GNN-based structural defense); ARGUS outperforms both on both MT and TSR (exact per-model table truncated in the provided text).

Limitations

  • Single-agent compromise threat model only; multi-agent coordinated injection is not evaluated.
  • The corrective agent \(a_\text{cor}\) itself relies on LLM parametric knowledge, which may itself contain errors or be outdated.
  • MisinfoTask is limited to 108 tasks; generalization to larger or domain-shifted MAS workloads is unverified.
  • LLM-based scoring (GPT-4o as judge) introduces evaluation bias and may not reliably distinguish subtle misinformation.
  • The top-\(k\) edge budget for monitoring is a hyperparameter whose sensitivity is not fully characterized in the visible text.

Relevance to Agentic AI / LLM Agents

This work directly addresses a critical safety gap in multi-agent orchestration: how factually incorrect but benign-looking information propagates and compounds through an agent graph to cause systemic failure. The graph-theoretic framing (edge betweenness centrality, dynamic re-localization) offers a principled approach to monitoring inter-agent communication that generalizes beyond misinformation to other information-integrity threats. For researchers building agentic pipelines with tool use, RAG, or multi-agent delegation, the MisinfoTask benchmark and the MT/TSR evaluation protocol provide directly applicable red-teaming infrastructure. ARGUS's training-free design is practically significant—it can be layered onto existing MAS architectures without retraining the underlying agents.