Skip to content

LLM-based Few-Shot Early Rumor Detection with Imitation Agent

🕒 Published (v1): 2025-12-20 12:42 UTC · Source: Arxiv · Venue: KDD 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper proposes a few-shot Early Automatic Rumor Detection (EARD) framework that pairs a lightweight autonomous neural agent — trained via imitation learning — with a training-free LLM detector. The agent uses an MDP formulation with GAIL to learn when to trigger the LLM for rumor classification, reducing LLM inference to a single call per claim while achieving superior accuracy and earliness over full-shot baselines.

Problem

Existing EARD methods tightly couple temporal decision-making (when to stop observing posts) with rumor classification, require large annotated datasets, and rely on manually crafted reward functions (DQN/Hawkes Process) that fail under data scarcity. LLMs, while powerful few-shot text classifiers, are ill-suited for sequential time-series decisions and incur prohibitive inference cost when queried at every timestep.

Method

EARD is formulated as an MDP where the binary action is continue/stop-and-query-LLM. A lightweight agent (BERTweet encoder + two-hidden-layer policy/value/discriminator networks) is trained via Generative Adversarial Imitation Learning (GAIL) with three expert trajectory types derived from the LLM's own prediction sequence over the post stream:

  • Conservative Expert (CE): earliest timestep where LLM prediction is stable (no subsequent reversals).
  • Early-Action Expert (EAE): earliest timestep where LLM prediction is correct, ignoring future stability.
  • Misleading Expert (ME): timestep where the LLM locks onto an incorrect prediction, used as a repulsion target.

The objective minimizes JS-divergence from CE and EAE occupancy measures while maximizing divergence from ME, plus a causal entropy regularizer. The discriminator provides implicit reward signals; policy is updated via PPO-clip with GAE. Only 50 labeled + 100 unlabeled instances are used for training; the LLM remains frozen throughout.

Key Contributions

  • First few-shot EARD method; only the lightweight agent requires training (LLM is training-free).
  • Three-expert imitation learning design (CE, EAE, ME) that simultaneously learns earliness, stability, and avoidance of misleading stop decisions.
  • Theoretical proof that the objective yields an optimal policy under the GAIL occupancy-measure framework.
  • Demonstrated compatibility with multiple LLM backends (Mistral-7B, Llama-3-8B, ChatGPT/GPT-4o-mini) across four datasets.

Results

  • Accuracy (macro-F1): Framework boosts base LLMs by an average of 8.2% (Mistral), 15.6% (ChatGPT), and 18.9% (Llama3) across four datasets relative to standalone LLM baselines.
  • vs. full-shot EARD baselines: On TWITTER, Ours (w/ Mistral) surpasses all baselines in F1 while achieving ER=0.043 vs. ERD=1.000, CED=0.617, HEARD=0.647.
  • ChatGPT on PHEME: 0.646 F1 (ER=0.205) vs. best baseline HEARD 0.501 F1 (ER=0.266).
  • Cross-dataset generalization (trained on PHEME/TWITTER, tested on Twitter-COVID-19): Ours (w/ Mistral) degrades only 2.8%/2.4% F1, vs. ERD/CED/HEARD drops of 11–68%.
  • vs. Preset Time Checkpoints baseline (ChatGPT): On Twitter-COVID-19, agent reduces ChatGPT's ER by >50% at the 12-hour checkpoint while maintaining or improving F1.

Limitations

  • Agent training still requires 50 labeled examples; true zero-shot EARD is not addressed.
  • Expert trajectory quality is LLM-dependent: if the LLM detector is weak, CE/EAE trajectories may be noisy or sparse.
  • Only 7B/8B open-source LLMs tested due to compute cost; generalization to larger models is asserted but not empirically verified.
  • High variance in ER across runs (some standard deviations > 0.3) suggests instability in early stopping decisions.
  • Trajectory generation requires running the LLM over all timesteps of each training instance, which has an upfront inference cost.

Relevance to Agentic AI / LLM Agents

This work is a direct instantiation of the decoupled agent+LLM architecture pattern: a lightweight learned controller decides when to invoke the expensive LLM, limiting its calls to a single inference per episode. The imitation-learning approach to training the decision-making agent without a hand-crafted reward function is broadly applicable to any LLM-tool-use scenario where triggering cost is high and timing matters. The three-expert GAIL formulation (attract to good demonstrations, repel from bad) is a reusable design pattern for training agents that must balance competing temporal objectives. The cross-dataset generalization results reinforce that decoupled architectures can achieve better few-shot transfer than tightly coupled full-shot systems.