MASTE: A Multi-Agent Pipeline for Zero-Shot Aspect Sentiment Triplet Extraction

🕒 Published (v1): 2026-07-09 03:20 UTC · Source: Arxiv · link

Why this paper was selected

Zero-shot triplet extraction via multi-agent pipeline; shows LLM agent decomposition for structured NLP

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

MASTE is a training-free, four-stage multi-agent pipeline for zero-shot Aspect Sentiment Triplet Extraction (ASTE) that decomposes the task into specialized sequential agents (aspect extraction → opinion extraction → sentiment reasoning → consistency verification). It achieves state-of-the-art F1 among LLM-based methods on four ASTE benchmarks, improving GPT-4o zero-shot by ~36 F1 points across all datasets. The framework generalizes across five backbone LLMs without requiring any labeled data or in-context demonstrations.

Problem

Single-pass LLM generation fails at ASTE's exact-match evaluation due to three systematic failure modes: span bloat (over-extended boundaries), triplet merging (collapsing multiple targets), and hallucination (spans absent from the input). Even GPT-4o zero-shot reaches only 35.4% F1 on 14RES — roughly 40 points below supervised SOTA. Existing remedies (few-shot ICL, chain-of-thought) offer marginal gains and require labeled demonstrations, which are unavailable in true zero-shot deployment.

Method

MASTE shares a single frozen LLM backbone \(M\) across four sequentially conditioned agents:

1. Aspect Extraction Agent \(f_\text{asp}(s; M) \to A\): extracts verbatim aspect spans; discards any span not found as a case-insensitive substring of \(s\) via deterministic post-hoc validation.
2. Opinion Extraction Agent \(f_\text{opn}(s, A; M) \to P\): conditioned on \(A\); applies (a) aspect-conditioned localization, (b) minimal-boundary calibration (strips degree-only modifiers, preserves polarity-bearing expressions), (c) one-to-many pairing so one aspect can link to multiple opinions, (d) verbatim span filtering.
3. Sentiment Reasoning Agent \(f_\text{sen}(s, P; M) \to T^{(3)}\): assigns \(p_i \in \{\text{POS}, \text{NEG}, \text{NEU}\}\) to each aspect–opinion pair using full-sentence context for negation/contrast.
4. Consistency Check Agent \(f_\text{con}(s, T^{(3)}; M) \to T\): triplet-set-level verifier performing span grounding, polarity calibration, duplicate consolidation, and output canonicalization.

Each downstream agent receives the original sentence plus all upstream structured outputs, propagating typed intermediates rather than raw text.

Key Contributions

• First training-free multi-agent pipeline for ASTE requiring no fine-tuning, no in-context demonstrations, and no domain-specific annotations.
• Span-aware opinion extraction with minimal-boundary calibration and one-to-many aspect–opinion mapping, directly targeting span-bloat and triplet-merging failure modes.
• Cross-triplet Consistency Check Agent that validates the entire predicted triplet set simultaneously (not per-triplet), enabling hallucination removal and polarity recalibration at the set level.
• Comprehensive evaluation across four ASTE-Data-V2 benchmarks and five backbone LLMs (GPT-4o, GPT-3.5-turbo, Claude Sonnet 4.6, Gemini-3-flash, DeepSeek-V3.2, Seed 2.0 Pro), demonstrating backbone-agnostic generality.

Results

vs. zero-shot GPT-4o (F1 gains): +36.94 on 14RES, +36.05 on 14LAP, +35.41 on 15RES, +36.60 on 16RES.

vs. few-shot GPT-4o (no demonstrations used by MASTE): +12.76, +13.15, +15.51, +12.77 on the four datasets.

vs. strongest pipeline supervised baseline (Li-unified-R+PD / MiniConGTS): +20.85 F1 on 14RES, +4.95 on 14LAP, +15.14 on 15RES, +19.19 on 16RES.

Ablation (average F1, GPT-4o): - Full MASTE: 67.28 - w/o Consistency Check: 62.47 (−4.81) - w/o Opinion Extraction: 59.20 (−8.08) - w/o Sentiment Reasoning: 59.09 (−8.19) - w/o Opinion + Consistency: 40.76 (−26.52)

Cross-backbone average F1 gains (MASTE vs. zero-shot, same backbone): - GPT-3.5-turbo: +9.85; Claude Sonnet 4.6: +5.10; Gemini-3-flash: +14.0; DeepSeek-V3.2: +16.4; Seed 2.0 Pro: +27.2.

Limitations

• MASTE misses predicate-like aspect spans (e.g., use as an aspect); Aspect Extraction Agent errors cascade irreversibly through all downstream agents.
• Exact-match evaluation penalizes semantically correct but surface-form-differing spans (e.g., normalizing misspelled baterry → battery counts as wrong).
• One-to-many aspect–opinion recovery fails for shared global opinion predicates that apply to multiple aspects simultaneously (e.g., pleased over a list of features).
• Polarity calibration does not handle annotation-convention ambiguity: absence expressions (e.g., no) are labeled NEU in ASTE-Data-V2 but MASTE infers NEG.
• Recall remains competitive but not best-in-class, especially on 14LAP, where supervised pipeline methods achieve higher recall via broader candidate generation.

Relevance to Agentic AI / LLM Agents

MASTE instantiates a core agentic design principle — decompose a monolithic task into specialized, sequentially-conditioned agents that pass typed structured outputs rather than free text — applied to a structured NLP extraction problem. It demonstrates empirically that a pipeline of role-specialized LLM agents with intermediate verification outperforms both chain-of-thought (single-pass internal decomposition) and debate-style multi-agent approaches for tasks requiring exact-span extraction. The Consistency Check Agent specifically illustrates the value of a dedicated verification/critic agent operating over a full structured output set, a pattern directly applicable to agentic pipelines that must ground outputs in source evidence. The backbone-agnostic results (+5 to +28 F1 across five LLMs) suggest this sequential-conditioning architecture is a robust substrate for zero-shot structured extraction in any agentic system that processes text with complex relational structure.