Skip to content

DanceHA: A Multi-Agent Framework for Document-Level Aspect-Based Sentiment Analysis

🕒 Published (v1): 2026-03-17 14:11 UTC · Source: Arxiv · Venue: AAAI 2026 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

DanceHA is a training-free multi-agent framework for document-level Aspect-Category-Opinion-Sentiment-Intensity (ACOSI) tuple extraction from informal user-generated text. It combines a divide-and-conquer agent pipeline (Dance) with a human-AI annotation loop (HA) to produce Inf-ABSIA, a new 2,714-document dataset with 23,024 fine-grained ACOSI tuples. Knowledge distillation from the harness into small student models (Qwen-14B) achieves performance competitive with the full Dance ensemble on GPT-4o.

Problem

Document-level Aspect-Based Sentiment Intensity Analysis (ABSIA) — extracting \((a, c, o, s, i)\) tuples (aspect, category, opinion, sentiment polarity, intensity on a 0–5 Likert scale) — is underexplored because no large, fine-grained labeled dataset exists at the document level, and long-context informal text (emoji, lengthened words like "loooove") confounds standard sentence-level pipelines.

Method

Dance (Divide-and-Conquer Teamwork) operates in three stages: 1. A Divider agent \(F_D\) segments each document \(d\) into aspect-based thought groups \(\{(a_j, g_j)\}\) by splitting multi-aspect sentences and combining consecutive sentences about the same aspect. 2. Three specialized agents process each \((a_j, g_j)\) in two independent streams: - Category Assignment Agent \(F_C\) uses a rule-based RAG tool to retrieve a domain-specific category list and assign \(c = F_C(a, g, \text{RAG}(dm))\). - Opinion Extraction Agent \(F_O\) extracts informal opinion strings \(\{o\} = F_O(a, g)\), preserving lengthened forms verbatim. - Sentiment & Intensity Agent \(F_S\) assigns polarity \(s\) and intensity \(i \in [0,5]\) given \((a, g, o)\). 3. A rule-based Merge step assembles final ACOSI tuples \(T = \{(g, a, c, o, s, i)\}\).

HA (Human-AI Collaboration for Annotation) runs multiple Dance teams with different LLM backbones, feeds their outputs to a Manager Agent (MA) that resolves conflicts and produces consensus labels, then passes results to human annotators who retain, correct, discard, or add tuples. Of 45,304 candidate tuples reviewed, 13,716 were retained, 8,233 revised, 1,075 added, and 16,994 discarded, yielding 23,024 gold ACOSI tuples.

Knowledge Distillation constructs a reasoning chain dataset by merging all expert-agent sub-prompts into a super-prompt and fine-tuning student models (Qwen3-4B, Qwen3-14B) with LoRA (rank 32, lr 2e-4, 3 epochs, 4096-token context).

Key Contributions

  • DanceHA framework: training-free multi-agent ACOSI pipeline with divide-and-conquer decomposition and a manager meta-agent layer for consensus annotation.
  • Inf-ABSIA dataset: 2,714 document-level reviews across 3 domains (Restaurant, Hotel, Laptop), avg. 90 words/doc, avg. 8.48 ACOSI tuples/doc — larger and more fine-grained than any prior document-level ABSA/ABSIA dataset.
  • Informal-style analysis: demonstrates that lengthening words (e.g., "amaaaazing") reliably correlate with higher sentiment intensity, a previously unstudied signal in ABSIA.
  • Distillation: reasoning fine-tuned Qwen-14B surpasses few-shot CoT GPT-4o on all three domains.

Results

  • Dance + GPT-4o vs. 5-shot CoT GPT-4o (F1): 47.80% vs. 35.04% (Rest); 44.18% vs. 31.40% (Hotel); 29.97% vs. 16.61% (Lap).
  • Manager Agent (MA) + DeepSeek-V3 reaches best overall F1: 63.18% (Rest), 59.07% (Hotel), 47.40% (Lap), with MAE ≤ 0.105 across all domains.
  • Expert agent accuracies (Dance + GPT-4o, Restaurant): Divider aspect accuracy 54.53%, Category 54.11%, Opinion 36.01%, Sentiment polarity 68.39%, SIS 73.43%.
  • Distilled Qwen-14B (reasoning fine-tuned) outperforms few-shot CoT GPT-4o across all domains; performance is comparable to Dance with Qwen2.5-72B.
  • Sentiment polarity sub-task accuracy reaches ~95% across all models and domains, making intensity (SIS) the main bottleneck.

Limitations

  • Evaluation is confined to Inf-ABSIA; no cross-dataset generalization experiments are reported.
  • The Manager Agent requires two strong backbone models (GPT-4o, DeepSeek-V3), making annotation expensive.
  • Human revision introduces annotator subjectivity; inter-annotator agreement statistics are not reported.
  • Context length is capped at 4,096 tokens for all LLMs, potentially truncating longer reviews.
  • The category list is domain-specific and predefined; open-domain or zero-shot category discovery is not addressed.
  • Divider-agent thought-group segmentation is approximated away in evaluation (Equation 1), so grouping quality is not directly penalized in the F1 metric.

Relevance to Harnesses / Meta-Harnesses

DanceHA is a clear instantiation of a multi-agent harness pattern: a Divider agent decomposes long-context input, specialized sub-agents run in parallel pipelines, and a Manager meta-agent integrates and arbitrates across multiple Dance team outputs — exactly the orchestrator-worker-aggregator topology central to harness design. The HA pipeline further layers a human-correction step above the automated meta-agent, demonstrating a hybrid harness that degrades gracefully from full automation to human-in-the-loop. The knowledge distillation result — compressing the harness's collective reasoning into a student model via structured reasoning chains — is directly relevant to the question of how to amortize harness costs at inference time, a key engineering concern for meta-harness practitioners.