Skip to content

Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoning

🕒 Published (v1): 2025-02-17 13:42 UTC · Source: Arxiv · Venue: ACL 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

Table-Critic is a multi-agent framework for table reasoning that decomposes the critique-and-refine loop into four specialized agents (Judge, Critic, Refiner, Curator) operating iteratively until convergence. A distinguishing component is a self-evolving template tree that accumulates and hierarchically organizes error-pattern knowledge across runs. On WikiTQ and TabFact, it outperforms all baselines including Chain-of-Table and Critic-CoT while degrading fewer correct solutions.

Problem

Decomposition-based table-reasoning methods (e.g., Binder, Dater, Chain-of-Table) lack correction mechanisms for intermediate reasoning errors, causing cascading error propagation. LLM self-reflection is unreliable for this domain: models either rationalize prior errors or over-critique correct steps. There is no systematic way to accumulate and reuse critique knowledge across reasoning instances.

Method

Table-Critic wraps any initial reasoning chain \(\tau = \{s_1, \ldots, s_n\}\) (here Chain-of-Table) in a four-agent loop:

  1. Judge (\(A_j\)): Given \((T, q, \tau, \mathcal{T})\), emits per-step error analysis \(E\), overall status \(P \in \{\text{Correct}, \text{Incorrect}\}\), and a routing path \(R\) through the template tree \(\mathcal{T}\).
  2. Critic (\(A_c\)): Receives sampled templates \(T_s\) located via \(R\), identifies the first error step \(I\), and generates critique \(C\).
  3. Refiner (\(A_r\)): Receives only the partial chain \(\tau_p = \{s_1,\ldots,s_I\}\) plus \(C\), and reconstructs the full refined chain \(\tau'\). Partial-chain truncation prevents the refiner from being anchored to prior erroneous steps.
  4. Curator (\(A_{cu}\)): Activated only after a successful full refinement (\(P = \text{Correct}\)); reviews the refinement history \(H\) and updates \(\mathcal{T}' = \pi(\mathcal{T}, H, \text{instruction}^{A_{cu}})\).

The self-evolving template tree \(\mathcal{T}\) is a hierarchy where internal nodes represent broad error categories and leaf nodes hold concrete critique templates. The Curator evolves it via (i) template enhancement (adding templates to existing leaves), (ii) vertical expansion (splitting a leaf into a parent + two children for finer-grained error types), and (iii) horizontal expansion (appending a sibling branch for entirely new error categories). The tree is initialized with only 2 templates. The loop runs for at most \(K=5\) iterations with greedy decoding (temperature 0).

Key Contributions

  • Multi-agent critique framework (Judge / Critic / Refiner / Curator) with clearly separated responsibilities to overcome LLM self-reflection unreliability.
  • Multi-turn refinement that processes errors sequentially, one per iteration, to prevent cascading correction failures.
  • Self-evolving template tree: a dynamically growing hierarchical knowledge base that routes error-type identification and stores distilled critique patterns across all past reasoning instances.
  • Demonstrated model-agnostic gains across Qwen2.5-72B, LLaMA3.3-70B, and GPT-4o-mini.

Results

  • WikiTQ (4,344 samples): Table-Critic averages 73.7% vs. Chain-of-Table 66.0% (+6.3%) and Critic-CoT 67.4% (+6.3%); best single-model result 77.2% (Qwen2.5-72B).
  • TabFact (2,024 samples): Table-Critic averages 91.7% vs. Chain-of-Table 89.5% (+2.2%) and Critic-CoT 88.2% (+3.5%).
  • Error correction rate \(\Delta_{i \to c}\) on WikiTQ (Qwen): +9.6% vs. +5.6% for Critic-CoT.
  • Solution degradation rate \(\Delta_{c \to i}\) on WikiTQ (Qwen): −0.7% vs. −4.9% for Critic-CoT; net gain +8.9% vs. +0.7%.
  • Convergence: most accuracy improvement is captured within 3 iterations; stabilizes by iteration 5–6.

Limitations

  • Evaluation uses only two benchmarks (WikiTQ, TabFact); generalization to other semi-structured or free-form reasoning tasks is untested.
  • The Curator updates only after a fully successful refinement; if the process never converges within \(K\) steps, no new templates are distilled from that instance.
  • Multiple agent calls per iteration substantially increase inference cost compared to single-pass methods, even if total API cost is reported as competitive with majority voting.
  • The template tree is initialized with only 2 hand-crafted templates and must bootstrap from scratch per experimental run; cross-run or cross-dataset transfer of learned templates is not evaluated.
  • Relies on Chain-of-Table for initial chains; the coupling to one upstream method is not fully ablated.

Relevance to Harnesses / Meta-Harnesses

Table-Critic is a concrete instance of a meta-harness pattern: a controlling loop that wraps an existing base solver (Chain-of-Table), drives multi-agent refinement to convergence, and accumulates reusable scaffolding (the template tree) that improves all future invocations within the same run. The self-evolving template tree is particularly relevant as a lightweight form of harness-level memory—structured, typed, and dynamically extended—contrasting with unstructured scratchpad or flat prompt history approaches. The Judge-routes-to-template-then-Critic pipeline mirrors harness dispatch patterns where a router selects specialized sub-agents based on runtime signal classification. For researchers building harnesses, the partial-chain truncation trick (feeding the Refiner only \(\tau_p\)) is a transferable design principle for preventing anchoring bias in any iterative correction loop.