ProACT: Towards Breakdown-Aware Proactive Agent in Multi-User Collaboration

🕒 Published (v1): 2026-07-04 06:36 UTC · Source: Arxiv · link

Why this paper was selected

Proactive breakdown detection in multi-user agent collaboration; novel harness intervention trigger

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ProACT is a breakdown-aware agent framework that wraps an LLM backbone with a three-stage pipeline: diagnose collaboration breakdowns in multi-user conversation history, decide whether to speak or stay silent, then route the detected breakdown to a targeted collaboration skill. Evaluated on 3,244 turn-level examples across five LLM backbones, ProACT consistently improves appropriateness, non-interruptiveness, conciseness, and intervention quality over a direct-chat baseline.

Problem

Conversational agents embedded in multi-user workflows are fundamentally reactive—they respond only to explicit requests. In group settings, coordination failures (goal conflict, forgotten constraints, discussion loops, participation imbalance, risky commitments) accumulate incrementally and require timely, unprompted intervention; a passive agent misses these moments entirely. Prior work on proactive agents targets single-user intent prediction, which lacks the social/normative complexity of multi-participant disagreement and conflicting goals.

Method

ProACT implements a three-step decision pipeline over the speaker-attributed conversation history \(H_t = (x_i)_{i=1}^{t}\):

1. Breakdown-aware diagnosis: A structured call \((b_t, z_t) = \text{Diagnose}(H_t)\) labels the detected breakdown category \(b_t\) (from an 8-type taxonomy: conflict, uncertainty, underspecified plan, forgotten constraint, goal drift, discussion loop, participation imbalance, risky commitment) and binary intervention warrant \(z_t \in \{0,1\}\).

2. Routing: If \(z_t = 1\), the breakdown is routed to a skill \(k_t = r(b_t)\) from a library of lightweight operating procedures. Each skill specifies a trigger condition, diagnostic cues, social constraints (neutrality, human agency preservation), and output contract.

3. Execution: The selected skill governs the final action \(a_t = \pi_{k_t}(H_t) \in \{\varnothing, u_t\}\), where \(\varnothing\) is silence and \(u_t\) is a concise, group-facing utterance. The model exposes only \(u_t\)—no skill names, rationale, or meta-commentary appear in the output.

The benchmark (3,244 examples) is built from GitHub issue discussions and the QMSum meeting dataset (2,444 real + 800 synthetic from BEAM scenarios with injected controlled breakdowns), balanced 50/50 between intervention and silence reference labels.

Key Contributions

• ProACT framework: diagnosis → routing → skill-execution pipeline for proactive multi-user collaboration, grounded in common-ground and coordination-work theory.
• Collaboration breakdown taxonomy: 8 operationalized failure categories derived from the CSCW/organizational-behavior literature.
• Collaboration skill library: reusable, lightweight skill procedures (trigger, cues, constraints, output format) for each breakdown type.
• Proactive collaboration benchmark: 3,244 turn-level examples spanning 11 topic domains, with both real and synthetic conversations.
• Cross-backbone evaluation: systematic comparison of Direct Chat vs. ProACT on GPT-5.4, Kimi K2.5, Claude Sonnet 4.6, Gemini 3.1 Pro Preview, and GPT-OSS-120B.

Results

• ProACT improves all four metrics (Appropriate, Non-interruptive, Concise, Intervention Quality) for every tested backbone over Direct Chat.
• Kimi K2.5 (largest gain): Appropriate \(0.222 \to 0.870\) (+0.648), Non-int. \(0.323 \to 0.942\) (+0.619), Concise \(0.129 \to 0.964\) (+0.835), Quality \(1.789 \to 3.450\) (+1.661).
• GPT-5.4 (smallest gain): Appropriate +0.121, Non-int. +0.183, Concise +0.591, Quality +0.502.
• On silence-labeled turns (GPT-5.4): Direct Chat responds 100% of the time with 34.8% judged inappropriate; ProACT responds only 39.3% of the time and raises overall appropriateness from 65.2% to 89.6%.
• On intervention-labeled turns (GPT-5.4): Inappropriate silence stays low at 4.7%; ProACT intervention quality rises from 3.486 to 3.988 on a 1–5 scale.

Limitations

• LLM-as-judge circularity: all metrics are scored by GPT-5.4-Nano; no human inter-rater reliability is reported for the full evaluation set (only 694 examples received human review at annotation time).
• Synthetic data gap: 800 of 3,244 examples are synthetic (BEAM-derived); breakdown injection may not capture the distributional subtlety of organic coordination failures.
• No deployment evaluation: the framework is only tested on offline turn-level examples; no study of latency, user acceptance, or interruption perception in live collaboration.
• Skill library scope: the 8-skill taxonomy is manually designed; coverage of out-of-taxonomy breakdowns is unaddressed.
• Single-channel assumption: the environment assumes all participants and the agent share one communication channel; side-channels or asynchronous collaboration are out of scope.

Relevance to Harnesses / Meta-Harnesses

ProACT is architecturally a harness: it wraps an arbitrary LLM backbone with a structured control pipeline (diagnose → route → execute) that steers the model's behavior without modifying its weights, exactly the pattern meta-harnesses use to compose and orchestrate base models. The skill library functions as a set of named, inspectable sub-routines—analogous to tools or sub-agents in multi-agent harnesses—that the top-level controller selects based on runtime context. The structured invocation interface (breakdown label → skill ID → constrained output format) mirrors the tool-routing and output-contract patterns common in agentic scaffolds such as ReAct and Magentic-One. For researchers building harnesses that must decide when to invoke an agent or sub-routine (not just how), ProACT provides a concrete, empirically validated model of context-sensitive activation gating over a typed skill registry.