Skip to content

CLAP: Closed-Loop Training, Evaluation, and Release Control for Domain Agent Post-training

🕒 Published (v1): 2026-07-02 08:07 UTC · Source: Arxiv · link

Why this paper was selected

Closed-loop post-training + offline/online mismatch fix; directly buildable domain-agent recipe

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

CLAP is a governance-oriented closed-loop framework for domain-agent post-training that converts raw business data into structured training assets, applies multi-stage release gating, and validates adapters against the actual application chain before promotion. On five batches of anonymized manufacturing data, QLoRA-style LoRA-SFT yields modest average gains but per-batch regressions and extreme GRPO KL instability demonstrate that training completion or a single offline score is insufficient for safe adapter release.

Problem

Domain agents post-trained on real business data face four compounding risks: (1) noisy, inconsistently formatted source data that is not directly reusable as training assets; (2) uncertain and uneven per-batch SFT gains that average scores conceal; (3) offline-to-application mismatch where holdout metrics do not predict application-chain behavior; and (4) unsafe adapter release driven by training completion rather than risk-conditioned gating. No prior system integrates data validation, training diagnostics, and application-chain replay into a single release-control loop.

Method

CLAP defines the post-training asset set \(\mathcal{A} = \{D_\text{sft}, D_\text{grpo}, D_\text{eval}, C, G, R\}\) and makes adapter promotion conditional on: $\(\text{promote}(M') \iff \text{quality}(M') \geq \text{quality}(M) \;\wedge\; \text{risk}(M') \leq \tau\)$

The pipeline follows the sequence: $\(\text{assetize} \to \text{materialize} \to \text{admit} \to \text{train} \to \text{evaluate} \to \text{gate} \to \text{app validate} \to \text{release/fallback}\)$

Key mechanisms include: - Target/evidence normalization: strips value-bearing numeric prefixes from metric labels, enforces evidence verifiability, and eliminates train/eval ID/input overlap. - Two-stage gating: a pre-training gate checks reward variance and train/eval overlap; a post-training gate compares quality, risk, latency, and regressions against baseline. - Runtime risk diagnosis: monitors KL divergence, reward trajectory, and sequence length during GRPO training; triggers early-stop or hold on anomalies. - Application-chain replay (ABCD experiment): runs 100 cases under four conditions (base±RAG × adapter±) on the same 3B backbone to separate RAG necessity from adapter contribution.

SFT uses QLoRA-style LoRA (rank 16, alpha 32, 4-bit base, lr 2e-5, 1 epoch); GRPO is used only as a failure-boundary probe, not a production recipe.

Key Contributions

  • Formal definition of the closed-loop post-training asset set and a release-gate predicate combining quality and risk bounds.
  • Target/evidence normalization procedure for noisy structured extraction targets in real business data.
  • Empirical demonstration that static GRPO admission (rank-pass rate 1.0, reward margin ≈0.225) is insufficient: all five batches pass static checks yet exhibit extreme KL peaks (up to 6712.0) at runtime.
  • Controlled ABCD application-chain replay showing RAG is necessary for factual value recovery and that an application-oriented LoRA-SFT adapter improves core-field and answer-evidence matching at a latency cost.
  • Release-decision taxonomy: promote / canary / hold / reject / rollback, each backed by a traceable evidence bundle (dataset version, config, adapter artifact, evaluation record, regressed samples).

Results

SFT (5-batch average, base vs. QLoRA-LoRA-SFT): - Overall score: 0.8371 → 0.8469 (+0.0098) - Pass rate: 0.5620 → 0.5860 (+0.0240) - Evidence accuracy: 0.6120 → 0.6400 (+0.0280) - Hallucination rate: 0.0260 → 0.0180 (−0.0080) - Wrong-fact rate: 0.0800 → 0.0740 (−0.0060) - P95 latency: 3082ms → 2785ms (−297ms) - Per-batch overall deltas: −0.0101, −0.0006, +0.0076, +0.0183, +0.0336 (only 3/5 batches improve)

GRPO risk (all 5 batches held): - KL peaks range from 168.2 to 6712.0; all classified as unstable and gated to hold despite rank-pass = 1.0

Application-chain replay (3B backbone, 100 cases): - A (base+RAG): value=0.83, core=0.71, AnsEv=0.25, P95=3799ms - B (App-LoRA+RAG): value=0.85, core=0.78, AnsEv=0.45, P95=4582ms (+783ms latency cost) - C (base, no RAG): value=0.00, core=0.00 - D (App-LoRA, no RAG): value=0.00, core=0.00

Limitations

  • Data volume is modest (359–360 training samples per batch) and confined to a single anonymized manufacturing scenario; generalizability is undemonstrated.
  • Application-chain replay uses only 100 pre-release cases, not live production traffic.
  • No comparison against full-parameter fine-tuning, RAFT, or Self-RAG; algorithmic ranking is explicitly out of scope.
  • GRPO is studied only as a failure-boundary probe; a stable positive GRPO recipe is not established.
  • KL thresholds and gate parameters are scenario-specific and not systematically calibrated.
  • The database/versioning layer is described architecturally but not experimentally evaluated.

Relevance to Agentic AI / LLM Agents

CLAP directly addresses a critical gap in deploying domain-specialized LLM agents in production: the absence of a principled, end-to-end governance loop between data, training, and release. For researchers tracking agentic AI, the paper's core finding—that per-batch regressions are hidden by averages, that GRPO KL instability is invisible to static reward checks, and that offline gains do not transfer to application chains without RAG—provides concrete empirical grounding for why agent post-training requires system-level controls rather than algorithmic optimization alone. The ABCD application-chain replay methodology is a reusable evaluation pattern for any RAG-augmented agent deployment. The work connects to the broader line of production agent reliability, complementing benchmarking papers (AgentBench, ToolLLM) by focusing on the pre-release governance and rollback infrastructure that operational agents require.