Eliminating Agentic Workflow for Introduction Generation with Parametric Stage Tokens

🕒 Published (v1): 2025-12-28 12:51 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

STIG (Stage Token for Introduction Generation) eliminates external agentic workflows for academic introduction writing by internalizing multi-stage generation logic as parametric special tokens, enabling single-inference structured output. Fine-tuned on 2,600+ ACL papers with eight sequential stage-token pairs, STIG outperforms agentic workflow baselines by up to 17.4% on structural rationality while consuming 3.3× fewer tokens than AutoSurvey.

Problem

Existing LLM-based agentic workflows for academic writing decompose tasks into discrete agent-handled stages (outline → subsection drafting → integration → refinement), causing cascading errors from sequential multi-turn interactions, high token overhead from opaque API calls, and structural drift from post-hoc merging of independently generated modules. Introduction writing has been largely neglected relative to survey/literature-review generation.

Method

STIG replaces the external workflow with eight special token pairs (<STAGE0>–<STAGE7>, each with a corresponding <ENDk>) embedded directly in the model's vocabulary and output sequence. These tokens correspond to outline and content phases for four introduction subsections (Background, Problem/Limitations, Method+Results, Contributions). The model is fine-tuned via supervised instruction tuning (SFT) on 2,600+ ACL 2021–2024 papers annotated by GPT-4o, training it to predict all eight stages autoregressively in a single forward pass under the joint factorization P(S0 ⊕ S1 ⊕ … ⊕ Sn) = ∏ P(St | S<t). At inference, stage delimiters are parsed to extract and concatenate the four content sections. Training uses LLaMA-Factory with ZeRO3 on 8×A800 GPUs; backbones are Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct.

Key Contributions

• Parametric stage tokens that internalize multi-stage workflow logic into LLM weights, eliminating runtime workflow orchestration
• An eight-stage SFT dataset derived from 3,800+ ACL main-conference papers with GPT-4o-annotated outline and content annotations per subsection
• Demonstration that end-to-end single-inference generation outperforms agentic and outline-based baselines on structural rationality (SR), content coverage (CC), and semantic similarity (SS)
• Efficiency analysis showing 3.3× token-use advantage over AutoSurvey and 2× over Stage Writing FT

Results

On 1,176 ACL 2025 test papers: - STIG (Qwen2.5-7B): SR=0.832 vs. AutoSurvey 0.658 (+17.4%), CC=0.442 vs. 0.333, SS=0.977 vs. 0.966 - STIG (Llama3.1-8B): SR=0.836, CC=0.472, SS=0.978; all outperform Pure Prompt, ELABORATE Prompt, Outline Writing baselines - GPT-4o with elaborate prompt achieves higher SR (0.903) and CC (0.496) but is a proprietary closed model - Human evaluation (50 papers, 7 methods): STIG avg rank 2.14, FT w/o Stage Writing 2.46, AutoSurvey 5.92 - Token efficiency: STIG consumes ~987 effective tokens vs. ~6,470 for AutoSurvey (effective token ratio 3.93% vs. 7.34%) - Eight-stage STIG outperforms four-stage variant on all metrics (SR 0.832 vs. 0.808) - Cross-domain generalization to 102 CVPR papers: STIG outperforms all open-model baselines

Limitations

• Training data is restricted to ACL NLP papers; generalization to other scientific domains (medicine, physics) is untested at scale
• Relies on high-quality GPT-4o annotations; annotation inconsistencies directly degrade model performance
• Closed-model GPT-4o with elaborate prompting still outperforms STIG on SR and CC despite STIG's efficiency advantage
• Stage granularity and subsection schema are fixed to introduction writing; adapting to other manuscript sections requires redesign
• Evaluation of factual accuracy (hallucination of experimental results) is not directly measured—only structural and semantic metrics

Relevance to Harnesses / Meta-Harnesses

This paper directly operationalizes the idea of eliminating the meta-harness layer entirely by collapsing what a multi-agent harness orchestrates (stage sequencing, role assignment, inter-agent handoff) into model parameters via special tokens—a form of workflow internalization rather than workflow automation. For harness builders, it establishes a concrete failure mode of externally-orchestrated pipelines (cascading errors, token overhead, structural drift) that parametric approaches can avoid. It also provides a reusable ablation methodology (FT w/o stage logic vs. stage logic w/o FT vs. combined) that directly quantifies what harness-style decomposition contributes versus what fine-tuning alone achieves—useful for evaluating when a harness is actually necessary.