Skip to content

TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development

🕒 Published (v1): 2026-07-21 09:14 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

TraceDev is a five-agent framework that converts structured use-case requirements into repository-level code by building and maintaining a heterogeneous traceability graph linking requirements, design models, and code artifacts. The traceability graph serves dual duty as a consistency checker and shared memory for all agents. On two real-world RE datasets it outperforms ChatDev and MetaGPT by up to 340.80% in execution success rate.

Problem

Existing multi-agent code-generation pipelines (ChatDev, MetaGPT) are evaluated on toy single-sentence prompts (e.g., "Create a Snake Game") and have no mechanism to verify that every requirement in a structured use case is actually implemented. This causes silent functional omissions and semantic deviations across the Requirement → Design → Code transformation chain.

Method

TraceDev orchestrates five role-specific LLM agents in a pipeline that mirrors the SDLC:

  1. Requirement Refiner — normalizes heterogeneous use-case terminology via a formal syntactic constraint (BNF grammar injected into the prompt) and runs iterative syntactic-parser feedback loops until the output is structurally valid.
  2. Designer — generates class and sequence diagrams from the refined use case, decoupling architecture from implementation.
  3. Developer — produces multi-file repository-level code guided by the design models.
  4. Tester — constructs and executes test cases; execution errors trigger a resolve-error loop back to the Developer.
  5. Validator — the central coordination agent. It builds and maintains a heterogeneous traceability graph whose nodes are individual requirements, design elements, and code artifacts, and whose edges encode coverage links. When a link is missing the Validator reports an error report back to the responsible agent for targeted refinement. The graph also serves as the shared structured memory context for all agents.

The entire loop is iterative: traceability gaps and test failures drive re-generation until coverage is verified or a budget is exhausted.

Key Contributions

  • First application of a traceability graph inside a code-generation multi-agent system, using it both for consistency enforcement and as structured shared memory.
  • TraceDev framework with five specialized agents that collectively automate the full SDLC from structured use-case input to verified repository-level code.
  • Empirical evaluation on the ETOUR and SMOS RE-domain datasets (125 use cases total), demonstrating large margins over state-of-the-art baselines across automatic, human, and code-statistical metrics.

Results

  • ETOUR dataset (success rate): TraceDev 53.63% vs. ChatDev (Gemini-2.5-Flash) and MetaGPT; margin up to +186.64%.
  • ETOUR dataset (semantic-coverage rate): TraceDev 71.72%; surpasses ChatDev by 51.66% and MetaGPT by 75.14%.
  • SMOS dataset (success rate): TraceDev 56.82%; surpasses baselines by up to 340.80%.
  • Baselines: ChatDev and MetaGPT, both run on the same LLM backbone (Gemini-2.5-Flash implied for ChatDev).

Limitations

  • Absolute success rates remain below 60%, meaning more than 40% of use cases still fail end-to-end; headroom for improvement is large.
  • Evaluated only on two Java-centric RE datasets (ETOUR, SMOS); generalization to other languages or requirement formats is unverified.
  • The paper text is truncated before the Designer, Developer, Tester, and Validator agent details are fully described, so implementation specifics of graph construction and the iterative repair protocol are not fully reported here.
  • Comparison baseline set is narrow (only ChatDev and MetaGPT); more recent agent frameworks are not included.
  • Computational overhead of iterative traceability-graph validation and formal syntactic checking is not quantified.

Relevance to Harnesses / Meta-Harnesses

TraceDev is a concrete instantiation of a pipeline meta-harness: it wraps five LLM agents in a structured orchestration layer that enforces inter-stage contracts (formal grammar checks, traceability link verification) and routes failures back to the appropriate upstream agent for repair — exactly the control-flow and error-routing patterns central to harness design. The Validator Agent acting as a cross-agent shared-memory manager is analogous to a harness coordinator that mediates state between pipeline stages. For researchers building harnesses for software-engineering LLM agents, TraceDev's traceability graph as structured inter-agent memory and its iterative formal-verification feedback loop are directly relevant design patterns.