On the risk of coding before testing: An empirical study on LLM-based test generation workflow

🕒 Published (v1): 2026-07-06 14:23 UTC · Source: Arxiv · link

Why this paper was selected

Papadakis group; empirical study of test-first vs code-first ordering in agentic SE workflows

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LLM-generated code biases subsequent test generation by the same model, causing faults in the implementation to be silently replicated in test assertions — a phenomenon the authors call error propagation. Empirical evaluation across five models, three benchmarks, and four prompting strategies shows that generating tests after faulty code yields only 14% fault detection, versus 25% when tests are generated from the specification alone. This structural coupling undermines the reliability of agentic code-and-test co-generation workflows.

Problem

Current LLM-based software engineering workflows — including "vibecoding" and agentic pipelines — routinely use a single model to generate both an implementation and its test suite. These workflows implicitly assume that generated tests act as independent oracles. Due to the autoregressive nature of LLMs, faulty intermediate outputs become part of the prompt context for downstream generation steps, violating this independence assumption. No prior work systematically characterizes how error propagation cascades across multi-step workflow stages.

Method

The study constructs a dataset of faulty implementations by prompting five LLMs on three benchmarks (HumanEval+, MBPP, BigCodeBench) with temperature 0.8 (ten samples per task), filtering to retain only implementations that fail ≤50% of the reference test suite (non-trivial faults), and selecting the hardest-to-detect fault per task.

Four test generation strategies are compared in two configurations — with and without the faulty implementation in the prompt: 1. LLM-Plain — iterative compilation-error repair loop (baseline) 2. Test via Summarization — code summarized to NL before test generation (ChatAssert-style) 3. Chain-of-Thought (CoT) — explicit pre-generation reasoning about expected behavior 4. Chain-of-Verification (CoVe) — post-generation self-verification pass

A fifth condition, the Agentic workflow, preserves the full conversation history from code generation through test generation within a single session, mirroring real-world multi-turn agent interactions.

Fault detection is measured by requiring a generated test to pass on the reference implementation and fail on the faulty one. Fault triggering is a weaker precondition (behavioral divergence detected regardless of oracle correctness). Statistical significance is assessed via Mann-Whitney U test (\(p < 0.05\)).

Key Contributions

• Introduces and operationalizes the concept of error propagation in LLM code-and-test co-generation, distinguishing it from mere prompt quality issues.
• Demonstrates that exposing faulty code during test generation significantly reduces fault detection (14% vs. 25%), a statistically significant and consistent result across five model families.
• Shows that CoT and CoVe prompting strategies provide no meaningful mitigation against error propagation when faulty code is in context.
• Characterizes the propagation pathway in multi-step agentic workflows, where intermediate faulty outputs are reused as context for downstream steps.
• Identifies a threat to validity in LLM evaluation studies that use coupled code-and-test generation pipelines, where performance measurements may be systematically distorted by within-session context dependencies.

Results

• Tests generated without the faulty implementation (specification-only): ~25% fault detection rate.
• Tests generated after the faulty implementation (code-aware workflows): ~14% fault detection rate.
• Effect is statistically significant across benchmarks and model families (Mann-Whitney U, \(p < 0.05\)).
• CoT and CoVe prompting strategies do not recover fault detection effectiveness when faulty code is included in the prompt.
• Agentic workflow (full conversation history preserved) exhibits propagation consistent with the code-exposed configurations.
• Results generalize across GPT-5-mini, GPT-4.1-mini, DeepSeek-V4-Flash, Claude Haiku 4.5, and Llama 3.3 Instruct (70B).

Limitations

• Evaluation is restricted to Python programming tasks from benchmark datasets; generalization to other languages or real-world codebases is not demonstrated.
• The study uses a single faulty implementation per task (hardest-to-detect), which may not represent the full distribution of fault types encountered in practice.
• The agentic workflow simulation controls the initial code generation output, which may not fully replicate the variance of unconstrained agent behavior.
• Only five models are evaluated; the degree to which error propagation scales with model capability is not characterized.
• The paper is truncated before the full results section; quantitative breakdowns per benchmark and per prompting strategy are partially unavailable from the provided text.

Relevance to Harnesses / Meta-Harnesses

This paper directly challenges a core design assumption of agentic coding harnesses: that a single LLM can serve as both implementer and test oracle within the same workflow session. For harness designers, the finding that multi-turn context contamination degrades validation reliability is a structural indictment of coupled generate-then-verify loops, as used in many meta-harnesses that chain LLM calls without context isolation. The work motivates harness architectures that enforce oracle independence — for example, by routing test generation to a separate model instance, a separate session with cleared context, or a specification-grounded verifier. It also surfaces a methodological threat: meta-harnesses that evaluate LLM coding performance using co-generated tests may systematically undercount failures.