Skip to content

Bridging Behavior and Implementation: Automated Java Glue Code Generation for Behavior-Driven Development

🕒 Published (v1): 2026-07-22 03:06 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

AutoGlue is a hierarchical multi-agent framework that automatically generates Java BDD glue code from natural-language Cucumber/Gherkin steps. It decomposes the task into behavior interpretation, context retrieval, and code generation, achieving 58.7% API F1 improvement over few-shot prompting and producing directly usable code for 46.1% of evaluated steps.

Problem

Behavior-Driven Development (BDD) glue code — the step-definition layer that binds natural-language Gherkin steps to Java method calls annotated with @Given/@When/@Then — is entirely hand-written. Writing it requires simultaneous understanding of (1) underspecified natural-language intent, (2) scenario/feature context, and (3) project-specific APIs scattered across large codebases. No prior automated approach generates this code end-to-end; existing tools either map steps to already-existing implementations or require manual modeling, and execution-based evaluation oracles are unavailable because BDD steps lack isolated pass/fail outcomes.

Method

AutoGlue is a three-stage hierarchical multi-agent pipeline:

  1. Behavior Interpreter agent: given the target step plus its enclosing feature/scenario, it generates a structured description of the step's intent (disambiguating e.g. "submit form" across roles, states, and subsystems).
  2. Developer agent (supervisor): coordinates two specialized sub-agents in parallel:
  3. BDD Context Retriever: retrieves similar existing steps and their glue code from the same project (RAG over BDD artifacts).
  4. Project Code Explorer: traverses the repository to locate relevant classes, methods, and domain objects.
  5. Glue Code Generator: synthesizes the behavior description and both retrieved contexts into a Java step-definition method with correct @Step annotation regex, parameter types, and project API calls.

The hierarchy is motivated by the need to avoid mixed-artifact reasoning: each agent handles one information type to reduce context noise and error propagation.

Key Contributions

  • First automated approach to generate BDD glue code from natural-language steps (not just map to existing implementations).
  • AutoGlue: a hierarchical multi-agent framework separating behavior interpretation, BDD-artifact retrieval, and project-code retrieval before generation.
  • Benchmark of 1,307 step–glue-code pairs from 8 open-source Java projects with ground-truth glue code.
  • LLM-as-a-Judge evaluation protocol for BDD glue code usability, bypassing the unavailability of execution-based oracles.
  • Ablation study demonstrating independent contributions of behavior interpretation, BDD context retrieval, and project code retrieval.

Results

  • API F1: +58.7% over few-shot prompting; +50%+ average across baselines.
  • CodeBLEU: +43.7% over few-shot prompting; +40%+ average across baselines.
  • Direct usability (LLM-as-a-Judge): 46.1% of generated steps require no modification.
  • Most partially-correct cases need only minor edits (missing actions or imprecise parameters), not rewrites.
  • Ablation: removing BDD context retrieval most strongly degrades similarity metrics; removing project code retrieval most strongly degrades usability; removing behavior interpretation causes consistent drops across all metrics.
  • Baselines: few-shot prompting and plain prompting (zero-shot); no prior automated glue-code tools exist for comparison.

Limitations

  • Java only; the annotation-binding mechanism (regex + typed parameters) is more rigorous than scripting-language BDD frameworks, so transferability to Python/Ruby is unvalidated.
  • No execution-based evaluation; LLM-as-a-Judge is used as a proxy, introducing its own reliability concerns.
  • 46.1% directly usable means the majority still require human editing.
  • Context retrieval depends on the quality and coverage of existing project glue code; new projects with sparse BDD artifacts will have weaker retrieval signal.
  • Costs (latency, API calls) of the multi-agent pipeline are discussed but not benchmarked against industrial-scale BDD suites (hundreds of thousands of steps).
  • Paper is truncated; detailed efficiency and failure-mode analyses may be incomplete in the provided text.

Relevance to Harnesses / Meta-Harnesses

BDD glue code is itself a test harness layer — the executable bridge between specification and system-under-test — making AutoGlue a meta-harness generator: a system whose output is harness code. The AutoGlue pipeline architecture (Behavior Interpreter → Developer supervisor → dual retrieval sub-agents → generator) is a concrete instance of a hierarchical agent harness, where a supervisor orchestrates specialized sub-agents and integrates their outputs before a terminal generation step. The paper's decomposition principle — separating reasoning over different artifact types into dedicated agents to reduce noise and error propagation — is directly applicable to designing modular meta-harnesses for other specification-to-code automation tasks. The evaluation methodology (LLM-as-a-Judge to replace unavailable execution oracles) also addresses a recurring challenge in harness quality assessment when ground-truth execution is infeasible.