BACE: LLM-based Code Generation through Bayesian Anchored Co-Evolution of Code and Test Populations¶

🕒 Published (v1): 2026-03-30 16:40 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

BACE reformulates LLM-based code generation as a Bayesian co-evolutionary process in which populations of code candidates and test cases reciprocally update each other's posterior correctness beliefs via a noisy-sensor model. Unlike prior multi-agent systems that either treat generated tests as ground truth (fragile) or abandon test generation entirely (losing signal), BACE retains generated tests as uncertain evidence and anchors belief propagation on the immutable public input/output examples from the problem specification. On LiveCodeBench v6 (post-March 2025), BACE achieves state-of-the-art results across 7B, 120B open-weight, and proprietary model scales.

Problem¶

Closed-loop code generation systems (e.g., AgentCoder) use generated tests as oracles, but those tests are themselves unreliable: incorrect code passes trivial or hallucinated tests ("false positives"), and valid code is degraded by faulty assertions ("false negatives"). The state-of-the-art response—abandoning test generation in favor of pure reasoning (MapCoder, CodeSIM)—discards a high-value signal. No prior framework provides a principled way to exploit noisy, unverified tests without being misled by them, and simple population-based methods still suffer from co-evolutionary drift when fitness is computed against an unreliable oracle.

Method¶

BACE maintains two evolving populations: $\mathcal{C} = \{c_1, \ldots, c_N\}$ (code candidates) and $\mathcal{T} = \{t_1, \ldots, t_M\}$ (test cases). Each individual carries a posterior belief $b(c_i) = P(X_i = 1)$ and $b(t_j) = P(Y_j = 1)$, where the binary latent variables denote ground-truth correctness/validity.

Noisy sensor model. Execution outcomes $d_{ij} \in \{0,1\}$ are treated as observations from a noisy sensor parameterized by three hyperparameters: false pass ($\alpha$), accidental pass ($\beta$), and coincidental pass ($\gamma$). Correct code passing a valid test is assumed deterministic ($P=1$); all other pass configurations are probabilistic.

Belief updates. Updates are performed in log-odds space for numerical stability and additive evidence aggregation with learning rate $\eta$: $$\text{logit}(b(c_i)) = \text{logit}(b^{\text{prior}}(c_i)) + \eta \sum_j \Delta^{\text{code}}_{ij}$$ where the Weight of Evidence $\Delta^{\text{code}}_{ij}$ is a log-likelihood ratio conditioned on $b(t_j)$ and the noise parameters (Eq. 3). A credibility threshold emerges: a "pass" from a low-belief test decreases belief in the interacting code, making the system self-correcting against junk tests.

Anchoring. The 1–3 public I/O examples from the problem specification form an immutable anchor set $\mathcal{T}_{\text{anc}}$ with fixed prior $b \approx 1$. Code failing any anchor receives an infinite log-odds penalty; code passing anchors gains immediate credibility. This prevents the system from drifting into a self-validating loop.

Alternating evolution. Even generations freeze $\mathcal{C}$ and evolve $\mathcal{T}$; odd generations freeze $\mathcal{T}$ and evolve $\mathcal{C}$. This stabilizes learning signals and avoids "red queen" cycling.

Evolutionary operators (LLM-driven, not syntactic mutation): - Code: Semantic Crossover (merge two parents), Debug (repair against high-belief failing tests), Re-implement (rewrite with different algorithm). - Unit tests: Discriminate (generate/refine tests that separate two divergent code candidates), Complementary Crossover (merge non-redundant tests), Edge Case Generation, Coverage Augmentation. - Differential tests: generated from diverging candidate pairs to expose latent bugs.

Diversity retention. Code elitism combines top-$k$ by belief with the best representative of each functional equivalence class (identical pass/fail row in the observation matrix $D$). Test elitism clusters functionally redundant tests (identical columns in $D$) and retains only the highest-belief representative per cluster, keeping tests as an orthogonal sensor set.

Key Contributions¶

Bayesian co-evolutionary framework: first formulation of code synthesis as reciprocal belief propagation between code and test populations under a noisy sensor model.
Credibility threshold property: mathematical derivation showing that low-belief code candidates invert the belief update direction, passively filtering unreliable tests without any explicit rejection step.
Belief anchoring mechanism: immutable public examples ground the search and prevent co-evolutionary drift, enabling safe exploitation of noisy generated tests.
Behavioral diversity retention: dual-objective elitism (exploitation + functional equivalence diversity for code; orthogonal sensor compression for tests) prevents premature convergence.
LLM-informed variation operators: semantically meaningful crossover, debugging, and discriminative test operators replace stochastic syntactic mutation.
State-of-the-art on LiveCodeBench v6 (post-March 2025, contamination-free benchmark).

Results¶

GPT-5-Mini: BACE outperforms CodeSIM by +2.5% absolute on LiveCodeBench v6.
Qwen2.5-Coder-7b: BACE outperforms CodeSIM by +5.4% absolute on LiveCodeBench v6.
GPT-OSS-120b: BACE outperforms CodeSIM by +5.0% absolute on LiveCodeBench v6.
AgentCoder (the prior closed-loop baseline) underperforms Direct prompting with GPT-OSS-120b, confirming the fragility of deterministic oracle approaches.
All results are averaged over three runs; baselines include Direct prompting, AgentCoder, MapCoder, and CodeSIM.

Limitations¶

Noise hyperparameters ($\alpha$, $\beta$, $\gamma$) and learning rate $\eta$ must be set in advance; sensitivity analysis and auto-tuning methods are not discussed.
Anchoring requires that the problem specification provides at least 1–3 public I/O examples; problems with no examples cannot be anchored.
CoCoEvo (most closely related prior work) could not be benchmarked due to unavailable source code and unreported standard-benchmark results.
The text is truncated before the full experimental section, so ablation results and per-operator contribution breakdowns are not available in the provided excerpt.
Each generation requires cross-execution of all code × test pairs ($N \times M$ runs), which may be computationally expensive for large populations.

Relevance to Harnesses / Meta-Harnesses¶

BACE is a self-contained multi-agent harness that orchestrates LLM calls across distinct roles (code generator, test generator, debugger, crossover operator, discriminator) under a unified probabilistic controller — precisely the meta-harness pattern of wrapping heterogeneous agent actions inside a coherent outer loop with shared state. Its Bayesian belief layer is an exemplar of principled harness-level aggregation: rather than passing raw agent outputs downstream as ground truth, the harness models each output as a noisy observation and accumulates evidence across iterations, a design pattern directly applicable to any meta-harness that must synthesize unreliable sub-agent outputs. The alternating freeze/evolve schedule and diversity-preserving elitism are harness-level control-flow mechanisms that prevent the failure modes (convergence collapse, self-validating loops) that plague naive iterative multi-agent pipelines. Researchers building research-digest or code-synthesis meta-harnesses should note the anchoring idea — fixing a small, high-confidence reference set to prevent the outer loop from drifting — as a broadly transferable stabilization primitive.