AI-for-Science Low-code Platform with Bayesian Adversarial Multi-Agent Framework¶

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Bayesian adversarial multi-agent for reliable AI-for-science automated code generation

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

This paper introduces a Bayesian Adversarial Multi-Agent Framework (LCP) for AI-for-Science tasks, where three specialized agents—Task Manager (Challenger), Solution Generator (Solver), and Evaluator—co-evolve generated code and test cases via a non-LLM Bayesian update rule. The framework addresses hallucination and error propagation in multi-agent LLM systems by treating both code quality and test case quality as jointly uncertain quantities. A 32B open-source model with the framework outperforms a 235B baseline on SciCode.

Problem¶

Multi-agent LLM systems for scientific code generation suffer from two compounding failure modes: (1) hallucinated code propagates through the pipeline unchecked, and (2) LLM-generated test cases are themselves unreliable—they inherit the same failure modes as the code they are meant to validate. Standard self-refinement and unit-test approaches cannot guarantee scientific correctness (physical law adherence, domain constraints), and domain scientists without CS backgrounds produce vague prompts that degrade output quality further.

Method¶

Three agents operate in a recursive adversarial loop:

Task Manager (TM / Challenger): Decomposes the task into sub-tasks, generates and refines test cases, and translates vague domain-expert prompts into structured task plans via interactive clarification.
Solution Generator (SG / Solver): Produces a batch of $N$ candidate code solutions per iteration given the current prompt.
Evaluator: Scores both generated codes and test cases using three non-LLM scalar signals.

The scoring mechanism: $$S_1(i)^{t+1} = (1-\alpha)\cdot S_1(i)^t + \alpha\left(\frac{\sum_{j':\text{pass}} S_2(j')}{\lvert\{C_{j'}\}\rvert} - \frac{\sum_{j^\dagger:\text{fail}} S_2(j^\dagger)}{\lvert\{C_{j^\dagger}\}\rvert}\right)$$ captures test case "true hardness." Code score $S_2(j)^t$ weights pass/fail against current test-case hardness scores $S_1$. Prompt score $S_3^t$ averages $S_1$ and $S_2$ across all cases and codes.

Prompt composition for the next iteration is governed by a Bayesian posterior: $$p(\text{Prompt}^{t+1}_{ij} | S_3^t) \propto p(S_3^t | \text{Prompt}^t_{ij})\, p(\text{Prompt}^t_{ij})$$ where the likelihood rewards prompt pairs (test case $i$, sample code $j$) that historically exceeded their expected score.

To avoid testing all $N$ candidates, a Bayesian Optimization surrogate estimates code performance from Abstract Syntax Tree (AST) structural embeddings, prioritizing expensive test execution on the most promising candidates.

Key Contributions¶

Bayesian adversarial co-evolution of code and test cases using three non-LLM scalar scores ($S_1$, $S_2$, $S_3$), replacing static LLM-based evaluation.
A Bayesian Optimization surrogate over AST embeddings that estimates code performance without executing every candidate, enabling tractable exploration of the solution space.
A Low-Code Platform (LCP) that iteratively refines domain-expert prompts into structured task plans and scientifically valid test cases, requiring no prompt engineering skill.
Model-agnostic design validated across models from 1.7B (Qwen3) to 235B, demonstrating consistent uplift at every scale.

Results¶

SciCode benchmark (Subproblem Resolve Rate): - Qwen3-8b: 13.2% → 24.7% (+87.1% relative, without knowledge) - Qwen3-32b: 18.4% → 33.0% (+79.3%) - Qwen3-14b with framework (30.6%) matches Qwen3-235B baseline (30.6%), a 16× size advantage - Claude-sonnet-4: 31.3% → 42.7% (+36.4%) - Deepseek-R1: 29.6% → 41.0% (+38.5%)

ScienceAgentBench (GPT-4o base): - Valid Execution Rate (VER, without knowledge): LCP 90.2% vs. Self-Debug 83.3%, OpenHands 78.4%, Direct 52.9% - Success Rate (SR, without knowledge): LCP 26.5% vs. Self-Debug 22.6%

Ablation: Adversarial Test Cases (ATC) diverge positively from the no-ATC condition starting at iteration 3; SciCode score climbs from 27.1 to 37.2 over 5 iterations monotonically.

Limitations¶

Maximum iteration count (default 3) is a manually chosen hyperparameter; the framework may terminate before convergence on very hard tasks.
Computational cost scales with $N$ (code batch size = 20) and test case pool size (minimum 20); large batches are expensive despite the Bayesian surrogate.
The Bayesian surrogate relies on AST structural similarity, which may poorly generalize to code solutions with structurally novel correct approaches.
Evaluation on ScienceAgentBench uses GPT-4o as the sole base model; generalizability of SOTA results to other models on that benchmark is unconfirmed.
The text is truncated before reporting robustness results for non-professional users (Section 3.4), so those findings are unavailable.

Relevance to Agentic AI / LLM Agents¶

This work directly addresses a core vulnerability in multi-agent LLM pipelines: error propagation from unreliable intermediate outputs. By decoupling evaluation reliability from LLM trustworthiness—using non-LLM Bayesian scoring to govern agent interactions—it offers a principled alternative to the dominant paradigm of LLM-as-judge. The adversarial Challenger/Solver dynamic with co-evolving test cases is a concrete instantiation of self-play-style training-free improvement, relevant to any agentic coding workflow. The LCP's prompt refinement component also speaks to the human-AI collaboration challenge in agentic systems, where bridging the gap between domain-expert intent and machine-executable specification is a persistent bottleneck.