Skip to content

AI-for-Science Low-code Platform with Bayesian Adversarial Multi-Agent Framework

🕒 Published (v1): 2026-01-01 · Source: ICLR · Venue: ICLR 2026 · link

Why this paper was selected

Bayesian adversarial multi-agent for AI-for-science low-code automation [ICLR 2026]

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

This paper introduces a Bayesian Adversarial Multi-Agent Framework (delivered as a Low-Code Platform) for scientific code generation, using three agents—Task Manager (Challenger), Solution Generator (Solver), and Evaluator—in a recursive adversarial loop. A non-LLM Bayesian updating rule governs prompt composition, co-evolving generated code and test cases to reduce error propagation. On SciCode, a 32B open-source model wrapped in this framework matches or exceeds the raw 235B baseline.

Problem

LLM-based multi-agent systems for AI-for-Science suffer from compounding hallucinations: flawed intermediate outputs propagate uncritically between agents, and LLM-generated test cases inherit the same reliability failures as the code they evaluate. Scientific correctness additionally requires domain constraints that standard unit tests cannot express, and domain scientists typically lack the prompt engineering skills to specify tasks precisely.

Method

The framework composes three agents with distinct roles. The Task Manager (TM) acts as Challenger: it decomposes tasks into sub-tasks, generates and refines test cases with a "True Hardness" score \(S_1\), and translates vague scientific prompts into structured plans via interactive clarification. The Solution Generator (SG) acts as Solver, producing \(N=20\) candidate codes per iteration. The Evaluator computes three non-LLM scores:

\[S_1(i)^{t+1} = (1-\alpha)\cdot S_1(i)^t + \alpha\left(\frac{\sum_{j'\,:\,\text{pass}} S_2(j')}{|\{C_{j'}\}|} - \frac{\sum_{j^\dagger\,:\,\text{fail}} S_2(j^\dagger)}{|\{C_{j^\dagger}\}|}\right)\]
\[S_2(j)^t = \frac{\sum_i \mathbf{1}[C_j \text{ passes } T_i]\cdot S_1(i)^{t-1}}{\sum_i S_1(i)^{t-1}}, \quad S_3^t = \frac{1}{N}\sum_j S_2(j)^t + \frac{1}{M}\sum_i S_1(i)^t\]

Prompt composition for iteration \(t+1\) follows a Bayesian update over (test case \(i\), sample code \(j\)) pairs:

\[p(\text{Prompt}^{t+1}_{ij}|S_3^t) \propto p(S_3^t|\text{Prompt}^t_{ij})\,p(\text{Prompt}^t_{ij})\]

To avoid executing all \(N\) candidates, a Bayesian Optimization surrogate uses AST-based structural embeddings to predict code scores, prioritizing expensive test execution for the most promising candidates. The sample code pool is dynamically updated: high-\(S_2\) codes are added as reference examples for subsequent iterations.

Key Contributions

  • Three-agent adversarial co-evolution loop where test cases and code mutually challenge each other, governed by a non-LLM Bayesian update rather than LLM-based judgment.
  • Bayesian Optimization surrogate over AST structural embeddings to estimate code performance without full execution, enabling efficient exploration of the solution space.
  • Low-Code Platform (LCP) that transforms vague natural-language scientific prompts into structured task plans with domain-specific sanity checks, accessible to non-CS researchers.
  • Model-agnostic design demonstrated across 1.7B–235B parameter LLMs (Qwen3 family, DeepSeek-v3/R1, Claude Sonnet 4, GPT-3.5/4o).

Results

  • SciCode (Subproblem Resolve Rate): Qwen3-8B: 13.2% → 24.7% (+87.1%); Qwen3-14B without knowledge reaches 30.6%, matching Qwen3-235B baseline; Claude Sonnet 4: 31.3% → 42.7% (+36.4%).
  • ScienceAgentBench (GPT-4o base): Valid Execution Rate (VER) 90.2% (w/o knowledge) and 87.3% (w/ knowledge), vs. Self-Debug 83.3%/71.6% and OpenHands CodeAct 78.4%/73.5%; Success Rate 26.5%/27.5%, SOTA or tied.
  • Iterative convergence: HumanEval and MBPP Pass@1 improve monotonically over 5 iterations, converging around iteration 4–5; SciCode score climbs from 27.1 to 37.2 over 5 iterations.
  • Adversarial test case ablation: With ATC diverges positively from without-ATC starting at iteration 3, confirming adversarial test refinement as a key driver.

Limitations

  • Maximum iteration count is a user-set hyperparameter (default 3); premature stopping may leave solutions unrefined.
  • Running \(N=20\) candidates per iteration plus Bayesian Optimization overhead may be computationally prohibitive for very expensive scientific simulations.
  • Earth Science case study is illustrative; generalization to other scientific domains with different constraint structures is not systematically evaluated.
  • The paper is truncated before the robustness-for-non-professional-users experimental section is completed, leaving quantitative accessibility claims unverified from the provided text.

Relevance to Harnesses / Meta-Harnesses

This work is a textbook meta-harness: rather than modifying any base model, it wraps arbitrary LLMs in a three-loop orchestration layer (plan approval, test-case evolution, code generation) whose control flow is governed by non-LLM probabilistic scoring. The Bayesian prompt distribution update is precisely the kind of mechanism a meta-harness uses to steer sub-agent behavior across iterations without trusting any single call. The recursive co-updating of prompts, test cases, and sample code pools demonstrates how a harness can accumulate and exploit intermediate execution feedback to systematically improve output quality—a pattern directly applicable to general-purpose agentic harness design. The model-agnostic wrapper also illustrates how meta-harnesses can decouple orchestration logic from model capability, enabling smaller models to match larger baselines.