Co-evolving Agent Architectures and Interpretable Reasoning for Automated Optimization¶

🕒 Published (v1): 2026-04-20 01:44 UTC · Source: Arxiv · link

Why this paper was selected

Co-evolves agent architectures and interpretable reasoning for automated optimization

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

EvoOR-Agent treats LLM-based operations-research (OR) agent workflows not as fixed pipelines but as evolvable architectures, co-evolving both a global AOE-style architecture graph and a population of reasoning trajectories instantiated on it. This enables task-adaptive coordination across problem interpretation, formulation, solver selection, code generation, and debugging without hand-crafted pipeline engineering. Evaluated on seven heterogeneous OR benchmarks, it consistently outperforms fixed-pipeline OR agents, fine-tuned OR models, and evolutionary agent baselines.

Problem¶

Existing LLM-based OR systems rely on predefined reasoning–execution workflows—either fine-tuned pipelines or handcrafted prompt scaffolds. These fixed architectures cannot adapt to the heterogeneous demands of real-world OR tasks (some require implicit-constraint extraction, others depend critically on solver routing, others need iterative modeling–execution revision). The internal workflow organization is implicit and cannot itself be optimized.

Method¶

EvoOR-Agent maintains a global activity-on-edge (AOE) architecture graph $G_t = \langle V_t, E_t, \pi, \rho, w_\text{fit}, w_\text{sparse} \rangle$, where nodes are reasoning states, edges are executable transitions typed as $\{work, reason, tool\}$, and edge weights encode empirical fitness and exploration sparsity.

Architecture-level evolution: Individual agents are abstracted into phase-wise AOE chains $C(a) = \langle C_\text{ana}(a), C_\text{mod}(a), C_\text{code}(a) \rangle$. Newly evaluated chains are merged into $G_t$ via phase-local state alignment (LLM-judged semantic equivalence within each OR phase). Edge fitness weights update as: $$w_\text{fit}(e) \leftarrow w_\text{fit}(e) + \alpha\!\left(\frac{1}{|A_t(e)|}\sum_{a \in A_t(e)} F_{t+1}(a) - w_\text{fit}(e)\right)$$ Sparsity weights $w_\text{sparse}(e) = \frac{1}{\log(2 + \text{count}_t(e)) + \epsilon}$ incentivize exploration of underused paths. Nodes and edges below pruning threshold $\tau$ for $\sigma$ consecutive iterations are removed.

Trajectory-level evolution: A population of reasoning individuals evolves via (i) hybrid KB-guided initialization using a domain knowledge base of retrieved OR papers and code, (ii) graph-mediated path-conditioned recombination that samples new trajectories from $G_t$ weighted by $\Phi(e)$, (iii) multi-granularity semantic mutation (individual- and phase-level), and (iv) elitist $(\mu + \lambda)$ multi-source selection. Both levels co-evolve: trajectories shape the graph; the graph constrains trajectory search.

Key Contributions¶

AOE architecture representation: converts implicit reasoning–execution organization into an explicit, evolvable, analyzable graph search space with typed edge partition and phase-ordering constraints.
Co-evolutionary framework: simultaneous evolution of the architecture graph (topology, edge weights, pruning) and a population of reasoning trajectories (recombination, mutation, elitist selection).
Knowledge-base-assisted operators: an LLM-agent workflow retrieves OR literature and code repositories to construct a structured KB used for initialization and knowledge-guided mutation, with leakage-control protocol.
Empirical validation on seven heterogeneous OR benchmarks with ablations confirming contributions of each component.

Results¶

Outperforms zero-shot LLMs, fixed-pipeline OR agents (OptiMUS, OR-LLM-Agent), specialized fine-tuned OR models (ORLM, LLMOPT, StepORLM), and representative evolutionary agent frameworks (EvoAgent, EvoPrompt-style baselines) across all seven benchmarks: IndustryOR, MAMO, NL4OPT, BWOR, NLP4LP, ReSocratic.
Ablation confirms: removing the AOE-style architecture representation degrades performance; removing knowledge-base-assisted operators also degrades performance.
Case studies show the evolved architecture graph captures interpretable structures: formulation decomposition, solver-routing decisions, and solver-status-based revision.
(Specific numerical scores not extractable from the provided truncated text.)

Limitations¶

Evolutionary search over an LLM-evaluated population is computationally expensive; each fitness evaluation requires LLM inference over full OR solution traces.
Architecture graph construction depends on LLM-judged semantic equivalence for state merging—noise or inconsistency in judgments could corrupt the graph topology.
Knowledge base construction relies on retrieval of external papers/code; quality and coverage of retrieved sources directly affect initialization and mutation quality.
The framework is evaluated only on OR domains; generalizability to other agentic task classes is untested.
Non-skipping phase-ordering constraint may limit discovery of workflows that genuinely benefit from out-of-order phase execution.

Relevance to Harnesses / Meta-Harnesses¶

EvoOR-Agent is a direct instance of a meta-harness: rather than hand-engineering a fixed agent pipeline, it automatically evolves the harness structure itself, treating workflow topology as a first-class optimization target. The AOE graph is precisely a structured, mutable harness representation—analogous to how meta-harnesses in software orchestration define routing, dependency, and branching logic—but with weights and topology learned from execution traces. The co-evolutionary loop (architecture $\leftrightarrow$ trajectory population) is a general pattern applicable beyond OR: any domain where the right orchestration of sub-agents/tools is unknown a priori could benefit from this approach. The knowledge-base-assisted mutation operator also demonstrates how retrieval-augmented experience can bootstrap harness initialization, a technique directly transferable to general agent-harness design.