Skip to content

OptiAgent: End-to-End Optimization Modeling via Multi-Agent Iterative Refinement

🕒 Published (v1): 2026-07-06 17:27 UTC · Source: Arxiv · link

Why this paper was selected

End-to-end multi-agent iterative refinement pipeline; generalizable orchestration pattern

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

OptiAgent is a six-agent LangGraph pipeline that converts natural-language Operations Research problem descriptions into solver-ready mathematical formulations and executable Python code. It introduces four targeted feedback loops that route control back to upstream agents upon detecting specific failure modes (misinterpretation, structural defects, mathematical inconsistencies, code errors). It achieves state-of-the-art solving accuracy on 3 of 4 benchmarks (LP, MILP, NLP).

Problem

Translating natural-language optimization problems into correct, executable mathematical models requires OR expertise. Prior LLM-based approaches (OptiMUS, CoE, NEMO) either treat the pipeline as a black box or lack structured self-correction, producing code that may yield a numerically correct answer from a mathematically wrong formulation—and providing no transparency for expert auditing or debugging.

Method

OptiAgent composes six specialized agents in a sequential pipeline sharing a mutable AgentState object: Interpretation → Formulation → Analysis → Validation → Code Generation → Solver. Each agent targets one cognitive subtask of expert OR modeling. Mathematical structure is decomposed into a five-element canonical form (Sets, Parameters, Variables, Objective, Constraints).

Four distinct feedback loops handle specific failure modes: - Loop 1: Validation detects mathematical errors (e.g., non-linearity in an LP) → reroutes to Formulation. - Loop 2: Formulation detects fundamental misinterpretation → reroutes all the way back to Interpretation. - Loop 3: Analysis detects structural defects (undefined variables, missing constraints) → reroutes to Formulation. - Loop 4: Solver detects syntax/runtime errors → reroutes to Code Generation.

Each loop is capped at 3 iterations to prevent infinite refinement. The system selects among Pyomo, PuLP, or OR-Tools based on problem type inferred by the Analysis agent. The framework is implemented in LangGraph as a directed acyclic graph (DAG) of stateful nodes.

Key Contributions

  • A modular, model-agnostic multi-agent architecture for end-to-end OR modeling with full intermediate transparency (each agent exposes its reasoning and feedback artifacts).
  • Four targeted feedback loops, each keyed to a distinct failure mode category, enabling autonomous self-correction without human intervention.
  • Adoption of the five-element formulation (Sets, Parameters, Variables, Objective, Constraints) to structure LLM reasoning about mathematical models.
  • State-of-the-art solving accuracy (SA) on ComplexOR, IndustryOR, and LogiOR; highly competitive on NLP4LP.
  • Empirical evidence of model-agnosticism: framework tested with both Claude Sonnet 4.5 and GPT-5.4 backbones.

Results

Metric: Solving Accuracy (SA) — whether the generated model's objective value matches ground truth.

  • ComplexOR: OptiAgent (GPT-5.4) = 100% vs. best prior (ORThought) = 77.78%, best single-LLM baseline (GPT-5.4 w/o) = 83.33%.
  • IndustryOR: OptiAgent (GPT-5.4) = 84.34% vs. NEMO = 63.0%, ORThought = 57.83%, best single-LLM = 72.3%.
  • LogiOR: OptiAgent (GPT-5.4) = 70.65% vs. CoE = 68.48%, ORThought = 46.01%, best single-LLM (GPT-5.4 w/o) = 60.87%.
  • NLP4LP: OptiAgent (GPT-5.4) = 87.12% vs. ORThought = 89.02% (not SOTA here); LLMOPT = 83.8%.
  • Feedback loops triggered in 27.8%–61.4% of problems depending on dataset complexity.
  • Runtime: 16.2 min (ComplexOR, 18 problems) to 3.20 hours (NLP4LP, 263 problems).
  • LogiOR triggered 105 loops across 49 problems; ComplexOR triggered only 9 loops across 5 problems.

Limitations

  • Single-run evaluation only — no variance or confidence intervals reported, precluding statistical significance claims.
  • No ablation studies completed; the relative contribution of task decomposition vs. feedback loops vs. five-element structure is unknown.
  • Solver runtime excluded from latency measurements, understating total wall-clock cost for complex problems.
  • NEMO and CoE results on LogiOR not reported (missing baselines), limiting full comparison.
  • Max-3-iterations cap on loops may still leave unresolved errors silently passed downstream.
  • No human-in-the-loop evaluation; trust and usability claims for non-expert users are unvalidated.

Relevance to Harnesses / Meta-Harnesses

OptiAgent is a direct instantiation of a domain-specific agent harness: a LangGraph-based stateful pipeline that wraps six LLM agents with typed handoffs, shared state, and automatic error-routing feedback loops — exactly the architectural pattern studied in harness / meta-harness research. The four-loop feedback mechanism is a concrete example of harness-level control flow (routing, retry caps, failure-mode dispatch) operating above any single agent's context, demonstrating how a harness can enforce correctness guarantees that no individual agent can provide alone. The shared AgentState object and the APPROVED/REJECTED signals from the Validation agent parallel supervisor/orchestrator patterns common in multi-agent harness designs. For researchers tracking this topic, OptiAgent offers a well-evaluated case study in harness design tradeoffs: transparency vs. latency, loop-cap policy, and the division of labor between harness routing logic and per-agent reasoning.