Skip to content

Lessons Learned: A Multi-Agent Framework for Code LLMs to Learn and Improve

🕒 Published (v1): 2025-05-29 18:56 UTC · Source: Arxiv · Venue: NEURIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

LessonL is a multi-agent framework where LLM agents collaboratively solve coding problems by generating, banking, and selectively sharing "lessons" — concise natural-language explanations of why a solution succeeded or failed. A team of small open-source LLMs using LessonL outperforms GPT-4o on code optimization benchmarks, demonstrating that structured inter-agent knowledge transfer can substitute for scale.

Problem

No single LLM dominates all coding sub-tasks: complementary strengths are distributed across models at fine granularity (e.g., Qwen7B outperforms GPT-4o by 2.5Ă— on geometry problems). Existing multi-agent frameworks either hard-code agent roles (requiring a priori knowledge of strengths) or aggregate solutions without enabling agents to learn from each other's reasoning. There is no principled mechanism for transferring interpretable, reusable problem-solving knowledge between agents during inference.

Method

LessonL operates in iterative rounds with a solicitation–banking–selection loop:

  1. Solicitation: After each agent generates/updates a solution, it is prompted to produce a lesson — an explanation of what worked or failed — conditioned on the outcome category (speedup, slowdown, functional incorrectness, syntax error) and measured performance.
  2. Banking: All lessons are deposited into a shared lesson bank.
  3. Selection: At each round, at most \(k\) lessons are selected for the next prompt. Selection uses a hybrid criterion: top \(\lceil k/2 \rceil\) by adjusted speedup score, top \(\lfloor k/2 \rfloor\) by cosine similarity (via CodeBERT embeddings) to the current code.
  4. Effectiveness adjustment: Each lesson \(z\) with baseline speedup \(s\) accumulates a correction factor \(f = c/n\) based on whether agents applying it achieved \(s_j^{(t)} > s\) or \(< s\); lessons are sorted by \(s \times f\) rather than raw \(s\).

Each agent uses the selected lessons as additional context to generate an improved solution. The framework is task-agnostic: for code generation, lesson selection switches from speedup to test-case pass count plus relevance.

Key Contributions

  • Empirical finding that LLM complementary strengths exist at fine sub-task granularity, motivating heterogeneous agent teams.
  • The solicitation–banking–selection mechanism for generating, storing, and filtering inter-agent lessons during inference.
  • Dynamic lesson effectiveness adjustment factor \(f\) to avoid over-relying on lessons that no longer generalize.
  • Demonstration that three small open-source models (7B–14B) with LessonL outperform GPT-4o and reasoning models (except OpenAI o3) on code optimization.
  • State-of-the-art on code generation benchmarks (HumanEval, HumanEval+, MBPP) among non-reasoning methods.

Results

Code optimization (ParEval, geometric mean speedup): - LessonL serial: 2.16Ă— vs. GPT-4o 1.72Ă—, MoA 1.76Ă—, MapCoder 1.85Ă—, OpenAI o3 2.21Ă— - LessonL OpenMP: 3.46Ă— vs. GPT-4o 2.93Ă—, MoA 2.77Ă—, MapCoder 3.43Ă—, OpenAI o3 3.55Ă—

Code optimization (PolyBench, geometric mean speedup): - LessonL serial: 1.32Ă— vs. GPT-4o 1.12Ă—, OpenAI o3 1.71Ă— - LessonL OpenMP: 3.40Ă— vs. GPT-4o 2.73Ă—, OpenAI o3 3.15Ă—

Code generation (pass@1): - HumanEval: 0.915 (vs. MoA 0.909, GPT-4o 0.884) - HumanEval+: 0.878 (vs. MoA 0.872, GPT-4o 0.872) - MBPP: 0.899 (vs. MoA 0.897) - MBPP+: 0.765 (MoA 0.770 wins marginally)

Ablation: Removing lessons entirely (variant 5) drops OpenMP speedup from 3.46Ă— to 3.01Ă—; removing speedup-based selection most hurts serial mode; removing relevance-based selection most hurts OpenMP mode.

Scaling: Performance increases monotonically from 1 agent (1.60Ă—) to 6 agents (2.21Ă—) in serial mode.

Limitations

  • Evaluated only on code optimization and code generation; applicability to other reasoning domains (math, planning) not demonstrated.
  • Lesson quality depends on the LLM's ability to articulate accurate causal explanations; hallucinated or irrelevant lessons are only partially filtered by cosine similarity.
  • The framework requires running profiler/compiler tooling to measure speedup for lesson grading, adding infrastructure overhead beyond standard LLM inference.
  • OpenAI o3 (single agent with native reasoning) still matches or exceeds LessonL on several benchmarks, suggesting test-time compute scaling is a competing approach.
  • Prompt capacity limits the number of lessons \(k\) per round; the bank can accumulate more lessons than can be surfaced, creating a retrieval bottleneck.
  • Only open-source 7B–14B models are used as agents; interaction with larger or heterogeneous closed-source agent pools is untested.

Relevance to Harnesses / Meta-Harnesses

LessonL is directly architecturally analogous to a meta-harness: it wraps multiple base LLM agents in an outer orchestration loop that manages a shared knowledge store (the lesson bank), controls agent invocation order and prompt construction, and routes information between agents based on computed relevance and effectiveness scores. The solicitation–banking–selection mechanism is a concrete, reusable pattern for any harness that needs to aggregate and distill agent outputs into reusable context for downstream agents. The effectiveness adjustment factor \(f\) operationalizes feedback-driven lesson weighting — a technique transferable to harnesses that iteratively re-rank retrieved context based on downstream outcomes. This paper also makes explicit the architectural choice to keep inter-agent communication compact (lessons rather than full code), which is a key design principle for scalable multi-agent harnesses managing context budgets.