Skip to content

Knowledge Starts with Practice: Knowledge-Aware Exercise Generative Recommendation with Adaptive Multi-Agent Cooperation

🕒 Published (v1): 2025-01-01 · Source: NeurIPS · Venue: NeurIPS 2025 · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

ExeGen is a four-agent LLM framework for Knowledge-aware Exercise Generative Recommendation (KEGR) in adaptive learning: it infers a student's knowledge state from historical responses and generates personalized, quality-refined exercises. Two core mechanisms—a RAG+ICL human-simulated perception loop and a GAN-inspired adversarial refinement loop—distinguish it from vanilla prompting baselines. Evaluated on MOOCCubeX and deployed in real college courses, ExeGen consistently outperforms Zero-shot, ICL, CoT, and ICL+CoT across all eight metrics.

Problem

Existing intelligent education pipelines treat Knowledge Tracing (KT) and Exercise Recommendation (ER) as separate tasks: KT predicts binary correctness but lacks interpretable state descriptions; ER matches students to static exercise pools while ignoring knowledge state dynamics. Neither task alone enables a closed-loop, adaptive system that simultaneously diagnoses and generates personalized content.

Method

ExeGen coordinates four specialized agents in an adaptive loop:

  1. Recommendation Manager — orchestrates the pipeline, decomposes KEGR into subtasks, delivers final exercises.
  2. Knowledge Perceiver — runs the human-simulated knowledge perception mechanism: (a) constructs demonstrations by RAG-querying a domain KG for knowledge-evidence triples, then generating CoT explanations for both correct/incorrect response samples; (b) retrieves top-K demonstrations per exercise record via hard (exact concept) or soft (BERT cosine) matching; (c) synthesizes an interpretable knowledge-state description via ICL.
  3. Exercise Generator — uses CoT prompting with the inferred knowledge state to produce stem+concept+answer for three objective exercise types (single-choice, multi-choice, judgment).
  4. Quality Evaluation Expert (group) — implements a generative-adversarial mechanism: multiple dimension-specific experts (linguistic fluency, erroneous-concept coverage, overall correctness) each assess generated exercises and feed structured critique back to the Generator; iterates up to 10 rounds until all thresholds are met.

Key Contributions

  • Defines the novel KEGR task unifying knowledge-state perception and personalized exercise generation in a closed-loop formulation.
  • Human-simulated knowledge perception mechanism: RAG-augmented demonstration construction with dual hard/soft matching for reliable ICL reasoning.
  • Exercise generative-adversarial mechanism: multi-expert, multi-dimensional iterative refinement inspired by GAN generator–discriminator dynamics.
  • Comprehensive three-step evaluation protocol: GPT-based scoring (6 metrics), statistical metrics (Error Hit Rate, Recall), student questionnaire, and teacher expert review.
  • Real-world deployment in college education on top of the MOOCCubeX dataset.

Results

All metrics scored 0–5 (GPT-based) or 0–1 (HR, Recall); ExeGen vs. best baseline ICL+CoT on multiple-choice exercises:

  • Knowledge Relevance: 4.82 vs. 4.25
  • Answer Accuracy: 4.79 vs. 4.39
  • Difficulty Appropriateness: 4.80 vs. 4.28
  • Error Hit Rate (HR): 0.79 vs. 0.21
  • Error Recall: 0.82 vs. 0.40
  • ExeGen outperforms all baselines (Zero-shot, ICL, CoT, ICL+CoT) across all three exercise types and all eight metrics.
  • Ablation: removing Knowledge Perceiver causes the largest drop; removing Human-simulated Perception or Adversarial mechanism each cause significant degradation across all metrics.
  • Framework generalizes across LLaMA-70B, Qwen-turbo-1101, and GPT-4o; GPT-4o and Qwen-turbo-1101 achieve strongest results.

Limitations

  • Restricted to three objective exercise types (single-choice, multi-choice, judgment); subjective/open-ended exercise generation is explicitly left as future work.
  • Evaluation relies heavily on GPT-based scoring, which introduces potential bias from the same model family used in generation.
  • Knowledge graph dependency: domain KG must exist for the RAG component; generalization to domains without structured KGs is untested.
  • Maximum refinement rounds capped at 10 without empirical analysis of convergence behavior or cost under failure cases.
  • No comparison against non-LLM exercise-recommendation baselines, since KEGR is framed as a new task.

Relevance to Agentic AI / LLM Agents

ExeGen is a concrete instantiation of role-specialized multi-agent cooperation with an adaptive loop control structure, directly relevant to research on how agents decompose complex tasks, handle domain-knowledge grounding, and use adversarial inter-agent feedback to improve output quality. The GAN-inspired generator–discriminator dynamic between agents is a transferable pattern for any generative-refinement pipeline. The human-simulated ICL mechanism (RAG → demonstration construction → knowledge inference) exemplifies how to mitigate LLM hallucination in knowledge-intensive agentic subtasks. For the broader agentic AI community, this paper offers a worked example of closing the perception–action loop within a multi-agent system deployed in a real application domain.