AgenticRecTune: Multi-Agent with Self-Evolving Skillhub for Recommendation System Optimization¶

🕒 Published (v1): 2026-04-21 23:15 UTC · Source: Arxiv · link

Why this paper was selected

Multi-agent with self-evolving skillhub optimizes multi-stage recommendation pipelines

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

TL;DR¶

AgenticRecTune is a five-agent LLM framework (Actor, Critic, Insight, Skill, Online) that automates end-to-end system-level configuration optimization for multi-stage industrial recommendation pipelines. Its core novelty is a self-evolving Skillhub that continuously distills domain knowledge from live A/B experiment results into reusable skills. Deployed on Google Discover, it replaces expensive manual tuning loops with an autonomous, closed-loop optimization cycle.

Problem¶

Multi-stage recommender systems (pre-ranking → ranking → re-ranking) require system-level configuration tuning (e.g., fusion weights, routing thresholds) that is non-differentiable, creates an alignment gap between offline proxy metrics and online North Star metrics, and must be re-optimized from scratch after every model change. Traditional AutoML handles numerical search spaces but cannot reason about evolving multi-objective production targets expressed in natural language.

Method¶

The optimization target is formalized as multi-task Compositional Optimization:

\[\Theta^* = \arg\max_{\Theta \in \mathcal{P}} \mathbb{E}_{(x, y_{\text{true}}) \sim \mathcal{D}} \left[ \mathcal{U}(\mathcal{M}(\mathcal{F}(x; w, \Theta), y_{\text{true}})) \right]\]

subject to cost constraint \(\mathbb{E}_{x \sim \mathcal{D}}[C(\Theta)] \le C_{\max}\), where \(\Theta = [\theta_{\text{pre}}, \theta_{\text{rank}}, \theta_{\text{re}}]\) spans all three pipeline stages.

The framework runs two interleaved loops:

Reasoning Loop: The Actor Agent constructs a structured prompt from Skillhub knowledge and elite candidates from Agent Memory, proposes \(k\) configuration candidates with explanations; the Critic Agent applies validity checks, guardrail filtering, and diversity selection, writing survivors to Agent Memory.
Online Experiment Loop: The Online Agent auto-generates executable code/configs, schedules A/B tests on the production platform (with human review gate before launch), collects North Star metric results via platform API, and writes them back to Agent Memory.
Self-Evolving Skillhub: The Insight Agent performs Map-Reduce over historical experiment logs—pruning dominated candidates, maximizing Pareto diversity—and extracts actionable patterns; the Skill Agent synthesizes these into updated task-specific skills, closing the loop for the next reasoning iteration.

Agent Memory stores structured task records (id, config string, explanation, timestamp, status, online results) and maintains a Pareto-elite subset via greedy diversity maximization after metric standardization.

Key Contributions¶

End-to-end agentic framework for system-level (non-differentiable) configuration optimization across pre-ranking, ranking, and re-ranking stages simultaneously.
Self-evolving Skillhub: a closed Insight→Skill loop that continuously converts raw A/B experimental outcomes into persistent, reusable domain knowledge, enabling cross-iteration learning without human curation.
Actor-Critic proposal filtering that enforces format validity, guardrails, and proposal diversity before any production deployment.
Online Agent that bridges abstract parameter candidates to production A/B infrastructure: code generation, traffic allocation, statistical significance management, and results ingestion.
Agent Memory with Pareto-pruning and diversity maximization, preventing stale or dominated configurations from polluting future proposals.

Results¶

The paper reports qualitative deployment success ("multiple successful production launches on Google Discover") but the quantitative results section was truncated in the provided text. No specific metric deltas (e.g., DAU lift, engagement gain) or baseline comparisons are present in the supplied excerpt.

Limitations¶

Human review is required before each A/B test goes live, partially breaking full autonomy.
Feedback is noisy and delayed (online experiments must run to statistical significance before the next iteration), limiting iteration speed.
The Skillhub and domain knowledge are pipeline-specific; transferability to recommendation systems outside Google Discover is undemonstrated.
The paper does not include ablations isolating the contribution of each agent or the Skillhub; claimed gains are attributed to the full system.
No comparison against classical HPO baselines (e.g., Bayesian optimization, evolutionary search) on the same production task.

Relevance to Harnesses / Meta-Harnesses¶

AgenticRecTune is a concrete production-scale meta-harness: it wraps an existing multi-stage ML pipeline in an outer orchestration layer that drives iterative improvement via specialized sub-agents, persistent structured memory, and a self-updating skill library—exactly the architecture pattern central to meta-harness research. The self-evolving Skillhub is a direct instantiation of a meta-harness capability: converting trial outcomes into reusable harness components rather than discarding run-specific knowledge. The Actor-Critic-Online separation maps closely to harness design principles of proposal, verification, and execution stages, and the Map-Reduce Insight loop shows how meta-harnesses can perform continual self-improvement without human-authored updates to the orchestration logic.