Skip to content

SR-Agent: An Experience-Driven Agentic Framework for Post-Ranking Strategy Refinement in E-Commerce Recommendation

🕒 Published (v1): 2026-07-20 09:12 UTC · Source: Arxiv · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

SR-Agent is an experience-driven multi-agent framework that automates the continuous refinement of post-ranking strategies in industrial e-commerce recommender systems. It chains a UserSim agent (bad-case detection), an Analysis agent (causal diagnosis), and a constrained Strategy Refinement Harness (bounded, guarded updates) into a self-evolving closed loop. Deployed on Kuaishou at 60M-user scale, it replaces manual human workflows and improves key business metrics in a one-month A/B test.

Problem

Industrial post-ranking strategies (diversity quotas, similarity suppression, exposure caps) are statically configured and degrade as the recommendation environment evolves—new items, shifting user intent, stale taxonomy labels. Refinement requires multi-role manual coordination (inspectors → domain experts → algorithm engineers → QA), making cycles slow, costly, expert-dependent, and non-reusable. Existing LLM-based agents (RecUserSim, SimUSER, Self-EvolveRec) simulate user responses or evolve models but do not close the full loop from bad-case detection through bounded strategy update and validated deployment.

Method

SR-Agent freezes the base ranker and operates only on the strategy state \(S \in \mathcal{S}\). The displayed list is \(L = \pi_S(u, c)\); the loop produces candidate updates \(\{a_t\} \subset \mathcal{A}\) that must pass a guarded admission pipeline before modifying \(S\).

UserSim Agent inspects each displayed list via three staged skills: (1) Exposure Profiler characterizes repeated category/shop/price-band concentration; (2) Relation Extractor identifies functional relations (visual similarity, substitution, complementarity) not captured by existing labels; (3) Experience Evaluator integrates both with user intent and history to produce a structured bad case \((y, I_b, e_b, q_b)\).

Analysis Agent clusters bad cases by defect type and runs a three-stage chain-of-thought: (i) consolidate shared evidence into a cluster-level hypothesis; (ii) contrast with deployed \(S\) to attribute an operational cause \(h\) (stale grouping, threshold miscalibration, etc.); (iii) infer the minimal defensible scope \(s\) and refinement direction \(v\). Each diagnosis is stored as \(d = (z, h, s, v, e_d)\).

Strategy Refinement Harness is the core constrained component. It exposes a typed action space \(\mathcal{A}\) with four action types and hard-bounded parameters (Table 1): relation-threshold adjustment (\(\Delta\tau_r \in [-0.15, +0.15]\), step 0.01), redundancy-suppression adjustment (\(\Delta\lambda \in [-0.50, +0.50]\), step 0.05), exposure-limit adjustment, and taxonomy correction over authorized scopes only. The LLM cannot generate serving code, modify model objectives, or emit actions outside \(\mathcal{A}\). Candidates are validated through a four-stage reward pipeline: (1) bad-case retest on labeled cases from \(\mathcal{M}_b\); (2) historical traffic replay on \(\mathcal{D}_{\text{hist}}\); (3) human verification; (4) online A/B validation with guardrail-triggered rollback. Outcomes are recorded in Strategy Memory \(\mathcal{M}_s\) as \((d_t, a_t, r_t, o_t)\) with \(o_t \in \{\text{invalid, rejected, deployed, rolled-back}\}\).

Three memory banks persist state across iterations: Bad-Case Memory, Diagnosis Memory (reusable abstracted failure patterns), and Strategy Memory (proposals, parameters, validation outcomes).

Key Contributions

  • Formulation of post-ranking strategy maintenance as an automated self-evolving closed loop connecting inspection → diagnosis → bounded refinement → verified deployment.
  • SR-Agent: two reasoning agents (UserSim, Analysis) coupled with a guarded harness that enforces typed, bounded, auditable strategy updates and prevents LLM from touching serving code or model objectives.
  • Three persistent memory banks enabling experience accumulation and cross-iteration reuse of diagnoses and strategy outcomes.
  • First agentic system deployed at production scale (60M users, 70M items on Kuaishou) for continuous post-ranking strategy refinement, validated by a one-month online A/B test.
  • A four-stage reward pipeline (bad-case retest → traffic replay → human verification → online A/B + rollback) as a graded admission mechanism for safe deployment.

Results

From the one-month online A/B test (2.5% traffic split, ~60M users, ~70M items on Kuaishou): - Order volume: +0.71% vs. control (existing manually configured post-ranking strategies, base ranker frozen in both groups). - Browsing depth: +0.34%. - Clicked-category diversity: +0.48%. - Refinement cycle shortened and operational cost reduced (quantitative comparison not included in the provided text excerpt). - Human audit validates quality of bad cases produced by UserSim at approximately 1% random sample rate (precision figures truncated in provided text).

Limitations

  • Results come from a single platform (Kuaishou e-commerce); generalizability to other RS architectures is undemonstrated.
  • The typed action space \(\mathcal{A}\) and authorized scopes must be manually configured and versioned by strategy owners before deployment—the system does not discover new action types autonomously.
  • Only ~1% of bad cases and diagnoses are human-audited; systematic quality degradation could propagate undetected.
  • The base ranker is frozen; SR-Agent cannot address experience failures that originate in ranking rather than post-ranking.
  • Online A/B test window is one month; longer-term drift behavior of the self-evolving loop is not reported.
  • Quantitative ablation and comparison against prior LLM-agent baselines are truncated in the provided text.

Relevance to Harnesses / Meta-Harnesses

The Strategy Refinement Harness is a canonical harness design: it interposes a hard-typed, parameter-bounded, scope-whitelisted interface between an LLM reasoner and production execution, ensuring the model can only emit operations within a pre-audited action space and cannot self-modify the harness itself. The four-stage guarded admission pipeline (retest → replay → human gate → online gate with rollback) is a concrete instance of a staged validation meta-harness pattern—each gate filters candidate actions before they advance, and failure at any gate stops propagation. The three-memory architecture (bad cases, diagnoses, strategy outcomes) further illustrates how harness-embedded memory enables closed-loop, self-evolving behavior without unbounded LLM autonomy—a key design tension for meta-harness researchers to study.