Skip to content

RecGPT-V3 Technical Report

🕒 Published (v1): 2026-07-17 00:00 UTC · Source: HuggingFace · link

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

RecGPT-V3 is a stateful, hybrid-modal LLM-based recommender system that addresses three core inefficiencies of its predecessors: redundant stateless user modeling, a lossy tag-to-item information bottleneck, and prohibitive chain-of-thought latency. Deployed on Taobao's "Guess What You Like" feed, it delivers simultaneous gains in engagement metrics and a 52.4% reduction in serving compute.

Problem

Prior LLM recommenders (including RecGPT-V1/V2) share three compounding flaws: (1) stateless behavior modeling — every request re-encodes the full user history (\(\sim\)55K tokens for active users), wasting \(\sim\)95% of compute on redundant pattern re-discovery; (2) tag-to-item bottleneck — the LLM outputs coarse free-text tags that are mapped downstream to items, creating a lossy, low-bandwidth interface that cannot convey fine-grained item-level evidence; (3) inefficient explicit reasoning — chain-of-thought rationales averaging \(\sim\)3,000 tokens per request impose unacceptable latency and cost at billion-user scale.

Method

RecGPT-V3 addresses each challenge with a distinct component:

Memory Hub maintains a persistent, structured user memory \(\mathcal{M} = \{m_1, \ldots, m_K\}\) distilled from the full behavioral sequence \(\mathcal{B} = \{b_1, \ldots, b_N\}\) via an LLM-based compression function \(F_\phi: \mathcal{B} \to \mathcal{M}\) (initial pass). Each memory unit contains a pattern label from a predefined taxonomy plus a free-text preference summary with provenance links. An incremental curation function \(G(\mathcal{M}^{(t)}, \Delta\mathcal{B}^{(t,t+\delta)}) \to (\mathcal{M}^{\text{update}}, \mathcal{M}^{\text{new}})\) selectively updates or retains existing units and extracts new ones from unmatched behaviors, running every two months without re-encoding the full history.

Hybrid-Modal Recommendation Foundation Model extends Qwen3-14B with 65,536 Semantic ID (SID) tokens derived via CN-CLIP multimodal embedding (text + image + side info fused by Q-Former, trained with InfoNCE) followed by two-level RQ-VAE quantization (2 × 32,768 codebook), yielding structured SIDs like <C_18921><C_40222>. Related items share coarse-level prefix codes. Training proceeds in two stages: continual pre-training for SID grounding (\(\sim\)90% SID-grounding data, \(\sim\)10% general-domain) and instruction tuning for mixed-modality task following. The LLM now emits retrieval-compatible SIDs directly alongside natural language tags.

Latent Intent Reasoning replaces explicit chain-of-thought with compact learnable <CoT> latent tokens that encode an entire reasoning trace in a few dense steps, reducing output token count by \(200\times\). The latent tokens can be decoded back into human-readable rationales on demand, preserving explainability.

Key Contributions

  • Memory Hub: stateful, condensed (94.5% token reduction vs. raw history), traceable, and incrementally evolvable user memory that cuts Global Planner compute by 55.8%.
  • Hybrid-Modal Foundation Model: unified LLM vocabulary over natural language and SIDs, enabling the model to reason jointly over generalizable intent (text) and concrete item grounding (SIDs), eliminating the tag-to-item bottleneck.
  • Latent Intent Reasoning: \(200\times\) output-token compression via learnable latent <CoT> slots, with on-demand decodability preserving explainability.
  • End-to-end deployment on Taobao at hundreds of millions of daily active users with verified A/B gains.
  • GPU cost reduced to 19% of RecGPT-V1's baseline; 52.4% reduction over RecGPT-V2.

Results

All numbers from live A/B tests on Taobao "Guess What You Like" (RecGPT-V3 vs. RecGPT-V2): - CTR: +1.00% - IPV (item page views): +1.28% - TC (transaction count): +1.97% - GMV: +3.97% - End-to-end serving compute: −52.4% - GPU cost vs. RecGPT-V1: −81% (RecGPT-V3 uses 19% of V1 compute) - Memory Hub alone: 55.8% reduction in user-modeling computation; 94.5% token compression of raw history - Latent Intent Reasoning: \(200\times\) reduction in output token cost vs. explicit chain-of-thought

Limitations

  • Incremental memory curation runs every two months; rapidly shifting user interests within shorter windows may not be captured promptly.
  • SID vocabulary (65,536 tokens) must be re-derived if the item catalog changes substantially, implying non-trivial re-training overhead.
  • The paper provides only relative percentage gains; absolute metric values and statistical significance details are not disclosed, limiting external reproducibility assessment.
  • Evaluation is limited to Taobao's e-commerce domain; generalizability to other recommendation domains (content, social, etc.) is unverified.
  • Latent token decodability is described qualitatively; no quantitative faithfulness metrics for the reconstructed rationales are reported in the visible text.

Relevance to Agentic AI / LLM Agents

RecGPT-V3 is a direct instance of a deployed multi-agent LLM system (planner + multi-expert sub-agents) solving a production-scale reasoning task, making it immediately relevant to agentic AI research on efficiency and statefulness. The Memory Hub operationalizes persistent agent memory — a core open problem in long-horizon agentic operation — by demonstrating that incremental, schema-structured memory compression can substitute for stateless full-context re-encoding at billion-user scale. Latent Intent Reasoning offers a concrete mechanism for collapsing expensive chain-of-thought into latent deliberation steps while preserving decodable explanations, directly addressing the latency-vs-reasoning tradeoff central to deploying reasoning agents in real-time systems. The hybrid SID-text modality architecture also illustrates how LLM agents can be grounded in structured, domain-specific symbolic spaces alongside natural language — a pattern transferable to tool-use and embodied agent settings.