Hierarchical Long-Term Semantic Memory for LinkedIn's Hiring Agent

🕒 Published (v1): 2026-04-29 00:53 UTC · Source: Arxiv · Venue: KDD 2026 · link

Why this paper was selected

Long-term semantic memory from LinkedIn's production hiring agent; large-scale industry result

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

HLTM (Hierarchical Long-Term Semantic Memory) is a production memory framework for LLM agents that organizes behavioral data into a schema-aligned tree, enabling privacy-scoped, low-latency retrieval via multi-view memory representations. Deployed in LinkedIn's Hiring Assistant, it improves answer correctness by >5% and retrieval F1 by >10% over baselines while pushing the Pareto frontier between query latency and correctness.

Problem

Existing agent memory systems (MemGPT, GraphRAG, RAPTOR, etc.) lack the combination of properties required for industrial-grade deployment: they fail to enforce strict per-tenant privacy isolation, suffer from semantic-clustering hierarchies that are brittle under incremental updates, incur high online latency from on-the-fly summarization, and offer no provenance tracking for governance and debugging. Most also rely on the LLM's world knowledge to decide what to memorize rather than adapting to observed query workloads.

Method

HLTM organizes memory as a schema-aligned tree \(T = (V, E)\) whose topology follows enterprise data-ownership boundaries (e.g., account → recruiter seat → hiring project), not semantic clustering. Each tree node \(v\) stores a three-view memory representation \(M_v = (F_v, Q_v, S_v)\): - Facet view \(F_v\): LLM-extracted key–value pairs (e.g., location: SF Bay Area), linearized and embedded. - Answerable-QA view \(Q_v\): offline-synthesized question–answer pairs whose embedded questions align with user query embeddings ("think-fast" serving). - Summary view \(S_v\): a natural-language paragraph plus a single-sentence embedding target.

Parent nodes are populated via bottom-up hierarchical aggregation: an LLM groups semantically similar child evidence, merges redundancies, and prunes low-salience details.

At serving time, identity-scoped subtree filtering restricts retrieval to \(\mathcal{T}_v = T\!\restriction_{\{v\} \cup \text{Desc}(v)}\) for the querying identity. Within that scope, three retrievers score nodes independently:

\[s_{\text{facet}}(q,v) = \frac{1}{k\,|F_q|} \sum_{f \in F_q} \sum_{f' \in \text{TopK}(f, F_v)} \cos\!\bigl(\text{emb}(f),\,\text{emb}(f')\bigr)$$ $$s_{\text{QA}}(q,v) = \max_{q' \in Q_v} \cos\!\bigl(\text{emb}(q),\,\text{emb}(q')\bigr)$$ $$s_{\text{summary}}(q,v) = \cos\!\bigl(\text{emb}(q),\,\text{emb}(S_v)\bigr)\]

Top-\(k\) results from each view are merged as context for in-context-learning answer generation with node-ID citations. Lossless incremental indexing recomputes only modified leaves \(V^\star\) and propagates updates along ancestor paths, reducing update cost from \(O(|V|)\) to \(O(\text{tree height})\). An adaptation module mines sliding-window query logs for recurring patterns and salient facet names, feeding them as priors to the QA and facet extraction agents.

Key Contributions

• Schema-aligned tree topology that enforces privacy via subtree-restricted retrieval without relying on semantic clustering.
• Three-view memory representation (facet/QA/summary) supporting diverse query types and a "think-fast" offline precomputation mode.
• Offline bottom-up hierarchical aggregation, avoiding on-the-fly summarization latency for broad-scope queries.
• Workload-driven adaptation loop that continuously refines extraction based on historical query patterns.
• Lossless incremental nearline indexing with minutes-scale freshness at a fraction of full re-indexing cost.
• Full production deployment in LinkedIn's Hiring Assistant serving millions of documents.

Results

• Answer correctness: HLTM improves by >5% over baselines on the LinkedIn Hiring Assistant dataset.
• Retrieval F1: improves by >10% over baselines.
• Latency–correctness Pareto frontier: HLTM consistently dominates all evaluated methods at given query latency budgets (Figure 2).
• Evaluated on two datasets: LinkedIn Hiring Assistant (1,341 queries; ~53M total history tokens) and LongMemEval-s (500 queries; ~51M total history tokens).
• Dataset contains 473 retrieval-style and 868 summarization-style queries with expert-curated gold answers (≥3 annotators, majority-vote resolution).

(Note: per-baseline numerical breakdowns are in the full paper's Table 2, which is truncated in the provided text.)

Limitations

• Evaluated in a single enterprise domain (recruiting); generalizability to substantially different ontologies requires new schema design.
• Schema-aligned topology requires upfront engineering to map domain entities to a business hierarchy; not plug-and-play for arbitrary domains.
• Adaptation module applies minimum-support thresholds and optionally requires human review before deploying pattern updates, adding operational overhead in sensitive settings.
• Multi-view ingestion relies heavily on LLM calls (three agents per node), making cold-start indexing token-intensive.
• Full baseline comparison tables (Section 4) are not included in the provided excerpt, preventing independent verification of all reported numbers.

Relevance to Agentic AI / LLM Agents

HLTM directly addresses a core unsolved problem for production LLM agents: how to maintain personalized, longitudinal memory at enterprise scale without sacrificing latency, privacy, or consistency. The three-view representation and offline aggregation strategy provide a concrete, deployable blueprint for augmenting agents with external long-term stores that go well beyond simple vector databases or unstructured conversation logs. The workload-driven adaptation loop is an important systems contribution—showing that memory schema can evolve from observed agent behavior rather than being hand-engineered. For researchers tracking agentic AI, this paper bridges the gap between academic memory proposals (MemGPT, RAPTOR, GraphRAG) and the engineering constraints (GDPR, sub-second latency, millions of documents) that actually govern real deployments.