Skip to content

Privacy-Preserving RAG via Multi-Agent Semantic Rewriting: Achieving Confidentiality Without Compromising Contextual Fidelity

🕒 Published (v1): 2026-06-23 14:21 UTC · Source: Arxiv · link

Why this paper was selected

Multi-agent semantic rewriting harness for privacy-preserving RAG; reusable coordination pattern for sensitive deployments

Ask a follow-up

Open an assistant pre-loaded with this paper's context.

💬 Ask ChatGPT✦ Ask Claude

TL;DR

A three-agent offline preprocessing harness sanitizes RAG retrieval corpora by decomposing the privacy-vs-utility tension across specialized agents: one extracts PII, one extracts semantic structure, and one reconstructs clean text. Retrieval still runs over the original corpus (asymmetric dual-index), but only sanitized payloads reach the generation LLM. On LLaMA-3-8B over ChatDoctor, targeted privacy leakage drops from 144 instances to 1 with BLEU-1 of 0.122 vs. SAGE's 0.117.

Problem

RAG over private corpora is vulnerable to prompt-injection attacks that cause LLMs to regurgitate sensitive retrieved text nearly verbatim. Single-pass redaction fails because private identifiers and semantically essential content co-occur in natural language, creating an objective entanglement that forces a zero-sum trade-off between suppression and utility. Existing defenses either inject differential-privacy noise (degrading coherence) or synthesize surrogate texts (losing factual specificity).

Method

Offline Multi-Agent Rewriting Pipeline — three agents run sequentially as a one-time preprocessing pass:

  1. Pri-Extra Agent — Rule-LLM Synergistic Extraction: deterministic NER + regex capture explicit identifiers \(P_\text{expl}\); an LLM conditioned on \(P_\text{expl}\) then infers quasi-identifiers \(P_\text{quasi}\) (e.g., rare-disease + occupation combinations). Union: \(P_\text{seq} = P_\text{expl} \cup f_\text{LLM}(T_\text{raw}, P_\text{expl})\).

  2. Sem-Extra Agent — Structured Attribute Deconstruction: maps \(T_\text{raw}\) into \(n\) knowledge-slot tuples \(K_\text{seq} = \{(e_i, r_i, v_i, w_i)\}_{i=1}^n\) (subject, predicate, value, importance weight), decoupling factual content from surface phrasing.

  3. Reconstruction Agent — Fine-grained Conflict Routing: synthesizes \(T_\text{safe} = H(P_\text{seq}, K_\text{seq})\) entirely from \(K_\text{seq}\) slots. Applies placeholder substitution for low-depth conflicts (explicit identifiers → [Patient_A]) and high-level abstraction for deep semantic conflicts (rare syndrome → "a rare genetic immune disorder"). Enforces \(\forall p_m \in P_\text{seq},\ M(p_m, T_\text{safe}) < \tau\), where \(\tau = 0\) for explicit identifiers and a semantic-non-entailment routing rule for quasi-identifiers.

Asymmetric Retrieval Architecture — two indices (BGE-M3 dense + BM25 sparse) are built over raw \(T_\text{raw}\) to preserve recall; both indices return \(T_\text{safe}\) as payload. Hybrid retrieval uses Reciprocal Rank Fusion: \(\text{RRF}(i) = \frac{1}{k + r_\text{dense}(i)} + \frac{1}{k + r_\text{sparse}(i)}\) with \(k=60\).

Key Contributions

  • Frames RAG sanitization as an objective entanglement problem, providing principled motivation for multi-agent decomposition rather than prompt engineering.
  • Three-agent pipeline with Fine-grained Conflict Routing that resolves privacy/utility conflicts at the slot level rather than the passage level.
  • Asymmetric Retrieval and Isolated Generation Architecture (dual-track storage) that preserves retrieval recall over raw text while ensuring the generation LLM never receives \(T_\text{raw}\).
  • Entirely offline, zero-added-latency design; preprocessing is one-time, not per-query.
  • Evaluated across six LLM backbones on two datasets (ChatDoctor: 200k medical dialogues; Wiki-PII: 500k email + public articles) under both targeted and untargeted adversarial settings.

Results

  • Targeted attack leakage (LLaMA-3-8B, ChatDoctor): reduced from 144 instances (baseline) to 1 instance.
  • Contextual fidelity (BLEU-1): 0.122 vs. SAGE's 0.117 (best among privacy-preserving baselines).
  • Outperforms SAGE and AttrPrompt on factual consistency metrics across evaluated LLMs.
  • Online inference latency: no increase (offline preprocessing; zero per-query overhead).
  • Results hold across both targeted and untargeted adversarial attack scenarios.

Limitations

  • BLEU/ROUGE scores (0.122 / evaluated range) remain low in absolute terms, indicating non-trivial semantic divergence from original text even in the best case.
  • \(\tau\) for quasi-identifiers is implemented as a prompt-level routing rule, not a formally verified bound; no provable worst-case privacy guarantee (unlike differential privacy methods).
  • Offline preprocessing cost at corpus scale (e.g., 500k documents) is not characterized in the text.
  • Quasi-identifier detection relies on LLM inference, which may be inconsistent across backbone models or domains outside the two tested datasets.
  • Evaluation limited to two datasets and six LLMs; generalization to other sensitive domains (finance, legal) is not demonstrated.

Relevance to Harnesses / Meta-Harnesses

This paper is a textbook example of a preprocessing harness inserted into a larger pipeline (RAG): it wraps the retrieval stage with a fixed, multi-step agent orchestration that transforms inputs before they reach the main LLM, invisibly from the perspective of the generation model. The three-agent decomposition — extract, analyze, reconstruct — mirrors the pattern of meta-harnesses that coordinate specialized sub-agents with explicit handoffs and structured intermediate representations (\(P_\text{seq}\), \(K_\text{seq}\)). The Asymmetric Retrieval Architecture further demonstrates how a harness can interpose without degrading the capabilities of the wrapped system, a key design challenge for any meta-harness. The conflict routing logic and dual-track storage are reusable harness primitives applicable beyond privacy to any scenario requiring staged transformation of retrieved context.