betaFeedback

WorksComputer Science

Protection Is Not One Event: A Survey of RAG Protection Research by Measured Endpoint

Abstract

Retrieval-augmented generation (RAG) systems face security, privacy, and copyright risks including content leakage, source reconstruction, unauthorized access, data poisoning, and prompt injection. Yet RAG protection studies use incompatible evaluation metrics, datasets, and threat models. We present a systematic survey and human-coded evidence map of 118 full-text-confirmed RAG defense papers, organized by measured endpoint rather than claimed goal or defense mechanism. Our seven-family taxonomy covers content disclosure, formal privacy proxies, access control and suppression, corpus-use detection, and pipeline integrity and attribution. Thirty-two annotators double-coded the papers kappa=0.867. The survey identifies three gaps. First, retrieval reranking and context rewriting each span five or six endpoint families, so mechanisms do not identify protection outcomes. Second, only 17.8% of studies use content disclosure or reconstruction as their primary endpoint; under the frozen copyright/ownership relation label, 73.3% of records have corpus-use detection as their primary endpoint, and none have an emitted-expression primary endpoint. Third, most evaluations use general-purpose question-answering datasets and non-adaptive attacks rather than sensitive data and adaptive, multi-turn extraction. Our controlled RAG testbed shows that preventing verbatim copying does not prevent private-fact recovery: a normalized 8-gram guard reduces 20-token copying to 0.0%, but a three-query adaptive attacker recovers 48.3% of hidden facts on Qwen3-8B. We release the taxonomy, dataset, coding protocols, and testbed to enable comparable, claim-aligned RAG security evaluation.

Introduction

Retrieval-augmented generation (RAG) connects language models to external document collections at inference time [63, 43]. Because this external datastore can be searched, updated, and quoted directly, it creates new risks around privacy, copyright, and system security. Even if a document never appeared in pre-training, a RAG system might still leak its private details, copy its wording verbatim, or fall victim to poisoned entries.

A rapidly growing literature aims to “protect” RAG systems against these risks, yet papers using this label rarely measure the same thing. Consider two studies that both claim to protect the copyright of a retrieval database. The first plants a watermark to prove after the fact that the database was queried, while the second blocks unauthorized users from retrieving restricted documents. A third study defends against poisoned passages to keep answers accurate, and a fourth rewrites retrieved text so the generator cannot copy it word for word. All four papers report successful “protection,” but they evaluate completely different technical events. Treating them as interchangeable, or pooling their scores to compare defenses, is like comparing a burglar alarm to a fireproof safe.

Existing surveys organize this literature by threat model, pipeline stage, or system architecture. However, knowing where a defense sits in the pipeline tells us very little about what it actually achieves. In this work, we take a different approach: we survey RAG protection research based on the exact empirical endpoint each study measures. We define a frozen seven-family taxonomy of measured endpoints (Table 1 and Figure 1) that separates actual content disclosure from access control, usage detection, and pipeline integrity. Using this taxonomy, 32 annotators double-coded 118 full-text-confirmed RAG protection papers, reaching high agreement on what each paper measures (κ=0.867\kappa= 0.867).

The frozen taxonomy and its 17 subfamilies

Figure 1. The frozen taxonomy and its 17 subfamilies. A record is assigned by its primary measured endpoint, not by its topic.

Looking at the literature through its measured endpoints reveals three major blind spots. First, defense mechanisms only weakly predict what a paper tests (§ 5.1). Popular techniques such as retrieval reranking and context rewriting are evaluated across five or six different endpoint families, meaning readers cannot infer what was protected just from how the system was built. Second, research attention is heavily skewed away from actual data leakage (§ 5). Only 21 of the 118 papers (17.8%) have a primary endpoint that measures whether protected content is emitted to the user or reconstructed from the system. Among studies with the frozen copyright/ownership relation label, 73.3% have corpus-use detection as their primary endpoint, whereas none have an emitted-expression primary endpoint. Third, evaluation setups often fail to stress-test the defenses they propose (§ 5.3). Most studies rely on standard question-answering datasets that contain little sensitive content to begin with, and very few test against adaptive attackers who can ask follow-up questions. Our own controlled experiments confirm why this distinction matters: defenses that drive word-for-word copying down to zero still allow attackers to recover the underlying private facts (Appendix § 10).

To help future work build on clear empirical foundations, we make four contributions:

  • An endpoint-based survey that organizes RAG defense mechanisms, evaluation metrics, datasets, and threat models by the exact technical event they measure (§ 4, § 5.3).

  • A validated seven-family event taxonomy that makes cross-paper comparisons reproducible (§ 3).

  • A human-coded evidence map of 118 direct RAG protection papers, complete with item-level annotations, operating points, and open-source verification tools (§ 3, Appendix § 11.§ 2).

  • A critical audit and controlled testbed examining reporting quality, and the gap between stopping surface copying and preventing fact recovery (§ 5, § 6, Appendix § 10).

Related Work

Prior surveys of RAG security and privacy organize the field around system architecture and attack surfaces. Broad reviews of language model privacy cover threats like membership inference and data poisoning ([37]) alongside cryptographic and differentially private defenses ([23]). Dedicated surveys on copyright and ownership focus on text and model watermarking ([134]) or fingerprinting techniques ([131]). Within the RAG literature specifically, existing reviews group studies by threat model, pipeline stage, or defensive module ([86, 92, 130]). While this architectural view shows where a defense intervenes, it obscures what the evaluation actually tests. Individual empirical studies already hint at this divergence: papers evaluating literal and non-literal text reproduction ([26]), copyright-sensitive activation steering ((Zhang et al., 2026c)), proprietary knowledge base defense ([64]), and knowledge extraction benchmarks ([101]) all define “protection” through different operational outcomes. Furthermore, recent adaptation methods train models to decide when to retrieve, critique context, or withhold answers ([8, 70]), blurring the boundary between parametric memory and external datastores. Rather than grouping papers by where their mechanisms sit in the pipeline, we human-code the exact technical events their experiments measure.

Our approach builds on meta-scientific audits of machine learning research ([71, 57]) and classical measurement theory, which requires clearly defining target concepts and validating how they are measured ([35, 18, 81]). These principles have shaped recent critiques of benchmark design and fairness evaluation in natural language processing ([51, 10, 102, 12]). However, while prior audits typically expose flaws inside an experimental setup, we examine the link between two separate components of a paper: its high-level protection claim and its measured empirical endpoint. This focus on claim-measurement alignment parallels recent findings in question answering, where standard lexical overlap metrics often fail to capture true semantic accuracy ([14, 56, 2]).

The Endpoint Taxonomy and Human-Coded Evidence Map

To compare RAG protection studies fairly, we first need a clear way to categorize what their experiments actually measure. Treating “protection” as a single outcome confuses very different technical events. Stopping a model from copying source text word for word is not the same as hiding personal identifiers, preventing attackers from inverting vector embeddings, proving a differential privacy bound, blocking unauthorized retrieval, detecting watermarks, or filtering out poisoned documents.

A Seven-Family Taxonomy of Measured Endpoints

We introduce a frozen seven-family taxonomy (Families A, A-pii, and B–F) that groups studies by the exact empirical quantity their primary metric evaluates, rather than where their defense mechanism sits in the RAG pipeline (Table 1 and Figure 1). This design deliberately cuts across the module-based categories used in prior security reviews (§ 4). For example, if a paper proposes an unlearning method but evaluates it solely by measuring how often the system refuses to answer forbidden queries, our taxonomy places it under access suppression (Family D) rather than removal.

Conversely, if a paper tests whether deleted data can still be reconstructed from residual index states, it belongs to source recovery (Family B).

The seven families fall into four broader asset groups based on what is being protected:

  • Content Disclosure and Formal Proxies (Families A, A-pii, B, and C): These endpoints concern the secrecy of the underlying data. Family A measures whether generated answers copy protected wording verbatim or through close paraphrase. Family A-pii tracks whether the output leaks specific personal identifiers like names or phone numbers. Family B measures whether an attacker can reconstruct source documents outside ordinary text generation, such as by inverting embeddings or inspecting index structures. Family C groups studies that report a mathematical privacy bound (such as differential privacy ε\varepsilon) or a perplexity shift without measuring actual leaked text. Because Family C reports a theoretical or statistical proxy rather than observed leakage, only Families A, A-pii, and B directly measure content disclosure to a user or adversary.

  • Access Suppression (Family D): Endpoints here test whether the system blocks unautho-

rized documents from being retrieved, routed, or used to answer a query, typically through role-based filters or refusal mechanisms.

  • Corpus-Use Detection (Family E): Rather than preventing data exposure, these endpoints test whether an auditor can prove after the fact that a document or database was used, either by detecting planted watermarks and canaries or by running membership inference audits.

  • Pipeline Integrity and Attribution (Family F): The largest group in our map evaluates whether the RAG system resists external manipulation (such as corpus poisoning or prompt injection) or correctly attributes generated answers back to their source documents. Family G is reserved for underdetermined cases where the reported packet does not establish a specific technical event.

Fixing category boundaries by the measured metric makes cross-paper comparison reproducible. When 32 annotators double-coded all 118 core records using this seven-family scheme, they reached high agreement on the primary endpoint family (κ=0.867\kappa= 0.867).

Literature Search, Human Coding, and Corpus Scope

To build the evidence map, we searched arXiv, OpenAlex, and the ACL Anthology through July 17, 2026, following standard scoping review protocols (Tricco et al., [115]; Peters et al., [99]). Our queries returned 11,972 total hits, which deduplicated to 8,978 unique records. Screening titles and abstracts narrowed this pool to 925 candidate papers, for which we recovered 824 full texts (Appendix § 9).

Thirty-two trained annotators with backgrounds in natural language processing, machine learning, security, privacy, and information science double-coded all 824 recoverable full texts. Any disagreements were resolved by an independent adjudicator under a frozen protocol. To verify that our initial screening did not miss relevant work, we also double-screened a stratified sample of 1,200 excluded records to test our stopping rule (Callaghan et al., [17]). To qualify as a direct RAG-protection study in our core map, a paper had to evaluate a retrieval-augmented system, intervene on its query, corpus, index, context, or output, and report a source-verified protection or attribution metric. Pure attack papers, benchmarks, surveys, and untested proposals were tracked separately, and reporting task utility alone was not enough for inclusion. Each confirmed paper was assigned a single primary endpoint family to prevent multi-module systems from being double-counted as separate studies, while defense mechanisms and intervention stages could receive multiple labels.

This process produced 118 full-text-confirmed direct RAG-protection records (112 from the main candidate pool and 6 recovered from the excluded-record audit). For each record, coders annotated 34 structured fields covering system architecture, protected assets, defense mechanisms, measured outcomes, evidence relations, and publication status, reaching a median agreement of κ=0.94\kappa= 0.94 across all released fields (range 0.56 to 1.00). We release the de-identified 8,024-row human label matrix alongside verification scripts that recompute agreement across every field (Appendix § 9), and we verified every cited operating point directly against the source PDF rather than external metadata (Appendix § 9.§ 3).

Our design-based audit of excluded papers estimates that roughly 27.2 additional confirmable studies may exist beyond our candidate pool under the frozen screening rule. Throughout the paper, we report proportions with 95% Wilson score confidence intervals and permutation-test pp-values as exact descriptions of our 118-record analyzed pool, rather than absolute population totals for the entire field (Appendix § 9.§ 3). As we show next, different endpoint families not only measure distinct events, but often move in opposite directions under the very same defense (Table 4).

(a) Primary endpoints by evidence relation across 118 records. Copyright/ownership is a technical coding relation, not a legal classification.
Protected assetPrimary reported endpointAll (%)Copyright relationOther relation
Content disclosureEmitted source expression8 (6.8)08
Emitted atomic identifiers6 (5.1)06
Non-output reconstruction7 (5.9)07
Likelihood or formal bound6 (5.1)24
Access suppressionAccess and retrieval suppression26 (22.0)125
Use detectionCorpus-use or ownership detection18 (15.3)117
Integrity / attributionIntegrity, provenance, or removal47 (39.8)146
Total11815103
(b) Copyright-boundary sensitivity: relation agreement 74.3% (κ=0.546\kappa= 0.546), centrality agreement 97.6% (κ=0.893\kappa= 0.893).
BoundaryRecordsEmittedDiagnostic change
Frozen evidence-relation label150Includes S-RAG; excludes RAGFort.
Relation minus personal-data S-RAG140Removes the disputed privacy audit.
Frozen copyright-centrality label161Includes both S-RAG and RAGFort.
Centrality minus personal-data S-RAG151Substitutes proprietary-KB extraction for the privacy audit.

Table 4. Evidence-relation sensitivity.

The Landscape of Defenses by Measured Endpoint

Every empirical RAG protection study specifies where it intervenes in the pipeline, what mechanism it applies, and what outcome metric it measures. While prior surveys organize the literature around the first two dimensions (Mu et al., [86]; Palanisamy et al., [92]; Xu et al., [130]), we organize our analysis around the third. Below, we examine each endpoint family in turn, showing how the defenses tested against it work, where they trade off utility or fail under pressure, and what blind spots their metrics leave behind. All counts refer to the 118 human-coded core records, and Appendix § 11.§ 2 lists the exact operating points reported by each study.

Emitted Text and Identifiers (Families A and A-pii)

Families A and A-pii are the only endpoints in our taxonomy that directly score the text a user receives. Family A measures whether the generated response reproduces protected source wording verbatim or through close paraphrase, while Family A-pii tracks whether the response leaks atomic personal identifiers such as names, phone numbers, or email addresses. Despite sitting at the heart of privacy and copyright concerns, these two families together account for just 14 of the 118 records in our map. The defenses evaluated here intervene at four distinct points along the generation path. First, retrieval-side thresholding drops passages whose embedding distance to the query falls outside a safe band, ensuring the generator never sees quotable source text [141]. Second, context transformation rewrites or condenses retrieved passages before generation, using either extractive compression (Zeng et al., 2025a) or multi-agent semantic rewriting that paraphrases sensitive spans while keeping the passage useful for answering questions (Zhao et al., 2027). Third, constrained decoding intervenes during token generation to block verbatim reproduction: privacy-aware decoding adds calibrated noise to next-token probabilities under a differential privacy budget [116], adaptive backtracking rolls the decoder back whenever a partial span begins to match a protected entity [137], and dual-path generation synthesizes answers from a draft model while consulting the protected source only through a restricted reference channel [64]. Fourth, studies in Family A-pii typically rely on a two-stage filter pair, combining an input judge that blocks extraction-style prompts with an output regular expression or named-entity filter that redacts identifiers before returning the final answer [65, 84].

Because emitted-content metrics penalize even partial text overlap, defenses in Families A and A-pii face a much stricter test than binary access checks. As a result, they report the lowest rates of unqualified success across the entire evidence map: 88% of records in Family A and only 50% in Family A-pii report net improvements, compared with 94% to 96% in the access, detection, and integrity families. They are also the two families where adaptive attackers most frequently recover the protected information, occurring in 38% of Family A records and 33% of Family A-pii records, versus just 4% for access suppression, 6% for usage detection, and 0% for pipeline integrity. Looking closely at their operating points explains why achieving complete protection is so difficult. When interventions push text reproduction toward zero, task utility often drops sharply: distance thresholding cuts source copying in Zeng et al. [141] at the cost of reducing task ROUGE-L from 0.39 to 0.13, entity-level anonymization can sacrifice up to 35% in Recall@k (Zhu et al., 2026), and privacy-aware decoding at strict budgets (ϵ=0.01\epsilon= 0.01 to 0.05) causes model perplexity to diverge Wang et al., [116]. Conversely, when systems relax these controls to preserve answer quality, noticeable leakage remains. Adaptive backtracking still leaves an average leakage rate (ALR) of 0.13 and a context leakage rate (CLR) of 24.3% Yao et al., [137], while dynamic anonymization leaves residual leakage rates of 0.17 on ChatDoctor and 0.33 on EnronQA (Zhu et al., 2026). Similarly, the multilingual input judge in Li et al. [65] blocks English extraction prompts but still leaves 7.5% residual leakage on Arabic queries and 17.5% on Swahili.

These divergent numbers show that there is no universal exchange rate between privacy and accuracy in RAG generation. An operating point that looks safe on aggregate copying scores can still fail on specific entity types, non-English queries, or multi-turn extraction prompts. Consequently, studies evaluating Families A and A-pii must report their residual leakage alongside exact workload conditions and utility costs, rather than treating a drop in average text overlap as proof that source content cannot be extracted.

Source Reconstruction Outside Generation (Family B)

Even if a RAG system never leaks a single word through its generated answers, an attacker can still reconstruct indexed documents through underlying system channels. Family B captures these non-generation reconstruction endpoints, accounting for 7 records in our map (and every record that reports a content-reconstruction metric). Research here targets three distinct leakage channels, each requiring its own defense style. The most widely studied channel is embedding inversion, where an adversary with access to stored vectors runs a trained decoder such as Vec2Text or a fine-tuned T5 model to recover the original text. Defenses against inversion either add entropy-driven perturbations to the vectors (Jin et al., [53]) or apply distance-preserving and homomorphic encryption so the index remains searchable without exposing raw embeddings (Ye et al., [138]; Mao et al., [79]). A second channel is index-structure and deletion residue: PRAG demonstrates that a semi-honest server observing HNSW graph traversal paths can reconstruct the underlying document graph without inspecting a single vector (Li et al., [66]), while Ghost Vectors reveals that soft-deleted embeddings linger in HNSW indices and remain invertible until the index is rebuilt (Chakraborttii et al., [20]). The third channel is parametric residue, which arises when retrieved knowledge is compressed into distilled LoRA weights rather than passed as text context, turning the adapter weights themselves into an inversion target (Chen et al., [22]). Reversing this threat model, one study treats the vector store as an exfiltration medium and uses outlier detectors to catch steganographically smuggled embeddings (Wanger, [123]).

Interpreting reconstruction scores in Family B requires paying close attention to both the reference object and the strength of the attacker’s decoder. Across the seven studies, defenses measure success against very different targets: distance-preserving encryption drives Vec2Text reconstruction BLEU down from 83 to 12 (Ye et al., [138]), path-oblivious retrieval cuts HNSW edge recovery from 18.3% to 6.8% alongside a measurable drop in retrieval recall (Li et al., [66]), and epoch-key rotation reduces ROUGE-L recovery from soft-deleted vectors from 0.207 to 0.000 (Chakraborttii et al., [20]). Because recovering source wording, graph edges, and deleted vectors are distinct events, these numbers cannot be compared on a single scale. More importantly, a near-zero reconstruction score is only meaningful if the baseline attack actually works on the unprotected system. In the clinical-note experiment of Chakraborttii et al. [20], for example, the unprotected baseline yields a Vec2Text ROUGE-L of just 0.009 because the off-the-shelf inverter struggles with domain-specific medical text; in such cases, a low defended score reflects a weak attacker rather than a strong defense. Finally, Family B endpoints are strictly orthogonal to generation endpoints: encrypting embeddings or hiding HNSW traversal paths protects the backend datastore, but it does nothing to stop the language model from quoting retrieved passages verbatim to an end user (Family A).

Mathematical Bounds and Likelihood Proxies (Family C)

Instead of measuring actual text leakage or document reconstruction, the 6 records in Family C evaluate privacy through theoretical guarantees or statistical proxies, and every record in this family pairs its privacy metric with a task utility score. These studies fall into two distinct groups. The formal sub-family injects calibrated noise into the RAG pipeline and tracks a mathematical privacy budget. Within this group, DP-RDM adapts retrieval-augmented diffusion models to private domains under a differential privacy budget of ε=10\varepsilon= 10 (Golatkar et al., [45]; Lebensold et al., [61]), while Private-RAG uses individual Rényi privacy filters to answer hundreds of queries at ε≈10\varepsilon\approx10 in settings where naive composition would require ε≈1000\varepsilon\approx1000 (Wu et al., [125]). Other formal approaches apply differential privacy directly to token voting during text generation (Koga et al., [59]) or use pure ε\varepsilon-DP SimHash bucketing when building the retrieval datastore (Aboueleinen and Torki, [1]). By contrast, the statistical sub-family measures privacy through changes in model likelihood rather than formal bounds: SILO evaluates nonparametric datastores by comparing the model’s perplexity on a held-out book when related books are present in the index versus when they are removed, treating the likelihood shift itself as the primary endpoint (Min et al., [82]).

While mathematical bounds and likelihood shifts offer clean theoretical targets, they do not measure what a deployed system actually emits to a user, and their numbers cannot be pooled with empirical leakage rates. Even within Family C, reported operating points depend heavily on the protected unit, the composition rule, and the query workload. For example, DP-RDM achieves its ε=10\varepsilon= 10 guarantee over T=100kT = 100\mathrm{k} queries with a generation FID of 155.67 (Lebensold et al., [61]), whereas Private-RAG maintains ε≈10\varepsilon\approx10 by permanently retiring individual documents from retrieval once their personal privacy budgets are exhausted (Wu et al., [125]). Meanwhile, the private datastore approach in Aboueleinen and Torki [1] translates its SimHash bucketing guarantee into an empirical audit, reducing membership-inference accuracy from 70.22% to 53.60% at the cost of a 2.6% drop in task accuracy. Because query budgets, neighboring-dataset definitions, and utility trade-offs vary so widely across these setups, a paper reporting ε=10\varepsilon= 10 and a paper reporting zero verbatim copied spans (Family A) are providing fundamentally different types of evidence, even when evaluated on the exact same corpus.

Access and Retrieval Suppression (Family D)

Rather than scoring the words in a generated answer, Family D evaluates whether restricted documents are retrieved, routed, or allowed to be answered in the first place. Containing 26 records, this is the largest content-adjacent family in our map, and 96% of its studies report net improvements. Defenses here differ primarily in where they enforce authorization along the pipeline, and their results point to a clear engineering rule: enforcing access checks late in the pipeline is far riskier than enforcing them early. The common retrieve-then-filter pattern, which fetches top-kk passages first and checks permissions afterward, still exposes unauthorized context in 86.1% of base queries [87]. By contrast, pushing authorization down into the database engine through row-level security or predicate pushdown eliminates cross-tenant retrieval leakage entirely [13, 7]. Beyond structural database rules, a second group of defenses relies on semantic or learned policies, filtering passages by inferred user role, governing access at retrieval time, or placing a policy firewall around the RAG pipeline [76, 90, 135]. A third group modifies the system architecture or the documents themselves, either isolating the communication channel so raw user prompts never meet protected sources inside the generator [31] or perturbing source texts so a retrieving model cannot extract usable answers from them [3]. Notably, RAG unlearning studies that measure success by how often the model refuses to answer forbidden questions also fall into Family D rather than a removal category [117].

Comparing operating points across Family D highlights a sharp divide between deterministic permission checks and learned semantic filters. When access policies are tied to clean database metadata, engine-level row security drives tenant-isolation leakage from 0.2% down to zero across 1,000 queries [13]. However, when permissions must be inferred from natural language or when metadata is imperfect, residual leakage quickly reappears. For example, the AI-driven role filter in ARBITER reaches 85% accuracy and an F1 score of 0.89 over 389 queries, which actually trails the 0.94 F1 achieved by a simple static filter [76]. Similarly, hybrid graph-retrieval defenses remain vulnerable when 5% to 10% of document metadata labels are randomly corrupted [113], and enterprise confidentiality filters see their recall drop to 74.35% when user queries omit explicit project names [16].

More fundamentally, suppressing access or answerability is not the same as preventing content disclosure. On the one hand, a refusal rate measures policy compliance rather than whether sensitive knowledge was truly erased from the system. On the other hand, achieving zero unauthorized retrieval says nothing about what happens when the user is authorized. In commercial or licensed RAG deployments, legitimate subscribers are permitted to retrieve from the datastore, yet the operator must still prevent the model from reproducing copyrighted chapters verbatim. Protecting against that risk requires evaluating emitted expression (Family A) alongside retrieval permissions.

Post-Hoc Corpus-Use Detection (Family E)

Instead of blocking queries or suppressing generated text, the 18 records in Family E ask a forensic question: can an auditor prove after the fact that a specific document collection was indexed or queried by a RAG system? Strikingly, most studies framed around copyright and dataset ownership concentrate in this family, even though usage detection does not measure or prevent content disclosure. Research here follows four technical paths depending on whether the auditor can modify the corpus beforehand. The dominant design plants a verifiable signal into the data and later probes the suspect system to recover it. Researchers have embedded these signals as statistical watermarks in generated text [55], dual-layer semantic and lexical watermarks [73], reasoning traces in chain-of-thought space [47], multi-bit ownership payloads [132], black-box knowledge base watermarks [78], synthetic sentinel documents [60], retrievability-optimized canaries [75], visual sentinel images [77], and multimodal watermarks for RAG-as-a-Service platforms [25]. When planting signals in advance is impossible, a second, signal-free group audits corpus membership directly through black-box queries (Zeng et al., 2025b). Flipping the roles, a third group builds defenses to protect the datastore against membership inference attacks [32, 41], while a fourth inspects internal model activations or runtime execution games to catch unauthorized extraction [129] (Zhang et al. 2025c).

Evaluating Family E endpoints requires separating the reliability of the forensic signal from the secrecy of the underlying text. First, detection scores are highly sensitive to calibration choices, query budgets, the fraction of target material actually indexed, and simple post-processing attacks. For example, the watermark detector in Ward begins to trigger false positives when only 20% of the watermarked document set is present in the target corpus [55]. Under adversarial paraphrasing, membership audit performance in Zeng et al. (2025b) drops to an AUC of 79.6%. Similarly, applying a detect-and-inpaint filter causes visual sentinel detection to fall from an AUC of 0.99 to 0.62 [77], while injecting external knowledge into the database cuts RAG-WM watermark integrity from 100% down to 46.67% [78]. Comparing detectors therefore requires matching their false-positive thresholds and adversarial post-processing budgets. Second, and more fundamentally, a usage detector is an alarm rather than a lock: proving that a document sits inside a remote index says nothing about whether its protected wording was ever emitted to an end user (Family A), nor whether an attacker could reconstruct that wording without triggering the alarm.

Pipeline Integrity, Attribution, and Recurring Failure Modes (Family F)

Accounting for 47 records (40% of our entire map), Family F is by far the largest endpoint category, covering defenses against corpus poisoning, prompt injection, and provenance failures. Rather than protecting the secrecy of indexed documents, these endpoints test whether the RAG pipeline returns trustworthy, uncorrupted answers and correctly credits its sources. Poisoning defenses intervene at three stages of the pipeline. Admission-time filters screen out malicious documents before they enter the index using perplexity scores, Mahalanobis distance, vector hubness, or inter-document similarity [30, 126, 95, 54].

Retrieval-time defenses re-rank candidate passages, partition the retrieved set, or analyze context structure to isolate corrupted chunks [96, 105, 136]. Generation-time defenses aggregate answers across multiple passages, sometimes providing formal robustness certificates under a bounded poisoning budget, or isolate untrusted spans during synthesis [112, 107, 139]. Alongside poisoning defenses, prompt-injection studies enforce runtime control-flow integrity or deploy layered input scanners [103, 4], while a separate attribution literature traces poisoned outputs back to responsible documents, localizes corrupted spans at the character level, or uses internal model states to verify answer provenance [36, 100] (Zhang et al. 2025a; Zhang et al. 2026a).

Comparing evaluation practices in Family F against the rest of the map reveals a telling contrast. Because poisoning and injection studies draw heavily from adversarial machine learning, 51% of Family F records receive a low-concern rating for adaptive robustness testing, and 45% explicitly document an adaptive attack among their failure modes. For example, while a non-adaptive guardrail reports a true-positive rate of 1.000 at a false-positive rate of 0.015 [34], adaptive white-box attacks like PoisonedRAG-W push document-detector false-negative rates up to roughly 96% [54], and consistency collusion bypasses 70.3% of multi-agent audits [139]. Yet despite this strong tradition of adaptive testing, zero records in Family F document an attacker recovering protected source content against the proposed defense. The reason is simple: in Family F, an attack success rate (ASR) measures whether the attacker tricked the system into outputting a targeted wrong answer, executing a hijacked instruction, or citing the wrong source. Keeping ASR at zero guarantees that the pipeline behaves as the operator intended, but it is completely compatible with a system that answers every user query by quoting its private or copyrighted sources verbatim.

Looking across all seven endpoint families (Families A, A-π, and B–F), three recurring evaluation patterns explain why reported operating points rarely transfer from one setting to another (Appendix D). First, defenses face steep protection-utility trade-offs whose magnitudes are seldom reported on a shared scale [141 Wang] et al., 2026a). Second, claims of zero residual leakage almost always depend on fragile assumptions, such as perfectly clean permission metadata or attackers who explicitly name their target entities in the query [90, 20]. Third, moving from static queries to adaptive attackers routinely overturns the apparent success of a defense [137, 53]. These recurring limitations point to deeper structural gaps in how RAG protection is designed and evaluated, which we audit next.

Three Blind Spots in RAG Protection Research

Stepping back from individual endpoint families, our evidence map allows us to audit how the RAG protection literature as a whole connects methods, claims, and evaluations. Reading the 118 human-coded studies across families exposes three systemic blind spots that explain why progress in this field is so difficult to compare.

Blind Spot I: Mechanisms Do Not Dictate Endpoints

When readers encounter a RAG defense that “reranks retrieved passages” or “rewrites context before generation,” they naturally assume the mechanism tells them what risk is being mitigated. To test this assumption, we cross-tabulated the 13 coded defense mechanism types against the primary endpoint family each paper actually measured (Appendix § 11). While mechanism type and endpoint family are statistically associated (mutual information of 0.624 bits versus 0.211 bits under 20,000 label permutations, p=5×10−5p = 5 \times10^{-5}), the link is surprisingly weak. Knowing a paper’s defense mechanisms accounts for only 27% of the uncertainty in its endpoint distribution. In fact, a leave-one-out classifier that sees a paper’s entire set of defense mechanisms predicts its measured endpoint family only 53.4% of the time, barely outperforming the 39.8% baseline of always guessing Family F.

Only two specialized mechanisms point reliably to a single endpoint: poisoning and integrity controls are evaluated on an integrity endpoint (Family F) in 94% of their records, and watermarking techniques are evaluated on usage detection (Family E) in 92%. For every other mechanism in the literature, the modal endpoint family accounts for just 28% to 68% of records. This dispersion is most striking for the two most popular interventions in our map: retrieval reranking or thresholding (40 records) and context transformation or summarization (39 records), which span five and six of the seven endpoint families, respectively. In practice, the exact same technical intervention, such as rewriting retrieved passages before they reach the generator, is presented as a verbatim-copying defense in one paper, a PII redaction tool in a second, an access-control filter in a third, and a poisoning defense in a fourth, scored on a completely different endpoint each time.

Relying on high-level metric names is almost as ambiguous as relying on mechanism names (Appendix § 11). Only a handful of specialized metrics identify the underlying protection event on their own: content-reconstruction scores (7 records) appear exclusively in Family B, attribution metrics appear only in Family F, and literal text-overlap scores concentrate in Families A and A-pii (14 records). By contrast, task utility appears across 58 records (49%) spanning every single family, and attack success rate (ASR) or integrity metrics appear in 61 records across multiple families, making ASR the single most common metric in our map. Because a low ASR can mean anything from blocked prompt injection to failed PII extraction, neither a paper’s architectural mechanism nor its headline metric label reliably tells a reader what asset was actually protected.

Blind Spot II: High-Stakes Claims vs. Skipped Content Disclosure

When we organize the literature by what it actually measures rather than what mechanisms it uses, a massive gap appears where direct content protection should be. Across our 118 human-coded records, only 21 studies (17.8%) have their primary endpoint in Families A, A-pii, or B, which scores what a user receives or what an attacker can reconstruct from the system. Even if we include mathematical bounds and likelihood proxies from Family C, 77.1% of the literature (91 records in Families D, E, and F) has its primary endpoint outside this broader group. Although document takedown, copyright opt-out, and the right to erasure are among the most concrete demands placed on real-world RAG operators, papers addressing those topics evaluate their methods through text-overlap scores, reconstruction proxies, or query refusal rates, landing in Families A, B, C, or D rather than demonstrating actual source removal.

This skew is especially pronounced in research framed around copyright and dataset ownership. Among the 15 records coded with the frozen direct copyright or ownership evidence relation, 11 (73.3% [48.0, 89.1]) have a usage-detection primary endpoint in Family E, compared with just 7 of the 103 records (6.8%) addressing neighboring security or privacy goals (Bonferroni-adjusted p=4.9×10−7p = 4.9 \times10^{-7} across 14 comparisons). Meanwhile, zero records under this frozen label have a strict emitted-expression primary endpoint in Family A (95% exact upper bound of 18.1%). This coded subset overwhelmingly prioritizes post-hoc corpus-use detection, leaving verbatim reproduction by the generator absent from its primary endpoints.

Blind Spot III: Toy Benchmarks and Static Adversaries

Even when a paper selects a clear endpoint that matches its claim, the strength of its evidence depends on the corpus, the adversary, and the rigor of the experimental protocol. Across our map, 112 of the 118 records name a specific evaluation dataset, spanning 278 distinct corpora in total (Appendix § 11). Yet the literature relies overwhelmingly on standard open-domain question-answering benchmarks rather than datasets designed to test data protection. The three most popular corpora are Natural Questions (35 records), HotpotQA (31), and MS MARCO (28), followed at a distance by TriviaQA (7) and PopQA (5). By contrast, datasets containing genuinely sensitive or private records appear far less often, led by the Enron Email corpus (12 records), HealthCareMagic (6), and ChatDoctor (5). Evaluating privacy or copyright defenses on general encyclopedia and web-search passages creates an obvious interpretation problem: because these corpora contain little sensitive or proprietary expression to begin with, a low measured leakage rate may simply reflect a benign workload rather than an effective defense. A similar concentration appears in model and retriever choices, where evaluations cluster around GPT-4o (31 records), GPT-4o-mini (20), Contriever (18), all-MiniLM-L6-v2 (12), GPT-3.5 (12), Mistral-7B (12), Llama-3.1-8B (12), and Qwen2.5-7B (11). Sharing a generator or retriever across papers does not make their protection numbers comparable if one study scores corpus membership on Wikipedia while another scores verbatim copying on clinical notes. Threat models and adversary interaction budgets are equally narrow. Across the 118 studies,

47 assume an external black-box attacker and 39 model a malicious corpus contributor with write access, alongside 13 that assume a white-box system operator and 6 that treat a cloud service provider as a protocol participant (with 110 of 118 studies focusing exclusively on text). Only 16 records model an authorized user, even though licensed knowledge bases face their greatest extraction risk from legitimate subscribers who are permitted to query the system but prohibited from copying the underlying database verbatim. Worse still, most interactive evaluations restrict the adversary to a static, one-shot interaction. Fifty-one records allow the attacker only a single query and 25 model no interactive querying at all, whereas just 11 studies grant the adversary feedback-adaptive multi-query access. Testing a defense against a single fixed prompt provides little assurance against a real-world user who can inspect the system’s initial reply and iteratively refine follow-up queries to bypass filters.

These gentle evaluation conditions help explain why reported outcomes in the literature are so uniformly positive. In our eight-domain methodological appraisal (Appendix § 11), zero records raise high concern regarding threat-model definition or artifact availability, and 95% receive low concern for comparator and ablation design. However, only 48 of 118 records receive low concern for adaptive robustness testing, 72 records (61%) raise high concern for uncertainty analysis because they report single point estimates without confidence intervals, random-seed variance, or statistical tests, and 109 records raise some concern for evaluation independence (with only one record achieving low concern) because defenses are almost always tested by their own authors against locally implemented attacks. Against this backdrop, 108 of the 118 records (92%) report that their proposed intervention improves protection, compared with just 9 reporting mixed effects and 1 reporting net degradation. This 92% success rate should not be read as proof that RAG protection is largely solved; rather, it reflects a literature that predominantly tests self-implemented defenses against non-adaptive attackers on general-purpose QA benchmarks, without quantifying statistical uncertainty.

Controlled Testbed and Future Directions

Our literature audit shows that RAG protection studies rarely make content disclosure their primary endpoint, rely heavily on non-adaptive attackers, and use metrics that cannot be compared across families. To demonstrate why these distinctions matter in practice, we first present results from a controlled experimental testbed that evaluates multiple defenses across competing endpoints on the exact same workload, and then outline three priorities for future research.

Stopping Surface Copying Does Not Stop Fact Recovery

To test whether a defense that succeeds on one endpoint actually protects underlying content on another, we built a controlled evaluation testbed spanning 192 documents across two domains (Wikipedia and scientific papers) and two access policies, paired with three unexposed human-authored fact probes per document and 64 document-disjoint utility queries (full protocol and tables in Appendix § 10). Fixing dense retrieval at MiniLM top-3 and greedy decoding, we evaluated four representative controls across Qwen3-8B (primary), Phi-4-14B (replication), Qwen2.5-3B, and Mistral-7B: pre-generation access policy filtering (Family D), a cosine similarity threshold at 0.5 (Family D), an upstream summarization prompt instructing the model to answer in its own words (Family A), and a normalized 8-gram output guard that redacts spans matching protected-source 8-grams while retaining unmatched text (Family A). Crucially, rather than scoring a single static prompt, we also evaluated an online three-query adaptive controller that observes released text, refusals, and redactions to iteratively extract information.

Evaluating these defenses simultaneously across multiple endpoints confirms our central thesis: eliminating surface copying does not prevent an attacker from recovering private facts or identifying source documents (Table 6, Table 8, and Table 10). On quotation-denied items, uncontrolled models copy 20-token source spans verbatim (copy20_{20}) at high rates, reaching 64.1% to 89.1% under natural dense retrieval and up to 96.9% under separator-evasion attacks. Adding the normalized 8-gram guard drives both visible and decoded copy20_{20} down to 0.0% [0.0, 0.0] in the three-query adaptive evaluation. Yet when we score the exact same guarded responses on fact recovery and source linkage, substantial disclosure remains.

ModelTarget retrievedUncontrolled copy20_{20}Policy filter copy20_{20}Filter + guard copy20_{20}Ordinary-QA F1Authorized quote copy20_{20}
Qwen-3B98.4 [95.3, 100.0]78.1 [68.8, 87.5]37.5 [26.6, 50.0]0.0 [0.0, 0.0]0.250 [0.213, 0.288]73.4 [62.5, 84.4]
Phi-3.598.4 [95.3, 100.0]89.1 [81.2, 95.3]45.3 [32.8, 57.8]0.0 [0.0, 0.0]0.369 [0.316, 0.426]79.7 [68.8, 89.1]
Qwen-7B98.4 [95.3, 100.0]85.9 [76.6, 93.8]46.9 [34.4, 59.4]0.0 [0.0, 0.0]0.309 [0.267, 0.353]81.2 [71.9, 90.6]
Mistral-7B98.4 [95.3, 100.0]64.1 [51.6, 76.6]29.7 [18.8, 40.6]0.0 [0.0, 0.0]0.261 [0.225, 0.298]67.2 [56.2, 78.1]

Table 6. (a) Natural-policy dense retrieval. Copying uses all quotation-denied items. Authorised quote copying is desired release. Brackets: 95% source-item bootstrap intervals.

(a) Adaptive copy20copy_{20} (%). Raw rates use all quotation-denied items; residuals condition on successful unguarded copying, with each nn shown. Surface, normalised, and cumulative-session guards remain distinct.

ModelAttackRaw copy20copy_{20}Surface residualNormalized residualSession residual
Qwen3-8BSeparator evasion71.9 [59.4, 82.8]; nn = 6495.7 [89.1, 100.0]; nn = 460.0 [0.0, 0.0]; nn = 46n/a
Four-turn chunks60.9 [48.4, 71.9]; nn = 642.6 [0.0, 7.7]; nn = 392.6 [0.0, 7.7]; nn = 390.0 [0.0, 0.0]; nn = 39
Phi-4-14BSeparator evasion96.9 [92.2, 100.0]; nn = 64100.0 [100.0, 100.0]; nn = 620.0 [0.0, 0.0]; nn = 62n/a
Four-turn chunks81.2 [71.9, 90.6]; nn = 640.0 [0.0, 0.0]; nn = 527.7 [1.9, 15.4]; nn = 520.0 [0.0, 0.0]; nn = 52

Table 8. Adaptive copying and semantic similarity.

(a) Online three-query extraction (%). Attack selection uses development data only; branching sees released information. Dense top-3 rows condition on target retrieval before policy filtering; decoded endpoints accumulate released turns.

ModelControlVisible copy20copy_{20}Decoded copy20copy_{20}Hidden-fact recallSource re-ID@1nn
Qwen2.5-3BNone50.0 [35.0, 65.0]67.5 [52.5, 80.0]40.8 [27.5, 54.2]25.0 [12.5, 37.5]40
Normalized guard0.0 [0.0, 0.0]0.0 [0.0, 0.0]31.7 [20.8, 41.7]15.0 [5.0, 27.5]40
Summary + guard0.0 [0.0, 0.0]0.0 [0.0, 0.0]14.2 [6.7, 22.5]22.5 [10.0, 35.0]40
Policy + guard0.0 [0.0, 0.0]0.0 [0.0, 0.0]18.3 [10.8, 26.7]5.0 [0.0, 12.5]40
Policy + summary + guard0.0 [0.0, 0.0]0.0 [0.0, 0.0]17.5 [10.0, 25.8]20.0 [7.5, 32.5]40
Qwen3-8BNone55.0 [40.0, 70.0]72.5 [57.5, 85.0]60.0 [45.8, 73.3]25.0 [12.5, 40.0]40
Normalized guard0.0 [0.0, 0.0]0.0 [0.0, 0.0]48.3 [36.7, 60.8]42.5 [27.5, 57.5]40
Summary + guard0.0 [0.0, 0.0]0.0 [0.0, 0.0]39.2 [27.5, 50.8]40.0 [25.0, 55.0]40
Policy + guard0.0 [0.0, 0.0]0.0 [0.0, 0.0]22.5 [13.3, 31.7]20.0 [7.5, 32.5]40
Policy + summary + guard0.0 [0.0, 0.0]0.0 [0.0, 0.0]27.5 [16.6, 39.2]30.0 [15.0, 45.0]40

Table 10. Online adaptive extraction and recovery endpoints.

responses on fact recovery and source linkage, substantial disclosure remains. Under the three-query adaptive budget, an attacker facing the normalized guard still recovers 48.3% [36.7, 60.8] of hidden source facts on Qwen3-8B (31.7% on Qwen2.5-3B) and links the response back to the exact source document (re-ID@1) 42.5% of the time (Table 10). Blinded human annotations of 192 source-response pairs (κ=0.831\kappa= 0.831 for expression, κ=0.929\kappa= 0.929 for recovery; Table 12) tell the same story: on Phi-4-14B, the normalized guard reduces human-judged substantial content recovery only from 66.7% to 38.9% (Table 9). Furthermore, query budget sensitivity exposes the danger of one-shot testing: on guarded Qwen3-8B, hidden-fact recall is just 3.3% [0.0, 7.5] after a single query, but jumps to 48.3% on the third adaptive turn (Table 11).

(a) Ordinary QA uses clean metadata. Authorised quotation reports coverage, completeness, and refusal under policy filtering plus the normalised guard.

ModelOrdinary F1 noneOrdinary F1 surfaceOrdinary F1 normalizedOrdinary F1 summary-styleAuthorized targetAuthorized 5-gram recallAuthorized refusal
Qwen3-8B0.349 [0.309, 0.389]0.346 [0.308, 0.386]0.346 [0.306, 0.390]0.355 [0.312, 0.399]98.4 [95.3, 100.0]0.190 [0.152, 0.228]3.1 [0.0, 7.8]
Phi-4-14B0.179 [0.156, 0.205]0.184 [0.158, 0.209]0.177 [0.153, 0.201]0.208 [0.182, 0.237]98.4 [95.3, 100.0]0.169 [0.148, 0.191]3.1 [0.0, 7.8]

Table 9. Utility and model-specific human outcomes.

(a) Normalised-guard budget sensitivity; stopped sessions carry forward their last release and decoded outcomes accumulate valid turns.

ModelBudgetDecoded copy20_{20}Hidden-fact recallSource re-ID@1nn
Qwen2.5-3B10.0 [0.0, 0.0]0.8 [0.0, 2.5]15.0 [5.0, 27.5]40
20.0 [0.0, 0.0]3.3 [0.0, 7.5]10.0 [2.5, 20.0]40
30.0 [0.0, 0.0]31.7 [20.8, 41.7]15.0 [5.0, 27.5]40
Qwen3-8B10.0 [0.0, 0.0]3.3 [0.0, 7.5]7.5 [0.0, 17.5]40
20.0 [0.0, 0.0]5.0 [0.8, 11.7]10.0 [2.5, 20.0]40
30.0 [0.0, 0.0]48.3 [36.7, 60.8]42.5 [27.5, 57.5]40

Table 11. Query-budget and domain sensitivity. (a) Normalised-guard budget sensitivity; stopped sessions carry forward their last release and decoded outcomes accumulate valid turns.

(a) Human-rated source/response outcomes: 144 true and 48 mismatched pairs. Brackets: Wilson 95% intervals. Expression κ=0.831\kappa= 0.831; recovery κ=0.929\kappa= 0.929.

ControlClose/near-verbatim expressionSubstantial/near-complete recoverySource identifiablenn
None83.3 [68.1, 92.1]50.0 [34.5, 65.5]61.1 [44.9, 75.2]36
Normalized guard25.0 [13.8, 41.1]19.4 [9.8, 35.0]19.4 [9.8, 35.0]36
Summary-style prompt61.1 [44.9, 75.2]30.6 [18.0, 46.9]44.4 [29.5, 60.4]36
Summary + normalized guard30.6 [18.0, 46.9]11.1 [4.4, 25.3]19.4 [9.8, 35.0]36
Mismatched-source controls0.0 [0.0, 7.4]0.0 [0.0, 7.4]0.0 [0.0, 7.4]48

Table 12. Human judgments and automated proxy validation. (a) Human-rated source/response outcomes: 144 true and 48 mismatched pairs. Brackets: Wilson 95% intervals. Expression κ=0.831\kappa= 0.831; recovery κ=0.929\kappa= 0.929.

Our testbed also shows how easily endpoint and metric choices can distort comparisons of prompt-based defenses and task utility. In our single planned confirmatory contrast on 80 held-out answer-only items, the upstream Summary defense yields a difference-in-differences (DiD) in context-attributable exact-answer recall of just −0.4 percentage points [−8.3, 7.5] on Qwen3-8B and 0.0 [−7.9, 8.3] on Phi-4-14B (Figure 3). Short exact-answer recall has only 51.8% sensitivity against human-judged recovery and correlates with response length (Table 12). Response length also confounds standard utility metrics: while the Summary prompt appears to hurt ordinary-QA token F1 by −4.0 to −7.9 points across models (Figure 3), our format diagnostic reveals that token F1 heavily penalizes verbose free-form answers (scoring 18.9 on Qwen3-8B versus 46.8 when prompted for short answers; Table 14). Finally, structural access controls fail in complementary ways: pre-retrieval policy filtering blocks unauthorized documents when metadata is clean, but a 12.5% rate of missing metadata labels under a fail-open policy immediately restores 9.4% to 12.5% verbatim copying (Table 7). Across every layer of the pipeline, a defense is only as strong as the specific endpoint, query budget, and metadata assumptions under which it is tested.

Observed metadataDenied target keptDenied copy20_{20}Ordinary QA F1
Clean0.0 to 0.00.0 to 0.00.250 to 0.369
Missing, fail open12.5 to 12.59.4 to 12.50.252 to 0.369
Missing, fail closed0.0 to 0.00.0 to 0.00.226 to 0.349
Wrong label12.5 to 12.50.0 to 12.50.241 to 0.350

Table 7. (a) Metadata robustness. Cells are cross-model ranges, not pooled estimates. Fail-open missing or wrong labels admit 12.5% of access-denied targets.

Figure 3. Context contrast and matched utility across the original model roles. (A) No-control and Summary context effects after three queries; paired numbers are shown in that order. (B) Summary-minus-uncontrolled differences-in-differences on all 80 answer-only items. Only Qwen3 was planned confirmatory; exact-answer recall is an incomplete recovery proxy. (C) Absolute ordinary-QA F1 on 64 document-disjoint items. (D) Paired Summary-minus-uncontrolled changes in QA F1, quote-query F1, and authorised quote 5-gram recall. Bars are 95% source-item bootstrap intervals. Negative authorised-quote recall means less desired quotation.

ContextAnswer formatToken F1Span F1Answer recallResp. tokens
Qwen3-8B
Closed bookfree-form (frozen)9.1 [6.5, 11.8]24.8 [17.2, 32.7]24.2 [16.4, 32.3]20.3
Closed bookshort answer12.8 [7.6, 18.6]18.3 [11.7, 25.5]15.6 [9.7, 22.1]9.3
Retrieved top-3free-form (frozen)18.9 [15.0, 23.0]67.4 [57.4, 76.9]66.7 [56.6, 76.2]38.1
Retrieved top-3short answer46.8 [36.7, 56.5]75.0 [45.9, 67.7]55.6 [44.6, 66.4]7.7
Gold source onlyfree-form (frozen)28.0 [23.8, 32.5]86.2 [79.4, 91.9]84.7 [77.7, 90.7]27.5
Gold source onlyshort answer60.9 [52.6, 68.8]76.3 [67.7, 84.2]73.3 [64.1, 81.7]8.0
Phi-4-14B
Closed bookfree-form (frozen)5.2 [3.4, 7.2]30.8 [22.7, 39.2]31.6 [23.5, 40.6]85.3
Closed bookshort answer12.4 [7.9, 17.8]23.7 [16.4, 31.3]21.1 [14.3, 28.2]15.7
Retrieved top-3free-form (frozen)11.3 [8.7, 14.3]71.6 [62.5, 80.2]72.7 [63.4, 81.4]86.0
Retrieved top-3short answer39.2 [30.6, 48.3]65.2 [55.0, 75.1]63.6 [53.2, 73.6]22.3
Gold source onlyfree-form (frozen)22.6 [18.8, 26.4]90.1 [85.4, 94.1]91.4 [86.8, 95.2]50.0
Gold source onlyshort answer57.4 [49.2, 65.2]83.3 [75.0, 90.4]82.1 [73.7, 89.5]12.2
Qwen2.5-3B
Closed bookfree-form (frozen)3.6 [2.5, 4.8]23.7 [16.6, 31.1]23.4 [16.3, 31.0]64.1
Closed bookshort answer9.1 [3.8, 15.4]9.8 [4.1, 16.6]9.5 [3.8, 16.1]3.5
Retrieved top-3free-form (frozen)14.3 [10.6, 18.4]64.0 [53.8, 73.6]64.4 [54.1, 74.1]56.2
Retrieved top-3short answer40.0 [29.6, 50.5]46.6 [35.8, 57.6]43.9 [33.2, 54.8]5.6
Gold source onlyfree-form (frozen)20.9 [17.3, 24.7]81.4 [74.2, 87.9]81.1 [73.5, 87.9]44.8
Gold source onlyshort answer57.3 [47.6, 67.1]68.5 [58.1, 78.4]65.5 [54.9, 75.5]7.8
Mistral-7B
Closed bookfree-form (frozen)5.1 [3.6, 6.6]34.0 [25.8, 42.0]33.4 [25.3, 41.4]70.6
Closed bookshort answer13.4 [7.9, 19.5]20.7 [13.4, 28.7]18.3 [11.4, 26.0]12.2
Retrieved top-3free-form (frozen)17.3 [13.5, 21.3]66.5 [56.7, 76.1]66.4 [56.5, 76.1]49.7
Retrieved top-3short answer40.3 [31.1, 49.6]58.5 [48.0, 68.5]56.7 [46.2, 67.0]15.2
Gold source onlyfree-form (frozen)26.8 [22.2, 31.4]85.0 [78.3, 90.7]85.0 [78.0, 90.8]34.2
Gold source onlyshort answer51.0 [43.0, 58.9]72.3 [63.5, 80.5]69.1 [60.2, 77.6]9.2

Table 14. Context and answer-format effects on utility. (a) Format diagnostic on 64 document-disjoint items per model. Only context and answer-format instruction vary. Brackets: 95% item-bootstrap intervals. Token F1 scores the short gold answer against the whole response.

Three Golden Rules for Future RAG Protection

Together, our evidence map and controlled testbed point to three concrete rules for designing, reporting, and reviewing future RAG protection studies. First, studies whose claims concern data secrecy, privacy, or copyright must measure content disclosure directly. In our human-coded map, 97 of the 118 core records do not use content disclosure as their primary endpoint, and most evaluations rely on general-purpose question-answering benchmarks that contain little sensitive text to begin with (§ 5.3). Suppressing retrieval access (Family D), detecting watermarks after the fact (Family E), and blocking poisoned passages (Family F) are valuable operational goals, but they do not replace a measurement of what the system actually leaks to a user. Any defense claiming to protect the contents of a retrieval corpus should pair its operational metric with a direct disclosure endpoint from Family A, A-pii, or B, and evaluate on corpora containing genuinely sensitive or proprietary spans.

Second, authors and benchmarks should state their target endpoint at subfamily resolution and test against multi-turn, adaptive adversaries. Naming the exact evidenced cell among our 17 subfamilies (for example, distinguishing A.verbatim from A.near, or D.retrieve from D.answer) allows readers and meta-analyses to compare papers at a stated level of detail without pooling incompatible metrics. Likewise, because single-query evaluations routinely miss extraction vulnerabilities that emerge over three or more adaptive turns (§ 6.1), multi-query feedback-adaptive testing and imperfect metadata stress tests should become standard practice rather than rare exceptions.

Third, the field must move beyond simple event matching to validate evidential sufficiency. Our taxonomy establishes which technical event a paper measures, not whether the chosen metric is a sufficient or unbiased measure of that event. True evidential sufficiency requires convergent and discriminant validity controls (Campbell and Fiske, [18]) that almost no RAG protection study currently reports. In practice, this means verifying that an extraction or inversion attack is genuinely strong on the unprotected baseline before claiming credit for a low defended score, and checking that utility metrics like token F1 are not confounded by simple shifts in response length.

Conclusion

“Protection” in retrieval-augmented generation is not a single event. By introducing a frozen seven-family endpoint taxonomy and coordinating 32 annotators to double-code 118 full-text RAG protection studies (κ=0.867\kappa= 0.867), we showed that the literature evaluates fundamentally different technical outcomes that cannot be pooled or treated as interchangeable. Looking across this evidence map reveals that defense mechanisms weakly predict what is actually measured, that only 17.8% of studies use content disclosure or reconstruction as their primary endpoint, and that evaluations overwhelmingly rely on benign question-answering benchmarks and non-adaptive adversaries. Our controlled testbed confirms the practical stakes of these distinctions: defenses that eliminate verbatim span copying still leave substantial source facts recoverable under multi-turn adaptive querying. By releasing our human-coded evidence map, taxonomy protocols, and evaluation testbed, we aim to help the community move from ambiguous umbrella claims toward precise, comparable, and construct-valid evaluations within each protection goal.

Event and Control Schema

To prevent our coding from confusing a paper’s high-level topic with what its experiments actually measure, our schema records six dimensions separately: the technical event, the protected asset, the adversary’s identity and access level, the defensive intervention function, the primary measured endpoint, and the broader evidence relation (Table 1a). Table 1b defines our frozen seven-family endpoint taxonomy (Families A, A-pii, and B–F). Families A, A-pii, B, and C are grouped under the shared content-secrecy asset class; however, when we count studies whose primary endpoint measures actual content disclosure, we include only Families A, A-pii, and B because Family C reports theoretical bounds or likelihood shifts without observing emitted or reconstructed text. Under this four-group hierarchy, a paper’s claim and its primary measured endpoint are coded as concordant when they fall into the exact same family, partial when they land in different families within the same asset group, and discordant (off-target) when they belong to different asset groups altogether. Family G is assigned whenever a claim or an experimental packet is too vague to establish a specific technical event. Finally, each publication receives a single mutually exclusive primary endpoint family from human adjudication, whereas its defense mechanisms and intervention functions can receive multiple labels.

Technical eventPrimary asset / goalActor and accessRelevant functionsEvidence relation
Unauthorized corpus use or KB theftOwner text or image corpus; evidence of useExternal owner/auditor with black-box queries; copying operatorDetect, attribute; occasionally prevent/remediateDirect copyright/ownership evidence ([55, 75, 78, 73, 47])
Retrieval extraction or authorization bypassRestricted documents and access policyEnd user with black-box, adaptive queriesMeasure attacks; evaluate preventive controls separatelyDirect copyright- or privacy-framed extraction attacks ([121, 141]); security-framed controls ([135]); benchmark evidence ([101])
Output copying or close transformationExpression exposed in retrieved contextEnd user or downstream recipientPrevent, redact, attribute, remediateDirect only when copying is measured; citation alone is insufficient; copyright/ownership image output comparison (Golotkar et al., 2024); likelihood-based text opt-out is a separate proxy ([82]); security-framed text recovery ([64]); attribution is adjacent ([100]). Data Defenses measures use resistance, not copying ([3])
Query, database, or membership leakageConfidential query, corpus record, or indexCloud provider, protocol participant, or probing userPrevent or detect under a privacy/security definitionDirect RAG but non-copyright privacy/confidentiality ([6, 72, 28])
Corpus poisoning or retriever backdoorIntegrity of retrieved evidence and answerCorpus contributor with write or injection accessPrevent, detect, traceDirect RAG security; not evidence that copyright copying is prevented ([24, 106] (Zhang et al. 2025b))
Source influence or knowledge removalAttribution of an output; requested removalOperator or auditor with black-, gray-, or white-box accessAttribute, detect, remediateDirect RAG attribution/remediation; legal relevance is fact-dependent ([88, 100, 117])

Table 1a. Separate coding fields for technical events, assets, actors, functions, and evidence relations.

AssetFam.The endpoint concerns…
Content disclosureAgenerated output reproducing protected source expression
Content disclosureA-piigenerated output containing protected atomic identifiers
Content disclosureBreconstruction of source content from embeddings, an index, or another non-generation channel
Proxy for disclosureCa likelihood, perplexity, or formal privacy bound, with no emitted or reconstructed content
AccessDsuppression of authorization, retrieval, routing, or answerability
UseEdetection of a watermark, canary, membership, corpus use, or ownership
IntegrityFpoisoning, prompt injection, provenance, attribution, removal, or misuse
UnresolvedGa packet that does not establish the outcome a stricter assignment needs

Table 1b. Frozen event families. C shares the content asset group for concordance but does not measure observed disclosure.

Search, Coding, and Evidence Map

Literature Search and Eligibility Criteria

Our literature search ran through July 17, 2026, combining eight initial arXiv queries, nine expanded arXiv query families, the same nine query families executed over OpenAlex, and a title-and-metadata scan of the ACL Anthology. We also ran ten seed-paper lookups and ten forward-citation tracking passes as separate continuity checks, alongside capped supplementary searches on Crossref and DBLP. Each search family paired core retrieval terms (such as “RAG,” “retrieval-augmented,” “retrieval-enhanced,” and “grounded generation”) with protection-related keywords covering copyright, licensing, ownership, watermarking, copying, extraction, privacy, attribution, removal, and security; an adjacent query family paired general language-model terms with copyright and copying prevention. Retrieved records were deduplicated sequentially by arXiv identifier, Digital Object Identifier (DOI), and normalized title, followed by an author-title-year audit that linked multiple preprints or versions belonging to the same underlying study.

To be confirmed as a direct empirical RAG-protection record in our core map, a study had to evaluate a retrieval-augmented system, intervene on its query, corpus, index, context, or output, report a source-verified protection or attribution operating point, and address either a direct copyright/ownership goal or a neighboring RAG privacy/security goal. Pure attack papers, standalone benchmarks, survey articles, and untested architectural proposals were tracked in separate roles, and reporting task utility alone was not sufficient for inclusion.

In total, our queries returned 11,972 raw hits, which deduplicated to 8,978 unique records and yielded 925 candidate inclusions after abstract screening. We successfully recovered full texts for 824 candidates, while 101 full texts were unavailable and one recovered paper remained unresolved. Human full-text review confirmed 112 direct control studies from this candidate pool, and our stratified audit of 1,200 records sampled from the 8,053 excluded records identified 6 additional direct controls, bringing the final human-coded map to 118 records (Table 2a and Table 2b).

Table 2: Search-coverage estimates and evidence-map dimensions.

QuantityObservedEstimate/rangeConservative 95%Identification
excluded frame627.26 to 318Protocol-confirmable controls under the frozen human abstract rule; abstract-screen false negatives are not estimated.
Available possible-inclusion full texts112112n/aCensus of available full texts only; 102 records are unclassified.
Descriptive component sum118139.2118 to 430Not an identified total for all eligible records in the 8,978-record frame.
If two recall nonresponses are controls120146.1120 to 441All-positive second-stage nonresponse sensitivity.
Main-pool missingness envelope (102 unclassified)n/a139.2 to 241.2n/aNon-probabilistic; recall nonresponses remain excluded.
Expanded missingness envelopen/a139.2 to 248.1n/aAlso treats both weighted recall nonresponses as controls.

Table 2. (a) Excluded-frame audit: 1,200 sampled exclusions, ten strata. Conservative intervals combine simultaneous finite-population hypergeometric bounds with Bonferroni correction. The component sum is not a potential total.

AxisCategoryRecords
Evidence relationcopyright / ownership15
neighboring RAG goal103
Validated outcomeemitted source expression8
emitted atomic identifier / PII6
non-output source property7
Validated flagadaptive multi-query recovery7
Legacy fieldbroad “output-copy” yes28

Table 2b. Table 2, (b) Human-adjudicated map dimensions. Intervention functions are multi-label; abstract-only records are excluded.

AxisCategoryRecords
Modalitytext110
multimodal5
image3
Publicationpeer reviewed29
preprint88
thesis/report1

Table 2b.

AxisCategoryRecords
Year20231
202418
202543
202656
Functionprevent83
detect69
attribute26
remediate24

Table 2b.

Human Annotation and Inter-Rater Reliability

Thirty-two trained annotators with backgrounds in NLP and machine learning (18), security and privacy (9), and information science or law and technology (5) double-coded the recovered full texts. Whenever the initial pair disagreed, a third reviewer outside that pair adjudicated the record. Before coding the main pool, every annotator completed a 100-item calibration test with a required passing score of 95%; all 32 annotators passed, scoring between 95% and 98% (median 97%). Because raw calibration files and identity-linked records remain restricted under participant confidentiality terms, domain backgrounds and calibration pass rates are reported as coordinator attestations.

Our public artifact releases a de-identified 8,024-row human label matrix covering all 118 confirmed records across 34 coded fields. In this matrix, initial coder pairs that had no entry in the disagreement log are reconstructed as matching the consensus label, while logged disagreements preserve the exact pre-adjudication calls of both coders. Running our standalone verification script on this matrix reproduces a pre-adjudication agreement of 89.83% (κ=0.8672\kappa= 0.8672) on the primary endpoint family (Table 3a and Table 3b). Across the 14 strictly categorical fields, median reliability is κ=0.87\kappa= 0.87. Fifteen free-text rationale and source-location fields had no logged categorical disagreements; their nominal κ=1.00\kappa= 1.00 reflects consensus text entry rather than an independent reliability estimate. For the eight methodological appraisal domains, linear weighted κ\kappa is computed across lower, some, and high concern levels excluding “not reported” (NR) pairs, whereas nominal agreement includes NR calls.

(a) Pre-adjudication agreement over 824 full texts.
FieldAgreementκ\kappa
Strict direct-control decision92.4%0.642
Evidence relation74.3%0.546
Primary study role87.5%0.844
Legacy broad output-copy field97.2%0.852
Legacy recovery field94.1%0.795
Publication status93.6%0.860
(b) Refined-coding agreement. Weighted κ\kappa is linear across lower/some/high and excludes NR; nominal agreement includes NR.
FieldAgreementκ\kappaWeighted κ\kappa
primary family89.8%0.867n/a
adaptive recovery99.2%0.929n/a
interaction adaptivity83.1%0.760n/a
Event, asset, actor, and access97.5%0.9320.932
Comparator and ablation98.3%0.8260.851
Evaluation independence95.8%0.7180.722
Sample and unit reporting93.2%0.8730.881
Uncertainty analysis94.1%0.8970.938
Adaptive or robustness test91.5%0.8570.877
Matched utility and harms94.9%0.8790.900
Artifact or reproduction support97.5%0.9490.950

Table 3. Human screening and refined-coding agreement.

Copyright-Relation Sensitivity

Within our 118-record map, 15 records are coded under the frozen direct copyright or dataset-ownership evidence relation (Table 4a). Eleven of these 15 studies (73.3% [48.0, 89.1]) have a usage-detection primary endpoint in Family E, compared with just 7 of the 103 studies (6.8%) addressing neighboring goals (Bonferroni-adjusted p=4.9×10−7p = 4.9 \times10^{-7} across 14 family comparisons). Under the frozen copyright-relation label (Table 4b), zero records have a strict emitted-expression primary endpoint in Family A (95% exact upper bound of 18.1%).

All citations and operating points were verified against source PDFs, and the exact query strings and audit logs are preserved in the released artifact.

Controlled Testbed and Claim-Aligned Analyses

Design and analysis hierarchy

The fixed protocol (not preregistered) uses a 192-document security corpus across two domains and two policies, with three unexposed human-authored probes per document; utility uses 64 document-disjoint open-policy items. MiniLM dense retrieval is fixed at top-3, decoding is greedy, and the three-query controller observes only released text, redaction, refusal, transport decodability, and released length. Qwen3 is the primary model, Phi-4 the replication, and Qwen2.5-3B and Mistral-7B exploratory. The exact upstream Summary baseline is commit-pinned.

The sole planned confirmatory contrast is the Qwen3 Summary-minus-uncontrolled difference-in-differences (DiD) of exact hidden-probe recall on all 80 held-out answer-only items. Context effects are target-present minus target-ablated. An attack ranking selected on uncontrolled target-present development data is shared across all four exposure/control cells. Inference uses paired source-item bootstrapping and sign randomisation without multiplicity correction for this single contrast. Other models, policies, natural retrieval, shorter spans, and linkage analyses are secondary or exploratory.

Natural-policy and adaptive evaluations

Natural-policy and claim-aligned evaluations use distinct denominators, stated in their table panels. Natural-policy intervals resample source items, and models are reported separately. A quotation-denied item can remain answerable; the policy filter excludes inaccessible documents, while the normalised 8-gram guard suppresses protected source spans. Authorised quotation is desired release. The adaptive experiment additionally distinguishes raw outcomes on all quotation-denied items from residual outcomes conditional on successful unguarded copying.

ModelLexicalDense
NoneFilterFilter+guardNoneFilterFilter+guard
Qwen-3B87.5/84.4/70.351.6/39.1/34.426.6/0.0/0.092.2/89.1/78.162.5/45.3/37.526.6/1.6/0.0
Phi-3.587.5/84.4/79.753.1/37.5/34.414.1/1.6/0.093.8/92.8/89.164.1/48.4/45.320.3/3.1/0.0
Qwen-7B85.9/85.9/79.748.4/40.6/37.514.1/1.6/0.096.9/93.8/85.960.9/50.0/46.915.6/1.6/0.0
Mistral-7B87.5/76.6/68.851.6/39.1/26.623.4/3.1/0.093.8/85.9/64.160.9/45.3/29.725.0/1.6/0.0

Table 6b. (b) Each cell is copy5_5/copy10_{10}/copy20_{20} (%) under the semantic extraction query with quotation forbidden.

ModelOrdinary triggerOrdinary Δ\DeltaF1F1 decreaseAuthorized non-target triggerAuthorized Δ$5\Delta\$5-gram recall
Qwen-3B16.7 [9.4, 25.0]-0.004 [-0.014, 0.004]5.2 [1.0, 9.4]0.0 [0.0, 0.0]0.000 [0.000, 0.000]
Phi-3.510.4 [4.2, 16.7]-0.006 [-0.020, 0.007]4.2 [1.0, 8.3]9.4 [3.1, 17.2]0.000 [0.000, 0.000]
Qwen-7B7.3 [2.1, 12.5]-0.011 [-0.023, -0.001]5.2 [1.0, 10.4]3.1 [0.0, 7.8]0.000 [0.000, 0.000]
Mistral-7B10.4 [5.2, 16.7]-0.008 [-0.021, 0.002]4.2 [1.0, 8.3]0.0 [0.0, 0.0]0.000 [0.000, 0.000]

Table 7b. (b) Paired guard-minus-unguarded side effects on the same response. Authorised target text is excluded from the guard set; triggers concern other non-quotable sources.

Table 8b, (b) Detailed-paraphrase cosine similarity and mismatched-source margin; neither is thresholded as copying or legal similarity.

Model / controlSource cosineTrue-negative margin
Qwen3-8B / None0.794 [0.758, 0.828]0.704 [0.665, 0.742]
Qwen3-8B / Surface guard0.679 [0.624, 0.732]0.583 [0.522, 0.638]
Qwen3-8B / Summary-style prompt0.777 [0.738, 0.815]0.691 [0.645, 0.733]
Qwen3-8B / Summary-style + surface guard0.666 [0.606, 0.723]0.576 [0.514, 0.638]
Phi-4-14B / None0.882 [0.858, 0.902]0.760 [0.732, 0.785]
Phi-4-14B / Surface guard0.816 [0.783, 0.844]0.696 [0.662, 0.728]
Phi-4-14B / Summary-style prompt0.870 [0.842, 0.894]0.752 [0.718, 0.785]
Phi-4-14B / Summary-style + surface guard0.789 [0.748, 0.828]0.679 [0.633, 0.719]

Table 8b.

Table 9, (b) Human-rated outcomes by model, using the definitions and pair sample in Table 12.

ModelControlExpressionRecoveryIdentifiablenn
Qwen3-8BNone72.233.350.018
Normalized guard5.60.00.018
Summary-style prompt55.65.622.218
Summary + normalized guard11.10.011.118
Phi-4-14BNone94.466.772.218
Normalized guard44.438.938.918
Summary-style prompt66.755.666.718
Summary + normalized guard50.022.227.818

Table 9b.

Table 10b, (b) Detailed-paraphrase recovery on quotation-denied items. Hidden-fact recall uses unexposed human-written probes; re-ID@1 ranks the response against 96 documents. These are recovery/linkage, not copying measures.

ModelControlcopy20copy_{20}Hidden-fact recallSource re-ID@1
Qwen3-8BNone3.1 [0.0, 7.8]14.6 [7.9, 21.2]87.5 [79.7, 95.3]
Normalized guard0.0 [0.0, 0.0]12.4 [6.6, 18.8]78.1 [67.2, 87.5]
Summary + normalized guard0.0 [0.0, 0.0]9.5 [4.8, 15.3]76.6 [65.6, 85.9]
Policy + summary + normalized guard0.0 [0.0, 0.0]4.2 [1.1, 7.9]43.8 [31.2, 56.2]
Phi-4-14BNone4.7 [0.0, 10.9]46.0 [37.3, 55.6]90.6 [82.8, 96.9]
Normalized guard0.0 [0.0, 0.0]38.4 [29.6, 47.4]90.6 [82.8, 96.9]
Summary + normalized guard0.0 [0.0, 0.0]34.1 [25.9, 42.3]85.9 [76.6, 93.8]
Policy + summary + normalized guard0.0 [0.0, 0.0]20.6 [13.8, 28.6]56.2 [43.8, 68.8]

Table 10b.

Table 11b, (b) Three-query domain results: 48 Wikipedia and 48 scientific documents, three hidden probes each; target-retrieved subsets contain 25 and 15 items.

ModelDomainGuard factsGuard re-ID@1Full stack factsFull stack re-ID@1nn
Qwen2.5-3BWikipedia42.7 [29.3, 57.3]8.0 [0.0, 20.0]20.0 [9.3, 32.0]8.0 [0.0, 20.0]25
Scientific papers13.3 [4.4, 24.4]26.7 [6.7, 53.3]13.3 [4.4, 24.4]40.0 [13.3, 66.7]15
Qwen3-8BWikipedia62.7 [46.7, 77.3]40.0 [20.0, 60.0]34.7 [20.0, 49.3]20.0 [8.0, 36.0]25
Scientific papers24.4 [11.1, 37.8]46.7 [20.0, 73.3]15.6 [4.4, 28.9]46.7 [20.0, 73.3]15

Table 11b. Query-budget and domain sensitivity. (b) Three-query domain results: 48 Wikipedia and 48 scientific documents, three hidden probes each; target-retrieved subsets contain 25 and 15 items.

Human construct checks

Two independent human annotators, blind to model, control, score, and pair status, rated 144 true-source and 48 mismatched-source pairs. True pairs balance two models, two prompts, and four controls. Agreement is 88.0% (κ=0.831\kappa= 0.831) for expression and 95.3% (κ=0.929\kappa= 0.929) for recovery. Wilson intervals describe this fixed sample. Automated proxy correlations use source-item-clustered bootstrap intervals; partial correlations rank-residualise response length. These are technical rather than legal judgments.

Source linkage ranks Qwen3 responses with BM25 and MiniLM over the 192-item corpus, 96-way topic-matched candidates, and expanded pools of 20,233 Wikipedia or 1,169 scientific documents. Uniform, question-only, answer-only, and length-matched shuffled controls separate topic and length effects from response signal.

Table 12b, (b) Proxy validation on those 192 human-rated pairs. Every proxy is scored against the displayed source. Spearman intervals cluster by source item; partial ρ\rho adjusts for response length; κ\kappa uses four-level exact recall.

QuantityTrue-source pairsMismatched-source pairs
Exact hidden-probe recall of displayed source (all probes), Spearman ρ\rho0.72 [0.60, 0.82]n/a
Exact hidden-probe recall of displayed source (≥\geq4-token answers), Spearman ρ\rho0.51 [0.27, 0.69]n/a
Full-source multiset token recall, Spearman ρ\rho0.87 [0.80, 0.92]n/a
Normalized character similarity, Spearman ρ\rho0.85 [0.78, 0.90]n/a
Response token count, Spearman ρ\rho0.84 [0.75, 0.90]n/a
Exact hidden-probe recall of displayed source (all probes), partial Spearman ρ\rho (controlling for response length)0.41 [0.22, 0.57]n/a
Exact hidden-probe recall of displayed source (≥\geq4-token answers), partial Spearman ρ\rho (controlling for response length)0.25 [0.04, 0.44]n/a
Full-source multiset token recall, partial Spearman ρ\rho (controlling for response length)0.46 [0.33, 0.58]n/a
Normalized character similarity, partial Spearman ρ\rho (controlling for response length)0.38 [0.24, 0.50]n/a
Linear weighted κ\kappa (human vs. discretized exact recall)0.490n/a
Sensitivity, any recovery vs. displayed-source exact-recall indicator (%)51.8 [42.7, 60.7]n/a
Specificity, any recovery vs. displayed-source exact-recall indicator (%)96.6 [82.8, 99.4]100.0 [92.6, 100.0]
Mismatched-control false-positive rate, human any recovery (%)n/a0.0 [0.0, 7.4]
Mismatched-control false-positive rate, automated any recovery on the displayed (wrong) source’s own probes (%)n/a0.0 [0.0, 7.4]

Table 12b. Human judgments and automated proxy validation. (b) Proxy validation on those 192 human-rated pairs. Every proxy is scored against the displayed source. Spearman intervals cluster by source item; partial ρ\rho adjusts for response length; κ\kappa uses four-level exact recall.

Primary contrast, utility, and sensitivity

ModelControlΔ\Delta Token F1Δ\Delta Span F1Δ\Delta Resp. tokens
Qwen3-8BSummary-4.5 [-7.3, -1.8]-3.4 [-10.3, +3.3]+6.0 [-2.1, +13.1]
SystemBlock-0.4 [-1.1, +0.3]-0.1 [-1.6, +1.6]+2.7 [-0.2, +5.7]
Threshold+0.0 [-2.7, +2.4]-11.5 [-19.6, -4.2]-17.1 [-24.8, -10.3]
Phi-4-14BSummary-4.0 [-6.2, -2.0]-4.9 [-10.5, -0.3]+5.6 [-2.1, +13.5]
SystemBlock+2.0 [+0.5, +3.7]-0.0 [-3.2, +3.1]-3.6 [-7.9, +0.4]
Threshold+3.6 [+0.7, +6.6]-13.3 [-20.8, -6.3]-18.5 [-27.6, -9.3]
Qwen2.5-3BSummary-8.3 [-12.4, -4.6]-23.1 [-33.7, -12.9]+15.9 [+6.1, +25.6]
SystemBlock+1.8 [+0.4, +3.3]-3.0 [-10.0, +3.5]-10.1 [-15.8, -4.8]
Threshold-0.9 [-3.4, +1.6]-11.7 [-19.4, -4.5]-14.2 [-22.9, -5.9]
Mistral-7BSummary-6.2 [-9.2, -3.4]+3.8 [-2.8, +10.9]+27.3 [+19.1, +36.0]
SystemBlock-1.2 [-2.7, +0.1]+1.1 [-2.1, +5.0]+3.8 [-2.2, +10.3]
Threshold-0.3 [-2.9, +2.4]-9.1 [-17.1, -2.1]+2.4 [-6.7, +10.6]

Table 13. Paired utility deltas against uncontrolled (5,000-draw paired item-bootstrap intervals). Token F1 and span F1 distinguish response-length sensitivity from answer coverage.

ModelControl±\pm5 pts±\pm10 pts
Qwen3-8BSummary,,
SystemBlock✓✓
Threshold,,
Phi-4-14BSummary,,
SystemBlock✓✓
Threshold,,
Qwen2.5-3BSummary,,
SystemBlock,✓
Threshold,,
Mistral-7BSummary,,
SystemBlock✓✓
Threshold,,

Table 13 (PDF p. 26). Post-hoc utility equivalence bounds: a checkmark requires the entire 95% paired span-F1 interval strictly inside the bound, a stricter criterion than 90%-interval TOST inclusion.

ModelF1 SDSpan SDRangeChanged
Qwen3-8B0.0130.0350.0326.2%
Phi-4-14B0.0170.0650.04318.8%
Qwen2.5-3B0.2500.0000.53017.2%
Mistral-7B0.0060.0000.0156.2%

Table 13 (PDF p. 26, second). Six replicates per model vary batch size (4, 8, 16) and row order, with seed, revision, and greedy decoding fixed.

(a) Exploratory construct sensitivity. DiD uses all 80 answer-only items except long-answer recall (n=57n = 57, 21 Wikipedia and 36 scientific), standardised to equal domain weights. Domain columns use 40 items each and forced target-present Summary exposure. Response length is in tokens; other columns are percentage points.

ModelLong-answer recall DiD a^{a}Neg.-control margin DiDFull-source token DiDResponse length DiD (tokens)Wikipedia exact recallb^{b}Sci. paper exact recallb^{b}
Qwen3-8B-17.5 [-30.6, -5.4]-1.3 [-9.2, 7.1]5.1 [-0.5, 10.5]57.7 [28.6, 85.9]75.8 [67.5, 83.3]11.7 [5.8, 18.3]
Phi-4-14B-8.4 [-21.0, 4.2]0.4 [-7.5, 8.8]3.4 [-1.9, 8.6]84.4 [49.8, 119.8]68.3 [57.5, 79.2]14.2 [7.5, 22.5]
Qwen2.5-3B-7.7 [-20.6, 5.7]-2.9 [-11.7, 5.8]3.5 [-2.3, 9.7]35.8 [14.1, 57.5]28.3 [18.3, 39.2]13.3 [6.7, 21.7]
Mistral-7B-0.3 [-13.8, 12.5]11.7 [2.5, 20.4]2.9 [-2.5, 8.2]11.7 [-16.4, 39.9]68.3 [58.3, 78.3]23.3 [16.7, 30.0]

Table 15, (b) Post-hoc TOST on exact-answer DiD. Bounds were chosen after the frozen analysis. pTOSTp_{\mathrm{TOST}} is the larger shifted one-sided sign-randomisation value (20,000 draws). Equivalence requires p<0.05p < 0.05 and the 90% paired bootstrap interval inside the bound.

ModelBoundDiD90% paired bootstrap CIpTOSTp_{\mathrm{TOST}}α\alphaInterpretation
Qwen3-8B±5pp-0.4[-7.1, 6.2]0.1390.05not established
Qwen3-8B±10pp-0.4[-7.1, 6.2]0.0110.05equivalent (within bound)
Phi-4-14B±5pp0.0[-6.7, 6.7]0.1190.05not established
Phi-4-14B±10pp0.0[-6.7, 6.7]0.0100.05equivalent (within bound)
Qwen2.5-3B±5pp-3.8[-11.2, 3.7]0.3950.05not established
Qwen2.5-3B±10pp-3.8[-11.2, 3.7]0.0940.05not established
Mistral-7B±5pp10.4[2.9, 17.9]0.8770.05not established
Mistral-7B±10pp10.4[2.9, 17.9]0.5280.05not established

Table 15. Exploratory construct and practical-equivalence checks.

Seven-query extension

The exploratory extension pins Summary, SystemBlock, and the threshold to the same upstream configuration. A complete-source request is followed by up to six requests using Base64, ROT13, reversed words or characters, fact tables, and paraphrases. The source intervention uses a shared attack ranking across three controls and two exposure arms. Natural retrieval uses control-specific development rankings over both policies and all 160 test items included in intent-to-treat analyses. Utility is evaluated on two query types and 64 document-disjoint items.

Budgets 1, 3, and 7 carry forward the last released state after stopping. Intervals use 5,000 paired source-item bootstrap draws and sign randomisation uses 20,000 draws. Models and the two attacker designs are not pooled. Long-answer analyses use the 57 eligible items with accepted answers of at least four tokens, standardised to equal domain weights. Practical-equivalence bounds are post-hoc: a large TOST pp value does not establish equivalence.

(a) Exploratory seven-query source intervention, shared attacker, 80 answer-only items. DiD is defence-minus-uncontrolled for the present-minus-ablated context effect. Brackets: 95% paired source-item bootstrap intervals.

ModelDefenseUncontrolled context effectDefense context effectDifference-in-differencesnn
Qwen3-8BSummary43.342.5-0.8 [-7.9, 6.2]80
Qwen3-8BSystemBlock43.345.82.5 [-3.8, 8.3]80
Phi-4-14BSummary40.440.80.4 [-7.1, 7.9]80
Phi-4-14BSystemBlock40.439.6-0.8 [-6.7, 5.0]80
Qwen2.5-3BSummary22.922.90.0 [-8.8, 8.3]80
Qwen2.5-3BSystemBlock22.924.61.7 [-5.4, 8.8]80
Mistral-7BSummary33.840.46.7 [-25, 16.2]80
Mistral-7BSystemBlock33.828.3-5.4 [-12.9, 2.1]80

Table 16, (b) Seven-query shared-attacker DiD by endpoint. Long-answer recall uses 57 eligible items with equal domain weights; other outcomes use 80. Values are percentage points except response-length tokens.

ModelDefenseExact-answer DiDLong-answer DiDa^{a}Full-source token DiDDecoded copy20_{20} DiDResponse-length DiD (tokens)
Qwen3-8BSummary-0.8 [-7.9, 6.2]-16.3 [-28.6, -4.0]6.5 [0.8, 12.1]-36.2 [-48.8, -23.8]110.4 [74.2, 146.4]
Qwen3-8BSystemBlock2.5 [-3.8, 8.3]-1.9 [-10.5, 6.1]1.8 [-1.3, 4.9]-12.5 [-21.2, -5.0]24.5 [17.0, 59.0]
Phi-4-14BSummary0.4 [-7.1, 7.9]-13.2 [-25.4, -1.4]1.1 [-3.8, 6.0]-28.7 [-40.0, -17.5]69.7 [22.3, 118.4]
Phi-4-14BSystemBlock-0.8 [-6.7, 5.0]-5.1 [-14.4, 3.1]3.6 [-0.4, 7.7]-3.8 [-12.5, 5.0]82.7 [31.5, 132.9]
Qwen2.5-3BSummary0.0 [-8.8, 8.3]-8.1 [-20.6, 4.1]3.5 [-2.8, 10.0]-42.5 [-55.0, -30.0]58.7 [13.4, 103.8]
Qwen2.5-3BSystemBlock1.7 [-5.4, 8.8]4.4 [-8.9, 16.7]1.7 [-2.7, 6.1]11.2 [1.2, 21.2]33.0 [-92.0, 26.3]
Mistral-7BSummary6.7 [-2.5, 16.2]-2.0 [-14.8, 10.4]0.4 [-5.0, 5.6]-32.5 [-45.0, -20.0]-54.6 [-99.9, -12.7]
Mistral-7BSystemBlock-5.4 [-12.9, 2.1]-10.7 [-22.4, 0.3]-2.3 [-7.2, 2.5]3.8 [-6.2, 13.8]-7.5 [-56.7, 39.8]

Table 16. Seven-query source-intervention results.

ModelDefenseUncontrolled recallDefense recallPaired Δ\DeltaTarget presence
Qwen3-8BSummary42.540.8-1.7 [-7.9, 4.6]72.5
Qwen3-8BSystemBlock42.543.81.2 [-4.6, 7.1]72.5
Qwen3-8BThreshold42.532.5-10.0 [-18.3, -2.5]46.2
Phi-4-14BSummary40.845.44.6 [-2.9, 12.1]72.5
Phi-4-14BSystemBlock40.839.6-1.3 [-6.7, 4.2]72.5
Phi-4-14BThreshold40.832.1-8.8 [-15.8, -2.1]46.2
Qwen2.5-3BSummary22.512.5-10.0 [-16.7, -3.8]72.5
Qwen2.5-3BSystemBlock22.527.95.4 [-1.2, 12.5]72.5
Qwen2.5-3BThreshold22.522.1-0.4 [-7.5, 6.7]46.2
Mistral-7BSummary30.437.57.1 [-1.3, 15.8]72.5
Mistral-7BSystemBlock30.423.8-6.7 [-13.8, -0.4]72.5
Mistral-7BThreshold30.429.2-1.2 [-6.7, 5.0]46.2
ModelUncontrolledSummarySystemBlockThreshold
Qwen3-8B18.6 [14.6, 22.8]14.1 [10.9, 17.4]18.1 [14.3, 22.3]18.6 [14.7, 22.6]
Phi-4-14B11.5 [8.8, 14.7]7.6 [6.0, 9.4]13.5 [10.2, 17.2]15.1 [11.1, 19.4]
Qwen2.5-3B14.9 [11.2, 19.1]6.6 [4.8, 8.7]16.7 [12.8, 21.0]14.1 [10.3, 18.0]
Mistral-7B17.2 [13.5, 21.1]11.0 [8.7, 13.5]16.0 [12.5, 19.6]16.9 [12.8, 21.6]

Table 17. Seven-query natural retrieval and ordinary QA utility. (a) Seven-query natural retrieval with control-specific attackers. Paired deltas use all answer-only test items, without conditioning on retrieval; target presence is reported separately. Brackets: 95% paired source-item bootstrap intervals. (b) Single-turn, document-disjoint ordinary-QA token F1 (percentage points) with 95% item-bootstrap intervals; utility does not vary with attack budget.

Figure 4. Exploratory exact-answer proxy changes at query budgets 1, 3, and 7 (percentage points). Top: source-intervention difference-in-differences under a shared attacker. Bottom: paired natural-retrieval changes under control-specific attackers, using all-item intent-to-treat analysis. All three original point estimates are plotted. Stopped interactions carry their last released state forward; models and attacker designs are not pooled. Intervals were not reported in the original budget table.

Survey Synthesis and Record-Level Operating Points

Mechanisms, intervention stages, and metrics are multi-label axes, so their counts need not sum to 118.

Interaction settings comprise 51 single-query, 31 fixed multi-query, 25 non-interactive, and 11 feedback-adaptive records. Attacker access includes external black box (47), corpus-contributor write (39), authorised user (16), operator white box (13), service-provider protocol party (6), corpus-owner auditor (4), and operator grey box (3). Intervention stages are generation (81), context construction (64), retrieval (60), post-generation (39), corpus acquisition (27), index construction (26), session (8), and removal/update (7). Mechanism counts are in Figure 5, with one additional “other” record; metric counts are in Figure 5.

Figure 5. Mechanisms and metrics by primary endpoint over 118 records. Cells and row totals share a count scale; multi-label rows overlap. Stage codes and numbered representative works retain the original annotations.

Representative works by mechanism row: 01 ([58, 32]); 02 ((Zeng et al., 2025b,a)); 03 ([111, 133]); 04 ([34, 140]); 05 ([100, 36]); 06 ([13, 21]); 07 ([87, 48]); 08 ((Abouelenein and Torki, 2026; Hu et al., 2024)); 09 ([65] (Zeng et al. 2025b)); 10 ([45, 116]); 11 ([132, 55]); 12 ([66, 123]); 13 ((Zhang et al., 2025d,a)).

Figure 6. Reporting and design appraisal over 118 records. Counts are lower/some/high concern; NR is zero throughout. These ratings are not efficacy weights.

Artifact and provenance

Reported operating points by endpoint family

Fam.RecordPrimary endpoint as reported
AZeng et al. [141]Repeat Prompt / Repeat Context / ROUGE Prompt+Context counts of retrieved chunks reproduced in output; summarization, distance threshold, re-ranking drive counts to 0
AWang et al. [116]Repeat Prompt/Context defined as response reproducing text verbatim from the retrieval corpus; ROUGE-L above threshold vs corpus text (264→127 etc.)
AZeng et al. (2025a)Repeat Prompt = at least 10 tokens exactly matching original dataset; ROUGE-L>0.5; targeted PII count driven to 0
ALiang et al. [68]Repeat Prompts / Repeat Length / Target Info under targeted and prefix extraction attacks (59 pct reduction in repeated prompts)
AZhu et al. (2026)Privacy leakage rate under the ’privacy extraction attack’ of He et al. (ChatDoctor 0.23 -> 0.17; EnronQA 0.63 -> 0.33, Table 3); BLEU/ROUGE-L reported as UTILITY
AZhang et al. (2026b)Repeat Prompts = >=10 consecutive tokens from retrieval corpus; ROUGE Prompts/Contexts>0.5; zero leakage across epsilon=0.1–0.10
AYao et al. [137]ALR/ASR/CLR privacy-entity leakage in generated output vs a GRPO-trained adaptive attacker (NormAvep 1.00 -> 0.13)
ALi et al. [64]Chunk Recovery Rate (ROUGE-L>0.5 generated output vs original chunk plus cosine) under Worm-Attack and RAG-Thief; relative mean 1x -> 0.51x
Fam.RecordPrimary endpoint as reported
A-piiLi et al. [65]Document-level any-success PII leak rate in outputs 0.875 -> 0.000 (en/zh/de) under output regex filter plus input judge
A-piiZhao et al. (2027)Repeat Prompt = outputs containing >=10 consecutive verbatim tokens from private text; ROUGE Prompt>0.5 vs original private text; adaptive attack table
A-piiCheng et al. [29]De-anonymization Rate under a three-tier threat model (ArcFace/CosFace/AdaFace 99.4 -> 1.8-3.1 pct; T2 top-1 4.2 pct), plus VQA accuracy / FID / LPIPS utility
A-piiMoulton et al. [84]Garak PII-leak failures 50 -> 0; Laplace-adjusted success probability 0.980 -> 0.019; ABAC arm renders the PII corpus unretrievable
A-piiBodea et al. [11]LLM-as-a-Judge privacy leakage score 0-100 comparing sensitive entities in the SOURCE text against those present in the GENERATED text, PRE and POST anonymization (Table 5), plus ROUGE-L/cosine/PPL utility
A-piiMori et al. [83]Count of patient-name leaks in model responses under 1000 benign and 100 targeted extraction queries (Table 3)
BJin et al. [53]ROUGE-2/BLEU-2/EMR/BiNLI of embedding-inversion (EIA) reconstructions of the user’s input text, driven to ~0
BChen et al. [22]ROUGE-2 recall of document reconstruction from generated LoRA weights by a trained T5 decoder (<9 pct)
BLi et al. [66]Edge recovery rate from observed HNSW traversal paths 18.3 -> 6.8 pct with false-positive rate and Recall@10 degradation at identifiable rho/E operating points
BWanger [123]Isolation-Forest / One-Class-SVM detector ROC-AUC (0.85-1.000) and TPR@1%/5% FPR against steganographic embedding exfiltration; VectorPin provenance protocol
BYe et al. [138]Vec2Text reconstruction BLEU/precision/recall/F1/ROUGE from encrypted embeddings (BLEU 83 -> 12) plus retrieval accuracy and throughput
BMao et al. [79]Resistance to ICA and graph matching; kNN-overlap rate between plaintext and ciphertext spaces; relative attention-output error 1.49-1.63 pct
BChakraborttii et al. [20]Vec2Text ROUGE-L recovery from soft-deleted HNSW vectors 0.207 -> 0.000 under epoch key rotation; ECDSA-signed erasure proof
CWu et al. [125]Formal (epsilon approx 10) DP guarantee across hundreds of queries via individual Renyi privacy filters, plus utility; no empirical leakage measurement in the packet
CLebensold et al. [61]Shutterstock watermark visibly copied into generated images (qualitative, UNDEFENDED RDM baseline, Figure 2b); protection outcome is DP epsilon=10 plus FID 10.9
CAboueleinen and Torki [1]Membership-inference attack accuracy 70.22 -> 53.60 pct (advantage Adv = Acc - 0.5) at a 2.6 pct accuracy cost under pure epsilon-DP SimHash bucketing
CGolatkar et al. [45]CLIP similarity between CPR-generated images and retrieved private images (Syn-Ret) held low while prompt similarity (Syn-Cap) stays high; empirical k_c / NAF Delta_max bit bound
CKoga et al. [59]S2MIA TPR-FPR curves with AUC collapsing to ~0.5 at epsilon=10 (BLEU used as the membership score), plus average match accuracy utility
CMin et al. [82]Perplexity on held-out Harry Potter book with vs without HP books in the datastore (Table 6); qualitative sentence-level attribution examples (Table 7)
DAgnew et al. [3]Judge-rated fraction of defended texts the attacker LLM is UNABLE TO USE to answer (news RAG benchmark n=75; >90 pct on many datasets; 75 pct transfer to unseen text)
DPallera et al. (2026)MBA exact mask-fill reconstruction leakage rate (5 masked tokens, membership signal) 29.5->0.0 pct; poisoning ASR 56->0 pct
DHu et al. (2024)HR@K / MRR separately for public answers and privacy-threatening (private-link-derived) answers; adversarial training suppresses the latter
DNamboothiri [87]Structural leak rate (unauthorized chunks RETRIEVED) 86.1->0 pct; keyword-matched answer leak 41.3/29.5 -> 0 pct; 100 pct -> 0 pct violation rate
DPanebianco et al. [94]Cluster purity, accuracy, precision, recall, F1 (0.77) and AUPRC 0.97 for DETECTING PII-leak and jailbreak interactions
DZhang et al. (2024)User study (N=36): GPT-4o-rated reduction of private information stored/input into LLM memory, plus subjective/objective usability metrics (Table 1)
DLorenzo et al. [76]Role-based access filtering accuracy 85 pct and F1 0.89 on 389 queries vs static-filter baseline (F1 0.93-0.94) and no-filter baseline (F1 0.32-0.43)
DXie et al. [128]Attack Success Rate of PII extraction (Bad Query / Adversarial / CopyBreakRAG) 81.9 -> 9.5 pct; detection P/R/F1 94.1/93.5/93.8
DZafar et al. [140]AUROC / FPR95 / FPR90 and abstain rate for DETECTING quasi-identifier cluster leakage in generated RAG outputs across three domains
DCherubin and Paverdi [31]Precision/recall for declined answers, K-precision/recall and Elo utility; architectural isolation means user text never reaches the generator
DArceo and Narsing [7]Cross-Tenant Leakage Rate = fraction of cross-tenant probes returning at least one unauthorized CHUNK (80->0 pct); AVR; injection block rate
DZhou et al. (2025b)LASR/NAV/LK all 0.00; LK = leaked tokens / total private tokens with leakage confirmed by ROUGE-L recall between generated output and private KB; MIA and Vec2Text/GEIA and PASR
DBudigi and Sirigiri [13]Tenant-isolation leakage rate 0.2 pct -> 0 pct over 1000 queries with engine-level row-level security, plus latency
DChen [21]Access-control compliance verified per test case (retrieval restricted to permitted profiles) plus relevance/correctness of retrieved information
DSalemi and Zamani [104]Attacker profile-identification accuracy (linkability) and attribute-inference accuracy: ~100 pct with full context vs 20 pct client-side / 58.5 pct query-only
Fam.RecordPrimary endpoint as reported
DWang et al. [122]Private-triple removal rate rremover_{\mathrm{remove}} and de-anonymization QA-attack success rate rattackr_{\mathrm{attack}}, plus PPL fluency and downstream RAG accuracy
DMasoud et al. [80]Privacy and completeness scores with and without prompt injection (Table I); up to 58 pct privacy-score improvement
DThornton [113]Retrieval Pivot Risk 0.95 -> 0.0 across three corpora with Leakage@k, Amplification Factor and per-attack RPR=1.0 baselines
DNutakki [90]Unauthorized context exposure 3.84 -> 0.00 pct; sensitive answer leakage 2.91 -> 0.04 pct; 98.6 pct red-team attacks blocked
DZhang et al. (2025d)Unlearning effectiveness on the forget set after excluding forgotten user data from the retrieval index, with HR@K / NDCG@K utility on remaining data
DMouzouni [85]Unauthorized-access outcome across four governance baselines (200 queries): results are never unauthorized regardless of 63-75 pct routing accuracy
DByun et al. [16]Confidentiality-filter recall 99.01 pct (82.31 pct accuracy / 74.35 pct recall with ambiguous queries) preventing security-sensitive queries reaching external LLMs
DWu et al. [124]98 pct accuracy detecting malicious, document-extraction and hallucination attacks; extraction requests denied
DOuyang and Hou [91]ASR matrix driven to zero across three attack families with lineage/provenance signing and a tau x K ablation, plus utility baseline
DWang et al. [117]Unlearning Success Rate; ROUGE-L recall of output vs short reference ANSWER; TPR@1%FPR residual memorization
DYao et al. [135]AUROC / matching accuracy for risk detection across five risks (0.96-0.997) and precision/recall/F1 for mitigation
EChen et al. [25]Verification Success Rate, CGSR and TPR/FPR of AQUA watermark verification; filter-resistance results
EZeng et al. (2025b)Shadow-RAG membership audit AUC 0.989 / Acc 0.957 / F1 0.956; robustness under prompt-modifying and paraphrasing defenses
EChoi et al. [32]Detection accuracy/precision/recall/F1 for member attack queries plus adjusted attack accuracy (adversary advantage) reduction
EXie et al. [129]Chunk Recovery Rate: chunk counted recovered iff ROUGE-L(generated output, original chunk)>0.5 AND cosine similarity; 97.4 -> 1.6; detection TPR
EKwok et al. [60]Effective Detection Rate with p<1e-5 at 0.1 pct sentinel injection; FDR<=1.0 pct across four datasets
EZhang et al. (2025c)Internal-state classifier ACC/F1 (up to 100/100) separating copyrighted-leakage from non-leakage inputs; IS-W/RAG vs LLM-as-a-Judge baselines
ELiu et al. [74]stylometric attribution F1 0.825 -> 0.221 (TF-IDF) and 0.691 -> 0.203 (BERT)
EHuang et al. [50]Watermark detection AUC (up to 100) and spoofing-AUC; Knowledge Corruption Rate; ROUGE-1/L and PPL reported as UTILITY of the watermarked answer
EYang et al. [132]Watermark recovery rate WRR=1.00 for multi-bit payload; robustness/detection outcomes under attacks
EBurnat [15]Coalition-detection TPR=1.00 over 2400 trials at FPR<=0.05, plus (eps_acc, delta_acc)-DP retrieval guarantee and a matching membership-inference AUC trend in k
EFu et al. [41]AUC / ASR / TPR @1 reduction across seven membership-inference attacks (SPV, LiRA, Recall, Log-Loss); up to 27.8 pct MIA-success reduction and 75 pct TPR@1 reduction
EPellicer et al. [98]Artifact-level disclosure risk on exported evidence: Membership-Inference Accuracy -9.8 pp, Attribute-Inference Accuracy -8.4 pp, top-1 Linkage success rate (Table 2)
EGuo et al. [47]CoT-space watermark ownership verification p-values (malicious p<0.01 vs independent p=1.00)
EJovanović et al. [55]Watermark-propagation p-values and TP/FP rates on the FARAD benchmark
ELiu et al. [75]ROC-AUC and TPR @1%FPR = 100 pct at a 12-query operating point for watermarked canaries
ELiu et al. [73]Interrogator-Detective dual-layer watermark detection accuracy and false-positive rates across LLMs and adversarial conditions
ELuo et al. [77]Detection AUC / TPR from sentinel-image key queries against the RAIG reference database
ELv et al. [78]WSN / WIRR watermark verification success and integrity across five datasets and four LLMs, including robustness to removal attacks
FYang et al. [133]Poisoned-context attack defense improvement ~20 pct; QA accuracy under varying poisoned/factual context ratios
FSaleem et al. [103]Macro-averaged ASR and FPR, detection rate and response-quality score for a three-layer prompt-injection defense (Table 8)
FCohen et al. [34]Virtual Donkey detection TPR=1.000 / FPR=0.015 (BLEU, ROUGE-L, METEOR input-output similarity used as classifier FEATURES); attack-side exfiltrated email-address F1
FCeraku [19]Poison Impact Rate reduction across ingestion/retrieval/generation layers on a 300-document CTI corpus (90 poisoned), plus latency
FCui and Liu [36]Character-level traceback overlap and over-selection metrics for locating poisoned spans; passage-level identification and ASR baselines
FSharma [107]Certified defense: ASR 93 pct -> 5.3 pct (n=150) for runtime memory poisoning, with provenance signing
FJin et al. [54]Malicious-document detection performance plus end-to-end ASR and precision
FKim et al. [58]ASR 0.04 against Gemini on MS MARCO with highest accuracy among defenses
Fam.RecordPrimary endpoint as reported
FWang et al. [120]Attribution precision/recall for identifying the responsible context texts, plus ASR with and without the defense
FNoughabi et al. [89]True-positive detection rates for poisoned subgraph elements vs perplexity-based filtering baselines
FDatta et al. [38]ASR reduced to near zero for sentence-relation-based sanitization of malicious instructions (from 82-83 pct baseline)
FYao et al. [136]Answer accuracy and ASR across poisoning settings for context-analysis-based security
FGerami et al. [44]Influence Score identifies the poisoned document with 86 pct success over 3000 samples (p<0.0001)
FTan et al. [112]Provably robust aggregation: ASR reduced to as low as 1 pct at 71 pct accuracy
FWang et al. [118]Adulterant Retrieval Rate 100 pct; Hijack Success >94 pct (unauthorized system answers wrongly); CDPA/CIRA 100 pct authorized fidelity; 80.2 pct robustness retention
FThornton [114]Best-F1 detection comparison (QPD, keyword anomaly) and ROC curves for corpus-dependent poisoning
FCheng et al. [30]Chunk-wise perplexity filtering detection of poisoned chunks plus text-similarity metrics
FLin et al. [69]Security Rate of generated code under two poisoning scenarios (average across LLMs drops 60.84 -> 50.57 without defense; CodeGuarder consistently improves SR)
FYu et al. [139]ASR increase of 4-16x when the isolation component is removed; certified-synthesis information-flow controls
FTan et al. [111]TPR/FPR tables for revealing poisoned responses via LLM activation analysis (metric definitions elided in packet)
FWang et al. [119]Attribution precision/recall for the injected/corrupted texts plus ASR with and without attack
FGowda [46]ASR-R / ASR-A under faithful evaluation with a certified detection radius for memory poisoning
FPathak and Sharma [95]Adversarial-hub attack recall 1.000 (universal/concept) and 0.91 on HotFlip at AUROC ~1.0 with ~1 pct benign FPR at an admission-time gate
FDekel et al. [39]ACC and ASR across poisoning attacks using sparse-attention corpus defense
FAlam et al. [4]Attack Pass-through Rate plus 0 pct FPR on the benign subset for priority-aware prompt control-flow integrity
FPathmanathan et al. [96]ASR (retriever returns at least one poisoned passage) plus utility under benign conditions for RAGPart/RAGMask
FChoudhary et al. [33]Attention-aware detectability of stealthy poisoning across epsilon regimes vs perplexity filters; end-to-end robust pipeline
FLiang et al. [67]Answer accuracy and ASR (0.00 across several settings) for a RAG-guards-RAG expert-module defense
FXian et al. [126]Mahalanobis-distance detection rate >98 pct for universally poisoned documents plus retrieval and attack success rates
FSaxena et al. [105]Corpus-poisoning detection precision/recall and F1 across datasets and evidence sizes vs perplexity baselines; 86 pct rationale accuracy
FPatil [97]ASR 0 pct (95 pct CI [0,1]) for numerical-claim manipulation with 99.8 pct entity detection rate
FJia et al. [52]Answer accuracy on a poisoned MS MARCO corpus using PageRank-based trust propagation
FBatista et al. [9]Sentence-level in-generation attribution accuracy per algorithm (Table 5) with pre-attribution ground truth
FZhou et al. (2025a)ACC / ASR across datasets for K-means conflict removal and knowledge consolidation against corpus poisoning
FSi et al. [110]Answer accuracy and ASR (percentage of poisoning queries/documents succeeding) for two-stage semantic filtering
FAlvarez et al. [5]Block F1 0.802 / 0.846 and top-1 source accuracy 0.858 (source-plus-relation 0.229) vs MiniCheck, RAGAlign, AlignScore baselines
FGao et al. [42]ASR reduced by up to 87 pct against PoisonedRAG with answer-accuracy preservation (Tables 1-Table 2)
FLeong [62]ASR by defense across nine models (360 runs): input-layer 77.8 pct, Memory Sandbox 0 pct for eight of nine, one model inverts to 100 pct via the RAG pathway
FShih and Kang [109]Attack-detection precision 1.00 with recall/accuracy 0.45 and F1 0.62 for the Reverse RAG version over 674 attacks; 100 pct block on a 301-sample high-confidence subset
FShen et al. [108]Provably robust generation accuracy and ASR under prompt injection at different attack positions
FXiang et al. [127]71 pct clean accuracy with 38 pct certifiable robust accuracy; ASR reduced from over 90 pct
FChen et al. [27]Answer accuracy and ASR (frequency of attacker-specified output) across datasets
FHong et al. [48]Discriminator classification F1 and exact-match robustness under counterfactual/corrupted retrieved context
FFei et al. [40]Harm driven to 0.000 by removing unauthenticated structural writes and rerunning the selector; accumulability criterion
FQi et al. [100]Agreement with human answer attribution on XOR-AttriQA; citation precision/recall/F1 (TRUE/NLI-based)
FZhang et al. (2025a)Traceback precision/recall for identifying poisoned texts in the knowledge database vs perplexity-based detection
FZhang et al. (2026a)Responsibility-attribution accuracy for poisoned knowledge vs PPL/norm baselines; RAG accuracy under benign and poisoned settings

Table 29.

ControlDeclared technical claimClaim-aligned evidenceNot established
Policy filterContexts unauthorized for the requesting principal do not reach generation.Target presence after filtering; access-control failures; authorized QA utility.Copying or recovery from contexts the principal is authorized to access.
Cosine thresholdRetrieved documents with MiniLM cosine below the pinned 0.5 operating point do not reach generation.Post-threshold target presence, all-item recovery, and matched QA utility.Confidentiality of relevant documents that remain above threshold.
Benchmark SummaryThe pinned upstream prompt requests a concise, query-relevant summary in the model’s own words and rejects irrelevant context.Response concision and relevance; full-source coverage and blinded human source-content recovery for the broader extraction-defense interpretation. Exact hidden-answer recall is retained only with a construct audit.Legal similarity, corpus-use detection, or universal resistance to semantic inference.
Benchmark SystemBlockThe pinned system prompt forbids exposing raw database content or facts available only from the database.Context-attributable source-content recovery, refusal behavior, and matched benign utility under a defense-aware attacker.Formal access control, legal confidentiality, or robustness beyond the tested models, prompts, and budget.
Normalized 8-gram guardNo released output contains a matching normalized 8-gram.Trigger behavior, shorter-span utility, and the implied absence of matching copy20_{20} under the same normalization.Suppression of facts, paraphrase, source identity, or reversible encodings outside its canonicalization.

Table 5. (a) Declared claims and their aligned measurements; other endpoints describe separate channels.

Figure 2. Response-to-source linkage calibration (top-1, %). Qwen3 answer-only responses are ranked by BM25 or MiniLM against the experimental corpus, question-selected topic-matched candidates, and expanded same-domain pools. Candidate-set sizes and the reported uniform and question-only baselines are retained. This measures source linkage, not copying or content recovery.

References

  1. [1]Abdelrahman Aboueleinen and Marwan Torki. 2026. Differentially private datastore generation for retrieval-augmented inference. In International Conference on Pattern Recognition, pages 624–638. Springer.
  2. [2]Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, Nicholas Meade, and Siva Reddy. 2024. Evaluating correctness and faithfulness of instruction-following models for question answering. Transactions of the Association for Computational Linguistics, 12:681–699.
  3. [3]William Agnew, Harry H Jiang, Cella Sum, Maarten Sap, and Sauvik Das. 2024. Data defenses against large language models.arxiv.org/abs/2410.13138
  4. [4]Md Takrim Ul Alam, Akif Islam, Mohd Ruhul Ameen, Abu Saleh Musa Miah, and Jungpil Shin. 2026. Prompt control-flow integrity: A priority-aware run-time defense against prompt injection in llm systems.
  5. [5]Ander Alvarez, Santhiya Rajan, Alessandro Genuardi, Oliver Wirjadi, Samuel Mugel, and Román Orús. 2026. Provenanceguard: Source-aware factuality verification for mcp-based llm agents.arxiv.org/abs/2606.18037
  6. [6]Maya Anderson, Guy Amit, and Abigail Goldsteen. 2024. Is my data in your retrieval database? membership inference attacks against retrieval augmented generation.DOI
  7. [7]Francisco Javier Arceo and Varsha Prasad Narsing. 2026. Securing the agent: Vendor-neutral, multi-tenant enterprise retrieval and tool use.DOI
  8. [8]Akari Asai, Zeqiu Wu, Yizhong Wang, Avi Sil, and Hannah Hajishirzi. 2024. Self-rag: Learning to retrieve, generate, and critique through self-reflection.arxiv.org/abs/2310.11511
  9. [9]João Eduardo Batista, Emil Vatai, and Mohamed Wahib. 2025. Safe: Improving llm systems using sentence-level in-generation attribution.arxiv.org/abs/2505.12621
  10. [10]Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. 2021. Stereotyping norwegian salmon: An inventory of pitfalls in fairness benchmark datasets.
  11. [11]Andreea-Elena Bodea, Stephen Meisenbacher, and Florian Matthes. 2026. A case study on the impact of anonymization along the rag pipeline.
  12. [12]Samuel Bowman and George Dahl. 2021. What will it take to fix benchmarking in natural language understanding?
  13. [13]Venkata Krishna Prasanth Budigi and Siri Chandana Sirigiri. 2026. Beyond similarity search: A unified data layer for production rag systems.arxiv.org/abs/2605.03275
  14. [14]Jannis Bulian, Christian Buck, Wojciech Gajewski, Benjamin Börschinger, and Tal Schuster. 2022. Tomayto, tomahto. beyond token-level answer equivalence for question answering evaluation.
  15. [15]Florian AD Burnat. 2026. Auditing privacy in multi-tenant rag under account collusion.arxiv.org/abs/2605.19847
  16. [16]Grace Byun, Shinsun Lee, Nayoung Choi, and Jinho D Choi. 2025. Secure multifaceted-rag: Hybrid knowledge retrieval with security filtering.DOI
  17. [17]Max Callaghan, Finn Müller-Hansen, Melissa Bond, Candyce Hamel, Declan Devane, Wojciech Kusa, Alison O’Mara-Eves, Rene Spijker, Mark Stevenson, Claire Stansfield, et al. 2024. Computer-assisted screening in systematic evidence synthesis requires robust and well-evaluated stopping criteria.DOI
  18. [18]Donald T Campbell and Donald W Fiske. 1959. Convergent and discriminant validation by the multitrait-multimethod matrix.pubmed.ncbi.nlm.nih.gov/13634291
  19. [19]Daniela Ceraku. 2026. Rag-h: A rag-hardened soc framework for trustworthy threat intelligence and ai defence.DOI
  20. [20]Chandranil Chakraborttii, Jackeline García Alvarado, Sitora Abdulofizova, and Shivanshu Dwivedi. 2026. Ghost vectors: Soft-deleted embeddings remain reconstructible in hnsw vector databases.arxiv.org/abs/2606.18497
  21. [21]Bingxiang Chen. 2024. An exploration of using retrieval-augmented generation with access control.
  22. [22]Jinwen Chen, Hainan Zhang, Liang Pang, Yongxin Tong, Haibo Zhou, Wei Lin, and Zhiming Zheng. 2026a. Privacy-preserving reasoning with knowledge-distilled parametric retrieval augmented generation.arxiv.org/abs/2509.01088
  23. [23]Kang Chen, Xiuzე Zhou, Yuanguo Lin, Shibo Feng, Li Shen, and Pengcheng Wu. 2025a. A survey on privacy risks and protection in large language models.arxiv.org/abs/2505.01976
  24. [24]Liuji Chen, Xiaofang Yang, Yuanzhuo Lu, Jinghao Zhang, Xin Sun, Qiang Liu, Shu Wu, Jing Dong, and Liang Wang. 2026b. Uncovering competing poisoning attacks in retrieval-augmented generation.arxiv.org/abs/2505.12574
  25. [25]Tianyu Chen, Jian Lou, and Wenjie Wang. 2025b. Safeguarding multimodal knowledge copyright in the rag-as-a-service environment.arxiv.org/abs/2506.10030
  26. [26]Tong Chen, Akari Asai, Niloofar Mireshghallah, Sewon Min, James Grimmelmann, Yejin Choi, Hanneah Hajishirzi, Luke Zettlemoyer, and Pang Wei Koh. 2024. Copybench: Measuring literal and non-literal reproduction of copyright-protected text in language model generation.
  27. [27]Ziyuan Chen, Yueming Lyu, Yi Liu, Weixiang Han, Jing Dong, Caifeng Shan, and Tieniu Tan. 2026c. Radar: Defending rag dynamically against retrieval corruption.arxiv.org/abs/2605.22041
  28. [28]Yihang Cheng, Lan Zhang, Junyang Wang, Mu Yuan, and Yunhao Yao. 2025a. Remoterag: A privacy-preserving llm cloud rag service.arxiv.org/abs/2412.12775
  29. [29]Zehua Cheng, Wei Dai, and Jiahao Sun. 2026. Identity-decoupled anonymization for visual evidence in multi-modal retrieval-augmented generation.DOI
  30. [30]Zirui Cheng, Jikai Sun, Anjun Gao, Yueyang Quan, Zhuqing Liu, Xiaohua Hu, and Minghong Fang. 2025b. Secure retrieval-augmented generation against poisoning attacks.
  31. [31]Giovanni Cherubin and Andrew Paver d. 2025. Highlight & summarize: Rag without the jailbreaks.arxiv.org/abs/2508.02872
  32. [32]Yujin Choi, Youngjoo Park, Junyoung Byun, Jaeweok Lee, and Jinseong Park. 2025. Safeguarding privacy of retrieval data against membership inference attacks: Is this query too close to home?DOI
  33. [33]Sarthak Choudhary, Nils Palumbo, Ashish Hooda, Krishnamurthy Dj Dvjotham, and Somesh Jha. 2025. Through the stealth lens: Attention-aware defenses against poisoning in rag.arxiv.org/abs/2506.04390
  34. [34]Stav Cohen, Ron Bitton, and Ben Nassi. 2024. Here comes the ai worm: Unleashing zero-click worms that target genai-powered applications.arxiv.org/abs/2403.02817
  35. [35]Lee J Cronbach and Paul E Meehl. 1955. Construct validity in psychological tests.
  36. [36]Huining Cui and Wei Liu. 2026. Needle-in-rag: Prompt-conditioned character-level traceback of poisoned spans in retrieved evidence.arxiv.org/abs/2605.01782
  37. [37]Badhan Chandra Das, M Hadi Amini, and Yanzhao Wu. 2025. Security and privacy challenges of large language models: A survey.arxiv.org/abs/2402.00888
  38. [38]Soumil Datta, Melissa Umble, Daniel S Brown, and Guanhong Tao. 2026. A sentence relation-based approach to sanitizing malicious instructions.arxiv.org/abs/2605.01078
  39. [39]Sagie Dekel, Moshe Tennenholtz, and Oren Kurland. 2026. Addressing corpus knowledge poisoning attacks on rag using sparse attention.arxiv.org/abs/2602.04711
  40. [40]Zeming Fei, Hongming Fei, Xiaoyang Wang, Prosanta Gope, Biplab Sikdar, Ying Zhang, et al. 2026. Selection integrity for llm graph memory: An accumulability criterion for information-flow-blind retrieval.arxiv.org/abs/2606.12290
  41. [41]Haowei Fu, Bo Ni, Han Xu, Kunpeng Liu, Dan Lin, and Tyler Derr. 2026. Ensemble privacy defense for knowledge-intensive llms against membership inference attacks.
  42. [42]Chengcai Gao, Zhihong Sun, Xiaochuan Shi, Qiufeng Wang, and Chao Liang. 2026. Bird: A bidirectional ranking defense mechanism for retrieval augmented generation.arxiv.org/abs/2605.20123
  43. [43]Yunfan Gao. 2023. Retrieval-augmented generation for large language models: A survey.arxiv.org/abs/2312.10997
  44. [44]Armin Gerami, Kazem Faghih, and Ramani Dura iswami. 2025. Quantifying document impact in rag-llms.arxiv.org/abs/2601.05260
  45. [45]Aditya Golatkar, Alessandro Achille, Luca Zancato, Yu-Xiang Wang, Ashwin Swaminathan, and Stefano Soatto. 2024. CPr: Retrieval augmented generation for copyright protection.arxiv.org/abs/2403.18920
  46. [46]Ishrith Gowda. 2026. Memsad: Gradient-coupled anomaly detection for memory poisoning in retrieval-augmented agents.arxiv.org/abs/2605.03482
  47. [47]Junfeng Guo, Yiming Li, Ruibo Chen, Yihan Wu, Chenxi Liu, Yanshuo Chen, and Heng Huang. 2025. Towards copyright protection for knowledge bases of retrieval-augmented language models via reasoning.arxiv.org/abs/2502.10440
  48. [48]Giwon Hong, Jeonghwan Kim, Junmo Kang, Sung-Hyon Myaeng, and Joyce Jiyoung Whang. 2024. Why so gullible? enhancing the robustness of retrieval-augmented models against counterfactual noise.
  49. [50]Hao Huang, JiaTang Luo, Ruihua Zhou, Yunpeng Li, and Yuling Liu. 2026. Don’t corrupt the fact: A trustworthy rag watermarking framework based on dual factual shield.
  50. [51]Abigail Z Jacobs and Hanna Wallach. 2021. Measurement and fairness.DOI
  51. [52]Austin Jia, Avaneesh Ramesh, Zain Shamsi, Daniel Zhang, and Alex Liu. 2025. Ragrank: Using pagerank to counter poisoning in cti llm pipelines.
  52. [53]Shuaifan Jin, Xiaoyi Pang, Zhibo Wang, He Wang, Jiacheng Du, Jiahui Hu, and Kui Ren. 2025. Safeguarding llm embeddings in end-cloud collaboration via entropy-driven perturbation.
  53. [54]Weifei Jin, Xilong Wang, Wei Zou, Jinyuan Jia, and Neil Gong. 2026. Cleanbase: Detecting malicious documents in rag knowledge databases.arxiv.org/abs/2605.00460
  54. [55]Nikola Jovanović, Robin Staab, Maximilian Baader, and Martin Vechev. 2025. Ward: Provable rag dataset inference via llm watermarks.arxiv.org/abs/2410.03537
  55. [56]Ehsan Kamalloo, Nouha Dziri, Charles LA Clarke, and Davood Rafiei. 2023. Evaluating open-domain question answering in the era of large language models.
  56. [57]Sayash Kapoor and Arvind Narayanan. 2023. Leakage and the reproducibility crisis in machine-learning-based science.DOI
  57. [58]Minseok Kim, Hankook Lee, and Hyungjoon Koo. 2025. Rescuing the unpoisoned: Efficient defense against knowledge corruption attacks on rag systems.arxiv.org/abs/2511.01268
  58. [59]Tatsuki Koga, Ruihan Wu, Zhiyuan Zhang, and Kamalika Chaudhuri. 2024. Privacy-preserving retrieval-augmented generation with differential privacy.arxiv.org/abs/2412.04697
  59. [60]Tsun On Kwok, Xi Yang, Ki Sen Hung, Chang Liu, and Yangqiu Song. 2026. Sentinlerag: Synthetic sentinel knowledge for rag database copyright protection.arxiv.org/abs/2606.05787
  60. [61]Jonathan Lebensold, Maziar Sanjabi, Pietro Astolfi, Adriana Romero-Soriano, Kamalika Chaudhuri, Mike Rabbat, and Chuan Guo. 2024. Dpr-dm: Adapting diffusion models to private domains without fine-tuning.
  61. [62]Jun Wen Leong. 2026. Defense effectiveness across architectural layers: a mechanistic evaluation of persistent memory attacks on stateful llm agents.
  62. [63]Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks.
  63. [64]Qinfeng Li, Miao Pan, Ke Xiong, Ge Su, Zhiqiang Shen, Yan Liu, Sun Bing, Hao Peng, and Xuhong Zhang. 2026a. Ragfort: Dual-path defense against proprietary knowledge base extraction in retrieval-augmented generation.
  64. [65]Yan­hang Li, Zhichao Fan, and Zexin Zhuang. 2026b. Where privacy risk lives in english-source multilingual rag: A stage-decomposed audit across five query languages.
  65. [66]Zhijun Li, Minghui Xu, Huayi Qi, Wenxuan Yu, Tingchuang Zhang, Qiao Zhang, GuangYong Shang, Zhen Ma, and Xiuzhen Cheng. 2026c. Prag: End-to-end privacy-preserving retrieval-augmented generation.arxiv.org/abs/2604.26525
  66. [67]Xun Liang, Mengwei Wang, Yuefeng Ma, and Simin Niu. 2026a. Safe rag by rag: Untying the bell that rag rang with the rag hand.
  67. [68]Zhilin Liang, Yuxiang Wang, Zimu Zhou, Hainan Zhang, Boyi Liu, and Yongxin Tong. 2026b. Fedmosaic: Federated retrieval-augmented generation via parametric adapters.
  68. [69]Bo Lin, Shangwen Wang, Yihao Qin, Liqian Chen, and Xiaoguang Mao. 2025. Give llms a security course: Securing retrieval-augmented code generation via knowledge injection.arxiv.org/abs/2504.16429
  69. [70]Victoria Lin, Xilun Chen, Mingda Chen, Weijia Shi, Maria Lomeli, Richard James, Pedro Rodriguez, Jacob Kahn, Gergely Szilvasy, Mike Lewis, et al. 2024. Ra-dit: Retrieval-augmented dual instruction tuning.arxiv.org/abs/2310.01352
  70. [71]Zachary C Lipton and Jacob Steinhardt. 2019. Research for practice: troubling trends in machine-learning scholarship.DOI
  71. [72]Mingrui Liu, Sixiao Zhang, and Cheng Long. 2025a. Mask-based membership inference attacks for retrieval-augmented generation.arxiv.org/abs/2410.20142
  72. [73]Peiyang Liu, Ziqiang Cui, Di Liang, and Wei Ye. 2025b. Who stole your data? a method for detecting unauthorized rag theft.arxiv.org/abs/2510.07728
  73. [74]Xuan Liu, Hefeng Zhou, Sicheng Chen, Chao Yang, Xingcheng Xu, Jingjing Qu, Jiong Lou, Jie Li, and Xia Hu. 2026. Privacy-preserving text sanitization for distributed agents collaboration via disentangled representations.arxiv.org/abs/2606.15335
  74. [75]Yepeng Liu, Xuandong Zhao, Dawn Song, and Yuheng Bu. 2025c. Dataset protection via watermarked canaries in retrieval-augmented llms.arxiv.org/abs/2502.10673
  75. [76]Michele Lorenzo, Idilio Drago, Dario Salvadori, and Fabio Romolo Vayr. 2025. Arbiter: Ai-driven filtering for role-based access control.arxiv.org/abs/2512.20535
  76. [77]Ziyuan Luo, Yangyi Zhao, Ka Chun Cheung, Simon See, and Renjie Wan. 2026. Imagesentinel: Protecting visual datasets from unauthorized retrieval-augmented image generation.arxiv.org/abs/2510.12119
  77. [78]Peizhuo Lv, Mengjie Sun, Hao Wang, Xiaofeng Wang, Shengzhi Zhang, Yuxuan Chen, Kai Chen, and Limin Sun. 2025. Rag-wm: An efficient black-box watermarking approach for retrieval-augmented generation of large language models.arxiv.org/abs/2501.05249
  78. [79]Chenxin Mao, Shangyu Liu, Zhenzhe Zheng, Fan Wu, Jie Wu, and Guihai Chen. 2026. An efficient and privacy-preserving architecture for cross-institutional collaborative rag.arxiv.org/abs/2605.25716
  79. [80]Aiman Al Masoud, Marco Arazzi, and Antonino Nocera. 2026. Sd-rag: A prompt-injection-resilient framework for selective disclosure in retrieval-augmented generation.
  80. [81]Samuel Messick. 1995. Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning.
  81. [82]Sewon Min, Suchin Gururangan, Eric Wallace, Weijia Shi, Hannaneh Hajishirzi, Noah Smith, and Luke Zettlemoyer. 2024. Silo language models: Isolating legal risk in a nonparametric datastore.arxiv.org/abs/2308.04430
  82. [83]Junki Mori, Kazuya Kakizaki, Taiki Miyagawa, and Jun Sakuma. 2026. Differentially private synthetic text generation for retrieval-augmented generation (rag).
  83. [84]Richard Helder Moulton, Austin O’Brien, and John D Hastings. 2025. Quantifying return on security controls in llm systems.arxiv.org/abs/2512.15081
  84. [85]Charafeddine Mouzouni. 2026. Context kubernetes: Declarative orchestration of enterprise knowledge for agentic ai systems.arxiv.org/abs/2604.11623
  85. [86]Yanming Mu, Hao Hu, Feiyang Li, Qiao Yuan, Jiang Wu, Zichuan Liu, Pengcheng Liu, Mei Wang, Hongwei Zhou, and Yuling Liu. 2026. Towards secure retrieval-augmented generation: A comprehensive review of threats, defenses and benchmarks.arxiv.org/abs/2603.21654
  86. [87]Rohith Namboothiri. 2026. Authorization-first retrieval: Enforcing least privilege in multi-agent rag systems.DOI
  87. [88]Ikhtiyor Nematov, Tarik Kalai, Elizaveta Kuzmenko, Gabriele F...ug...noli, Dimitris Sacharidis, Katja Hose, and Tomer Sagi. 2025. Source attribution in retrieval-augmented generation.arxiv.org/abs/2507.04480
  88. [89]Havva Alizadeh Noughabi, Fattane Zarrinkalam, and Ali Dehghantanha. 2026. Defense against knowledge poisoning attack on graphrag.
  89. [90]Sandeep Nutakki. 2026. SENTINEL: Retrieval-time governance for enterprise RAG a controlled red-team benchmark for policy-aware retrieval, privacy enforcement, and audit replay. Journal of Advances in Developmental Research, 17(1).
  90. [91]Ciyan Ouyang and Rui Hou. 2026. Memlineage: Lineage-guided enforcement for llm agent memory.arxiv.org/abs/2605.14421
  91. [92]Balamurugan Palanisamy, GSS Chalapathi, Vikas Hassija, and Rajkumar Buyya. 2026. Security and privacy in retrieval-augmented generation: Architectures, threats, defenses, and future directions for building trustworthy systems.
  92. [94]Francesco Panebianco, Stefano Bonfanti, Francesco Trovò, and Michele Carminati. 2025. Leaksealer: A semisupervised defense for llms against prompt injection and leakage attacks.
  93. [95]Prashant Kumar Pathak and Tarun Kumar Sharma. 2026. When global gating is enough: Admission-time hubness control in anisotropic vector retrieval systems.
  94. [96]Pankayaraj Pathmanathan, Michael-Andrei Panaitescu-Liess, Cho-Yu Jason Chiang, and Furong Huang. 2025. Ragpart & ragmask: Retrieval-stage defenses against corpus poisoning in retrieval-augmented generation.
  95. [97]KrishnaSaiReddy Patil. 2026. Ragshield: Detecting numerical claim manipulation in government rag systems.
  96. [98]Alvaro Lopez Pellicer, Plamen Angelov, Marwan Bukhari, Yi Li, Eduardo Soares, and Jemma Kerns. 2026. Protomedagent: Multimodal clinical interpretability via privacy-aware agentic workflows.
  97. [99]Micah Dl Peters, Casey Marnie, Andrea C Tricco, Danielle Pollock, Zachary Munn, Lyndsay Alexander, Patricia McInerney, Christina M Godfrey, and Hanan Khalil. 2021. Updated methodological guidance for the conduct of scoping reviews.
  98. [100]Jirui Qi, Gabriele Sarti, Raquel Fernández, and Arianna Bisazza. 2024. Model internals-based answer attribution for trustworthy retrieval-augmented generation.
  99. [101]Zhisheng Qi, Utkarsh Sahu, Li Ma, Haoyu Han, Ryan Rossi, Franck Dernoncourt, Mahantesh Halappanavar, Nesreen Ahmed, Yushun Dong, Yue Zhao, et al. 2026. Benchmarking knowledge-extraction attack and defense on retrieval-augmented generation (rag).
  100. [102]Inioluwa Deborah Raji, Emily M Bender, Amandalynne Paullada, Emily Denton, and Alex Hanna. 2021. Ai and the everything in the whole wide world benchmark.
  101. [103]Gulshan Saleem, Nisar Ahmed, Muhammad Imran Zaman, Ali Hassan, and Umar Mujahid. 2026. A layered security framework against prompt injection in rag-based chatbots.
  102. [104]Alireza Salemi and Hamed Zamani. 2026. Improving user privacy in personalized generation: Client-side retrieval-augmented modification of server-side generated speculations.
  103. [105]Yash Saxena, Ankur Padia, Mandar S Chaudhary, Kalpa Gunaratna, Srinivasan Parthasarathy, and Manas Gaur. 2025. Ranking free rag: Replacing re-ranking with selection in rag for sensitive domains.
  104. [106]Yangguang Shao, Chengshang Hou, Gaopeng Gou, Gang Xiong, and Junzheng Shi. 2026. Poisoncraft: Practical poisoning of retrieval-augmented generation for large language models.
  105. [107]Tarun Sharma. 2026. Smsr: Certified defence against runtime memory poisoning in persistent llm agent systems.
  106. [108]Zeyu Shen, Basileal Imana, Tong Wu, Chong Xiang, Prateek Mittal, and Aleksandra Korolova. 2026. Reliabilityrag: Effective and provably robust defense for rag-based web-search.
  107. [109]Yu-Kai Shih and You-Kai Kang. 2025. Design and implementation of a secure rag-enhanced ai chatbot for smart tourism customer service: Defending against prompt injection attacks—a case study of hsinchu, taiwan.
  108. [110]Xiaonan Si, Meilin Zhu, Simeng Qin, Lijia Yu, Lijun Zhang, Shuaitong Liu, Xinfeng Li, Ranjie Duan, Yang Liu, and Xiaojun Jia. 2026. Secon-rag: A two-stage semantic filtering and conflict-free framework for trustworthy rag.
  109. [111]Xue Tan, Hao Luan, Mingyu Luo, Xiaoyan Sun, Ping Chen, and Jun Dai. 2024. Revprag: Revealing poisoning attacks in retrieval-augmented generation through llm activation analysis.
  110. [112]Xue Tan, Yi Zheng, Chang Huo, Yunruo Zhang, Yu Liu, Hao Luan, Zhu yang Yu, Jun Dai, Xiaoyan Sun, and Ping Chen. 2026. Pra-rag: Provably robust aggregation in retrieval-augmented generation against retrieval corruption.
  111. [113]Scott Thornton. 2026a. Retrieval pivot attacks in hybrid rag: Measuring and mitigating amplified leakage from vector seeds to graph expansion.
  112. [114]Scott Thornton. 2026b. Semantic chameleon: Corpus-dependent poisoning attacks and defenses in rag systems.
  113. [115]Andrea C. Tricco, Erin Lillie, Wasifa Zarin, Kelly K O’Brien, Heather Colquhoun, Danielle Levac, David Moher, Micah DJ Peters, Tanya Horsley, Laura Weeks, et al. 2018. Prisma extension for scoping reviews (prisma-scr): checklist and explanation.DOI
  114. [116]Haoran Wang, Xiongxiao Xu, Baixiang Huang, and Kai Shu. 2026a. Privacy-aware decoding: Mitigating privacy leakage of large language models in retrieval-augmented generation.
  115. [117]Shang Wang, Tianqing Zhu, Dayong Ye, and Wanlei Zhou. 2025a. When machine unlearning meets retrieval-augmented generation (rag): Keep secret or forget knowledge?arxiv.org/abs/2410.15267
  116. [118]Weijie Wang, Peizhuo Lv, Yan Wang, Weiqing Huang, Rujie Dai, Guokun Xu, Qiujian Lv, Hangcheng Liu, Hao Hu, and Jiacheng Zhang. 2026b. Making theft useless: Adulteration-based protection of proprietary knowledge graphs in graph systems.arxiv.org/abs/2601.00274
  117. [119]Yanting Wang, Runpeng Geng, Ying Chen, and Jinyuan Jia. 2026c. Attentrace: Contextual attribution of prompt injection and knowledge corruption.arxiv.org/abs/2508.03793
  118. [120]Yanting Wang, Wei Zou, Runpeng Geng, and Jinyuan Jia. 2025b. {TracLLM}: A generic framework for attributing long context {LLMs}.arxiv.org/abs/2506.04202
  119. [121]Yuhao Wang, Wenjie Qu, Shengfang Zhai, Yanze Jiang, Zichen Liu, Yue Liu, Yinpeng Dong, and Jiacheng Zhang. 2025c. Silent leaks: Implicit knowledge extraction attack on rag systems through benign queries.arxiv.org/abs/2505.15420
  120. [122]Yujing Wang, Jinwen Chen, Hainan Zhang, Liang Pang, Yongxin Tong, Binghui Guo, Hongwei Zheng, and Zhiming Zheng. 2025d. Learning to erase private knowledge from multi-documents for retrieval-augmented large language models.arxiv.org/abs/2504.09910
  121. [123]Jascha Wanger. 2026. Vectorsmuggle: Steganographic exfiltration in embedding stores and a cryptographic provenance defense.
  122. [124]Pingyu Wu, Daiheng Gao, Jing Tang, Huimin Chen, Wenbo Zhou, Weiming Zhang, and Nenghai Yu. 2025a. Mes-rag: Bringing multi-modal, entity-storage, and secure enhancements to rag.arxiv.org/abs/2503.13563
  123. [125]Ruihan Wu, Erchi Wang, Zhiyuan Zhang, and Yu-Xiang Wang. 2025b. Private-rag: Answering multiple queries with llms while keeping your data private.arxiv.org/abs/2511.07637
  124. [126]Xun Xian, Ganghua Wang, Xuan Bi, Jayanth Srinivasa, Ashish Kundu, Charles Fleming, Mingyi Hong, and Jie Ding. 2024. On the vulnerability of applying retrieval-augmented generation within knowledge-intensive application domains.arxiv.org/abs/2409.17275
  125. [127]Chong Xiang, Tong Wu, Zexuan Zhong, David Wagner, Danqi Chen, and Prateek Mittal. 2026. Certifiably robust rag against retrieval corruption.arxiv.org/abs/2405.15556
  126. [128]Jin Xie, Songze Li, and Guangk Cheng. 2026a. Seal-tag: Self-tag evidence aggregation with probabilistic circuits for pii-safe retrieval-augmented generation.arxiv.org/abs/2603.17292
  127. [129]Yuanbo Xie, Yingjie Zhang, Yulin Li, Shouyou Song, Xiaokun Chen, Zhihan Liu, Liya Su, and Tingwen Liu. 2026b. Detecting rag extraction attack via dual-path runtime integrity game.
  128. [130]Yuming Xu, Mingtao Zhang, Zhuohan Ge, Haoyang Li, Nicole Hu, Yongqi Zhang, Zhiyuan Wen, Jason Chen Zhang, Qing Li, and Lei Chen. 2026. Securing retrieval-augmented generation: A taxonomy of attacks, defenses, and future directions.
  129. [131]Zhenhua Xu, Xubin Yue, Zhebo Wang, Haobo Zhang, Qichen Liu, Xixiang Zhao, Jingxuan Zhang, Wenjun Zeng, Wengpeng Xing, Dezhang Kong, et al. 2025. Copyright protection for large language models: A survey of methods, challenges, and trends.arxiv.org/abs/2508.11548
  130. [132]Ke Yang, Shuguang Yuan, Jing Yu, and Chi Chen. 2026. Knowledge-infused multi-bit watermarking for rag knowledge bases.DOI
  131. [133]Shu Yang, Jiayuan Su, Han Jiang, Mengdi Li, Keyuan Cheng, Muhammad Asif Ali, Lijie Hu, and Di Wang. 2024. Dialectical alignment: Resolving the tension of 3h and security threats of llms.arxiv.org/abs/2404.00486
  132. [134]Zhiguang Yang, Gejian Zhao, and Hanzhou Wu. 2025. Watermarking for large language models: A survey.DOI
  133. [135]Hongwei Yao, Haoran Shi, Yidou Chen, Yixin Jiang, Cong Wang, and Zhan Qin. 2025a. Controlnet: A firewall for rag-based llm system.arxiv.org/abs/2504.09593
  134. [136]Ruobing Yao, Yifei Zhang, Shuang Song, Neng Gao, and Chenyang Tu. 2025b. Ecosafear: Efficient security through context analysis in retrieval-augmented generation.arxiv.org/abs/2505.13506
  135. [137]Zhihao Yao, Yuxuan Gu, Xiaochong Feng, Weitao Ma, Bo Li, Xiaocheng Feng, and Bing Qin. 2026. Adaptive backtracking for privacy protection in large language models.arxiv.org/abs/2508.06087
  136. [138]Huanyi Ye, Jiale Guo, Ziyao Liu, and Kwok-Yan Lam. 2025. Efficient privacy-preserving retrieval augmented generation with distance-preserving encryption.
  137. [139]Zhe Yu, Wenpeng Xing, Gaolei Li, Shuguang Xiong, Hongzhi Wang, Xuyang Teng, and Meng Han. 2026. Cordon-mas: Defending rag against knowledge poisoning via information-flow control.arxiv.org/abs/2605.26754
  138. [140]Osama Zafar, Alexander Nemecek, Yiqian Zhang, Wenbiao Li, Debargha Ganguly, Vikash Singh, Vipin Chaudhary, and Erman Ayday. 2026. Privacy policy enforcement guardrails for data-sensitive retrieval-augmented generation.arxiv.org/abs/2605.17034
  139. [141]Shenglai Zeng, Jiankun Zhang, Pengfei He, Yiding Liu, Yue Xing, Han Xu, Jie Ren, Yi Chang, Shouqiang Wang, Dawei Yin, et al. 2024a. The good and the bad: Exploring privacy issues in retrieval-augmented generation (rag).arxiv.org/abs/2402.16893

Paper details

vlnc:W_yn796ebq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 4

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents