Retrieval-Augmented Generation (RAG) is often sold as a safety upgrade — ground the model in trusted text and hallucinations largely disappear. That promise is not just casual hype. A 2024 Stanford research review of AI legal research tools notes that vendors market RAG as authoritative grounding, including a Thomson Reuters executive’s claim that it can reduce hallucinations “to nearly zero.”
That story can be true in small, stable domains where edge cases are rare and yesterday’s answer is usually safe today. But as an organization grows and its corpus expands, the ground truth stops being a single body of text and becomes a set of variants. Policies split by region, plan tier, product version, enrollment cohort, and effective date. Each document remains “correct,” yet the system produces the most expensive kind of failure: an answer that is well cited, authoritative, and completely wrong.
The core question shifts. Not “is this statement supported by a source?” but “does this supported statement apply here?”
The problem is applicability: whether correct information governs this situation, right now.
One Question, Many Realities
Imagine an electric utilities company that sells small appliances and administers warranty and replacement programs. They ship a support chatbot. A customer asks:
“My toaster isn’t working — can I get a replacement?”
In a small corpus, retrieval feels like safety. There is only one version of the answer, so finding the right topic means finding the right answer. In a mature support corpus, the same question hides the variables that determine the correct answer — the model of the toaster, when it was bought, where the customer lives, and which program issued it.
A naive RAG system retrieves a generally relevant warranty chunk — “fails within 24 months” — and produces a confident “yes” with steps and a citation. The claim is then rejected by the actual eligibility workflow because a different, still valid, policy branch applies.
Nothing was fabricated. The failure is selection: the system never chose which policy branch it was answering for.
This is why “relevance” stops meaning “right” as the corpus grows. Similarity search is optimized for topical alignment — text about “toaster replacement” — not for compatibility with an eligibility state, configuration, or timeline.
At scale, the most misleading outputs often look the most professional. Retrieval pulls multiple correct chunks that assume different conditions, and the model blends them into one fluent procedure — a “franken-answer” where each chunk is true on its own, but the combined answer becomes incoherent because it spans mutually incompatible assumptions.
What a Franken-Answer Looks Like
Franken-answer (bad): Yes — you can get a replacement toaster if your unit failed within 24 months of purchase. Complete the in-chat troubleshooting checklist, and we’ll confirm eligibility. If approved, we’ll ship a replacement or issue a voucher if shipping isn’t available in your area. Appliances from Program X are covered for 12 months from enrollment and must be registered first.
Next Steps:
- Share your region and purchase details.
- Confirm retail purchase or Program X.
- Provide proof of purchase and registration (if applicable).
- After troubleshooting, we’ll ship your replacement (or send a voucher).
Sources: Warranty policy (24 months); Region B fulfillment (voucher); Program X addendum (12 months + registration); Updated workflow (Jan 1).
Why this fails: it promises a 24-month replacement and 12-month Program X coverage without choosing which applies. It offers shipping and vouchers without selecting which. It mixes “purchase date” and “enrollment date” logic. It blends the post-January troubleshooting requirement into all cases, even where it doesn’t apply.
Nothing is invented, everything is cited, yet no real customer can follow the combined procedure.
The Compatibility Envelope
When teams say “RAG works,” the claim often means: given the right page, the model can summarize it. At scale, the hard work moves earlier — the system must decide which page is eligible to be right for the current case.
What most RAG architectures lack is a compatibility envelope: the set of conditions that make an answer applicable to a specific case. Outside that envelope, a retrieved document is not “less relevant” — it is a disallowed truth. And most systems have no representation of this envelope at all.
The gap between what retrieval optimizes for (topical similarity) and what correctness requires (scope compatibility) only widens as the corpus grows. Large commercial systems routinely encode policy in configuration, cohorts, and feature-flag rules that never appear verbatim in customer-facing documentation. The retrieval layer may return impeccably written policy text while missing the operational condition that selects which policy governs.
The Facets of Applicability
The toaster example illustrates branching by customer attributes, but applicability fractures along many axes simultaneously.
Temporal applicability: A corpus often contains both a current policy and the outdated policy it replaced. Because older documents have existed longer, they tend to have richer detail and more embedding weight from historical usage. Stale truth actively outranks current truth because the retrieval layer has no concept of temporal validity windows.
Compositional applicability: The franken-answer demonstrates this. A single response assembles facts from multiple retrieved chunks, each carrying its own implicit scope. No individual chunk is wrong — the act of combining them is the error. Per-chunk evaluation will never catch failures that live in the seams between documents.
Implicit conditions: Often, the information needed to make a branching decision does not exist in the retrieval corpus at all. Whether a customer qualifies under the standard warranty or the utility program may depend on an enrollment flag in a CRM. No document says “if enrollment_flag = PROGRAM_X, use the 12-month policy.” The documentation describes each program’s terms separately, assuming the reader already knows which one they’re in. When the knowledge that determines applicability does not exist in the text being retrieved, no amount of retrieval optimization can surface it.
Ambiguity: “My toaster is broken, can I get a replacement?” feels like a complete question. It is massively underspecified — but only relative to the corpus’s branching structure, not relative to everyday language. Standard query disambiguation handles linguistic ambiguity (“bank” means financial institution or riverbank). Applicability disambiguation is different: the question is linguistically clear, but the answer space branches in ways the user cannot anticipate because they don’t know the topology of the policy corpus. The system must recognize underspecification that the user has no reason to suspect exists.
Path convergence: Multiple policy branches can produce the same surface-level answer — “yes, you qualify for a replacement” — via entirely different logic paths, with different downstream consequences: shipping versus vouchers, different documentation requirements, different timelines. The system can appear correct while having reasoned through the wrong branch. The error surfaces only later, in a rejected claim or a confused follow-up. Checking the final answer is not enough — you must verify that the answer was derived from the correct scope.
Applicability Is an Infrastructure Problem
The most common response to these failures is more prompt engineering — more context, step-by-step reasoning, instructions to prefer recent documents. These are upstream fixes that address output behavior without changing the fundamental problem of candidate selection.
RAG retrieves what is written. Applicability decides what is allowed to be true. That distinction sounds philosophical until you try to fix it with prompting or reranking alone.
Solving the applicability problem at its root means treating scope as a first-class retrieval constraint. Attributes like region, user tier, software version, and effective date must become machine-usable filters applied to vector search before generation begins. Metadata is not a luxury — it is the infrastructure that prevents the model from looking at the wrong branch of truth.
The RAG systems that hold up at scale are not necessarily those using the most sophisticated LLMs. They are systems that treat knowledge as structured data — with explicit conditions, validity windows, and authority levels — allowing the retrieval layer to enforce them strictly instead of guessing.
The central metric for a mature RAG system is no longer whether the answer is grounded. It is whether the system chose the correct scope before it started generating.
