A knowledge base that checks permissions before the model

A company knowledge base with a language model has a failure mode that typical demos overlook: a document the user is not allowed to open can still shape the answer. It may appear as a citation or influence the wording more subtly. Even a refusal can reveal that a restricted document exists and indicate what it concerns.

Filtering results after generation does not solve the problem. If a restricted document entered the model's context, removing its citation does not undo its influence on the answer.

Order as architecture

In the knowledge reading room we built, identity and scope are resolved before retrieval begins. The controlled corpus contains 600 generated documents, five fictional identities, and no client content or credentials.

OIDC login issues a signed session with the user's identity and groups. Search combines full-text and semantic matching, but always includes an explicit group filter. A document outside the user's scope never appears among the candidate results. That is the first control.

The second control is independent of the first. Row-level security (RLS) enforces the same scope inside the database, outside the application code. A query bug, a skipped filter, or a regression after a release is therefore insufficient to expose a restricted document.

The generative model receives only material that passed both controls. Citations can point solely to documents available to the user, and a refusal reveals nothing about documents beyond that scope.

Preparing permissions before indexing

The hardest implementation work is often mapping the organization's existing access model. Identify the authoritative identity source, group membership, document owners, and the lifecycle for removing access after a role change or departure. The search index should not maintain a manual second copy of permissions when an authoritative system already exists.

Each document needs a stable identifier, owner, current-version status, access scope, and update date. An email attachment and a file-share copy may contain identical text but have different audiences. Deduplication must not widen access by merging them into a single representation with broader permissions.

Define what happens after access is revoked. Sessions and caches need an acceptable propagation time, and saved answers must not become permanent copies of material the user can no longer open.

A useful pilot sequence

Start with a limited corpus and several intentionally different roles. Build a matrix that asks the same questions under authorized and unauthorized identities. Include answers, citations, absent sources, conflicting documents, and direct attempts to open a closed record.

Measure retrieval only among documents each identity may see. The set should use employee vocabulary, abbreviations, former product names, and questions without document identifiers. Report access enforcement and relevance separately. A safe refusal is a correct security result and can still be a poor user outcome.

Add answer drafting last. Returning cited passages first makes evaluation simpler. Generation earns its place when users need a synthesis of several consistent sources and the workflow can detect conflict and stop.

Reading the results and cost

Zero out-of-scope releases applies to the prepared identity-question matrix. It does not establish that every corporate group is configured correctly or that revocations propagate quickly. Test identity integration, sessions, caches, service accounts, and administrative access separately.

The 51.1% first-result score on the harder paraphrase set shows that strong access control does not create a strong search engine. Define minimum relevance for critical categories and a conservative refusal threshold before launch. Returning more passages may improve coverage while increasing latency, infrastructure use, and the amount of permitted material passed to the model.

The recorded local drafting run incurred no external API fees. Infrastructure, indexing, identity integration, and document-owner work still carry cost. Measure time to a usable answer, the share of answers that open the correct source, and questions routed to document owners.

Knowledge-base checklist

1. Which system is authoritative for identity and group membership?
2. How quickly does a revocation reach sessions, the index, and caches?
3. Is scope enforced in the query and independently in the database?
4. Is the refusal identical whether a restricted document exists or not?
5. How are current versions and conflicting sources identified?
6. Do saved answers respect later loss of access?
7. Is relevance measured separately by role, vocabulary, and document category?
8. Who owns stale content and approves wider access?
9. Are administrators and service accounts subject to controlled scope?

Conflicts remain visible

Knowledge bases have a second failure mode: contradictory sources. An old price list beside a new one, two reports with different numbers. A model that averages them into one smooth answer produces credible-sounding untruth.

In this reading room, a source conflict is an explicit status. The system identifies the conflicting sources, and their owners resolve the disagreement. When no accessible source answers the question, the result states that gap explicitly.

The measurement

The test asks 150 reference questions through five identities with different access scopes, producing 750 question-and-identity combinations. Answer, citation, and refusal checks all passed, with zero releases outside the asker's scope. A harder paraphrase check put the intended document first in 51.1% of 45 eligible pairs, which is too weak for broad semantic search. The set also includes a report conflict, an outdated price list, a closed HR document, and an injection attempt. Search plus the access policy completed with a 95th-percentile time of 17.758 ms, excluding login, interface, and local drafting.

The 600 documents are generated, and the matrix measures permission enforcement. Search quality depends on the client's vocabulary and document set, so semantic-matching choices and relevance testing remain deployment work. The optional local drafting check incurred PLN 0 in external API fees.

The question worth asking a vendor

If you are considering an AI knowledge base, ask what happens to a restricted document before the model starts generating. Filtering only the finished result is not a sufficient access control.

Scope and accuracy are separate measurements. A system can enforce permissions flawlessly and still fail to surface documents the asker has every right to see. The quality of retrieval itself is the subject of why your AI knowledge base cannot find answers.

The system's details and the full measurement matrix are on the case page, and a free process scan is a good place to test this pattern against your permission structure.