Internal legal knowledge search: design the access test before the search box

I would begin with a collection that has a clear owner and a known permission model, such as approved internal templates. Ingesting the entire document management system creates a permissions project, a records project, and a search project at once. A smaller collection makes it possible to establish whether the controls actually work.

For each document, record its source identifier, version, owner, permitted groups, current status, and last successful permission refresh. Keep that metadata with every searchable chunk. Splitting a document into smaller pieces must not detach a paragraph from the restrictions that apply to its parent.

Make collection membership explicit. A file appearing in a convenient shared folder does not establish that it belongs in a firm-wide knowledge base. Someone with responsibility for that material should approve its inclusion. A rejected or retired template should disappear from ordinary results while its records lifecycle remains governed separately.

Microsoft documents security filters that exclude search results based on the calling user's identity or group membership. It also describes native document-level permission approaches, some marked preview. Those distinctions matter when selecting a deployment path; a product capability is not evidence that your own integration configured it correctly. Microsoft's document access guidance

My design would derive identity from the authenticated session on the server. A browser-supplied group name or a sentence in a chat message should never grant access. The search service should receive only the permissions the application verified for that request.

Filter before sending retrieved text to the language model. Hiding an unauthorized citation in the interface after generation is too late: the answer may already contain the restricted information. Apply the same boundary to document titles, snippets, suggested questions, download previews, and exports.

The source system should remain authoritative. If the index cannot determine whether a permission record is current enough, the application needs an explicit behavior: exclude the affected material, route the request for help, or decline the answer. Quietly using yesterday's access list is a product decision with consequences, not a harmless cache optimization.

A user can be authorized to read an obsolete document. Permission checks therefore cannot establish that a clause is current, approved, or appropriate for the matter at hand.

I would give the answer a visible source date and document status. If the collection contains two approved versions with conflicting wording, show the conflict and link both. Avoid asking the model to silently choose which policy is authoritative. The document owner should resolve that ambiguity in the source collection.

An internal answer should also distinguish a quotation, a summary, and an inference. A reviewer needs to open the relevant passage without searching through a fifty-page document. Page references or anchored sections are helpful, but only if they point to the exact version used in the answer.

Create a synthetic collection with invented matters and no client information. Give test accounts deliberately different access. One account can see a template; another can see a restricted matter; a third can see neither. Place a unique invented phrase in the restricted file so accidental disclosure is detectable.

Ask direct questions, paraphrases, and indirect questions that might reveal the matter's existence. Test search suggestions and related-document cards as well as the main answer. A denial test that looks only for the complete source text can miss disclosure through a title or a summary.

Then change the account's permissions and repeat the queries. Revoke access while a conversation remains open. Check whether old answer history, exports, and saved links remain accessible under the firm's chosen policy. The expected behavior must be agreed before the implementation is called complete.

I would also test a document containing instructions such as “ignore prior restrictions.” That text is document content, not an authorization instruction. The application should continue to enforce its verified permission rules regardless of what a retrieved file says.

Search systems often create copies beyond the original index. There may be a reranking cache, conversation history, generated summaries, error logs, and analytics events. Each can become a second place where restricted material survives.

Keep a data-flow inventory that identifies which stores contain document text and which contain only operational metadata. For each store, name who can read it, how long it persists, and how revocation or deletion reaches it. This inventory should include development and support tooling, not just production databases.

For debugging, I would prefer request identifiers, permission decision codes, and document identifiers over full prompt capture. If content logging is necessary for a specific investigation, scope it explicitly and review its access. A restricted answer copied into an unrestricted error dashboard defeats the original access design.

A permission sync failure needs a person who can act on it. The alert should identify the affected collection, the last successful refresh, and whether the application has stopped returning that content. A generic “sync failed” notification leaves the operator guessing about exposure and availability.

The runbook should explain how to disable a collection, refresh permissions, invalidate affected caches, and verify recovery using the synthetic denial tests. It should also distinguish a connector outage from a policy rejection. Users need different guidance for “your account cannot access this material” and “the source is temporarily unavailable.”

Measure successful authorized answers and unauthorized disclosures separately. A strong answer score cannot offset a failed access boundary. Track missing-source answers, stale-source answers, and questions the system declined so the collection owner can improve usefulness without relaxing permissions casually.

The first release would include one approved collection, an identity integration, document-level retrieval filters, source-linked answers, a revocation test, and a recovery runbook. It would exclude autonomous legal decisions and external client communication.

Before accepting that release, I would ask the firm's owner of the collection to perform the access tests with the implementation team. They should see the allowed answer and the denied answer, then watch the same account lose access and fail safely. That demonstration is more informative than another polished search result.

A later release can expand collections once the permission lifecycle is understood. The useful question is whether another collection uses the same access model or introduces new rules that need separate engineering.

If your firm wants internal knowledge search, bring the proposed collection, the source system, and the people responsible for its access rules.