The usual advice about agents and access control is to get your document permissions right before you connect anything. It is good advice and it is not enough. Getting them right is table stakes, and most enterprises did it years ago. The folder is locked, the SharePoint site has the correct group on it, the Confluence space is restricted, Active Directory says exactly who sits in which group. Then an agent answers a question using three paragraphs from inside that file, and the boundary everyone trusted turns out to have been a filing convention rather than a control.
We have said elsewhere that an auditable agent has to enforce permissions at the chunk. That post makes the claim. This one is the mechanism, because a claim about access control that nobody can inspect is worth about as much as no claim at all.
We think the reason the folder fails is structural, and it has two halves. The first half is where the permission decision is made. The second half is when it binds. Almost everything written about this fixes one half and calls it finished.
The folder was never a boundary
Ingestion flattens the document. A forty page policy PDF goes in as one object with one owner and one access control list, and it comes out as three hundred chunk records with vectors attached. From that point the file does not exist as far as retrieval is concerned. What exists is three hundred paragraphs, each independently reachable by similarity to a question nobody has asked yet.
Every production pattern handles this the same way. The ACL is copied onto each record at index time. Azure AI Search is explicit that once a skillset chunks a document, the permission metadata “move from indexer field mappings to index projections,” which in plain terms means the ACL has to be stamped onto every chunk record or it cannot be evaluated at all. Their index schema carries a fixed permissionFilter enum with exactly three permitted values, userIds, groupIds and rbacScope, and the service applies the filter itself from the caller’s token rather than trusting the application to do it.
Elastic takes the same shape and writes a field called _allow_access_control into a separate access control index. It is worth reading what their connectors actually do with it, because the default is more interesting than the design.
# elastic/connectors @ f4f91b4 · access_control.py L7–27 · Elastic License 2.0
ACCESS_CONTROL = "_allow_access_control"
DLS_QUERY = """{
"bool": { "should": [
{ "bool": { "must_not": { "exists": { "field": "_allow_access_control" } } } },
{ "terms": { "_allow_access_control.keyword": {{#toJson}}access_control{{/toJson}} } }
] } }"""
Read the first clause in that should. A document with no _allow_access_control field matches, which means it is visible to everyone. Permission filtering fails open by design, and it fails open for exactly the documents that were indexed before document-level security was switched on, or by a connector that does not populate the field. That is a sensible default for a search product where the alternative is an empty result set. It is a poor default for an agent that will happily quote what it finds.
The general point is that the copy is now authoritative. The source system still holds the real answer, but nothing in the query path consults it. What you have is a cache with no invalidation strategy, and the thing being cached is an authorization decision.
Two axes, not one
Call the first axis granularity. The chunk is the unit an agent retrieves, quotes, and summarizes, so the chunk is the unit the decision has to cover. That is what we mean by Chunk-Level RBAC, and the industry has already conceded the point quietly, in documentation rather than in marketing. It is the default behaviour of the leading enterprise retrieval products. Most teams simply never wire the projection and keep assuming the folder holds.
Call the second axis binding time, and it is the one nobody closes. Glean’s indexing documentation says permissions and memberships “are processed asynchronously, there might be a small delay before documents are visible to groups / users.” Azure documents a timing lag before permission changes are recognised, and ACL changes on Data Lake Storage require a resync. Every vendor says some version of sync often.
Sync often is not a bound. A bound is a number. The ConfusedPilot research out of UT Austin measured revoked and deleted documents still surfacing through a commercial enterprise assistant for between 38 and 183 seconds after revocation, because the deleted document stays in the cache until the next index pass. Their finding included a detail worth more than the number: the compliance check did not verify whether the content came from a deleted document. The link was dead and the text was still there.
If that sounds like a lab result, here is an engineer at Notion describing the same window in their own production system, in public, in 2024:
“When you change permissions, we kick off an online recursive reindex job for that page and its recursive subpages. While the job is running, the index has stale entries with outdated permissions. When you search… Because the index permissions can be stale, we also reload the result set from Postgres and apply our normal online server-side permission checks to filter out pages you lost access to but that have stale permissions in the index.”
That is the honest version of the design, and note what it costs them. They accept that the index is wrong for a while, and they pay for a second permission check against the source of truth on every search to compensate. Most teams building retrieval have not decided which of those two things they are doing.
So the honest statement of the problem is that chunk grained plus index bound is still exposed. You need the decision to cover the chunk, and you need to know how long your copy of the answer is allowed to be wrong.
Where the filter goes, and what it costs
There are three ways to place the permission check, and each one is a real position with a real cost.
Filter after retrieval. Retrieve the top ten, drop what the user cannot see, return the rest. Simple, and it changes the answer rather than the result set. Unauthorized chunks consumed slots in your top ten, so the user does not get a denial. They get a thinner answer, or a confident wrong one, built from whatever survived. Descope calls this starvation. Engineers usually diagnose it as a retrieval quality bug and go tune the embeddings.
Post filtering also has a cost people forget to price. If your authoritative check is an authorization service, you are making that call once per candidate. OpenFGA’s default batch size is fifty checks per request, set in DefaultMaxChecksPerBatchCheck in their server config. Post filtering a candidate set of a thousand is twenty sequential round trips before the user sees anything.
Filter before the search, from outside it. Compute the permitted set, then hand the vector index a predicate it did not plan for. This is where the graph fights back. Qdrant’s write up on filterable HNSW explains the mechanism in percolation terms: below a critical threshold the graph “decomposes into small connected components,” and traversal stops reaching neighbours it should have found. They describe a threshold rather than a curve, and they publish no percentages, so neither will we. A permission filter is typically the most selective filter in the system, which puts it on the wrong side of that threshold more often than any other filter you run.
Filter inside the search. Give the index a predicate it can evaluate natively while it traverses. MongoDB Atlas documents this directly: “Pre-filtering ensures that documents are eliminated before MongoDB Vector Search traverses the HNSW graph.” Two things to know before relying on it. The operator set inside a $vectorSearch filter is narrower than general MQL, covering equality, range, set membership, existence and the logical operators, with no $regex, $elemMatch or $expr. And Atlas states plainly that “filtered queries are typically slower than an otherwise equivalent unfiltered query.”
There is a subtler cost, and it is visible in the library most teams reach for. This is the stage builder LangChain’s MongoDB integration uses on every filtered vector query:
# mongodb-labs/pymongo-search-utils @ 673265f · pipeline.py L82–91 · Apache 2.0
stage = {
"index": index_name,
"path": search_field,
"queryVector": query_vector,
"numCandidates": top_k * oversampling_factor,
"limit": top_k,
}
if filter:
stage["filter"] = filter
return {"$vectorSearch": stage}
numCandidates is set to top_k times an oversampling factor, which defaults to ten. The filter is attached afterwards and the candidate count is not widened to compensate. Ask for ten results with a filter that admits one document in fifty, and the index is still only asked to consider a hundred candidates. You will get an answer. It will be short, it will look like a retrieval quality problem, and nothing anywhere will tell you that the filter is what shortened it.
What chunk boundary enforcement actually requires
Six things, and the cost of each is real.
A permission attribute on every chunk, not a pointer to a document that has one. The cost is write amplification. One ACL edit fans out to every chunk derived from that file.
A permission field the index can actually evaluate, in that index’s own dialect. The abstraction layers do not save you here. LlamaIndex exposes a dozen filter operators and carries a # TODO add more operators in the source, and every one of them has to be lowered onto whatever the underlying store supports. Atlas supports a subset of MQL. Pinecone allows 40 KB of filterable metadata per record and caps $in and $nin at ten thousand values. A user who sits in four hundred nested groups does not flatten neatly into any of them.
Identity resolved for the human on the request, not for the service account that runs the pipeline. The failure here is easy to ship and hard to see. The retrieval layer authenticates as itself, so it can read everything, and the application filters afterwards. That works until the second application arrives, and the third, each reimplementing the filter, and one of them getting it wrong. An agent has two identities to keep straight: its own principal, with whatever standing access an administrator configured, and the identity of the person it is acting for. Azure’s answer is to take the caller’s token in a request header and apply the filter inside the service. The cost of doing it properly is that the acting user’s identity has to travel with every call, which means the surrounding application has to have it.
Candidate depth chosen with the filter in mind. If the predicate admits one document in fifty, the oversampling factor has to know that. Otherwise the control silently truncates the answer.
A staleness bound you are willing to state out loud. Whatever refreshes your permission copy, know its cadence. “It syncs” is not an answer to an auditor. Notion’s answer is to assume the index is stale and re-check against Postgres, which is a real answer with a real price.
A showable boundary. Storage, and the discipline to log the decision rather than only the outcome. A trace that records the question, the retrieval strategy selected, the scoping applied and to whom, the context actually passed to the model, and the citations returned is a trace you can hand an auditor. One that records only the answer is a receipt. This is the same argument as the audit trail on the agent itself, one layer down.
There is a shortcut people reach for, which is to give each role its own index and skip the filtering problem entirely. It is worth knowing what that costs first. The HoneyBee work measured dedicated indices per role at 27 times the storage of a unified index, and per user partitioning at more than 50 times the storage for less than 2 times the latency improvement. Partitioning is not free and it does not scale with the org chart.
How we do it
Our position is that this belongs in the Information Layer rather than in the application. The reason is the one above: an application-layer filter is a control that every new consumer has to reimplement correctly, and the enforcement should sit at the layer the data lives in, so the agent inherits it rather than bypasses it.
That has a structural consequence worth stating plainly. There is one retrieval layer and many consumers. Downstream agents query it as a service rather than each building their own path to the corpus, which means the permission decision is made once, in one place, by the component that owns the index.
Three specifics, in the words we publish on our own security page and in our product documentation. Every chunk indexed by the retrieval engine carries a permission set inherited from the source document. Permission lookup happens at retrieval time rather than at ingestion, and the requesting user’s permissions are matched against each candidate chunk before retrieval rather than after. And the enforcement is applied by the retrieval layer rather than trusted to the model, which matters more than it sounds: a model instructed to ignore what it should not have seen has already seen it.
The vector store underneath is MongoDB Atlas by default, which is why the Atlas filter semantics above are not an academic detour, and other stores are supported where a team already has one. Role membership comes from the directory you already run, through Active Directory, Azure AD or Okta over SSO, so there is no second permission system to keep in step with the first.
The phrase doing the work in all of that is “inherited from the source document,” and it is also the honest limit.
The part inheritance does not solve
Inheritance is how we and everyone else do this today. That model has two edges it cannot reach, and I would rather write them down than pretend they are handled.
The first is sub document sensitivity. Three paragraphs inside a file being restricted while the rest is not is a policy nobody can express in an inheritance model, and not one of the vendor systems above can express it either. Manual paragraph tagging is the obvious answer and no organisation will sustain it. The tractable version is inheritance by default with overrides driven by classification: sensitivity labels, detected personal data, section structure. That is a real piece of work and it is ahead of us rather than behind us.
The second is derived artifacts. Summaries, extracted entities, and graph nodes are built from chunks and then indexed as new objects in their own right. If the derivation does not carry the intersection of its sources’ permissions, a restricted paragraph launders itself into an unrestricted node, and the lineage that would have caught it was never recorded. We ship graph based retrieval, so this is our problem and not a hypothetical one.
Neither edge has a good public answer yet. Both get worse as more of the estate becomes derived rather than ingested.
For engineering leaders: ask your team to produce the list of chunks a named person was allowed to retrieve last Tuesday. Not the policy, the list. How long that takes tells you which of the two axes you are actually enforcing.
For engineers: open your vector store’s stage builder and find where numCandidates is set. Then find out what fraction of the corpus your permission filter admits. If the second number is small and the first one does not know about it, your retrieval quality problem is a permissions problem wearing a costume.
The folder was a boundary because no human ever read that far into that file looking for that specific thing. Agents read that far on every query.