Your chatbot correctly answers questions about a customer's uploaded files. That proves the retrieval workflow works for that customer. It does not establish what happens when another customer asks the same question.
To test customer isolation in a Supabase RAG chatbot, create separate test workspaces, authenticate as their actual users, and verify the entire information path: retrieved chunks, model context, answers, citations, downloads and cached conversations. Include requests that attempt to access another workspace directly, and repeat the checks after membership changes.
The goal is to demonstrate that each request can use only the information its requester is allowed to access. A system prompt asking the model to respect privacy is not an access-control mechanism.
This guide provides a practical test plan for a startup building a document-based AI product. The examples use fictional workspaces and synthetic documents. They are proposed tests, not results from an audited application.
What customer isolation means in a RAG application
Retrieval-augmented generation, or RAG, retrieves external information and supplies it to a model when generating an answer. In a multi-tenant product, that information may belong to different customer organizations.
A tenant is a customer or workspace boundary. A user can belong to one tenant, several tenants or none. Some documents may also have narrower permissions within a tenant.
A useful access rule is:
The requester must have access to the selected workspace and permission to read the specific document before any of its content enters the answer-generation path.
Write the rule for your product explicitly. Membership in a workspace may permit reading every document in a simple MVP; a more complex product may need project, team or document-level restrictions.
Supabase's RAG permissions guide demonstrates restricting document-section retrieval through Postgres Row Level Security, or RLS. This provides a foundation for permission-aware vector search. Your own membership model and application paths still need verification.
Create a two-workspace test fixture
Use a staging environment you control and synthetic information. Create users who have intentionally different access rights.
| Identity | Membership | Expected access |
|---|---|---|
| Alice | Workspace A only | Authorized A documents |
| Bilal | Workspace B only | Authorized B documents |
| Casey | Workspaces A and B | Documents permitted in the selected workspace |
| Dana | No workspace membership | No private workspace documents |
Upload a file named support-policy.pdf into each workspace. Using the same filename helps expose code that mistakenly treats filenames as globally unique identifiers.
Give the files similar structure but clearly different content:
| Workspace | Synthetic document fact | Private test marker |
|---|---|---|
| A | Standard onboarding takes 3 business days | A-MARKER-7QX |
| B | Standard onboarding takes 8 business days | B-MARKER-9ZT |
The markers help detect accidental exposure in traces. Do not put the forbidden marker into the test question and then count its appearance in the answer as proof of leakage: the model could simply repeat the question. Inspect where information actually came from.
Include a document that Alice is allowed to read and, if your product supports restricted documents, another A document she cannot read. Tenant separation alone does not test finer permissions.
Test the identity used by the real request
Run the chatbot through the same authentication path used by customers. A dashboard query or test executed as an administrator can have different privileges from a request made by Alice.
Record the authenticated user, selected workspace, endpoint and relevant database execution context. Keep credentials out of test reports.
For requests routed through an application backend, verify that authorization is enforced there and in the data path as intended. Do not assume that a server-side Supabase client automatically represents the browser user.
Supabase documents RLS behavior, privileged service access and exceptions such as views and security-definer functions in its Row Level Security guide. An RLS policy is effective only when the query actually executes under a context to which that policy applies. Keep privileged credentials out of frontend code.
A tenant ID supplied by the browser can identify the requested workspace. It cannot, by itself, prove membership in that workspace.
Check retrieval before checking the final answer
Ask Alice's chatbot:
How long does standard onboarding take?
Inspect the retrieved document IDs and chunks before they enter the model context. Under the fixture above, the relevant evidence should come from Workspace A and describe 3 business days.
Repeat as Bilal. The corresponding evidence should come from Workspace B and describe 8 business days.
Do not judge isolation from the answer alone. A model might receive an unauthorized passage and still omit it from its reply. The passage has already crossed the intended boundary if it was passed to a model or reranking service on behalf of an unauthorized requester.
Restrict access to diagnostic traces. Otherwise, an application can protect its main interface while exposing private content through an internal log viewer.
A tenant-isolation test matrix
The expected outcomes below assume private, workspace-scoped documents with no cross-workspace sharing. Adjust the matrix when your product intentionally supports sharing, and document those exceptions.
| Test | Request | Expected outcome |
|---|---|---|
| Allowed retrieval | Alice asks about an A document | Authorized A evidence is returned |
| Cross-tenant question | Alice asks for B's private policy | No B evidence enters retrieval output or model context |
| Workspace tampering | Alice changes the request's workspace ID to B | Access is denied; B content is not returned |
| Direct document lookup | Alice supplies a known B document ID | No B document content or private metadata is disclosed |
| Direct chunk lookup | Alice requests a B chunk through an exposed route | Access is denied |
| Missing identity | An unauthenticated request searches private files | No private evidence is returned |
| No membership | Dana selects A | No A evidence is returned |
| Workspace switch | Casey switches from A to B | Retrieval follows the selected, authorized workspace |
| Restricted document | Alice requests an A document outside her role | The narrower document permission is enforced |
| Prompt instruction | Alice asks the bot to ignore workspace limits | Authorization remains unchanged |
“Access is denied” does not require every endpoint to return the same HTTP status. Depending on the interface, the correct result may be an error or an empty result. The important assertions are that unauthorized information is unavailable and the application's behavior matches its documented contract.
Add positive checks alongside negative checks. A system that returns nothing to everyone may pass an incomplete denial-only suite while being unusable.
Inspect functions, views and privileged routes
A vector-search function can become the critical access boundary. Review its execution privileges, parameters and query behavior alongside its similarity calculation.
For every exposed search path, ask:
- Which identity executes the database query?
- Is the supplied workspace validated against that identity?
- Can the caller omit or replace a workspace filter?
- Are document permissions checked before content leaves the retrieval boundary?
- Do alternate endpoints, views or database functions expose the same records differently?
A function that runs with elevated privileges needs deliberate authorization design. A name such as match_documents says nothing about the privileges or filtering inside it.
Keep administrative ingestion and end-user search paths clearly defined. Background workers may need wider access to process documents, but that does not justify making all documents available to a customer-facing search request.
Test citations and file downloads separately
A filtered answer is not enough if its source viewer lets another user open the underlying file.
Take a document link generated for Bilal and attempt to use the application's document-viewing route as Alice. Also test direct object access where your product exposes it. Check filenames, titles, extracted text and previews as well as the original PDF.
Supabase Storage has its own access-control model. Database policies on a custom document table do not automatically establish all the controls needed for stored objects.
If the application uses signed URLs, treat them as temporary access capabilities. A URL issued before permission removal may remain usable until expiry, depending on the serving mechanism. Do not promise immediate revocation without testing that behavior. For stricter requirements, evaluate an authenticated serving path that rechecks current access and document the remaining caching boundaries.
When returning an answer with citations, verify that both the retrieved evidence and the citation destination belong to the authorized source. See RAG for PDF tables and page citations for the separate question of whether a cited location actually supports an answer.
Warm the cache before testing another user
Some isolation failures appear only after one customer has asked a question.
Start as Bilal and ask a question that produces an answer from B's document. Then ask the same question as Alice. Inspect answer caches, retrieval caches and any shared conversation state.
A cache keyed only by question text can return a response generated from another user's documents. A tenant key helps, but tenant membership alone may not be enough when users within a tenant have different permissions.
Choose a cache design that reflects the actual authorization scope and handles permission changes. Depending on the product, this may involve user-scoped entries, an access-policy version or fresh authorization checks against cached evidence.
Do not reuse an old answer merely because the current requester can access one of its sources. Verify the required access to every private source contributing to the answer.
Repeat the tests after removing membership
Remove Alice from Workspace A, then repeat the read paths using her existing session. Test another request with a refreshed session as well.
This distinguishes behavior based on current membership records from behavior that depends on previously issued authentication claims. Supabase notes that JWT-backed authorization information may remain stale until the token is refreshed; account for that when defining your revocation requirements.
| After-removal check | What to observe | Decision to document |
|---|---|---|
| New chat question | Whether fresh retrieval is denied | When membership changes take effect |
| Existing conversation | Whether earlier answers remain visible | History access and retention behavior |
| Cached answer | Whether old evidence is reused | Cache invalidation or reauthorization |
| Existing source link | Whether a previously issued link still opens | Link lifetime and serving policy |
| In-flight response | Whether a running request continues | Handling of permission changes during execution |
Revocation cannot undo information that a user already downloaded or read while authorized. Define the behavior your application can enforce for future access, rather than claiming that previously delivered information disappears.
Check that uploads cannot cross workspace boundaries
Read permissions are only part of the design. A user should not be able to assign a new document to another tenant, move an existing document into a forbidden workspace or attach chunks to a document they do not control.
Test create and update routes with altered tenant IDs and document IDs. Inspect the background ingestion job as well: it should use validated ownership information rather than trusting arbitrary client metadata.
When authorization information is copied onto chunk records, establish how consistency is maintained when document ownership or permissions change. Otherwise, the document and its derived chunks can disagree about who may read them.
Deletion and reindexing deserve the same attention. For the document lifecycle, see why a RAG chatbot can keep using outdated information.
What to record in the test report
A useful report makes failures reproducible without exposing customer data or credentials.
| Report field | Example content | Why it matters |
|---|---|---|
| Application build | Commit or release identifier | Ties findings to a tested version |
| Identity and membership | Alice, A-only membership | States the expected access scope |
| Request path | Chat endpoint or document viewer | Identifies the boundary tested |
| Test input | Question and requested document ID | Allows a controlled replay |
| Expected behavior | A evidence only; B inaccessible | Defines the assertion |
| Observed behavior | Authorized IDs, denial or unexpected exposure | Separates expectations from results |
| Supporting trace | Redacted retrieval and context records | Shows where an issue occurred |
| Follow-up | Fix owner and regression-test reference | Makes the result actionable |
Record “not tested” when a path was not exercised. Do not turn an untested cache, direct connection or source-download route into an implied pass.
Automate the deterministic assertions where practical: forbidden record IDs should be absent, unauthorized downloads should fail, and allowed requests should still work. Model wording may vary, so access-control checks should not depend only on exact final-answer text.
Questions founders should ask before launch
Ask the team to demonstrate one permitted request and one denied request with real test-user sessions. Follow both through the database, model context and source viewer.
Agree on the meaning of workspace switching, document sharing and access revocation. Those product decisions determine what the tests should assert.
A passing fixture gives evidence about the scenarios it covers. It does not establish that every route, deployment configuration or future change is secure. Rerun the relevant checks when changing database policies, retrieval functions, caching or identity handling.
Frequently asked questions
Is adding a tenant ID filter enough?
A filter can be part of the implementation, but the server must validate that the requester is allowed to use the tenant ID. Test alternate read paths and ensure the effective authorization boundary cannot be bypassed by changing request parameters.
Can RLS protect vector search?
Supabase documents using Postgres RLS with pgvector-backed document sections. Whether your particular search path respects those policies depends on its execution context, privileges and implementation. Verify the actual function called by your application.
Should each customer have a separate database?
A separate database is one architectural option, not a universal requirement. Shared tables with carefully enforced permissions can support multi-tenant applications. Choose based on operational needs and the isolation requirements, then test the complete request path either way.
Does a refusal prove the chatbot never accessed private data?
No. Inspect the retrieval output and model context. A refusal can occur after unauthorized content has already been retrieved or sent to a processing service.
What should happen when a user leaves a workspace?
Define future-access behavior for retrieval, saved chats, caches and source links. Test existing sessions as well as refreshed ones, and state any delay or temporary-link lifetime explicitly.
Plan a RAG assistant around customer boundaries
Zee AI's RAG development services can help you plan document-based applications around the way customers access information.
For a useful initial discussion, prepare your workspace roles, document-sharing rules and a small synthetic example of who should see what. That gives the architecture and acceptance tests a concrete starting point.
Technical references
Technical references checked on September 22, 2026. The fixture and checklist describe proposed application tests; no benchmark or security certification is claimed.
