Design a multi-tenant enterprise knowledge assistant

knowledge, rag, multi-tenancy, hybrid-search

Prompt

Your company sells a knowledge assistant to enterprise customers: each customer, a tenant, uploads its own documents, and the assistant answers questions by retrieving from that tenant's corpus only, including exact lookups like SKU numbers and ticket IDs. Design the retrieval architecture and the tenant isolation guarantees.

Interviewer's answers to clarifying questionsopen after you have asked your own

Scale? 200 enterprise tenants, corpora ranging from 500 to 500,000 documents each, shared infrastructure across tenants for cost reasons.
Query types? A mix of free-text troubleshooting questions and exact-identifier lookups (SKU, error code, ticket number) that must match precisely.
Isolation requirement? Zero cross-tenant document leakage, even under a misconfiguration, is a contractual requirement with financial penalties.
Current architecture under review? A single shared vector index with a tenant_id metadata field, filtered after similarity search.
Observed problem? Exact-identifier queries ("error code TS-999") sometimes retrieve topically related but wrong chunks, because the embedding model treats the identifier as noise.
Cost constraint? Per-tenant dedicated infrastructure is too expensive at this scale; some sharing is required.

Follow-upsafter your high-level design

A new engineer suggests filtering by tenant_id after the similarity search runs, to simplify the query path. What's wrong with that compared to filtering before ranking?
How would you test, with confidence, that cross-tenant leakage is actually zero rather than just "not observed yet"?
One large tenant's corpus grows to 2M documents while most tenants stay under 10,000. Does your shared-infrastructure design still hold, or does this tenant need to be split out?
A tenant asks for a corpus-wide synthesis answer, "what are our three most common failure causes across all our incident reports," that no single retrieved chunk set can answer. What changes?

Answer first: out loud, on paper, or with the interview prompt above. Then compare with the rubric.