Tenant Isolation: Per-Tenant Qdrant Collections, KMS Keys, and Migration

The query log was 847,000 lines long.

Arjun had pulled it to answer one question: in the three months since the sixth tenant had been onboarded, had any retrieval ever returned a chunk from the wrong tenant's catalog?

He found the answer on page four of the scrollback.

The Valid Failure: A 0.02 Margin Is Not Architecture

A query from the Surat saree house — do you have a Benarasi with a red border, not too heavy? — had scored against thirty-one chunks in the shared collection before the cross-encoder reranked them. Twenty-eight were from Surat's catalog. Three were from zUdyog Fashion's. The top Surat chunk reranked at 0.83. The highest zUdyog chunk ranked at 0.81.

The customer received the correct answer because of a 0.02 gap between a score pointing at the right catalog and a score pointing at the wrong one.

That margin was not architecture. It was the cross-encoder's preference on that day, for that query, given the particular distribution of chunks that happened to be most similar across the two tenants. On a different day, with different phrasing, with different recently-upserted chunks — the margin could be 0.01. It could be negative. The cross-encoder's preference for on-tenant chunks was a side effect of on-tenant training distribution, not a guarantee. At six tenants the luck had held. At sixty it would not hold, and there was no way to know where the boundary of the luck was.

The legal consequence: If a customer of Tenant A receives a product from Tenant B's catalog — even once — Tenant B's pricing, specifications, and product details have been disclosed outside the scope of Tenant B's data processing agreement. Under GDPR and India's Digital Personal Data Protection Act (DPDP Act), neither tenant had consented to their catalog being used to answer another tenant's customers.

None of this had happened. The 0.02 margin had held. The luck needed to become architecture.

First attempt: Qdrant payload filtering. Qdrant supports payload filters applied at query time — each chunk had a tenant_id field written at ingest, and adding a filter to the query call restricted results to on-tenant chunks:

results = client.search(
    collection_name="shopbot_shared",
    query_vector=query_vector,
    query_filter=Filter(
        must=[FieldCondition(key="tenant_id", match=MatchValue(value=tenant_id))]
    ),
    limit=top_k
)

This worked. Every test passed. The problem: the filter lived in the query call, not in the collection. If any code path omitted it — a debugging session, a new pipeline branch, a midnight hotfix written by someone who had not read the three-month-old query log — the shared collection would return unrestricted results. A filter is a rule. Rules can be forgotten. The architecture needed topology: a shape in which there is no path from Tenant B's query to Tenant A's chunks, regardless of what the query code says.

Second attempt: separate collections, shared encryption. Separate Qdrant collections per tenant provided real logical isolation — a query against shopbot_surat cannot return chunks from shopbot_zudyog by the database's own guarantees. But both collections lived on the same storage volume, encrypted by the same key. A storage-level breach, a misconfigured backup, an infrastructure provider incident — any of these would expose every tenant's data simultaneously. Under both GDPR and DPDP, the system would need to report that breach to every tenant and every affected customer at once. Logical isolation without cryptographic isolation is adjacency with a label.

The Valid Source: Per-Tenant Collection + Per-Tenant KMS Key

The solution required both: separate collections and separate encryption keys.

Provisioning a tenant collection — each tenant received a dedicated Qdrant collection named by tenant slug, with vector parameters matching bge-small's 384-dimensional output:

VECTOR_SIZE = 384  # bge-small INT8 output dimension

def provision_tenant_collection(tenant_id: str) -> str:
    collection_name = f"shopbot_{tenant_id}"
    client.recreate_collection(
        collection_name=collection_name,
        vectors_config=VectorParams(size=VECTOR_SIZE, distance=Distance.COSINE)
    )
    client.create_payload_index(
        collection_name=collection_name,
        field_name="product_id",
        field_schema="keyword"
    )
    return collection_name

Provisioning a Customer Managed Key in AWS KMS — each tenant received an independent encryption key. Revoking Tenant B's key leaves Tenant A's collection intact and completely inaccessible to anyone without Tenant A's key:

def provision_tenant_key(tenant_id: str) -> str:
    response = kms.create_key(
        Description=f"ShopBot data key — {tenant_id}",
        KeyUsage="ENCRYPT_DECRYPT",
        Tags=[
            {"TagKey": "tenant_id", "TagValue": tenant_id},
            {"TagKey": "service",   "TagValue": "shopbot-rag"},
        ]
    )
    key_id = response["KeyMetadata"]["KeyId"]
    kms.create_alias(
        AliasName=f"alias/shopbot-{tenant_id}",
        TargetKeyId=key_id
    )
    return key_id

scroll-and-upsert migration — moving each tenant's chunks from the shared collection into their dedicated collection ran as a scrolling batch, filtering by tenant_id, upserting into the target collection:

def migrate_tenant(tenant_id: str, source: str = "shopbot_shared") -> int:
    target  = f"shopbot_{tenant_id}"
    offset  = None
    migrated = 0

    while True:
        results, next_offset = client.scroll(
            collection_name=source,
            scroll_filter=Filter(must=[
                FieldCondition(key="tenant_id", match=MatchValue(value=tenant_id))
            ]),
            limit=200, offset=offset,
            with_vectors=True, with_payload=True
        )
        if not results:
            break
        client.upsert(collection_name=target, points=results)
        migrated += len(results)
        if next_offset is None:
            break
        offset = next_offset

    return migrated

The migration ran across all six tenants in 43 minutes. The shared collection held 47,291 chunks. Each tenant's run logged its count. Total migrated: 47,291. The shared collection was retired after verification.

KNOWN_TENANTS whitelist — after the migration, the query path no longer accepted a collection name from the caller. It derived the collection name from the authenticated tenant ID and validated against a closed set before executing any search:

KNOWN_TENANTS = {
    "zudyog_fashion",
    "surat_saree_house",
    "jaipur_boutique",
    "national_retailer",
    "vastralaya",
    "kolkata_silks"
}

def retrieve(query: str, tenant_id: str, top_k: int = 5) -> list:
    if tenant_id not in KNOWN_TENANTS:
        raise ValueError(f"Unrecognised tenant: {tenant_id}")
    collection_name = f"shopbot_{tenant_id}"
    query_vector = embedder.encode(query)
    return client.search(
        collection_name=collection_name,
        query_vector=query_vector,
        limit=top_k
    )

The wall was not a filter. It was the absence of a path.

The Valid Knowledge: What Isolation Actually Means

The cross-tenant test confirmed the change at the collection level:

def test_cross_tenant_isolation():
    surat_query = "Benarasi with a red border, not too heavy"
    surat_results = retrieve(surat_query, tenant_id="surat_saree_house")
    for result in surat_results:
        assert result.payload["tenant_id"] == "surat_saree_house"
    try:
        retrieve(surat_query, tenant_id="unknown_tenant_xyz")
        assert False, "Should have raised ValueError"
    except ValueError as e:
        assert "Unrecognised tenant" in str(e)

test_cross_tenant_isolation ... PASSED
Surat results: 5 chunks, all tenant_id=surat_saree_house
Cross-tenant attempt: ValueError — Unrecognised tenant: unknown_tenant_xyz

Before the migration: the same Surat query had ranked three zUdyog chunks inside the top ten at scores of 0.81, 0.79, and 0.76. After the migration: five on-tenant Surat chunks, and the invalid-tenant attempt raised at the collection lookup — before the vector search was ever executed.

RAGAS after per-tenant collections (zUdyog baseline):

Context Precision improved 2 points. Removing cross-tenant chunks from the pool the cross-encoder ranked against reduced noise at the top of the results list. Faithfulness was unchanged — the model speaks from what is retrieved, and what is retrieved had already been honest. The isolation made the retriever's scope exact.

The filter was advisory. The collection is topology. Two different things.

This article covers concepts from Book 4, Chapter 2 of the RAG Mastery Series. The series builds ShopBot — a production RAG system — across four books, from first embeddings to a fleet of six isolated, compliant, auditable tenants.