What Is a Vector Database? ChromaDB, FAISS, and pgvector Explained

A Python loop can search five product embeddings in milliseconds. At a thousand products, it takes 80 milliseconds — tolerable. At ten thousand, 800 milliseconds. At a hundred thousand — not unusual for a platform serving many boutiques — eight seconds. The customer has already left.

The embedding approach is correct. The storage and retrieval strategy is wrong. A vector database is what fixes it.

Why a Regular Database Cannot Do This

A regular database query is exact. It finds rows where a column equals a value, falls within a range, or matches a pattern. It is designed for exact lookups — fast, reliable, predictable.

Finding the nearest neighbour in a high-dimensional vector space is not an exact lookup. There is no index structure in a standard database that supports it efficiently. PostgreSQL has an extension called pgvector that allows storing vectors and computing similarity in SQL — but a naive implementation still degrades toward a full table scan as data grows. The problem is structural, not a configuration issue.

The problem has a name in computer science: Approximate Nearest Neighbour search — ANN. Finding the exact closest vector in high-dimensional space is expensive. Finding a very-close-to-exact closest vector is achievable in milliseconds, even at scale, if you use the right data structure. Vector databases are built around ANN algorithms. They sacrifice a tiny amount of theoretical accuracy — returning the nearest neighbour 99.5% of the time instead of 100% — in exchange for search times that stay in single-digit milliseconds regardless of collection size.

Three Options and Why One Wins at This Stage

FAISS — Facebook AI Similarity Search — is a library for efficient similarity search. Fast, well-tested, used at hyperscale. It is also not a database. It is an in-memory index. Persisting it to disk, loading it reliably, managing updates when new products are added, handling concurrent reads — all of this requires building infrastructure around FAISS that the library does not provide. For a research team at hyperscale, FAISS is the right tool. For a developer building a product that needs to be updatable, it adds complexity that does not serve the project.

pgvector stores vectors in Postgres and computes similarity in SQL. Useful when you want to keep vector storage inside an existing database. A naive implementation still degrades toward a full table scan under heavy load. The advantage is familiarity; the limitation is scale.

ChromaDB runs entirely locally — no Docker, no cloud account, no server configuration. It persists to disk, so embeddings survive between sessions. It uses HNSW internally and returns results in milliseconds regardless of collection size. For a first system, ChromaDB is the right choice: local, free, zero configuration. When the system needs to scale to production with multiple tenants, migration to Qdrant Cloud carries the same concepts forward. The architecture does not change. The infrastructure does.

What a Vector Database Actually Does

A vector database stores embeddings as its native data type and answers proximity queries as its primary operation.

When you add a product to ChromaDB, you provide the product text and its embedding — the 1,536 numbers that represent its meaning. ChromaDB stores both. When a customer query arrives, you provide the query embedding. ChromaDB calculates the similarity between the query vector and every stored product vector and returns the closest matches, ranked by similarity score.

It does not scan rows looking for matching keywords. It navigates the geometry of the embedding space and returns the nearest neighbours — the products whose meanings are closest to the query's meaning.

Under the hood, ChromaDB uses HNSW — Hierarchical Navigable Small World — a graph-based algorithm that builds a layered structure of connections between vectors. Search navigates from broad clusters down to precise matches in logarithmic steps rather than scanning everything. The important thing is not the algorithm name but what it produces: search that stays fast regardless of how many vectors are stored.

One implementation detail matters in practice. ChromaDB returns distance, not similarity. To convert: similarity = 1 - distance. A similarity of 0.91 means the query and the retrieved chunk are very close in meaning. A similarity of 0.51 means they are adjacent, not genuinely relevant.

The Vocabulary Gap, Made Visible

The same query — "something light and breathable for hot weather" — against keyword search returns zero results. The word hot appears nowhere in the catalog. The word light appears nowhere. The keyword search cannot bridge the vocabulary gap between how a customer speaks and how a product is described.

Against ChromaDB, the same query returns the cotton kurta at 0.893 and the linen co-ord set at 0.871. Two correct answers, retrieved in 180 milliseconds, from a store that will scale without changing a line of retrieval code.

This is the difference between a system that matches characters and a system that understands meaning.

The Boundary Is the Guarantee

In the Pramana Framework, the vector store is the archive of valid sources — the repository of verified evidence the system is permitted to speak from. What is not in ChromaDB cannot be retrieved. What cannot be retrieved cannot be cited. What cannot be cited cannot be answered.

That boundary is not a limitation. It is the guarantee.

A vector database without a retrieval threshold — the minimum similarity score below which a result is discarded — becomes a noise pipe. It returns something for every query, regardless of how relevant that something is. The threshold and the vector database together form the Grounding Layer: evidence is retrieved only when it earns retrieval.

This article covers concepts from Book 1, Chapter 4 of the RAG Mastery Series. The series builds ShopBot — a production RAG system — across four books, from first embeddings to cloud-native deployment.