Cosine Similarity vs Dot Product: Which One Does RAG Use?

Both cosine similarity and dot product measure the relationship between two vectors. In production RAG systems, cosine similarity is almost always the right choice. Here is why.

The Difference

Dot product multiplies each pair of corresponding dimensions and sums the results. It captures both direction and magnitude — a longer vector will score higher even if it points in the same direction as a shorter one.

Cosine similarity normalises both vectors to unit length before computing the dot product. The result is a score from -1 to 1 that reflects only the angle between the vectors — direction without magnitude.

Why Magnitude Is a Problem in Retrieval

In embedding models, magnitude reflects token count and repetition, not semantic richness. A 500-word product description produces a longer embedding than a 20-word product title. If retrieval uses raw dot product, longer documents are systematically rewarded, regardless of relevance.

Cosine similarity removes this bias. A short, precise product specification that answers a query perfectly should score higher than a verbose document that mentions the answer once in passing.

When Dot Product Is Appropriate

Dot product retrieval makes sense when embeddings are already normalised to unit vectors — which is how most modern embedding models output them by default (text-embedding-3-small, all-MiniLM-L6-v2, etc.). In that case, cosine similarity and dot product produce identical rankings, and dot product is computationally cheaper.

Qdrant, ChromaDB, and FAISS all support both. If you are using pre-normalised embeddings, dot product is fine. If your embeddings are not normalised, use cosine.

The Practical Rule

If you are retrieving from a corpus where documents vary significantly in length — which describes almost every production RAG use case — use cosine similarity. If you have verified that your embeddings are already L2-normalised, dot product gives the same result with slightly lower latency.

Cosine similarity and retrieval scoring are covered in depth in Book 1, Chapter 3 of the RAG Mastery Series. Book 1 is free — no sign-in needed.