Lesson 11: Your AI Found 20 Products. But Which 3 Actually Match? Inside Cross-Encoder Reranking

Subtitle: Discover how Varastra uses a cross-encoder to rerank search results, find the most relevant products, and give its AI assistant better information before it generates an answer.

1. The Problem: Finding Products Isn't Enough

Imagine a customer asks Varastra, an Indian ethnic-wear store:

“Find me a comfortable cotton kurta for a summer wedding.”

The search system retrieves 20 possible products. Some are cotton kurtas, some are festive outfits, and others may only partially match the request.

Which products deserve to reach the AI model?

Retrieval is designed to find promising candidates quickly. Reranking takes those candidates and examines their relevance more carefully.

That's the job of the cross-encoder in this diagram.

2. Understand the Architecture

Follow the arrows:

Customer query → Hybrid retrieval → 20 query-candidate pairs → Cross-encoder → Sort by relevance score → Top 3 sent to the prompt

Each stage has a specific purpose.

3. Step 1: The Customer Asks a Question

The customer submits a query, such as:

“Find a breathable cotton kurta under ₹2,000.”

The system must find products that match the customer's needs.

The query is passed to hybrid retrieval, which combines search methods such as dense semantic search and sparse keyword search.

Why it matters: The system needs relevant candidates before it can decide which ones are best.

4. Step 2: Hybrid Retrieval Finds the Top 20 Candidates

The retriever searches the available product information and returns 20 candidate results, as shown in the diagram.

For example, the candidates might include:

• Cotton kurtas.
• Linen outfits.
• Festive kurta sets.
• Products containing related keywords but missing an important requirement.

These are possible matches, not 20 guaranteed correct answers.

Why not rerank every product?

A catalog can contain thousands of products. Evaluating every product with a more computationally expensive model can increase latency and cost.

Retrieval narrows the search space first. Reranking then focuses on a smaller set.

Remember: If a relevant product never enters the candidate list, the reranker cannot recover it.

5. Step 3: Create 20 Query-Candidate Pairs

The diagram shows 20 query-candidate pairs entering the cross-encoder.

Each pair contains:

1. The customer's query.
2. One candidate product or document.

For example:

Query: “Breathable cotton kurta under ₹2,000.”

Candidate A: “Lightweight cotton kurta, ₹1,799.”

Candidate B: “Heavy embroidered silk kurta, ₹3,500.”

The cross-encoder evaluates each candidate in relation to the query.

Unlike a basic keyword comparison, it can use relationships between words across the query and candidate text to judge relevance.

The quality of this judgment depends on the model and the information supplied about each product.

6. Step 4: The Cross-Encoder Scores Each Candidate

The diagram specifies this model:

ms-marco-MiniLM-L-6-v2

A cross-encoder processes the query and candidate together and produces a relevance score.

Unlike a typical bi-encoder retrieval setup, which embeds the query and documents separately for efficient search, a cross-encoder evaluates each query-document pair directly.

This makes it useful for reranking a limited set of candidates, although it is generally more expensive to run per candidate.

Example

Imagine the model produces these illustrative scores:

The higher scores suggest that Products A and C are more relevant to the query than Product B.

Important: These numbers are illustrative, not actual scores from the diagram. A relevance score is not automatically a probability or a percentage of correctness.

7. Step 5: Sort the Results and Keep the Top 3

After scoring the 20 candidates, the system sorts them by relevance score.

The diagram specifies:

Sort by relevance score → Top 3 to prompt

Only the three highest-ranked candidates are passed to the LLM prompt.

This reduces the amount of retrieved material supplied to the model and focuses its attention on the strongest candidates.

The top three are not automatically correct. The system still needs appropriate grounding and validation, especially for changing facts such as price and stock.

8. Why Not Use Only the Retriever?

The retriever and reranker solve different problems.

The diagram's central lesson is:

The retriever is fast and broad. The reranker is slower and more careful.

Using both creates a practical balance between search efficiency and relevance.

9. What Can Go Wrong?

For Varastra, relevance scores alone should not enforce critical business constraints. Price limits, stock availability, and other mandatory requirements should be checked against authoritative data.

The team should evaluate retrieval recall, reranking quality, latency, and the relevance of the final top-three results.

10. Key Takeaways

1. Hybrid retrieval finds 20 candidate products.
2. Each candidate is paired with the customer's query.
3. The cross-encoder model is ms-marco-MiniLM-L-6-v2.
4. The model scores the relevance of each query-candidate pair.
5. The system sorts candidates by score.
6. The top three results are passed to the LLM prompt.
7. Reranking improves candidate ordering but cannot guarantee correctness.
8. Good results depend on both candidate retrieval and reranking quality.

The big lesson: Finding possible answers is only the first step. A reliable AI search system must also identify which retrieved information deserves the model's attention.