Lesson 13: Your AI Forgot What the Customer Just Said? How Varastra Gives Its Shopping Assistant a Memory

Subtitle: Discover how Varastra combines recent conversation turns, a rolling summary, and active product IDs to understand follow-up questions, prioritize products customers are discussing, and generate more context-aware answers.

1. The Problem: Customers Don't Repeat Everything

Imagine a customer is chatting with Varastra, an Indian ethnic-wear store.

The conversation goes like this:

• Customer: Show me a festive kurta.
• Assistant: Here are a few options.
• Customer: What about it in mint?
• Assistant: Let me check the available options.
• Customer: And size L?

The last two questions depend on earlier messages. Without conversation context, the assistant may not know which product “it” refers to.

This is why Varastra maintains structured session memory.

Instead of relying on the current message alone, it keeps three useful pieces of information in Redis.

2. Understand the Architecture

The diagram shows this flow:

Current customer turn → Hybrid retrieval + reranking → Cross-encoder scores → Active-SKU bonus → Re-sort top 3 → Prompt builder → GPT-4o-mini → Answer and session update

Meanwhile, Redis session memory supplies context to the prompt builder and active-SKU ranking stage.

The memory contains:

1. The last three conversation turns.
2. A rolling two-sentence summary.
3. Up to three active product SKUs.

A SKU (Stock Keeping Unit) is an identifier used to distinguish a product or product variant.

3. Step 1: Store the Last Three Turns

The diagram labels the first memory component:

Last 3 turns · verbatim

These are the three most recent conversation turns, retained in their original wording.

For example:

• “Show me the mint-green kurta.”
• “Does it come in size L?”
• “Is it suitable for a festive occasion?”

Keeping recent turns verbatim preserves details that a summary might omit.

Why it matters: The assistant can refer to the customer's actual wording instead of trying to reconstruct every detail from memory.

4. Step 2: Maintain a Rolling Two-Sentence Summary

The second component is a rolling two-sentence summary, refreshed every four turns.

A summary preserves useful context without keeping the entire conversation in the active memory section.

For example:

“The customer is looking for festive ethnic wear in light colors. They are interested in a mint-green kurta and want to check size availability.”

This is an illustrative summary, not a fixed output from the system.

The diagram specifies that the summary is refreshed every four turns.

Why not rely only on recent messages? Older preferences or decisions may remain useful after they fall outside the last-three-turn window.

Why not keep the entire conversation? A compact summary helps control how much context is supplied to the model.

5. Step 3: Track Up to Three Active SKUs

The third component stores:

Active SKUs · up to 3 · most recent first

Suppose the customer has discussed these products:

• K-1299 — a mint-green kurta.
• S-4299 — a festive suit.
• K-2088 — a cotton kurta.

The system can keep these identifiers in session memory, ordered by recency.

The diagram shows a maximum of three active SKUs. It does not specify the exact rules for adding, removing, or expiring them.

Why it matters: Product IDs provide a concrete reference to items under discussion. The assistant doesn't have to rely entirely on ambiguous phrases such as “that one.”

6. Step 4: Retrieve and Rerank Candidates

When a new customer message arrives, it enters the retrieval pipeline.

The diagram specifies:

Hybrid + rerank · top-20 → CE scores

Hybrid retrieval finds candidate information using different search methods. A cross-encoder (CE) then scores each candidate's relevance to the current query.

For example, the customer asks:

“What about it in mint?”

The search pipeline retrieves possible product matches and assigns relevance scores.

However, a search result may not rank highly enough even if it refers to a product the customer has already been discussing.

That is where the active-SKU bonus comes in.

7. Step 5: Boost Products Already in the Conversation

The diagram shows this rule:

+0.05 if c.sku ∈ active_skus → re-sort → top-3

Here:

• c represents a candidate.
• c.sku is the candidate's product identifier.
• active_skus is the list of recently discussed products.
• +0.05 is the ranking bonus shown in the diagram.

Example

Imagine the cross-encoder produces these illustrative scores:

The active-SKU bonus changes the ordering. The system then re-sorts the candidates and selects the top three.

These scores are examples to explain the mechanism, not measured system results.

Why this matters: A product already under discussion can remain prominent even when another candidate receives a slightly higher original relevance score.

The bonus does not prove that a product matches the customer's current request. Relevance and mandatory constraints still matter.

8. Step 6: The Prompt Builder Combines the Context

The prompt builder receives the retrieved and reranked candidates alongside the relevant session memory.

The diagram shows the prompt builder receiving information from the memory components and the ranking pipeline.

Together, these inputs can help the model understand:

• What the customer asked most recently.
• What the conversation has been about.
• Which products are currently active.
• Which retrieved candidates are most relevant.

For example, “And size L?” can be interpreted using the earlier product discussion rather than treated as an isolated question.

Why it matters: Retrieval provides relevant information, while session memory helps preserve conversational continuity.

9. Step 7: GPT-4o-mini Generates the Answer

The diagram specifies:

Model: GPT-4o-mini · temperature 0

The model uses the prompt and supplied context to generate a response.

For example:

“You're asking about the mint-green kurta we discussed. I'll check whether size L is currently available.”

The system should verify actual size availability against authoritative product data before claiming that the size is in stock.

Temperature 0 makes generation more deterministic, but it does not guarantee correctness.

10. Step 8: Update Session Memory After the Answer

After generating the response, the system updates the session.

The diagram's feedback path says:

Push turn · refresh summary · update active SKUs

This closes the loop:

1. Add the latest conversation turn.
2. Refresh the rolling summary when required.
3. Update the active SKU list.
4. Use the updated memory on subsequent requests.

The next customer question can then benefit from the latest conversation context.

11. What Can Go Wrong?

Memory should be updated consistently, and the system should handle missing or invalid session data safely.

12. Key Takeaways

1. Redis stores three structured memory components in this architecture.
2. The last three turns preserve recent conversation wording.
3. A rolling two-sentence summary preserves broader context and refreshes every four turns.
4. Up to three active SKUs are maintained, most recent first.
5. The retrieval pipeline finds candidates and assigns cross-encoder scores.
6. An active SKU receives a +0.05 bonus before the candidates are re-sorted.
7. The prompt builder combines memory and retrieved context before GPT-4o-mini generates an answer.
8. Session memory improves continuity but does not replace retrieval, validation, or current product data.

The big lesson: Good AI memory isn't simply storing every message. It's preserving the right information so the assistant can understand the next question, remember the products under discussion, and respond with useful context.