The Problem With Brilliant Liars

It is 11 PM on a Thursday.

Arjun has thirty-one support tickets open in one browser tab and the chatbot conversation logs in another. He has been cross-referencing them for forty minutes. The chai on his desk went cold an hour ago. He is not looking for a pattern; he already found it. He is reading to understand how bad it is.

The first ticket: "Your chatbot told me the kurta had a cotton lining. I bought it. It doesn't."

He opens the log. Finds the conversation. The customer had asked, in plain words, "Is this kurta cotton-lined?"

The chatbot had replied: "Yes, this kurta features a soft cotton lining for added comfort, making it ideal for all-day wear."

He opens the product page. No lining mentioned anywhere. Not in the description. Not in the specifications. Not in the care instructions. The chatbot invented it. Fluently. Confidently. In the tone of a helpful assistant who knew exactly what they were talking about.

He reads the next ticket.

"The bot said I could return within 30 days. Your policy says 7."

Log. Conversation. The customer asked about returns. The chatbot said thirty days. zUdyog's policy page says seven, in bold, at the top of the help section.

He reads six more. Each one is the same shape. The customer asked a factual question. The chatbot gave a factual-sounding answer. The answer was wrong. The customer acted on it. One had driven forty minutes to a pickup point for a delivery the chatbot said had arrived. It had not. The chatbot had given them a timeline it had constructed from nothing.

By the eighteenth ticket, what he was reading had stopped being information and started being an accusation, and he could not look away.

Three returns filed this week. Two angry emails. One customer who said, in writing, that she would not shop at zUdyog again.

Arjun closes the ticket tab. Opens his notebook. Writes one line.

"The chatbot is a brilliant liar. Figure out why. Then fix it."

He pauses. Adds one more line beneath it, because honesty matters even when it is uncomfortable.

"I built it. This is my problem."

What the Chatbot Was Actually Doing

Arjun spends an hour reading before he writes a single line of code in response.

The chatbot was not broken. That was the thing he had to accept before he could fix it. A broken system fails visibly — it crashes, it returns an error, it produces gibberish that no one would trust. This chatbot did none of those things. It answered. Every time. Smoothly. In complete sentences. In the correct register for a customer service interaction.

The problem was not that it malfunctioned. The problem was what it did when it did not know something.

Here is the thing about the AI system powering that chatbot. It was trained on text. Enormous amounts of it — product descriptions, customer reviews, support conversations, fashion articles, thousands of pages of clothing-catalog content from across the internet. During training it learned, at a deep level, the patterns of how product-related questions are answered. How kurta descriptions sound. How return policies are typically phrased. How customer-service agents explain delivery timelines.

What it did not learn — could not have learned — was the specific contents of zUdyog's catalog. zUdyog's actual return policy. The actual specifications of this specific kurta. These were not in its training data. They are private business data that existed nowhere the model had access to.

So when a customer asked "Is this kurta cotton-lined?" — the model did not consult a database. It did not retrieve the product specification. It predicted: given this question, in this context, what is the most plausible continuation of this conversation?

And the most plausible continuation — based on everything it had learned about how product Q&A conversations go — was to confirm the feature confidently.

"Yes, this kurta features a soft cotton lining..."

The model did not know the answer. It only knew what answers sound like.

That is the sentence Arjun writes in his notebook twenty minutes into his reading. Not as a metaphor. As a precise technical description of what happened.

The model did not know. It generated. And generation trained on plausible text produces plausible text — whether or not that text is true.

Why This Is Dangerous

There is a particular kind of failure that is harder to catch than visible failure.

When a system crashes, everyone knows it crashed. When a system returns an error, the user sees the error. When a system produces gibberish, the user recognises gibberish. But when a system produces a confident, well-formatted, grammatically perfect answer that is simply wrong — no alarm sounds. No error is thrown. The HTTP response is 200 OK. The answer appears in the chat window. The customer reads it, trusts it, acts on it.

This failure is invisible in form. The system that is failing looks identical, from the outside, to the system that is working.

And there is something else. The model does not fail more on the questions it knows less about. It does not hedge on uncertain ground. It answers with the same fluency and confidence whether it is describing something it has absorbed deeply from thousands of examples or constructing something from scratch because the real answer was never in its training data.

This is what Arjun later came to call Confident Drift.

The system drifts — from what it can actually verify into what sounds right — without any signal that the drift is happening. Confidently. Fluently. Wrongly.

Every wrong answer in those thirty-one tickets was a drift. Not a malfunction. A drift. The model moved from the edge of its knowledge into the territory of plausibility, and it did so without hesitation, without caveat, without a single word that told the customer: I am not certain about this.

Understanding drift precisely — what causes it, why it looks the way it does — is the only way to build something that prevents it.

Three Things Arjun Tried Before Finding the Answer

He does not arrive at the solution immediately. He tries three other approaches first, because the obvious fixes are wrong in ways worth understanding.

Attempt one: a more careful prompt.

The instinct is natural. If the chatbot is being too confident, tell it to be less confident. Arjun rewrites the system prompt: "Only answer if you are completely certain. If you are not certain, say you don't know."

He tests it. The next few answers come back with phrases like I believe, to the best of my knowledge, and you may want to verify this.

Then a ticket arrives: "Your chatbot said 'I believe this comes in size S, but you may want to verify' — so I ordered it assuming it probably did. It didn't come in S."

A hedge phrase on a drift is not a fix. It is a disclosure attached to an invention. The customer who reads I believe and decides that means probably yes is not protected. The model still does not have access to zUdyog's actual inventory. It is still generating from pattern. The prompt made it slightly more polite about its uncertainty without making it any more accurate. Stricter prompts reduce confidence. They do not reduce wrongness.

Attempt two: paste the entire catalog into the prompt.

If the model does not know zUdyog's data, give it the data. Every product description, every size listing, every policy paragraph — copied into the system prompt so the model can read from it.

Arjun tries this. Accuracy improves. He is briefly optimistic.

Then he checks the API bill for the week. Every single customer query — regardless of what was asked — is sending the entire catalog to the model. Thousands of tokens per request, even when the customer is asking about one product. The cost per query has increased by a factor of twenty. And the accuracy, while better, is still not reliable. When the catalog is long enough, the model's attention drifts across it inconsistently. Products near the beginning and end of the prompt fare better than those in the middle. Ask about a product buried in the fifteenth paragraph of a twenty-paragraph catalog and the model sometimes answers from a different product's details.

Pasting the catalog into the prompt is a patch. It addresses the symptom — the model not having the information — without addressing the cause. It is also expensive, increasingly unreliable as the catalog grows, and completely unscalable. When zUdyog adds two hundred new products for the festive season, the prompt that was barely affordable becomes impossible.

Attempt three: fine-tune the model on zUdyog's data.

Train a version of the model specifically on zUdyog's products and policies. Teach it, at the weight level, what zUdyog sells and how.

This sounds right. It fails for a reason that is easy to miss.

Fine-tuning changes how a model talks, not what it knows in a reliably current way. A fine-tuned model learns to sound like a zUdyog assistant. But its knowledge of product details is encoded in weights — static parameters that do not update when the catalog changes. A new collection launches. Sizes change. A return policy is updated. The fine-tuned model does not know any of this. Its knowledge is frozen at the moment of training.

Fine-tuning a model on a live business's data is like training an employee thoroughly in January and never updating them again. By April, half of what they confidently know is out of date.

Each of these three attempts fails for the same underlying reason. They try to put better information inside the model. The correct solution puts real information in front of the model, at the moment of each query, every time.

The Missing Piece

Arjun draws a diagram in his notebook. Not code. A diagram.

On the left: the customer's question. On the right: the model's answer. In between: nothing. A gap.

That gap is where the problem lives. The question arrives, the model has no bridge to real data, so it generates from pattern. The answer emerges from statistical probability rather than verified fact.

What needs to go in that gap is obvious once you see the gap clearly. Real data. Retrieved from the actual catalog. For this specific query. Before the model generates a single word. Not the entire catalog — that was attempt two's mistake. The relevant portion of the catalog, retrieved specifically for what this customer is asking about, right now.

The model's job changes completely. Instead of being asked to know things — which it cannot reliably do — it is asked to read a specific piece of real information and express it clearly. That is a task it is exceptionally good at.

A translator given a document to work from does not need to know the content from memory. They read the document. They render it accurately in the target language. Their skill is in the rendering, not in the remembering. The model is the translator. It renders evidence into language. The gap needs to be filled with evidence — retrieved, current, specific — before the rendering begins.

He explains it to Krishna the next morning, in the second-bedroom-that-was-still-calling-itself-an-office, with a single analogy.

"Imagine you hire someone to answer customer calls," he says. "Day one, you train them on zUdyog for a week. Then they start answering."

"Reasonable," Krishna says.

"Three months later — two hundred new products. A changed return policy. New sizes. Does the person trained in week one know any of that?"

"No."

"And if they answer anyway — because customers are calling and silence is not an option — what happens?"

Krishna considers. "They guess. Plausibly. Based on training."

"That is every LLM without RAG," Arjun says. "Now imagine: before every call, you give the same person a printed sheet with the specific product information relevant to that caller's question. They read the sheet. They answer from the sheet only. They cannot answer from memory."

"Then they are accurate," Krishna says.

"Even if the product changed yesterday. Because the sheet is current."

Krishna is quiet for a moment. "And RAG is the system that prints the sheet."

"Every time. For every question."

The Grounding Layer

Every AI system has a point where it decides what is true. In this book, we will call that point the Grounding Layer. In classical Indian philosophy, there is a word for a valid source of knowledge: Pramana. Not a guess. Not a pattern. A valid source. This book is about building AI systems that have one.

In zUdyog's broken chatbot, that point was inside the model — in the probability distributions learned from training data. The model decided what was true by calculating what was most plausible. The consequences are in those thirty-one tickets.

In a grounded system, that point is in the retrieval step that happens before generation. The system retrieves the actual truth from the actual data source — the real catalog, the current policy, the verified specification — and places it in front of the model as its context. The model reads it. It answers from it. It does not reach beyond it.

When retrieval succeeds, the answer is grounded. When retrieval fails — when the relevant evidence is not found, or the data was never stored, or the chunk was structured poorly — the gap reopens and drift resumes.

The Grounding Layer is not a technology you install. It is a design decision — the decision that between every question and every answer, a retrieval step happens. Real data is found. Real data is provided. The model translates evidence, not imagination.

What This Looks Like in Practice

Same question. Two different systems.

Without a Grounding Layer. A customer asks: "Does the embroidered anarkali suit come with a dupatta?" The chatbot answers: "Yes, most embroidered anarkali suits include a matching dupatta as part of the ensemble. This gives a complete, coordinated look suitable for festive occasions." The answer sounds correct. It is based on the general truth that many anarkali suits come with dupattas. But it is not based on zUdyog's specific product. The answer might be right by coincidence. It might be wrong.

With a Grounding Layer. The system retrieves the actual product entry for zUdyog's embroidered anarkali suit. It finds this text: Embroidered anarkali suit with dupatta. Teal. Sizes S to XXL. Price ₹3,499. Care: dry clean only. ShopBot answers: "Yes, the embroidered anarkali suit comes with a dupatta included. It is available in teal, in sizes S to XXL, priced at ₹3,499."

Same question. Different architecture. The second answer is not a guess — it is a reading of actual product data. If the product did not come with a dupatta, the retrieved text would say so, and ShopBot would say so.

That is the entire promise of RAG. Not a smarter guess. A grounded answer.

What This Means for ShopBot

Arjun closes his notebook. Opens a terminal.

He will build ShopBot — a product assistant for zUdyog Fashion that cannot drift. Not because it has been instructed not to. Because the architecture does not give it the opportunity.

When a customer asks whether the silk saree needs dry cleaning, ShopBot will retrieve the silk saree's care-instruction chunk — the actual text Arjun wrote when he structured the catalog — and express it. If the instruction says dry clean only, ShopBot says dry clean only. If the catalog entry mentions nothing about a lining, ShopBot says it has no information about a lining and offers the support contact.

The model behind ShopBot is, if anything, less capable than the one behind the chatbot that failed. It is gpt-4o-mini rather than a larger model. It does not matter. A smaller model reading accurate evidence produces better answers than a larger model drifting through plausible patterns.

ShopBot will know the answer because, before it speaks, it will have read it.

This book is not about making models smarter. It is about making them honest.

A Reader Exercise

Before reading Chapter 2, do one thing.

Pick a product you own — a phone, a garment, an appliance. Something with a specific verifiable detail: a battery capacity, a fabric composition, a warranty period, a return window.

Ask any publicly available AI assistant: "What is the [specific detail] of [your product]?"

Then check the real answer — the box, the manual, the manufacturer's website.

Note two things. First — was the AI's answer correct? Second — whether correct or wrong, did the AI signal any uncertainty? Did it hedge? Did it say I'm not sure, but... or you should verify this? Or did it answer with the same fluency and confidence it would use if it actually knew?

This exercise makes Confident Drift personal. You will see exactly what it looks like — an answer that arrives with authority, with specificity, with the full vocabulary of confidence — regardless of whether it is grounded in actual knowledge or constructed from pattern.

Remember the feeling. Every design decision in this book exists to prevent it from appearing in systems you build. [Effort: 10 minutes.]

What Just Happened

An AI chatbot drifted — from the edge of what it knew into what sounded right — and produced thirty-one support tickets, three returns, and one customer who left. The model did not malfunction; it did exactly what it was trained to do, which was generate plausible text. Three fixes failed because they tried to put better information inside the model. The correct fix puts real information in front of the model, per query, before generation begins. That is the Grounding Layer — the Pramana of an AI system, its valid source of knowledge.

This book builds one.

Chapter 2 turns the empty directory into a working project. The virtual environment. The dependencies. The verified setup. And the first test that turns a single sentence into 1,536 numbers — the numbers where everything begins.