Preface

The 9% Problem

ShopBot was no longer being built. It was being run.

That sentence had ended Book 2. By the time it was written, zUdyog Fashion's chatbot answered seven thousand queries on a campaign Friday and ninety-one out of every hundred customers walked away with the answer they came for. For three weeks the system had survived without anyone touching it. Arjun checked the dashboard each morning and went back to his other work. Krishna stopped asking about cost. The 22% from Book 1 had become 9%, and 9% felt — for a stretch of weeks Arjun would later remember as the quietest of the project — like a place to stop.

It was not a place to stop.

The three weeks he spent watching the dashboard hold without doing anything to it were, he later came to think, the part of the project he was least proud of — not because the work was bad but because he had let himself believe the work was finished. The chai beside his keyboard had cooled twice that morning while he scrolled through the failed-query log for what he told himself was the last time. He did not reach for it.

The 9% sat in the log as four discrete piles, each one a different kind of failure that could not be addressed by tuning what was already there. Hybrid retrieval and reranking had taken Book 2's piles down. Vocabulary gaps. Single-intent literal misses. Conversational context. Repeated computation. What remained was different in kind.

A query like something nice for the weekend had no token to match and no obvious meaning to embed. The retriever returned semantically reasonable chunks that were not on-target — the model answered, but the answer felt generic in a way that made customers click away. A query like what jewellery would suit the silk saree returned the saree's chunk and asked the model to answer about jewellery from it. Type mismatch. The model received product information and was asked a styling question; the prompt constraint from Book 1 caught some of these, not all. A query that began and the smaller size? in turn three of a conversation depended on a memory the system had — but the system did not have a way to decide whether to retrieve again, refuse, or hand off to a different code path; the chain ran the same shape on every turn whether or not the shape was right. And the embedding bill — small per query, real in aggregate — was a line item Krishna had stopped asking about for the wrong reason. He had stopped because Arjun had said, in November, that the per-query cost was a paisa and a half. At fifty thousand queries a day, that paisa and a half was ₹750 a day, which was ₹22,500 a month, which was a number Krishna noticed when he looked at the spreadsheet that lived in the same folder as the whiteboard photographs.

Krishna had written four lines on the whiteboard at the end of Book 2, in his careful capitals, the photograph dated and saved.

Vague queries — HyDE. Type mismatch — CRAG. Routed pipeline — LangGraph. Embedding cost — fine-tuned bge-small → ONNX INT8.

The lines on the board are the agenda. They have not yet become work.

This is the book where they do.

Each one of the four is a different kind of architectural move from the moves in Book 2. Hybrid retrieval was a signal added; reranking was a judge added; memory was a state added; caching was a layer added. None of those changed the shape of the pipeline. The pipeline in Book 2 is still a chain — query enters, retrieves, reranks, generates, returns. Each component does its job and hands off to the next.

The four problems on the whiteboard cannot be solved by adding another component to that chain. They require the chain to become something other than a chain. HyDE generates a hypothetical answer first and retrieves against the answer rather than the question — the chain bends back on itself. CRAG checks whether the retrieved context actually addresses the query and re-retrieves with a transformed query if it does not — the chain branches. LangGraph turns the pipeline into a routed graph that can take different paths depending on what the query looks like — the chain stops being a chain at all. And the fine-tuned bge-small in INT8 is an admission that the off-the-shelf embedding model is too general for a five-hundred-product fashion catalog and too expensive to keep paying for at scale — the chain replaces a component that has not been touched since Book 1 Chapter 3.

These are the four. They are not optional. The 9% will not become 5% by adjusting thresholds or chunking more carefully. Each fix in this book is a different shape of architecture, and learning to choose between them — when to bend the chain back, when to branch it, when to graph it, when to retrain it — is what distinguishes a Book 2 system from a Book 3 system.

The Pramana Framework, by the end of this book, will not only refuse honestly when retrieval is silent. It will also know how to reach for a different kind of evidence when the question itself is too thin to anchor on, when the question's type does not match the catalog's shape, or when the right answer requires the system to ask itself a question before reaching for the catalog at all.

The book has nine chapters and an appendix. The chapters work through the four whiteboard lines, each in its own architectural register, with the labelled RAGAS dataset from Book 2 Chapter 8 as the standard the work is measured against. The appendix carries the wiring — the new clients, the fine-tuning script, the LangGraph state schema, the ONNX export pipeline.

By the end, the 9% will not be zero. Software does not become zero. But it will become a number that fits in single digits and that the dashboard can name, and what the system speaks from will be a Grounding Layer that knows how to reach harder when the obvious reach fails.

This is the book about what comes after the architecture works. The work that begins after the work is reliable.

Chapter 1 — Nine of Every Hundred — opens on the morning Krishna re-photographs the four-line whiteboard, hands the printout to Arjun, and asks the question that has been waiting since the last campaign Friday closed. How long until the first one?