What 1536 Numbers Actually Mean
On the third day of building ShopBot, Arjun runs into a wall he did not expect.
Not a bug. Not a misconfigured environment. A fundamental problem with the assumption he had been building toward without examining it.
He had assumed — without questioning the assumption — that finding relevant products from zUdyog Fashion's catalog was a search problem. You take the customer's words, you look for those words in the catalog, you return the matches. He had done keyword search a dozen times on other projects. He knew how it worked. It was not a sophisticated technique, but it was fast and it was proven.
He spends twenty minutes building a basic keyword search over zUdyog's five products. Eleven lines of Python. He tests it with six queries — the kind of queries real customers would type.
Four of the six return nothing.
He stares at the terminal for a long time, the chai beside his laptop going cold while he reads the empty result lines and tries to understand them.
He opens his notebook and writes down the four queries that failed, and why.
"Light fabric for summer" — fails. The catalog says breathable and cotton. Not light fabric.
"Something for a festive occasion" — fails. The catalog says festive wear. Not occasion.
"Warm outfit for cold weather" — fails. The catalog says woolen and winter evenings. Not cold weather.
"Traditional Indian clothing" — fails. The saree, the anarkali, the kurta — all traditional. But the words traditional Indian appear nowhere in any product description.
He puts the pen down. The realisation arrives quietly, the way realisations do when you have been working with a wrong assumption for too long.
The queries are completely reasonable. A real customer would type any of them without hesitation. The products that would answer each query exist in the catalog — the right answer is there. The problem is that the customer's words and the catalog's words are different. And keyword search only finds something when the words match exactly.
This is the problem ShopBot needs to solve before it can retrieve anything worth giving to the language model. A Grounding Layer built on keyword search will fail four customers in six. That is not a Grounding Layer. That is a different kind of drift — silent retrieval failure, followed by the model generating answers from nothing because nothing was retrieved.
Why Keyword Search Fails at This Job
Keyword search has been the backbone of information retrieval for decades. It works well for many things. This is not one of them.
The reason is precise. Keyword search operates on the surface of language. It asks "are these characters present in this document?" — not "does this document address what the searcher means?"
Consider the query "light fabric for summer." Keyword search looks for documents containing the string light, the string fabric, the string summer. zUdyog's linen co-ord set description says relaxed fit, summer casual — it contains summer, so it might surface. But the breathable cotton kurta, which is perhaps the most relevant product, says breathable cotton, ideal for summer — it does not contain light or fabric as separate terms. The woolen shawl, which is the wrong answer, also surfaces if any of the query words happens to appear anywhere in its longer description.
Now consider "warm outfit for cold weather." The woolen shawl entry says Heavyweight woolen shawl for winter evenings. The word warm does not appear. The word cold does not appear. A keyword search returns nothing — despite the woolen shawl being the obvious answer to that customer's question.
The gap is between vocabulary and meaning. Keyword search lives in vocabulary. Customers communicate in meaning. For a clothing catalog where warmth, lightness, formality, and occasion can each be expressed in dozens of different words, keyword search is the wrong foundation.
Arjun knows this clearly by the time he closes the terminal. He needs something that understands what a question is about — not just which characters it contains.
Three Approaches, Two Dead Ends
Before he reaches for the advanced solution, Arjun considers the alternatives. Drona's influence: never reach for the sophisticated fix before understanding why the simpler ones fail. The simple ones are worth understanding because they reveal what the problem actually requires.
Approach 1: Synonym expansion.
Build a synonym dictionary. Light maps to lightweight, breathable, airy, thin. Warm maps to woolen, thermal, insulated, cozy. When a query arrives, expand it with synonyms before searching.
He sketches it in fifteen minutes. It works better. Light fabric for summer now finds the breathable cotton kurta.
Then he tests something for a mehendi function.
Mehendi is a pre-wedding ceremony in Indian culture — a celebration where guests typically wear ethnic clothing in bright colours. The synonym dictionary has no entry for mehendi. It has no entry for sangeet, tilak, or any of the dozens of Indian occasion words a customer of zUdyog Fashion might reasonably use. Building a dictionary comprehensive enough for Indian fashion e-commerce is not a weekend task. It is months of work that becomes outdated the moment new terminology enters the market.
Synonym expansion shifts the problem from vocabulary mismatch to incomplete dictionary. Better. Not solved.
Approach 2: Filters instead of search.
Add structured dropdowns. Category: ethnic, western, fusion. Occasion: casual, formal, festive, wedding. Fabric: cotton, silk, linen, wool. The customer selects filters and ShopBot returns products that match.
This works for customers who know exactly what they want and understand the catalog's taxonomy. It fails for the customer who types: "something for a mehendi, not too heavy, the bride's sister is wearing mustard so nothing clashing."
That is a real customer query. Dropdowns cannot parse it. They cannot understand that not too heavy eliminates wool and silk in large quantities, that nothing clashing with mustard narrows the colour range, that mehendi implies ethnic rather than western. Natural language carries all of that simultaneously. Filters carry none of it.
Filtering is a complement to search. It is not a replacement for natural language understanding.
Both approaches address symptoms. The root problem — the gap between vocabulary and meaning — requires a different kind of solution. One that does not match characters, but understands concepts. That solution is embeddings.
What an Embedding Is
An embedding is the answer to one question: how do you put language into mathematics in a way that preserves meaning?
The answer, developed over decades of research and refined through modern deep learning, is this: represent each piece of text as a point in a high-dimensional space, where pieces of text with similar meaning are positioned near each other.
In practice, OpenAI's text-embedding-3-small model converts any piece of text — a word, a sentence, a paragraph — into a list of 1,536 numbers. That list is a coordinate. A precise location in a 1,536-dimensional space.
Here is what makes this useful: the space is structured by meaning, not by letters.
Lightweight summer kurta and breathable cotton top for warm weather produce coordinates that are close to each other in this space — because they describe similar things, even though they share almost no words. Woolen shawl for winter evenings and thick warm wrap for cold nights are also close to each other. Lightweight summer kurta and woolen shawl for winter evenings are far apart — opposite ends of the warmth spectrum, placed in opposite regions of the space.
Distance in this space is semantic distance. Closeness means similarity of meaning. This is called semantic similarity, and it is the engine behind everything ShopBot does.
When a customer types light fabric for summer, the embedding model converts that query into a coordinate. The retriever then finds the product embeddings that are closest to that coordinate in the 1,536-dimensional space — the products whose meaning is most similar to the query's meaning. Not the products that share the most words. The products that address the same concept.
That is what the Grounding Layer needs: not a word-matcher, but a meaning-finder. Embeddings are the meaning-finder.
Seeing It in Code
Arjun writes a demonstration. Not ShopBot yet — just proof that the concept works on real zUdyog products.
# src/embed_demo.py
from langchain_openai import OpenAIEmbeddings
from dotenv import load_dotenv
import numpy as np
load_dotenv()
embeddings = OpenAIEmbeddings()
# Five products from zUdyog Fashion's catalog
products = [
{
"id": "p001",
"text": "Breathable cotton kurta, ideal for summer. "
"Available in white, sky blue, mint green. Sizes S to XXL. Price ₹1,299."
},
{
"id": "p002",
"text": "Heavyweight woolen shawl for winter evenings. "
"Charcoal grey and maroon. One size fits all. Price ₹2,199."
},
{
"id": "p003",
"text": "Printed silk saree with zari border, festive wear. "
"Available in red, gold, emerald. Dry clean only. Price ₹4,299."
},
{
"id": "p004",
"text": "Linen co-ord set, relaxed fit, summer casual. "
"Colours: beige, sage, dusty rose. Sizes XS to XL. Price ₹1,899."
},
{
"id": "p005",
"text": "Embroidered anarkali suit with dupatta. "
"Teal. Sizes S to XXL. Price ₹3,499."
}
]
def cosine_similarity(vec_a, vec_b):
"""
Measure how similar two vectors are.
Returns 1.0 for identical meaning, 0.0 for unrelated, -1.0 for opposite.
For text embeddings, scores typically range from 0.5 to 1.0.
"""
a, b = np.array(vec_a), np.array(vec_b)
return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))
print("Embedding products...")
product_vectors = embeddings.embed_documents([p["text"] for p in products])
# Test queries — using customer language, not catalog language
test_queries = [
"light fabric for summer",
"something warm for cold evenings",
"festive Indian outfit for a wedding",
"traditional silk drape for celebration",
"casual everyday Indian wear",
]
for query in test_queries:
query_vector = embeddings.embed_query(query)
scored = []
for i, product in enumerate(products):
score = cosine_similarity(query_vector, product_vectors[i])
scored.append((score, product["id"], product["text"][:55]))
scored.sort(reverse=True)
print(f"\nQuery: '{query}'")
print(f" Best match (score {scored[0][0]:.3f}): {scored[0][2]}...")
print(f" 2nd match (score {scored[1][0]:.3f}): {scored[1][2]}...")
He runs it. Output:
Embedding products...
Query: 'light fabric for summer'
Best match (score 0.891): Breathable cotton kurta, ideal for summer...
2nd match (score 0.847): Linen co-ord set, relaxed fit, summer casual...
Query: 'something warm for cold evenings'
Best match (score 0.878): Heavyweight woolen shawl for winter evenings...
2nd match (score 0.641): Embroidered anarkali suit with dupatta...
Query: 'festive Indian outfit for a wedding'
Best match (score 0.912): Printed silk saree with zari border, festive wear...
2nd match (score 0.887): Embroidered anarkali suit with dupatta...
Query: 'traditional silk drape for celebration'
Best match (score 0.903): Printed silk saree with zari border, festive wear...
2nd match (score 0.791): Embroidered anarkali suit with dupatta...
Query: 'casual everyday Indian wear'
Best match (score 0.869): Linen co-ord set, relaxed fit, summer casual...
2nd match (score 0.832): Breathable cotton kurta, ideal for summer...
Every query finds the right product as its top result. Not one shared word between light fabric for summer and the cotton kurta's description — yet the match score is 0.891. Not one shared word between traditional silk drape and Printed silk saree — yet the match is 0.903.
The vocabulary–meaning gap is closed. The embedding understands that drape and saree are related. That celebration and festive are related. That warm and woolen are related.
Arjun saves the file and sits back. He feels, quietly and privately, that this is the moment the project became real.
Understanding Cosine Similarity
The scoring function — cosine similarity — deserves a clear explanation, because it will appear throughout this book and the next three.
Two vectors can be compared by measuring the angle between them. If the angle is zero — the vectors point in exactly the same direction — the similarity is 1.0. If the angle is 90 degrees — the vectors are perpendicular, completely unrelated — the similarity is 0.0. If the angle is 180 degrees — the vectors point in opposite directions — the similarity is -1.0.
The mathematical formula computes this without actually measuring angles.
similarity = (A · B) / (|A| × |B|)
The dot product of two vectors, divided by the product of their magnitudes. The result is always between -1.0 and 1.0.
For text embeddings in practice, scores cluster in a narrower range. Above 0.9 means very similar meaning — paraphrases, near-synonyms. Between 0.7 and 0.9 means related meaning — same topic area, overlapping concepts. Between 0.5 and 0.7 means loosely related — shared context but different focus. Below 0.5 means unrelated for practical purposes.
The score of 0.847 for light fabric for summer against the linen co-ord set is meaningful — the linen set is a legitimate second choice for a summer outfit, and the score reflects that. The cotton kurta wins because it scores 0.891. Neither is wrong; the higher number is simply closer.
This score becomes load-bearing in Chapter 6, when ShopBot needs to decide not just which product is most relevant, but whether any product is relevant enough to return. A system that always returns the top result — even when the top result scores 0.51 — will answer questions the catalog cannot answer. The score is the signal that tells the Grounding Layer when to retrieve and when to admit it has nothing.
A Grounding Layer that retrieves confidently when it should admit failure produces a different kind of Confident Drift — not from the model's training data, but from a retrieved document that was not actually relevant. The similarity score is the first line of defence against that failure. This is why the Pramana Framework treats retrieval quality as a first-class concern — a Grounding Layer that retrieves noise is no Grounding Layer at all.
What the 1,536 Numbers Actually Represent
A reasonable question: why 1,536? Why not 100, or 10,000?
The dimensionality is a design decision made during model training. More dimensions allow the model to encode finer distinctions — the difference between formal festive and casual festive, between winter warmth and monsoon warmth, between machine-washable cotton and dry-clean silk. Fewer dimensions lose those distinctions but are cheaper to compute and store.
1,536 is the dimensionality OpenAI chose for text-embedding-3-small — a balance between expressiveness and cost. It is expressive enough to capture the semantic distinctions that matter for product retrieval. It is compact enough that storing thousands of product embeddings in ChromaDB costs a negligible amount of disk space and memory.
What each individual number in the 1,536 represents is not interpretable by a human. The model learned the dimensions during training — they are not labelled warmth or formality or price range. The meaning is distributed across all 1,536 values simultaneously. The only way to read meaning from an embedding is to measure its distance from other embeddings. Distance is the language of this space.
What Krishna Understands From This
Arjun shows Krishna the output the same evening, in the second-bedroom-that-was-still-calling-itself-an-office.
Krishna reads it carefully. He is not a developer, but he reads output the way he reads a balance sheet — looking for what the number means, not how it was produced.
"This found the silk saree for the wedding query," he says. "Without the word silk or wedding in the query."
"Without either word," Arjun confirms.
"And the score — 0.912 — that means what exactly?"
"The meaning of festive Indian outfit for a wedding and the meaning of printed silk saree with zari border, festive wear are 91.2% aligned in the embedding space. They are describing the same kind of thing in different words."
Krishna looks at the second result for the same query — the anarkali at 0.887.
"The anarkali is also a good answer for a wedding query," he says. "Is 0.887 too close to 0.912? How do we know which one to recommend?"
It is the right question. Arjun does not have the full answer yet.
"That is what the retriever decides," he says. "We return the top three. The model reads all three and picks the most relevant one for the specific question."
"And if the model picks wrong?"
"Chapter 4," Arjun says.
The Cost of Not Having This
Before embedding was available at this price point and accessibility, the alternatives were poor.
A custom synonym dictionary required months to build, permanent maintenance, and was always incomplete. Real cost: one to two engineering months per catalog overhaul. Manual tagging — a human taxonomist reading every product and adding controlled-vocabulary tags — was trivial for five products and indefinite for five hundred. A commercial search engine licensed for e-commerce, like Elasticsearch or Solr with custom analyzers, was powerful but complex to configure for semantic search, required machine-learning expertise to tune, and ran into lakhs of rupees in annual licensing for a small business.
OpenAI's embedding API, used correctly, costs roughly ₹3–4 to embed zUdyog's starting catalog of five products. As the catalog grows to five hundred products, the one-time embedding cost grows to roughly ₹300. Each customer query — one embedding call per question — costs less than 0.01 paise.
This is not a tradeoff. At this price point, the embedding approach is simply better — more accurate, lower cost, no maintenance overhead — than every alternative that came before it.
A Reader Exercise
Run the embed_demo.py script with three additional queries.
queries = [
"something for a wedding",
"warm wrap for cold evenings",
"easy care everyday wear",
]
For each query, record which product ranks first and what its similarity score is. Then ask yourself: does the ranking match what you would recommend to a customer who typed that query?
If the ranking feels wrong for any query, examine the product description that ranked first. The embedding model is matching meaning to meaning — if the match feels off, the issue is usually in how the product is described, not in the model. Poorly written product descriptions produce poor retrieval. This is the first hint of what Chapter 5 addresses: the structure of what you store determines the quality of what you retrieve. [Effort: 15 minutes.]
What Just Happened
Keyword search failed because vocabulary and meaning are different things — a customer who types light fabric for summer and a catalog that says breathable cotton share no words but share a concept. Synonym expansion addressed the vocabulary gap for known terms but collapsed under the breadth of Indian fashion language. Filters handled structured queries but could not parse natural-language intent. Embeddings solve the root problem: they convert text into coordinates in a 1,536-dimensional space where semantic similarity is geometric proximity. Texts that mean similar things are positioned near each other regardless of shared words. Cosine similarity measures that proximity, and the score becomes the signal that tells the Grounding Layer in Chapter 6 when to retrieve and when to withhold.
The woolen shawl ranked last for a summer query not because of any rule — but because its meaning lives in a different region of the space. Keyword search needed words to match. Embeddings need meaning to be close. They are different tools solving different problems, and only one of them can power the Grounding Layer.
Chapter 4 builds the memory that holds these embeddings — ChromaDB, where zUdyog's catalog will live as a searchable vector store, and where the Grounding Layer will go to find evidence every time a customer asks a question.