The idea that turns words, sentences, even images into numbers a computer can compare — and the foundation of semantic search and "chat with your documents." Simpler than full generation, and immediately useful.
Ordinary keyword search is dumb about meaning. Search "how to fix a flat" and it may miss a page titled "repairing a punctured tire" — same idea, different words. Computers match letters, not meaning.
Embeddings fix this. They turn text into numbers that capture meaning, so a machine can tell that "flat" and "punctured tire" are basically the same thing. That single capability powers modern search, recommendations, and letting an AI answer questions from your documents.
You met this in the attention guide: an embedding is a list of numbers — a vector — that represents a piece of text. A short one might be 384 numbers; larger models use 1,000+. Each number is a dimension.
You don't set these numbers by hand. A model learns them from reading enormous amounts of text, tuning the numbers so that things used in similar ways end up with similar vectors.
The key intuition: picture every vector as a point in space. Related meanings land near each other; unrelated meanings land far apart. Real embeddings live in hundreds of dimensions (impossible to draw), but the idea is the same as a simple 2-D map.
If meaning is a point in space, "how similar are two texts?" becomes "how close are their two points?" The most common measure is cosine similarity: it looks at the angle between two vectors.
Practically: to find the best match for a query, embed the query, then find the stored vector with the highest cosine similarity. That's semantic search in a nutshell.
Here's the demo that made embeddings famous. Because meaning is captured as numbers, you can do arithmetic on it. Take the vector for "king," subtract "man," add "woman" — and you land almost exactly on "queen."
The model learned that a consistent direction in the space means "royalty," and another means "gender." Moving along those directions actually changes meaning in a sensible way. It's a striking sign that these numbers encode real structure, not random noise.
Single words are just the start. Modern embedding models take a whole sentence, paragraph, or document and produce one vector for the entire thing — a summary of its overall meaning.
This is what makes real applications possible: you can embed a support article, a product review, a legal clause, or a user's question, and then compare them all in the same space. "Find documents similar to this question" becomes a distance lookup.
Semantic search — find results by meaning, not exact words. ("flat" finds "punctured tire.")
Recommendations — "more like this": items whose vectors sit near ones you liked.
Clustering — automatically group thousands of reviews or tickets by topic, with no labels.
Classification — is this email spam? Compare its vector to known examples.
Deduplication — spot near-duplicate texts even when the wording differs.
RAG — let an LLM answer using your private documents (next section — the big one).
An LLM only knows what it saw in training — not your company handbook or last week's notes. Retrieval-Augmented Generation (RAG) fixes that by fetching the right documents first, then handing them to the LLM to answer with. Embeddings are the "fetching" part.
It runs in two stages. Setup (done once):
Answering a question (every time):
Because the LLM is handed the actual relevant text, it can answer accurately about private or up-to-date material it was never trained on — and cite where the answer came from. This is one of the most common real-world uses of AI today.
If you have a million chunks, comparing a query to every single one can be slow. A vector database (or a library like FAISS) stores vectors cleverly so it can find the closest ones without checking all of them — this is approximate nearest-neighbor search. It trades a tiny bit of accuracy for a huge speed-up.
Everything above usually runs in the cloud — which means your documents leave your device. Doing it on-device flips that: you embed and search your own files locally, so nothing private ever gets uploaded.
/v1/embeddings endpoint produces these vectors right on the phone or laptop. Combined with a small local vector store, that's the foundation for private, offline "chat with your documents" — RAG with none of your data leaving the device. It's a natural pairing with the shared-model and memory tricks from the first guide.Pick an answer and it'll explain why. Score updates as you go.
1. What problem do embeddings solve that keyword search doesn't?
Meaning, not words. "flat" can match "punctured tire" because their vectors are close.
2. In embedding space, two texts with similar meaning have vectors that are…
Close together. Similarity = nearness. Cosine similarity measures it via the angle between vectors.
3. A cosine similarity near 1.0 between two vectors means…
Very similar. Near 0 = unrelated; near −1 = opposite.
4. Why is "king − man + woman ≈ queen" impressive?
Real structure. Consistent directions in the space encode ideas like "royalty" and "gender."
5. In RAG, what is the job of the embeddings?
Retrieval. Embeddings fetch the relevant chunks; the LLM then answers using them.
6. What does a vector database speed up?
Nearest-neighbor search. It indexes vectors so you only search "nearby," trading a little accuracy for speed.
7. What's the main benefit of doing embeddings on-device (as in EdgeLM)?
Privacy + offline. Nothing gets uploaded; you can search your own files with no internet.
/v1/embeddings) so private data never leaves the device.