Detailed content
🔢 What embeddings are
Computers don’t understand words—they understand numbers. Embedding is the trick that turns text into numbers while preserving the meaning: each sentence becomes a long list of numbers (a vector), and texts with similar meaning end up nearby of each other in an imaginary space. "Cat" and "feline" stay close; "cat" and "truck" are far apart.
🆕 New here?
"Embedding" (or "vector") is a list of numbers—like [0.12, -0.03, 0.88, ...], generally with 384 to 1024 positions—which represents the meaning of text. A "embedding model" is a small AI trained only to produce these vectors (it doesn’t generate text, it just “measures meaning”). "Similarity" = how close two vectors are; close = similar meaning.
🔑 The central idea
Embeddings turn “searching for text” into a problem of geometry:
- •Each piece of text becomes a point in a space with many dimensions.
- •Similar meanings stay next; distant meanings become far.
- •Searching for “what matches the question” becomes finding the nearest points — it doesn’t matter whether the exact words match.
Each word becomes a point. The question (in cyan) lands near the "animals" cluster and far from "transportation" and "food"—so the search returns cat, feline, kitty, even if the question doesn’t use any of those words. This is a search for meaning, not by exact word.
📚 Why RAG
The local model knows a lot about the world — but it knows nothing about your world: your PDFs, your notes, your company’s internal documentation. Training the model again with this data is expensive and slow. RAG solve it externally: when the question comes in, it search the right excerpts from your documents and paste in the prompt as context. The model responds by reading it — no retraining needed.
🆕 New here?
RAG = Retrieval-Augmented Generation (retrieval-augmented generation). In plain English: "before answering, go find the relevant passages and read them". The model stays the same — you just enrich the prompt with the right material at the right time.
✗ Model only (no RAG)
- ✗Doesn’t know your private documents
- ✗"Hallucinates" when it doesn’t know — makes things up confidently
- ✗Knowledge frozen as of the training date
- ✗Can’t cite the source
✓ Model + RAG
- ✓Answers questions about YOUR files
- ✓Fewer hallucinations — responds based on the retrieved passage
- ✓Updates instantly: change the document, change the response
- ✓You can show where the answer came from
🆚 RAG isn’t fine-tuning
Fine-tuning changes the weights of the model (teaches it a new way to respond) — expensive, slow, and hard to update. RAG doesn't touch the model: it only decides what to show in the prompt. For “answer questions about my data,” RAG is almost always the simplest, cheapest, and most updatable option.
💡 Practical tip
RAG is what provides memory long-term memory to a local agent (Track 4): you store conversations, notes, and facts as vectors, and the agent “remembers” by retrieving what matters — instead of trying to cram everything into the context window. Memory = RAG over what has already happened.
🔧 Anatomy of a RAG
Every RAG has two halves: the ingestion (done once, when you add documents) and query (done for each question). Understand the diagram and the rest becomes a code detail.
🆕 New here?
"Chunk" (chunk) is a section of the document—a paragraph, around 200–500 tokens. You split the text into chunks because (1) the embeddings model has a size limit and (2) you want to retrieve only the section relevant, not the whole file. A good chunk is self-contained (makes sense on its own) and neither too large nor too tiny.
Read in two rows: above (purple), the ingestion — document → chunks → vectors → database. Below (in cyan), the query — question → vector → find the neighbors in the database → the Local LLM receives the passages and writes the answer. The same embeddings model is used on both sides.
The 6 steps, in order
Chunk — split up the document
Split the text into self-contained chunks (e.g., 1 paragraph). Only during ingestion.
Embed — turn into a vector
Pass each chunk through the embedding model; each one becomes a vector.
Store — save to the vector database
Save the vector + original text in the vector store. Done: document “indexed.”
Query — embed the question
When you ask a question, embed its text with the same model.
Retrieve — find the nearest neighbors
Ask the database for the N chunks closest to the question vector.
Generate — respond with context
Paste the excerpts into the prompt and ask the local LLM to answer using only them.
⚡ Running embeddings locally
Embedding models are small ones — some are only a few hundred MB and run smoothly even on budget hardware, without a GPU. The nomic-embed-text is a great default to start with: lightweight, good at Portuguese, and already packaged in Ollama. Let’s download it and generate the first vector.
⌨️ Practical example: your first embedding
Objective: download an embeddings model and turn a sentence into a vector. Paste it into the terminal and then into Python.
How to check: o curl returns a JSON object with a long list of numbers — that's the vector. In Python, len(...) prints a dimension (e.g., 768). If it printed a number, your embeddings are running 100% locally. Save that number: all vectors from the same model have the same size.
💡 Practical tip
Use the same embeddings model to index the documents and embed the question. Mixing models (or switching models midway) breaks search because the vectors no longer live in the same space. Did you switch embedding models? Re-index everything.
🗄️ Simple vector database
You could store the vectors in a list and compare them one by one—it works with 10 chunks, but becomes painfully slow with 100,000. A vector database is built for this: it stores vectors and quickly answers the question "which are the N most similar to this one?".
🆕 New here?
"Vector store" / "vector database" is a database specialized in finding similarities between vectors. Common local options: Chroma (easiest to get started, stores on disk with 2 lines), FAISS (Meta's library, incredibly fast, no server) and sqlite-vec (an extension that adds vector search to the SQLite you already know). They all run locally, without the cloud.
🟣 Chroma
Simpler to get started. Persists to disk on its own, user-friendly API.
Use when: wants to set up a RAG in minutes.
🟣 FAISS
Meta’s search library. Very fast, great at scale.
Use when: there are many vectors, and speed matters.
🟣 sqlite-vec
Vector search inside a SQLite file. No server.
Use when: already uses SQLite and wants a single file.
⌨️ Minimal setup with Chroma
Objective: store a vector and find the closest match—the heart of RAG, in isolation.
# pip install chromadb import chromadb db = chromadb.PersistentClient(path="./meu_rag") # guarda em disco col = db.get_or_create_collection("docs") # guardar: id + texto + o vetor que veio do nomic-embed-text col.add(ids=["1"], documents=["A senha do wifi do escritório é roxo-1234"], embeddings=[[0.12, -0.03, 0.88]]) # <vetor real do passo 4> # buscar: os 3 mais parecidos com o vetor da pergunta hits = col.query(query_embeddings=[[0.10, -0.01, 0.90]], n_results=3) print(hits["documents"]) # → o(s) trecho(s) recuperado(s)
How to check: run it twice. The PersistentClient writes to ./meu_rag, so on the second run the document is already there without needing to add it again. The query should return the text you saved.
💡 Practical tip
Always keep the original text alongside the vector (the field documents). The vector is only used to find; what you paste into the prompt is the text. A vector database without the text stored is a search that finds the right spot but can’t show you the content.
🚀 A minimal end-to-end RAG
Time to put it all together. This script handles the complete RAG workflow: it reads a .txt, breaks it into chunks, embeds and stores them, then answers a question using only what's in the document — everything in your Ollama. The sections marked with <...> are what you swap.
⌨️ Practical example: full RAG in ~25 lines
Objective: answer a question whose answer exists only in your file. Save it as rag.py and run.
# pip install chromadb openai (Ollama já rodando) from openai import OpenAI import chromadb client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") EMB = "nomic-embed-text" LLM = "qwen2.5:7b" # <troque pelo seu modelo> def embed(txt): return client.embeddings.create(model=EMB, input=txt).data[0].embedding # 1) INGESTÃO: lê o .txt, quebra em chunks, embeda e guarda texto = open("notas.txt").read() # <seu arquivo .txt> chunks = [c for c in texto.split("\n\n") if c.strip()] # 1 parágrafo = 1 chunk col = chromadb.Client().create_collection("notas") col.add(ids=[str(i) for i in range(len(chunks))], documents=chunks, embeddings=[embed(c) for c in chunks]) # 2) CONSULTA: embeda a pergunta, busca os vizinhos, monta o prompt pergunta = "Qual a senha do wifi do escritorio?" # <sua pergunta> viz = col.query(query_embeddings=[embed(pergunta)], n_results=3)["documents"][0] contexto = "\n".join(viz) resp = client.chat.completions.create(model=LLM, messages=[ {"role": "system", "content": "Responda SO com base no contexto. Se nao houver, diga 'nao sei'."}, {"role": "user", "content": f"Contexto:\n{contexto}\n\nPergunta: {pergunta}"} ]) print(resp.choices[0].message.content)
How to check: create one notas.txt with a fact that the model there'd be no way to know — e.g., the line The office wifi password is roxo-1234. Run it and ask the question above: if the answer includes roxo-1234, RAG worked—the model got it right only because the passage came from the your document. Now ask something that no is in the file: it should respond I don't know instead of making things up.
✓ RAG that works
- ✓Self-contained chunks of the right size
- ✓Same embedding model for ingestion and queries
- ✓System prompt says to use ONLY the context
- ✓Shows the source for every answer
✗ RAG that fails
- ✗Huge chunk (entire document) or too fragmented
- ✗You switched the embedding model and didn’t re-index
- ✗No “don’t make things up” rule — the model hallucinates
- ✗Puts too much text in the prompt and overflows the context
Self-check (optional): what does RAG do in practice?
📌 Module summary
Next:
Track 4 — Advanced & 24/7 Agents · you’ll move from “running and serving models” to putting them to work on their own, all the time.