Most people I see learning RAG skip the foundations. They jump straight from “I want a chatbot over my docs” to GraphRAG, Self-RAG, agentic retrieval, ColBERT, every flavor in the patterns zoo, without ever shipping the dumbest possible version first.
Then they wonder why their evals are unstable, why hybrid search “didn’t help,” why the reranker is “broken.” It’s because they don’t have a baseline to measure against. They don’t have the vocabulary to describe what they’re seeing. And they don’t have a mental model of the pipeline that lets them reason about which knob to turn.
So before I touch a single advanced pattern in my RAG mastery plan, here’s what Milestone 1, RAG Foundations, actually means.
Milestone 1, at a glance
| # | Foundation | What it covers | Mastery signal |
|---|---|---|---|
| 1 | The pipeline | Offline: documents → chunker → embedder → vector store. Online: query → embedder → retriever → top-k → prompt template → LLM → response. | Can sketch it from memory and explain why each step exists. |
| 2 | The vocabulary | Embedding, chunk overlap, BM25, RRF, MMR, recall@k, cross-encoder, HNSW, groundedness, faithfulness. | Can read a RAG paper without stalling on terminology. |
| 3 | The naive baseline | ~100 lines: load docs → fixed-size chunks → off-the-shelf embeddings → simple vector store → top-k cosine → prompt-stuff → LLM. No reranking, hybrid, or agents. | Can ship over a 50 to 100 PDF corpus by end of day. |
| 4 | The retrieval primitives | Dense, sparse, hybrid, reranking, metadata filtering, MMR. | Can explain when each helps and what it costs. |
The four sections below unpack each row.
The canonical pipeline
RAG has two phases.
Indexing (offline): documents → loader → chunker → embedder → vector store
Querying (online): query → embedder → retriever → top-k chunks → prompt template → LLM → response
You should be able to draw this on a whiteboard from memory and explain why each step exists.
Why chunk at all? Documents are too big for context windows, and finer-grained chunks let you retrieve only the relevant parts. Why embed instead of keyword search? Embeddings handle paraphrase and semantic similarity; pure keywords miss them. Why top-k instead of top-1? A single best chunk is rarely enough; you usually need a few candidates to give the LLM enough context. Why a prompt template instead of just concatenating? Templates give you a consistent contract for the LLM to follow, including instructions to ground in the context and refuse when it can’t.
If you can answer those without stalling, you’ve got the pipeline. If you can’t, the rest of RAG is going to feel like cargo culting.
The vocabulary
You don’t need to memorize a glossary, but you do need fluency. Here’s the minimum set:
- Embedding, dense vector, sparse vector — the numerical representations of text
- Chunk, chunk size, chunk overlap, stride — how you split documents
- Top-k, similarity score, cosine vs. dot product vs. Euclidean — how you measure closeness
- Vector store, vector index, ANN, HNSW, IVF, flat index — where and how vectors live
- BM25, TF-IDF — the sparse retrieval baselines worth knowing
- Hybrid search, RRF (reciprocal rank fusion) — how dense and sparse combine
- Reranker, cross-encoder vs. bi-encoder — how you re-sort retrieved candidates
- MMR (maximal marginal relevance) — how you add diversity to retrieved chunks
- Recall@k, precision@k, MRR, nDCG — how you measure retrieval quality
- Context window, prompt stuffing — what fits and how
- Groundedness, faithfulness, hallucination, citations — what makes an answer trustworthy
My rule: if a sentence in a RAG paper has a term I can’t immediately define, it goes on the list.
The naive baseline
This is the most important thing you’ll build in M1. Not because it’s impressive (it isn’t) but because everything else gets compared to it.
Concretely, the naive baseline is roughly 100 lines of code that:
- Load documents (PDF, markdown, whatever)
- Fixed-size chunk them (500 to 1000 tokens, around 10% overlap)
- Embed with one off-the-shelf model (text-embedding-3-small, Cohere, BGE, it doesn’t matter)
- Store in something simple (FAISS in memory, Chroma local, or pgvector)
- Embed the query, retrieve top-k by cosine similarity
- Stuff the chunks into a prompt template: “Answer using only the context below: {context}\n\nQuestion: {question}”
- Call the LLM, return the answer
No reranking. No hybrid. No query rewriting. No metadata filtering. No agent loop.
That’s the point. It’s the floor. When someone tells you GraphRAG improved their accuracy by 15%, the only meaningful question is: 15% over what? If they don’t have a naive baseline to compare against, the number is noise.
The retrieval primitives
Every advanced pattern is composed from a small set of primitives. Master these and you can read any RAG paper without getting lost:
- Dense retrieval — vector similarity over embeddings
- Sparse retrieval — BM25 over tokens
- Hybrid — dense plus sparse, fused with RRF or weighted scoring
- Reranking — run a cross-encoder over the top-N to re-sort into top-k
- Metadata filtering — pre-filter by structured fields before vector search
- MMR — re-pick top-k for diversity, not just relevance
Knowing the names isn’t the bar. The bar is being able to say when each helps and what it costs.
Example: dense retrieval alone misses exact-match terms like model numbers, error codes, and proper nouns. BM25 alone misses paraphrase; “how do I cancel my subscription” doesn’t match “ending your plan.” Hybrid handles both, but now you maintain two indices and tune fusion weights.
That kind of trade-off thinking is what M1 is really teaching you.
How I’ll know I’ve mastered M1
The plan says: “Mastered when you can explain the canonical pipeline and ship a naive RAG in less than one day.”
In practice, that breaks into two tests.
The whiteboard test. I can sketch the offline / online diagram from memory and answer questions like “why does chunk overlap matter?” or “when would you reach for a reranker?” without stalling.
The shipping test. Given a folder of 50 to 100 PDFs, I can have a working CLI or notebook answering questions over them by end of day, with basic citations. Looking up SDK syntax or pgvector setup is fine. Looking up what RAG is or how chunking works means I’m not there yet.
What’s next
After M1, I move into the patterns zoo: all 17 design patterns from naive through Self-RAG, CRAG, GraphRAG, and the rest. Each one ships as code. Each one gets compared to the naive baseline.
But none of that matters if the foundation is shaky. So this week, I’m building the baseline against a corpus I actually care about and timing how long it takes me to ship it.
If you’re starting your own RAG mastery path, I’d suggest the same. Don’t skip M1. The naive baseline isn’t the boring prerequisite. It’s the measuring stick that makes every advanced pattern meaningful.