Build retrieval-augmented generation systems with effective chunking, search, and evaluation.
An LLM's knowledge stops at its training cutoff and it can't access your private data. ReAct and RAG are the two prompt engineering frameworks that fix both problems — turning a plain LLM into an agent that can act and retrieve.
RAG is how you give LLMs accurate answers from documents they've never seen. This post covers the full architecture: chunking, embeddings, similarity search, vector databases, and a working implementation with ChromaDB.
A complete hands-on implementation of RAG using ChromaDB — persistent storage, collections, metadata filtering, custom embedding models, and a full end-to-end pipeline that answers questions from a private document.
Bad retrieval in RAG almost always traces back to bad chunks. Learn why fixed-size chunking destroys context, when it's acceptable, and what the alternatives are.
Fixed-size chunking breaks sentences mid-word. Sentence-based chunking fixes that by treating each complete sentence as its own chunk — better context, better vectors, better retrieval.
Recursive character splitting is the most practical chunking strategy for real documents — it respects natural boundaries like paragraphs and sentences, falls back gracefully, and uses overlap to preserve cross-boundary context.
Sliding window chunking ignores paragraph and sentence boundaries entirely. Instead it moves a fixed-size window forward by a configurable stride — creating dense, overlapping chunks that preserve context across every split.
Every chunking strategy so far splits by size. Semantic chunking splits by meaning — grouping sentences that discuss the same topic into one chunk, regardless of character or word count.
Basic RAG works for demos. Production RAG needs more — query expansion to handle ambiguous inputs, hybrid search for keyword precision, re-ranking to filter noise, and feedback loops to improve over time.