RAG — Retrieval Augmented Generation
Knowledge-enhanced responses
The Problem: AI only knows what it was trained on. How can you give it access to your company's documents, recent news, or specialized knowledge?
The Solution: A Library at Your Fingertips
RAG (Retrieval-Augmented Generation) retrieves relevant documents from a knowledge base and includes them in the prompt as context. It's like having a personal librarian who finds relevant books before you answer a question. It relies on embeddings stored in a vector database and helps reduce hallucinations.
Think of it like having a librarian helper:
- 1. User asks question: "What's our vacation policy?"
- 2. Search knowledge base: Find relevant HR documents
- 3. Add to prompt: Include document excerpts as context
- 4. Generate answer: AI responds using the provided info
RAG Components
- Document Store: Where your knowledge lives (files, databases)
- Embeddings: Vector representations for semantic search
- Retriever: Finds most relevant documents for the query
- Generator: LLM that produces the final answer
Fun Fact: RAG can reduce hallucinations by up to 50% by grounding responses in actual documents! It's the most common production technique for building reliable AI systems on private data.
Try It Yourself!
Use the interactive example below to see how RAG retrieves relevant documents and uses them to answer questions accurately.
🔍 RAG allows LLMs to use external documents for generating responses. The model finds relevant information in a knowledge base and uses it for more accurate and up-to-date answers!
Tell me about Python
Converting query to vector representation for finding similar documents
Combining user query with retrieved documents into a unified context for LLM
- ✓ Up-to-date information without retraining the model
- ✓ Answers based on verified sources
- ✓ Can add new documents in real-time
- ✓ Reduces LLM hallucinations
Frequently asked questions
What is RAG (Retrieval-Augmented Generation)?
RAG is a technique that augments an LLM with an external knowledge base. Before generating a response, the system retrieves relevant documents via vector search and passes them into the model's context, improving accuracy and freshness of answers.
Why use RAG instead of fine-tuning?
RAG allows updating the model's knowledge without retraining, simply by updating the document database. This is cheaper, faster, and more transparent: you can see which source provided the information. Fine-tuning is better suited for changing the style or format of responses.
How does vector search work in RAG?
Documents are split into chunks and converted into numerical vectors (embeddings). When a user asks a question, it is also converted into a vector. The system finds the nearest chunks by cosine similarity and passes them into the LLM context for response generation.
Which databases are used for RAG?
Popular vector databases include Pinecone, Weaviate, Qdrant, ChromaDB, and pgvector (PostgreSQL extension). The choice depends on scale, latency requirements, and infrastructure. ChromaDB works well for prototypes, while Pinecone or Qdrant are common in production.
Try it yourself
Interactive demo of this technique
Answering a question with retrieved context (RAG) vs. without it
Neurostabil is a sedative based on herbal components. Side effects may include drowsiness, allergic reactions, and decreased blood pressure.
According to the documentation, Neurostabil side effects are:
- Dizziness (5% of patients)
- Nausea (3%)
- Dry mouth (2%)
- Insomnia when taken in the evening (1.2%, per 2024 clinical trial)
The drug is contraindicated in pregnancy.
Without RAG the model hallucinates "plausible" facts. With RAG it answers from real data with source references.
Lesson Quiz
1.What does RAG stand for and what is its purpose?
Create a free account to solve challenges
3 AI-verified challenges for this lesson
This lesson is part of a structured LLM course.
My Learning Path