What is RAG (Retrieval-Augmented Generation)?

RAG is a technique that augments an LLM with an external knowledge base. Before generating a response, the system retrieves relevant documents via vector search and passes them into the model's context, improving accuracy and freshness of answers.

Why use RAG instead of fine-tuning?

RAG allows updating the model's knowledge without retraining, simply by updating the document database. This is cheaper, faster, and more transparent: you can see which source provided the information. Fine-tuning is better suited for changing the style or format of responses.

How does vector search work in RAG?

Documents are split into chunks and converted into numerical vectors (embeddings). When a user asks a question, it is also converted into a vector. The system finds the nearest chunks by cosine similarity and passes them into the LLM context for response generation.

Which databases are used for RAG?

Popular vector databases include Pinecone, Weaviate, Qdrant, ChromaDB, and pgvector (PostgreSQL extension). The choice depends on scale, latency requirements, and infrastructure. ChromaDB works well for prototypes, while Pinecone or Qdrant are common in production.

Lesson 4

RAG — Retrieval Augmented Generation

Knowledge-enhanced responses

The Problem: AI only knows what it was trained on. How can you give it access to your company's documents, recent news, or specialized knowledge?

The Solution: A Library at Your Fingertips

RAG (Retrieval-Augmented Generation) retrieves relevant documents from a knowledge base and includes them in the prompt as context. It's like having a personal librarian who finds relevant books before you answer a question. It relies on embeddings stored in a vector database and helps reduce hallucinations.

Think of it like having a librarian helper:

1. User asks question: "What's our vacation policy?"
2. Search knowledge base: Find relevant HR documents
3. Add to prompt: Include document excerpts as context
4. Generate answer: AI responds using the provided info

RAG Components

Document Store: Where your knowledge lives (files, databases)
Embeddings: Vector representations for semantic search
Retriever: Finds most relevant documents for the query
Generator: LLM that produces the final answer

Fun Fact: RAG can reduce hallucinations by up to 50% by grounding responses in actual documents! It's the most common production technique for building reliable AI systems on private data.

Try It Yourself!

Use the interactive example below to see how RAG retrieves relevant documents and uses them to answer questions accurately.

RAG — Retrieval Augmented Generation

🔍 RAG allows LLMs to use external documents for generating responses. The model finds relevant information in a knowledge base and uses it for more accurate and up-to-date answers!

Choose a query:

User Query

Tell me about Python

Query Embedding

Converting query to vector representation for finding similar documents

Retrieving Relevant Documents

Knowledge Base:

What is Python

Python is a high-level programming language created in 1991. Used for web development, data analysis, and ML.

JavaScript History

JavaScript was created in 1995 for browsers. Now used everywhere: frontend, backend (Node.js), mobile apps.

React Basics

React is a library for building user interfaces. Uses components and virtual DOM for efficient UI updates.

Machine Learning

ML is a field of AI where computers learn from data without explicit programming. Popular libraries: TensorFlow, PyTorch, scikit-learn.

Web Frameworks

Frameworks simplify development: Django and Flask for Python, Express for Node.js, Spring for Java, Rails for Ruby.

Context Augmentation

Combining user query with retrieved documents into a unified context for LLM

Response Generation

Benefits of RAG

✓ Up-to-date information without retraining the model
✓ Answers based on verified sources
✓ Can add new documents in real-time
✓ Reduces LLM hallucinations

Try it yourself

Interactive demo of this technique

Technique Comparison

Demo Mode

Pre-recorded responses

TaskBeginnerAnalysis

Answering a question with retrieved context (RAG) vs. without it

Without technique

Prompt

What are the side effects of the drug "Neurostabil"?

Response

Neurostabil is a sedative based on herbal components. Side effects may include drowsiness, allergic reactions, and decreased blood pressure.

Tokens:16/38

Time:260ms

Quality:

With Context + RAG

With technique

Prompt

Context from medical knowledge base: --- [DOC-1] Neurostabil (reg. #12345): Side effects: dizziness (5%), nausea (3%), dry mouth (2%). Contraindicated in pregnancy. Compatible with most NSAIDs. [DOC-2] Clinical trial 2024: 1.2% of patients reported insomnia when taken in the evening. --- Question: What are the side effects of the drug "Neurostabil"? Answer strictly based on the provided documents. If information is insufficient, say so.

Response

According to the documentation, Neurostabil side effects are:

Dizziness (5% of patients)
Nausea (3%)
Dry mouth (2%)
Insomnia when taken in the evening (1.2%, per 2024 clinical trial)

The drug is contraindicated in pregnancy.

👁️Without RAG the model "guesses" — produces plausible but potentially incorrect facts

👁️RAG added two relevant documents with exact data and percentages

🧠The "answer based on documents" instruction prevents hallucinations

✅In medicine accuracy is critical — RAG turns the LLM into a reliable tool

Tokens:135/62

Time:380ms

Quality:

Why this works

Without RAG the model hallucinates "plausible" facts. With RAG it answers from real data with source references.

1 / 2

Practice Challenges

Create a free account to solve challenges

3 AI-verified challenges for this lesson

Related lessons:Rag System Prompts

This lesson is part of a structured LLM course.

My Learning Path

Lesson 4

RAG — Retrieval Augmented Generation

Knowledge-enhanced responses

The Problem: AI only knows what it was trained on. How can you give it access to your company's documents, recent news, or specialized knowledge?

The Solution: A Library at Your Fingertips

Think of it like having a librarian helper:

1. User asks question: "What's our vacation policy?"
2. Search knowledge base: Find relevant HR documents
3. Add to prompt: Include document excerpts as context
4. Generate answer: AI responds using the provided info

RAG Components

Document Store: Where your knowledge lives (files, databases)
Embeddings: Vector representations for semantic search
Retriever: Finds most relevant documents for the query
Generator: LLM that produces the final answer

Fun Fact: RAG can reduce hallucinations by up to 50% by grounding responses in actual documents! It's the most common production technique for building reliable AI systems on private data.

Try It Yourself!

Use the interactive example below to see how RAG retrieves relevant documents and uses them to answer questions accurately.

RAG — Retrieval Augmented Generation

🔍 RAG allows LLMs to use external documents for generating responses. The model finds relevant information in a knowledge base and uses it for more accurate and up-to-date answers!

Choose a query:

User Query

Tell me about Python

Query Embedding

Converting query to vector representation for finding similar documents

Retrieving Relevant Documents

Knowledge Base:

What is Python

Python is a high-level programming language created in 1991. Used for web development, data analysis, and ML.

JavaScript History

JavaScript was created in 1995 for browsers. Now used everywhere: frontend, backend (Node.js), mobile apps.

React Basics

React is a library for building user interfaces. Uses components and virtual DOM for efficient UI updates.

Machine Learning

ML is a field of AI where computers learn from data without explicit programming. Popular libraries: TensorFlow, PyTorch, scikit-learn.

Web Frameworks

Frameworks simplify development: Django and Flask for Python, Express for Node.js, Spring for Java, Rails for Ruby.

Context Augmentation

Combining user query with retrieved documents into a unified context for LLM

Response Generation

Benefits of RAG

✓ Up-to-date information without retraining the model
✓ Answers based on verified sources
✓ Can add new documents in real-time
✓ Reduces LLM hallucinations

Try it yourself

Interactive demo of this technique

Technique Comparison

Demo Mode

Pre-recorded responses

TaskBeginnerAnalysis

Answering a question with retrieved context (RAG) vs. without it

Without technique

Prompt

What are the side effects of the drug "Neurostabil"?

Response

Neurostabil is a sedative based on herbal components. Side effects may include drowsiness, allergic reactions, and decreased blood pressure.

Tokens:16/38

Time:260ms

Quality:

With Context + RAG

With technique

Prompt

Response

According to the documentation, Neurostabil side effects are:

Dizziness (5% of patients)
Nausea (3%)
Dry mouth (2%)
Insomnia when taken in the evening (1.2%, per 2024 clinical trial)

The drug is contraindicated in pregnancy.

👁️Without RAG the model "guesses" — produces plausible but potentially incorrect facts

👁️RAG added two relevant documents with exact data and percentages

🧠The "answer based on documents" instruction prevents hallucinations

✅In medicine accuracy is critical — RAG turns the LLM into a reliable tool

Tokens:135/62

Time:380ms

Quality:

Why this works

Without RAG the model hallucinates "plausible" facts. With RAG it answers from real data with source references.

1 / 2

Practice Challenges

Create a free account to solve challenges

3 AI-verified challenges for this lesson

Related lessons:Rag System Prompts

This lesson is part of a structured LLM course.

My Learning Path