Fine-tuning vs Prompting

The Solution: Tutor vs Textbook

Fine-tuning means taking a pretrained language model and continuing its training on a smaller, focused dataset so it adapts to your task, domain, or style. The model already knows grammar, facts, and reasoning from its original pretraining; fine-tuning nudges its weights toward your patterns — your tone of voice, your output format, your jargon. Imagine you need to learn a new subject. Before fine-tuning, try improving your results with better prompt engineering. If that's not enough, you have two main paths:

• Textbook (prompting): Read the chapter before each test. Quick to start, but you have to re-read every time.
• Tutor (fine-tuning): Study with a teacher until you truly understand. Takes time upfront, but knowledge stays with you.

Full fine-tuning vs LoRA / PEFT

Full fine-tuning updates every weight in the model. It gives the most control but is expensive: a 7-billion-parameter model needs tens of gigabytes of GPU memory just to hold the optimizer state, and you end up storing a complete copy of the model for each task. PEFT (Parameter-Efficient Fine-Tuning) avoids this by freezing the original weights and training only a tiny set of new parameters. The most popular PEFT method is LoRA (Low-Rank Adaptation): it injects small trainable matrices into each layer, so you adjust well under 1% of the parameters. QLoRA goes further, loading the frozen base model in 4-bit quantization so a single consumer GPU can fine-tune a large model. The resulting LoRA adapter is just a few megabytes — you can swap adapters per task without duplicating the whole model.

When to fine-tune vs prompt or RAG

Reach for fine-tuning when you need a consistent style, format, or behavior that prompting can't reliably enforce — for example, always emitting strict JSON, classifying support tickets, or matching a brand voice across thousands of calls. It also lowers per-request cost, because the instructions are baked into the weights instead of repeated in every prompt. But fine-tuning is the wrong tool for injecting fresh facts. To teach a model knowledge that changes often, use Retrieval Augmented Generation (RAG): store your documents in a searchable vector database, find the passages relevant to each question using embeddings, and paste them into the prompt automatically. Updating RAG means editing a document; updating a fine-tuned fact means retraining. A practical rule: start with prompting, add RAG for knowledge, and fine-tune only for behavior you can't prompt your way to.

Concretely, suppose you want a model that turns customer emails into structured tickets with fields like priority, category, and summary. You collect 500–1,000 example email-to-ticket pairs, format them as prompt/completion data, and fine-tune with LoRA. The job runs in an hour or two on one GPU, and afterward the model emits clean JSON without a long instruction prompt — cheaper and more reliable at inference time than prompting alone.

Types of Fine-tuning

• Full fine-tuning: retrain all weights. Maximum quality, but expensive (needs 8+ GPUs)
• LoRA / QLoRA: train only small adapters. Cheap and fast, 90% of the quality
• RLHF: train using human feedback. For teaching preferences, like being helpful vs harmful
• Instruction tuning: teach to follow instructions. Transforms a base model into a chatbot

Try It Yourself!

Answer a few questions about your use case, and see which approach is recommended for you!