Chain-of-Thought — Step-by-Step AI Reasoning
Think step by step
The Problem: AI often jumps straight to answers, making mistakes on complex problems that require step-by-step reasoning. How can we make it think more carefully?
The Solution: Think Out Loud
Chain of Thought (CoT) prompting asks the model to spell out its reasoning step by step before committing to a final answer. Just like a math teacher who tells students to "show their work," you are forcing the model to lay out intermediate steps instead of guessing. Unlike zero-shot prompting — where the model jumps straight to an answer — CoT produces a visible reasoning trace, and that trace is what makes the difference on multi-step problems.
Why does it work? A language model generates one token at a time, and each token it writes becomes part of the context for the next one. When you ask it to reason out loud, the early steps ("first I add these two numbers...") act as scratch paper that the later steps can build on. In effect, the model spends more compute "thinking" before answering, and intermediate results stop it from skipping a step. There are two common flavors: zero-shot CoT, where you simply append a phrase like "Let's think step by step," and few-shot CoT (see few-shot), where you show a couple of fully worked examples so the model imitates the reasoning style.
A worked example. Ask plainly: "A shop had 100 apples, sold 23 in the morning and 31 in the afternoon — how many are left?" and a model may blurt out a wrong number. Add "think step by step" and it writes: "Morning sales: 23. Afternoon sales: 31. Total sold: 23 + 31 = 54. Remaining: 100 − 54 = 46." The visible arithmetic catches errors the snap answer would hide. Tradeoffs: CoT costs more tokens and latency, and the explanation can sound convincing while still being wrong — a plausible-looking chain is not a proof, so do not treat it as a guarantee against hallucinations. For the hardest problems, combine it with Self-Consistency — sample several independent reasoning paths (with a higher temperature) and take the majority answer.
Think of it like solving a math problem out loud:
- 1. Read the problem: "I need to find how much money is left..."
- 2. Break it down: "First, let me calculate the total spent..."
- 3. Work through steps: "$20 + $15 + $8 = $43"
- 4. Reach conclusion: "$100 - $43 = $57 remaining"
Where Is This Used?
- Math Problems: Multi-step calculations, word problems
- Logical Reasoning: Puzzles, deduction tasks
- Complex Questions: Questions requiring multiple facts to combine
- Debugging Code: Walking through code execution step by step
Fun Fact: The phrase "Let's think step by step" is so powerful that it alone can boost accuracy by 10-40% on reasoning tasks! This was discovered by Google researchers in 2022 and is called "zero-shot CoT."
Try It Yourself!
Use the interactive example below to compare regular prompts vs Chain of Thought prompts. Watch how asking for step-by-step reasoning changes the quality of answers.
🔗 Chain of Thought (CoT) — a technique where the model "thinks out loud" step by step. This helps solve complex problems more accurately!
Mary had 5 apples. She gave 2 to Pete, then bought 3 more. How many apples does Mary have?
6 apples
How to write CoT prompts:
1. Zero-shot CoT (simplest way)
Just add the magic phrase at the end of your prompt, and the model will start reasoning step by step
Solve this problem: Mary had 5 apples. She gave 2 to Pete, then bought 3 more. How many apples does Mary have? Let's think step by step.
2. Explicit CoT (with structure)
Provide explicit reasoning structure with numbered steps
Task: A train travels at 60 km/h. How long will it take to travel 180 km if it stops for 15 minutes every 60 km? Think step by step: 1. First determine pure travel time 2. Then calculate number of stops 3. Calculate total stop time 4. Add travel time and stop time Give the final answer.
3. CoT with output format
Specify format to separate reasoning from final answer
Task: A class has 30 students. 40% are boys. How many boys are present if 3 are absent? Reason like this: <thinking> [your step-by-step reasoning here] </thinking> <answer> [final brief answer here] </answer>
CoT works because it forces the model to "show its work" — intermediate steps help avoid errors in complex reasoning. Especially useful for math, logic, and multi-step problems.
Frequently asked questions
What is Chain-of-Thought (CoT) prompting?
Chain-of-Thought prompting is a technique where you ask the LLM to show its reasoning step-by-step before giving the final answer. This dramatically improves accuracy on complex tasks like math, logic, and multi-step reasoning.
How much does CoT improve accuracy?
Research shows CoT can improve accuracy by 30-50% on complex reasoning tasks. On the GSM8K math benchmark, GPT-3 accuracy jumped from 17% to 58% using CoT. The improvement is most significant on tasks requiring multi-step reasoning.
When should I use Chain-of-Thought?
Use CoT for: math problems, logical reasoning, multi-step tasks, decision-making with multiple factors, debugging code, or any task where intermediate reasoning steps are valuable. Skip it for simple, straightforward tasks where it adds unnecessary overhead.
What's the difference between Zero-Shot CoT and Few-Shot CoT?
Zero-Shot CoT uses simple prompts like 'Let's think step by step' without examples. Few-Shot CoT provides 2-5 examples of step-by-step reasoning before the actual question. Few-Shot typically performs better but requires more tokens.
Try it yourself
Interactive demo of this technique
Mary had 5 apples. She gave 2 to Pete, then bought 3 more. How many apples does Mary have?
6 apples.
Let's solve this step by step:
- Starting amount: Mary had 5 apples
- First action: She gave 2 to Pete → 5 - 2 = 3 apples
- Second action: Bought 3 more → 3 + 3 = 6 apples
Answer: Mary has 6 apples.
Even for simple problems, CoT shows the reasoning process, which helps verify correctness and understand the logic.
Create a free account to solve challenges
4 AI-verified challenges for this lesson
This lesson is part of a structured LLM course.
My Learning Path