Lesson 2Hands-on

Image Analysis

Practical applications

The Problem: Beyond just describing images, we need AI that can deeply analyze visual content — identify patterns, compare images, and provide expert insights.

The Solution: From Pixels to Structured Data

While Vision Basics covers general image understanding (describing photos, visual Q&A), Image Analysis focuses on extracting structured data from documents, charts, medical images, and technical diagrams. It's like the difference between describing a painting and reading a patient's X-ray — precision matters. Results are returned as structured output (JSON, tables) for downstream processing.

Think of it like a specialist reading an X-ray:

1. Identify document type: Is it a chart, a form, a medical scan, or a receipt? The prompt strategy differs for each
2. OCR + layout parsing: Extract text while preserving structure — columns, headers, table cells, not just raw text
3. Structured extraction: Ask for JSON output: {"patient": "...", "diagnosis": "...", "medications": [...]}
4. Validation & grounding: Mark extracted data as [VERIFIED] or [UNVERIFIED] — LLMs can hallucinate entity names from documents

Where Is This Used?

Document Processing: Extract names, dates, amounts from scanned contracts, invoices, receipts — with structured JSON output
Chart & Graph Reading: Interpret bar charts, line graphs, pie charts — extract data points and trends
Medical Report Analysis: Parse lab results, radiology reports — extract diagnosis, measurements, recommendations
Technical Diagrams: Read architecture diagrams, flowcharts, circuit schematics — describe components and connections

Fun Fact: Vision models can now spot things humans might miss! In medical imaging, AI has detected early-stage cancers that radiologists overlooked. The combination of AI + human review is often more accurate than either alone.

Try It Yourself!

Use the interactive example below to perform detailed analysis on different types of images and see the depth of AI understanding.

Prompt Quality Matters

Generic prompt

"Describe this image"

Result:

This is a medical form with patient information and test results.

Structured prompt

"Extract from this medical form: 1) Patient name 2) Date 3) All test results as JSON {test: value, unit, reference_range}"

Result:

{"patient": "Jane Doe", "date": "2025-01-15", "results": [{"test": "Glucose", "value": 95, "unit": "mg/dL", "range": "70-100"}]}

For advanced OCR techniques — table extraction, multi-page documents, and handwritten text — see Document Understanding.

Confidence Markers

Always ask the model to mark extracted data with confidence levels. This helps catch hallucinated values.

For each extracted field, mark as:
[VERIFIED] — clearly visible in the image
[UNVERIFIED] — partially visible or inferred
[NOT_FOUND] — not present in the image

Try it yourself

Interactive demo of this technique

Technique Comparison

Demo Mode

Pre-recorded responses

TaskBeginnerAnalysis

OCR and document understanding: raw text extraction vs structured analysis

Without technique

Prompt

Extract text from this receipt image. [Photo: supermarket receipt]

Response

GROCERY STORE 24 MILK 2.5% 89.90 BORODINSKY BREAD 65.00 RUSSIAN CHEESE 450G 389.00 BANANAS 1.2KG 95.88 EGGS C1 10PCS 109.00 TOTAL 748.78 CARD PAYMENT THANK YOU FOR YOUR PURCHASE

Tokens:290/65

Time:2100ms

Quality:

With Image Analysis

With technique

Prompt

Analyze the receipt photo and extract data in structured format: 1. Store: name and address (if available) 2. Items: JSON array [{name, quantity, unit, price_per_unit, total}] 3. Totals: {subtotal, tax, total, payment_method} 4. Purchase date and time 5. Verification: do item totals match the grand total? [Photo: supermarket receipt]

Response

1. Store: Grocery Store 24

2. Items:

[
  {"name": "Milk 2.5%", "quantity": 1, "unit": "pcs", "price_per_unit": 89.90, "total": 89.90},
  {"name": "Borodinsky Bread", "quantity": 1, "unit": "pcs", "price_per_unit": 65.00, "total": 65.00},
  {"name": "Russian Cheese", "quantity": 1, "unit": "450g", "price_per_unit": 389.00, "total": 389.00},
  {"name": "Bananas", "quantity": 1.2, "unit": "kg", "price_per_unit": 79.90, "total": 95.88},
  {"name": "Eggs C1", "quantity": 10, "unit": "pcs", "price_per_unit": 10.90, "total": 109.00}
]

3. Totals: {"subtotal": 748.78, "tax": 0, "total": 748.78, "payment_method": "card"}

4. Date: Not visible on the receipt.

5. Verification: 89.90 + 65.00 + 389.00 + 95.88 + 109.00 = 748.78 ✓ Matches.

👁️Raw OCR gives text as-is — no structure, no typing, no validation

🧠Structured schema (name, quantity, unit, price) turns text into data for DB/Excel

🔍Verification step (item sum = total?) catches OCR and extraction errors

✅OCR + structure + validation = reliable pipeline for document data extraction

Tokens:340/300

Time:3600ms

Quality:

Why this works

OCR without structure is just text. A prompt with JSON schema + verification (sum = total?) turns a document photo into data ready for system import.

1 / 2

Practice Challenges

Create a free account to solve challenges

4 AI-verified challenges for this lesson

Related lessons:Vision Basics Information Extraction

This lesson is part of a structured LLM course.

My Learning Path

Lesson 2Hands-on

Image Analysis

Practical applications

The Problem: Beyond just describing images, we need AI that can deeply analyze visual content — identify patterns, compare images, and provide expert insights.

The Solution: From Pixels to Structured Data

Think of it like a specialist reading an X-ray:

1. Identify document type: Is it a chart, a form, a medical scan, or a receipt? The prompt strategy differs for each
2. OCR + layout parsing: Extract text while preserving structure — columns, headers, table cells, not just raw text
3. Structured extraction: Ask for JSON output: {"patient": "...", "diagnosis": "...", "medications": [...]}
4. Validation & grounding: Mark extracted data as [VERIFIED] or [UNVERIFIED] — LLMs can hallucinate entity names from documents

Where Is This Used?

Document Processing: Extract names, dates, amounts from scanned contracts, invoices, receipts — with structured JSON output
Chart & Graph Reading: Interpret bar charts, line graphs, pie charts — extract data points and trends
Medical Report Analysis: Parse lab results, radiology reports — extract diagnosis, measurements, recommendations
Technical Diagrams: Read architecture diagrams, flowcharts, circuit schematics — describe components and connections

Try It Yourself!

Use the interactive example below to perform detailed analysis on different types of images and see the depth of AI understanding.

Prompt Quality Matters

Generic prompt

"Describe this image"

Result:

This is a medical form with patient information and test results.

Structured prompt

"Extract from this medical form: 1) Patient name 2) Date 3) All test results as JSON {test: value, unit, reference_range}"

Result:

{"patient": "Jane Doe", "date": "2025-01-15", "results": [{"test": "Glucose", "value": 95, "unit": "mg/dL", "range": "70-100"}]}

For advanced OCR techniques — table extraction, multi-page documents, and handwritten text — see Document Understanding.

Confidence Markers

Always ask the model to mark extracted data with confidence levels. This helps catch hallucinated values.

For each extracted field, mark as:
[VERIFIED] — clearly visible in the image
[UNVERIFIED] — partially visible or inferred
[NOT_FOUND] — not present in the image

Try it yourself

Interactive demo of this technique

Technique Comparison

Demo Mode

Pre-recorded responses

TaskBeginnerAnalysis

OCR and document understanding: raw text extraction vs structured analysis

Without technique

Prompt

Extract text from this receipt image. [Photo: supermarket receipt]

Response

GROCERY STORE 24 MILK 2.5% 89.90 BORODINSKY BREAD 65.00 RUSSIAN CHEESE 450G 389.00 BANANAS 1.2KG 95.88 EGGS C1 10PCS 109.00 TOTAL 748.78 CARD PAYMENT THANK YOU FOR YOUR PURCHASE

Tokens:290/65

Time:2100ms

Quality:

With Image Analysis

With technique

Prompt

Response

1. Store: Grocery Store 24

2. Items:

[
  {"name": "Milk 2.5%", "quantity": 1, "unit": "pcs", "price_per_unit": 89.90, "total": 89.90},
  {"name": "Borodinsky Bread", "quantity": 1, "unit": "pcs", "price_per_unit": 65.00, "total": 65.00},
  {"name": "Russian Cheese", "quantity": 1, "unit": "450g", "price_per_unit": 389.00, "total": 389.00},
  {"name": "Bananas", "quantity": 1.2, "unit": "kg", "price_per_unit": 79.90, "total": 95.88},
  {"name": "Eggs C1", "quantity": 10, "unit": "pcs", "price_per_unit": 10.90, "total": 109.00}
]

3. Totals: {"subtotal": 748.78, "tax": 0, "total": 748.78, "payment_method": "card"}

4. Date: Not visible on the receipt.

5. Verification: 89.90 + 65.00 + 389.00 + 95.88 + 109.00 = 748.78 ✓ Matches.

👁️Raw OCR gives text as-is — no structure, no typing, no validation

🧠Structured schema (name, quantity, unit, price) turns text into data for DB/Excel

🔍Verification step (item sum = total?) catches OCR and extraction errors

✅OCR + structure + validation = reliable pipeline for document data extraction

Tokens:340/300

Time:3600ms

Quality:

Why this works

OCR without structure is just text. A prompt with JSON schema + verification (sum = total?) turns a document photo into data ready for system import.

1 / 2

Practice Challenges

Create a free account to solve challenges

4 AI-verified challenges for this lesson

Related lessons:Vision Basics Information Extraction

This lesson is part of a structured LLM course.

My Learning Path