How do I get reliable JSON output from an LLM?

Use the API's built-in JSON mode (OpenAI's response_format, Anthropic's tool_use), provide a JSON Schema in the prompt, always say 'Return ONLY valid JSON', and implement client-side validation with a retry mechanism for malformed responses.

What is the difference between JSON Mode and Function Calling?

JSON Mode guarantees valid JSON output but doesn't enforce a specific schema. Function Calling (tool_use) lets you define exact schemas with required fields and types, giving you both valid JSON and guaranteed structure. Use Function Calling when you need specific fields.

Can LLMs output XML, YAML, or CSV reliably?

LLMs handle JSON most reliably since it dominates training data. XML and YAML work well with clear examples. CSV is less reliable for complex data. For best results, always specify the exact format, provide an example, and validate the output programmatically.

How do I handle parsing errors in structured output?

Implement a retry strategy: catch JSON parse errors, send the malformed output back to the LLM with 'Fix this JSON', and retry 1-2 times. Also use try-catch blocks, validate against your schema, and log failures for monitoring.

TechniqueJSON

Structured Output

JSON, XML, YAML

The Problem: AI often returns free-form text, but your code needs structured data like JSON or specific formats. How can we get consistent, parseable output?

The Solution: Fill Out the Form

Structured Output prompting specifies the exact format AI should use for its response. It's like giving someone a form to fill out instead of asking them to write a free-form letter. Essential for Prompt Chaining where each step needs parseable data, and few-shot examples help the model learn the exact format.

Think of it like filling out a structured form:

1. Define schema: Specify fields: name, email, department
2. Provide example: Show the exact JSON structure expected
3. Request data: "Extract contact info from this email"
4. Get structured result: Clean JSON ready to parse

Where Is This Used?

API Responses: Generating JSON for web applications
Data Extraction: Pulling structured info from unstructured text
Form Generation: Creating structured records from descriptions
Code Generation: Specific function signatures or class structures

Fun Fact: Modern APIs like OpenAI and Anthropic now support "function calling" which guarantees valid JSON output! This eliminates parsing errors and makes AI much more reliable for production use.

Try It Yourself!

Use the interactive example below to see how specifying output format leads to consistent, machine-readable responses.

Structured Output

Get data in a predictable format for programmatic processing

Popular output formats:

JSON

YAML

XML

Unstructured Output

The review is positive overall. The customer liked the fast delivery (5/5) but mentioned the packaging could be better (3/5). They would recommend this product. Main pros: quality and price. Main cons: packaging.

• Hard to parse programmatically
• Format may vary
• Requires NLP to extract

Structured Output (JSON)

Prompt Template

Analyze the following text and return the result ONLY in JSON format.

Text: "{TEXT}"

Use the following structure:
{
  "field1": "type and description",
  "field2": "type and description"
}

Important: return ONLY valid JSON without additional text.

API Examples

OpenAI JSON Mode

response = client.chat.completions.create(
    model="gpt-4",
    response_format={ "type": "json_object" },
    messages=[
        {"role": "system", "content": "Return JSON only"},
        {"role": "user", "content": "Analyze this review..."}
    ]
)

Anthropic Tool Use

response = client.messages.create(
    model="claude-3-opus",
    tools=[{
        "name": "extract_data",
        "input_schema": {
            "type": "object",
            "properties": {
                "sentiment": {"enum": ["positive", "negative"]},
                "score": {"type": "number"}
            }
        }
    }],
    tool_choice={"type": "tool", "name": "extract_data"}
)

Best Practices

• Always specify "return ONLY JSON without additional text"
• Use JSON Schema for client-side validation
• Provide an example of expected structure in the prompt
• Use response_format or tool_use for guaranteed JSON
• Handle parsing errors — model may not follow instructions

Method Comparison

Method	Reliability	Flexibility	Complexity
Prompt Instructions	⭐⭐	⭐⭐⭐	Low
JSON Mode	⭐⭐⭐	⭐⭐	Low
Function Calling	⭐⭐⭐	⭐⭐⭐	Medium
Structured Outputs API	⭐⭐⭐⭐	⭐⭐⭐⭐	Medium

Try it yourself

Interactive demo of this technique

Technique Comparison

Demo Mode

Pre-recorded responses

TaskBeginnerCoding

Extract contact information from text and return it as JSON

Without technique

Prompt

Extract contact information from this text: "Hi! My name is Alex Johnson, I work at DataFlow Inc. My email is a.johnson@dataflow.com, phone +1 (415) 555-0123. Office at 42 Main Street."

Response

Contact information:

Name: Alex Johnson
Company: DataFlow Inc.
Email: a.johnson@dataflow.com
Phone: +1 (415) 555-0123
Address: 42 Main Street

Tokens:65/52

Time:380ms

Quality:

With Structured Output

With technique

Prompt

Extract contact information from text and return ONLY valid JSON without explanations. JSON schema: ```json { "name": "string", "company": "string", "email": "string", "phone": "string", "address": "string | null" } ``` Text: "Hi! My name is Alex Johnson, I work at DataFlow Inc. My email is a.johnson@dataflow.com, phone +1 (415) 555-0123. Office at 42 Main Street." JSON:

Response

{
  "name": "Alex Johnson",
  "company": "DataFlow Inc.",
  "email": "a.johnson@dataflow.com",
  "phone": "+1 (415) 555-0123",
  "address": "42 Main Street"
}

👁️Exact JSON schema with field types is specified

🧠Instruction "ONLY valid JSON" excludes free-form text

🔍Result can be directly parsed via JSON.parse()

✅Structured output is ready for machine processing

Tokens:115/58

Time:350ms

Quality:

Why this works

Without specifying a schema, the model returns a dashed list — readable for humans but not machine-parsable. A JSON schema + "only JSON" instruction produce machine-readable output that can be used directly in code.

1 / 2

Practice Challenges

Create a free account to solve challenges

3 AI-verified challenges for this lesson

Related lessons:Rag Information Extraction

This lesson is part of a structured LLM course.

My Learning Path

TechniqueJSON

Structured Output

JSON, XML, YAML

The Problem: AI often returns free-form text, but your code needs structured data like JSON or specific formats. How can we get consistent, parseable output?

The Solution: Fill Out the Form

Think of it like filling out a structured form:

1. Define schema: Specify fields: name, email, department
2. Provide example: Show the exact JSON structure expected
3. Request data: "Extract contact info from this email"
4. Get structured result: Clean JSON ready to parse

Where Is This Used?

API Responses: Generating JSON for web applications
Data Extraction: Pulling structured info from unstructured text
Form Generation: Creating structured records from descriptions
Code Generation: Specific function signatures or class structures

Try It Yourself!

Use the interactive example below to see how specifying output format leads to consistent, machine-readable responses.

Structured Output

Get data in a predictable format for programmatic processing

Popular output formats:

JSON

YAML

XML

Unstructured Output

• Hard to parse programmatically
• Format may vary
• Requires NLP to extract

Structured Output (JSON)

Prompt Template

Analyze the following text and return the result ONLY in JSON format.

Text: "{TEXT}"

Use the following structure:
{
  "field1": "type and description",
  "field2": "type and description"
}

Important: return ONLY valid JSON without additional text.

API Examples

OpenAI JSON Mode

response = client.chat.completions.create(
    model="gpt-4",
    response_format={ "type": "json_object" },
    messages=[
        {"role": "system", "content": "Return JSON only"},
        {"role": "user", "content": "Analyze this review..."}
    ]
)

Anthropic Tool Use

response = client.messages.create(
    model="claude-3-opus",
    tools=[{
        "name": "extract_data",
        "input_schema": {
            "type": "object",
            "properties": {
                "sentiment": {"enum": ["positive", "negative"]},
                "score": {"type": "number"}
            }
        }
    }],
    tool_choice={"type": "tool", "name": "extract_data"}
)

Best Practices

• Always specify "return ONLY JSON without additional text"
• Use JSON Schema for client-side validation
• Provide an example of expected structure in the prompt
• Use response_format or tool_use for guaranteed JSON
• Handle parsing errors — model may not follow instructions

Method Comparison

Method	Reliability	Flexibility	Complexity
Prompt Instructions	⭐⭐	⭐⭐⭐	Low
JSON Mode	⭐⭐⭐	⭐⭐	Low
Function Calling	⭐⭐⭐	⭐⭐⭐	Medium
Structured Outputs API	⭐⭐⭐⭐	⭐⭐⭐⭐	Medium

Try it yourself

Interactive demo of this technique

Technique Comparison

Demo Mode

Pre-recorded responses

TaskBeginnerCoding

Extract contact information from text and return it as JSON

Without technique

Prompt

Extract contact information from this text: "Hi! My name is Alex Johnson, I work at DataFlow Inc. My email is a.johnson@dataflow.com, phone +1 (415) 555-0123. Office at 42 Main Street."

Response

Contact information:

Name: Alex Johnson
Company: DataFlow Inc.
Email: a.johnson@dataflow.com
Phone: +1 (415) 555-0123
Address: 42 Main Street

Tokens:65/52

Time:380ms

Quality:

With Structured Output

With technique

Prompt

Response

{
  "name": "Alex Johnson",
  "company": "DataFlow Inc.",
  "email": "a.johnson@dataflow.com",
  "phone": "+1 (415) 555-0123",
  "address": "42 Main Street"
}

👁️Exact JSON schema with field types is specified

🧠Instruction "ONLY valid JSON" excludes free-form text

🔍Result can be directly parsed via JSON.parse()

✅Structured output is ready for machine processing

Tokens:115/58

Time:350ms

Quality:

Why this works

1 / 2

Practice Challenges

Create a free account to solve challenges

3 AI-verified challenges for this lesson

Related lessons:Rag Information Extraction

This lesson is part of a structured LLM course.

My Learning Path