Structured Output

The Solution: Taming Free Text into Structured Data

Structured output is a way of constraining an LLM's response so it follows a specific schema — a JSON object, a typed set of function parameters, or any validated data structure — instead of producing free-form prose. Think of it as giving the model a form to fill out rather than a blank page. A normal prompt might come back as "Sure! The customer's name is Anna and her order total was about $42." — perfectly readable for a human, but a nightmare for your code. Structured output forces that same answer into something like { "name": "Anna", "total": 42.0 }, every single time.

How it works

Under the hood there are two main mechanisms. JSON mode tells the model that its entire reply must be a single valid JSON document, which removes the "helpful" chatter around the data. Function calling (also called tool calling) goes a step further: you describe a function and its parameters as a schema, and the API constrains generation so the model can only emit arguments that match the field names and types you defined. Many providers implement this with constrained decoding — at each step the model is only allowed to pick tokens that keep the output syntactically valid against the schema, so a malformed response becomes structurally impossible rather than merely unlikely.

When to use it — and the catch

Reach for structured output whenever an LLM's answer feeds directly into other code: extraction pipelines, API responses, database writes, agent tool calls, or config generation. The big payoff is that you can drop the fragile regex and string-parsing and trust the shape of the data. The crucial pitfall: valid JSON is not the same as correct data. A schema guarantees the shape (a field called total that is a number) but never the meaning — the model can still hallucinate a wrong number or invert a boolean. For example, ask for an invoice as { "amount": number, "currency": string, "paid": boolean } and you might get perfectly valid JSON where amount is off by a digit or paid is flipped. Always validate values with a tool like Pydantic or Zod, not just the schema, and on failure feed the error back to the model for one or two retries.

Where Is This Used?

• API Response Formatting: Guaranteed JSON responses that parsers can consume without error handling for format issues
• Data Extraction Pipelines: Pulling structured fields (name, date, amount) from unstructured documents at scale
• Form Filling Automation: Converting free-text intake forms or emails into validated database records
• Configuration Generation: Turning natural language feature descriptions into typed config objects or infrastructure-as-code
• Common Pitfall: Semantically Wrong Values: Even with JSON mode, LLMs may produce valid JSON with wrong values (hallucinated dates, inverted booleans) — always validate values, not just schema shape, with Pydantic or Zod

Try It Yourself!

Use the interactive demo below to see the difference between free-text and structured output, and build your own schemas to extract data reliably.