Structured Outputs: Getting Reliable JSON From AI

Structured JSON From AI

Structured Outputs refers to techniques that constrain an AI model to emit data in a predictable format, most often JSON that matches a schema. The practical goal is simple: downstream code should parse the output without guesswork, even when the model is uncertain about the content. For health information use cases, this matters because a single malformed field can break a pipeline that summarizes symptoms, extracts medication names, or maps free text to coded categories.

In practice, you combine three layers: a schema that defines fields and types, a generation constraint that steers the model toward that schema, and a validation step that rejects outputs that do not pass. If you have ever seen a response that looks like JSON but includes trailing commas or unescaped quotes, you have already met the failure mode Structured Outputs tries to reduce.

Main Problems And Pain Points

People often treat JSON as a formatting choice rather than a contract. Models can produce valid-looking text that still fails strict parsing, especially when the prompt mixes instructions with long context or when the model tries to be helpful by adding commentary outside the JSON block.

A common dependency is the schema itself. If the schema is underspecified, the model fills gaps with plausible guesses, and your validator has to decide whether guesses are acceptable. If the schema is overspecified, the model may refuse to comply or may omit fields it cannot confidently infer. I have seen teams set a “required” field list too aggressively, then spend days debugging why the model returns partial objects.

Another pain point is type drift. A field you expect to be a number may arrive as a string like ""12"". A date may arrive in multiple formats, such as 2026-09-01 versus 09/01/2026. Even when the JSON parses, type drift can break downstream logic that expects consistent types.

Finally, toolchains add their own failure modes. Some systems wrap the JSON in extra text, some stream tokens and cut off mid-object, and some retry automatically without preserving the original context. Those behaviors can produce outputs that are syntactically incomplete, which validation catches but does not fix by itself.

Solutions And Advice

Design A Tight Schema

Start by writing a schema that mirrors what your downstream code actually needs. Use explicit types for each field, and decide which fields are required versus optional. For health-related extraction, keep the schema narrow at first: for example, store medication name, dosage amount, unit, and frequency as separate fields rather than one free-text blob.

When you define enumerations, use them sparingly. If you must map to codes, include a fallback like ""unknown"" for cases where the model cannot map confidently. In a small internal test I ran with a schema that required 10 fields, the model often returned fewer fields when the input text lacked detail; after reducing required fields to 4 and making the rest optional, parse success improved noticeably.

Use constraints that match real-world variability. Dates should accept a single canonical format in your output, even if the input uses multiple formats. If you need canonicalization, add a rule: output ISO 8601 dates only, and if the date is missing, return null.

Constrain Generation And Parse

Structured output features in modern AI APIs typically accept a schema and instruct the model to produce output that conforms. Even with these features, you should still parse with a strict JSON parser and reject anything that fails. If your system supports it, request a response format that returns only the JSON object, not surrounding text.

In a practical pipeline, treat the model output as untrusted input. Parse the JSON, then validate it against the schema using a validator such as JSON Schema (draft 2020-12 is common). If validation fails, do not “best-effort” patch the output unless you can justify the patch rules and log them for audit.

Versioning matters. If you change the schema, record the schema version in your logs and keep a mapping from old versions to new ones. I once saw a team upgrade their schema from draft-07 to 2020-12 and forget that one validator option changed how it handled additional properties.

Validate Semantics, Not Just Syntax

Syntax validation catches malformed JSON, but health workflows also need semantic checks. For example, dosage amount should be a finite number, units should match a known set, and frequency should follow a pattern such as ""once daily"" or a structured representation. If you store symptom onset dates, reject outputs where the onset date is in the future relative to the document date.

These checks can be simple. A few guardrails catch many issues: ensure required fields are present, ensure numeric fields are within reasonable ranges, and ensure strings do not contain unexpected control characters. When a check fails, capture the raw model output and the validation errors so you can improve prompts or schema rather than guessing.

Some teams add a second pass that asks the model to repair its own JSON. That approach can work, but it also risks compounding errors if the model “fixes” the wrong thing. A safer pattern is to repair only formatting issues, like missing quotes, while keeping the original extracted values unchanged.

Handle Failures With Retries And Fallbacks

Retries should be deliberate. If validation fails, retry with a shorter prompt that includes the schema again and the validation error summary. Avoid resending the entire long context if it increases the chance of truncation; keep the retry input focused on the original text and the schema.

Set a retry budget. For example, allow 1 retry for formatting errors and 0 retries for semantic violations that indicate a misunderstanding, like mapping “ibuprofen” to “insulin.” If you do retry, log the failure category so you can measure which errors dominate.

When you cannot get valid structured output, fall back to a controlled mode: store the raw text and mark the structured fields as null. That keeps the pipeline running without silently fabricating data.

Case Examples

Medication Extraction From Notes

An anonymized scenario: a clinic note includes free text such as “Started metformin 500 mg twice daily after meals.” The goal is JSON with fields for medication_name, dosage_amount, dosage_unit, frequency, and timing_notes. The first model attempt returns valid JSON but sets dosage_amount as the string ""500"" and frequency as ""twice daily"" without normalization.

The validator rejects type drift, and the retry asks for dosage_amount as a number and frequency in a defined set. The second attempt returns dosage_amount: 500, frequency: ""twice daily"", and timing_notes: ""after meals"". The pipeline then stores the structured record and keeps the original sentence for audit.

Symptom Timeline Normalization

An anonymized scenario: a patient message says “Fever started last Thursday, cough since Monday.” The schema expects onset_date as ISO 8601 and symptom_duration_days as an integer. The model outputs onset_date as “Thursday” and leaves symptom_duration_days blank, which passes JSON parsing but fails semantic checks.

The system uses the document’s reference date (for example, the message received date) to compute a canonical onset_date when the input provides relative dates. If the reference date is missing, the system returns onset_date: null and symptom_duration_days: null, while preserving the original relative phrase in a separate field.

Comparison Table And Checklist

Checklist for decision support:

1. Define the schema fields based on downstream needs, not on what the model can guess.
2. Mark only truly required fields as required; make the rest optional or nullable.
3. Use strict JSON parsing and reject any output that fails.
4. Validate types and formats (numbers, ISO dates, enumerations).
5. Add semantic checks that match the domain (ranges, date logic, unit consistency).
6. Log raw outputs and validation error categories for each failure.
7. Retry only when the failure category suggests a formatting issue, not a misunderstanding.
8. When retries fail, store raw text plus null structured fields rather than fabricating.

Common Mistakes

One mistake is trusting “almost JSON.” A response that includes a trailing comma or wraps JSON in markdown fences will fail strict parsers. Some teams add a regex to extract the first {...} block; that can hide truncation and produce partial objects that still parse.

Another mistake is mixing presentation instructions with extraction instructions. If the prompt asks for “a short explanation” and “return JSON,” the model may include explanation text that breaks strict parsing. Keep the output contract separate from any human-readable commentary.

Teams also overfit to a single test set. If you only test with clean inputs, you miss failures caused by typos, abbreviations, or missing context. I have seen a schema that worked on discharge summaries fail on patient messages because the messages used relative dates and informal dosage wording.

Finally, people forget to measure. Track parse success rate, validation failure rate by category, and retry counts. Without those metrics, you cannot tell whether schema changes help or whether the model version changed behavior; for example, a model update around 2026-03 could alter formatting tendencies even when prompts stay the same.

FAQ

What Is Structured Outputs?

It is a set of prompting and API features that constrain model output to a defined structure, usually a JSON object that matches a schema.

Why Does JSON Parse Fail Even With Constraints?

Constraints reduce formatting errors, but streaming truncation, extra text, or invalid escaping can still produce output that strict parsers reject.

Should I Validate With JSON Schema?

Yes for reliability: JSON Schema validation catches missing fields, wrong types, and invalid formats, which prevents silent downstream errors.

How Do I Handle Missing Medical Details?

Use nullable or optional fields and return null when the input does not contain enough information; store the original phrase for audit instead of guessing.

Can I Ask The Model To Repair Its Output?

You can, but restrict repair to formatting and use validation feedback; semantic corrections can drift, so log and review repair outcomes.

Author's Insight

Reliable JSON from AI comes from treating the model as a generator of candidate data, not as a trusted database. Schema design, strict parsing, and validation together form a practical safety net: parsing catches syntax, schema checks catch structure, and semantic rules catch domain mismatches. Tools like JSON Schema validators and logging of validation error categories make failures diagnosable rather than mysterious.

When you test, include messy inputs that resemble real health text: abbreviations, relative dates, and partial sentences. I also recommend tracking outcomes per schema version, because small schema changes can shift model behavior even when prompts stay constant.

Key Takeaways

• Structured Outputs reduces formatting failures, but strict parsing and validation still matter.
• Design schemas around downstream needs, with optional or nullable fields for missing medical details.
• Validate both syntax and semantics, then log raw outputs and error categories.
• Retry only for fixable formatting errors; fall back to raw text plus null structured fields when validation keeps failing.