Structured outputs: having AI fill in a schema
How a JSON schema turns free text into precise fields, what the feature guarantees, and the habits for checking the values.
On this page
In 30 seconds
- Where it happens
- In tools that ask an AI to sort free text into fields, for example to pull the name of an event, its date and its participants out of a sentence[1].
- What it guarantees
- A response that conforms to the schema[2]. Microsoft[1] contrasts this feature with the older JSON mode, which guaranteed valid JSON but could not ensure that the supplied schema was respected.
What it is
An email or a meeting report is free text. To pull precise fields out of it, an application can require the AI to follow a JSON schema. Microsoft[1] presents structured outputs as a feature that makes the model follow a JSON Schema definition supplied in the API call, and lists the extraction of structured data among its uses[1].
A response schema specifies the names of the fields and the type of data expected for each one[2]. Google[2] writes that with a schema, the model's response always follows the defined schema[2]. This article separates two questions: the shape of the response, and the accuracy of its content.
Straight answers
What is a structured output?
It is an AI response that follows a schema supplied with the request. Microsoft[1] distinguishes it from the older JSON mode, which guaranteed valid JSON but could not ensure that the schema was respected. Google[2] states that JSON mode without a schema is only a strong nudge and does not give 100% valid JSON. To get 100% valid JSON, the request must contain both a response schema and the application/json type[2].
Does a respected schema mean accurate values?
No. NIST[3] explains that content errors stem from the design of generative models: they produce outputs that approximate the statistical distribution of their training data, which can yield outputs that are accurate as well as inaccurate[3]. OWASP[5] describes hallucination as content that seems accurate but is made up[5].
What does the model do when the information is missing from the text?
According to Google[2], fields are optional by default: the model may fill them in or skip them. They can be declared required to force it to supply a value. If the context of the request is insufficient, the model answers mostly from the data it was trained on[2]. With Microsoft[1], all fields must be declared required, and an optional field is emulated with a type that also accepts the value null.
Can the output be executed as is?
No. OWASP[4] names this flaw "Improper Output Handling": insufficient validation of model outputs before they are passed to other components. Because the generated content can be steered by the prompt, OWASP compares this to giving users indirect access to additional functionality[4]. Its examples include an output passed directly into a command shell (the exec or eval functions), which leads to remote code execution, and SQL queries that are generated and run without parameterization, which lead to SQL injection[4].
When should a person review the output?
OWASP[5] proposes human oversight and fact-checking, especially for critical or sensitive information, with reviewers trained not to rely blindly on the AI[5]. NIST[3] gives the example of healthcare: a made-up summary of patient records could lead doctors to make a wrong diagnosis or recommend a wrong treatment[3]. It adds that the risk is particularly important to monitor in applications where the AI feeds high-consequence decisions[3].
How it works
Azure Databricks[6] describes the feature as generating responses in a defined JSON format, through a response format field in the chat request. It recommends it for extracting data from a large number of documents (for example, classifying product reviews as negative, positive or neutral), for batch processing that requires a given format, and for turning unstructured data into structured data[6]. Its example schema extracts the title, authors, abstract and keywords from a piece of research text[6].
Vendors do not support all of JSON Schema, only a subset. With Microsoft[1], a schema can have up to 100 object properties and five levels of nesting. The keywords pattern, minLength, maxLength, minimum and maximum are not supported[1]. Azure Databricks[6] sets the maximum at 64 keys, says it does not enforce length or size constraints on objects and arrays, and notes that a deeply nested schema gives lower-quality generation: it advises flattening it if possible[6]. It also specifies that the techniques used to improve quality change the number of tokens consumed, and therefore the billing[6].
With Google[2], the model's response uses the name and description of each field in the schema. Google recommends clear field names and a description that defines the role of each one[2]. It advises against copying the schema into the prompt: output quality may suffer[2]. It also notes that using structured output on tuned Gemini models may reduce model quality[2].
Research is working on the same problem. A study published on arXiv in February 2025[7] built a benchmark of about 40,000 different JSON schemas and finds that the most recent LLMs still struggle to produce a valid JSON string[7]. Another, from May 2025[8], measures schema accuracy and content fidelity separately[8].
What you do
These steps are for whoever sets up an extraction or uses its result.
- Plan for the empty value. For any field whose information may be missing from the text, use the option described by Microsoft[1]: a type that accepts the value null. A required field forces a value[2].
- Name and describe each field. Google[2] recommends unambiguous names and a description field for the role of each field.
- Validate the result. If the schema cannot be defined in advance, Google[2] recommends a client-side JSON validator with a retry on failure. OWASP[5] recommends automatically validating key outputs, especially in high-stakes environments[5]. With the databricks-claude-opus-5 model, a request may return a content filter stop reason when the model refuses the request, and the output may then not respect the schema: Azure Databricks[6] says to check this stop reason before parsing the output.
- Execute nothing without a check. OWASP[4] advises treating the model like any other user (a zero-trust approach) and validating its responses before passing them to the application's functions. It also recommends parameterized queries for all database operations that involve a model output[4].
- Cross-check against reliable sources. OWASP[5] encourages users to compare outputs with trusted external sources.
- Have a person review it when the stakes are high, as in the review question above[5].
Pitfalls
A required box pushes toward a plausible value. Google[2] states that a field declared required forces the model to supply a value, and that, lacking context, it answers mostly from its training data[2]. NIST[3] notes that these errors are presented with confidence: the risk arises when users believe false content because of the confident tone of the response[3].
A justification is not proof. NIST[3] points out that models sometimes provide logical steps to explain an answer even when that answer is wrong, which can lead people to trust them too much[3]. An "explanation" field filled in by the model therefore proves nothing.
A valid shape and faithful content are two separate measures. In the May 2025 study[8], a Mistral-7B model fine-tuned with constrained decoding reaches a schema accuracy of 99.5% and a content similarity of 94.0%[8].
Overconfidence makes the error worse. OWASP[5] defines overreliance on AI as excessive trust in generated content without verifying its accuracy, and notes that users may then build incorrect data into critical decisions or processes[5].
A well-formed output can still be dangerous. OWASP[4] gives the scenario of a user asking, through a chat-type feature, for an SQL query that deletes all the tables in a database: if the query produced by the model is not examined, all the tables are deleted[4].
Sources
- Microsoft, “How to use structured outputs with Azure OpenAI in Microsoft Foundry Models - Microsoft Foundry”. https://learn.microsoft.com/en-us/azure/foundry/openai/how-to/structured-outputs (accessed 14 November 2026)
- Google, “Structured output | Gemini Enterprise Agent Platform | Google Cloud Documentation”. https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/control-generated-output (accessed 14 November 2026)
- NIST, “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile”. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf (accessed 14 November 2026)
- OWASP, “LLM05:2025 Improper Output Handling”, 10 April 2024. https://genai.owasp.org/llmrisk/llm052025-improper-output-handling/ (accessed 14 November 2026)
- OWASP, “LLM09:2025 Misinformation”, 10 April 2024. https://genai.owasp.org/llmrisk/llm092025-misinformation/ (accessed 14 November 2026)
- Microsoft, “Structured outputs on Azure Databricks - Azure Databricks”. https://learn.microsoft.com/en-us/azure/databricks/machine-learning/model-serving/structured-outputs (accessed 14 November 2026)
- arXiv, “Learning to Generate Structured Output with Schema Reinforcement Learning”, 26 February 2025. https://arxiv.org/abs/2502.18878 (accessed 14 November 2026)
- arXiv, “SLOT: Structuring the Output of Large Language Models”, 6 May 2025. https://arxiv.org/abs/2505.04016 (accessed 14 November 2026)
Cite this article
Brain, “Structured outputs: having AI fill in a schema”, Cyberpedia, 2026. https:/
BibTeX
@misc{brain_cp515_en_2026,
author = {Brain},
title = {Structured outputs: having AI fill in a schema},
year = {2026},
howpublished = {Cyberpedia},
url = {https://startbrain.io/cyberpedia/en/articles/ai-structured-outputs-json-schema/},
note = {CC BY 4.0}
}