Structured output with response_format: JSON you can parse every time

The short version

• Put a JSON Schema in response_format with "type": "json_schema" and the model's content is JSON that matches it: no code fences, no preamble.
• In the schema, list every field in required, set additionalProperties: false and strict: true. Allow null instead of making a field optional.
• Parsing can still fail in two cases: the output hit max_tokens (finish_reason is length), or the model ignores json_schema. Check the first on every call and test the second once per model.
• If you only need data back, use structured output. Tool calling is for running your own code.

Asking for JSON in the prompt works most of the time. The rest of the time the reply comes wrapped in a ```json fence, opens with a sentence, or ends with a trailing comma, and the parser you write to clean that up becomes the most fragile code in the project. response_format removes the need for it.

A first request

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["FASTMETAL_API_KEY"],
    base_url="https://api.fastmetal.ai/v1",
)

resp = client.chat.completions.create(
    model="gemini-flash-lite-free",
    max_tokens=300,
    messages=[
        {"role": "user", "content": "Extract the details: Taro Tanaka, 42, from Shinjuku."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "person",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"},
                    "city": {"type": "string"},
                },
                "required": ["name", "age", "city"],
                "additionalProperties": False,
            },
        },
    },
)
print(resp.choices[0].message.content)

We ran this request on gemini-flash-lite-free in August 2026, with the sentence written in Japanese. The content that came back was exactly this, and went straight into json.loads():

{
  "name": "田中太郎",
  "age": 42,
  "city": "新宿"
}

That is Taro Tanaka, 42, Shinjuku, with no fence and no sentence around it.

gemini-flash-lite-free costs $0, so every example here runs without credit. It is shared and rate-limited, and Google may use free-tier prompts for training, so use made-up data like the above and check the model's data handling in the catalog before sending anything real.

The same call with curl

curl https://api.fastmetal.ai/v1/chat/completions \
  -H "Authorization: Bearer $FASTMETAL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gemini-flash-lite-free",
    "max_tokens": 300,
    "messages": [
      {"role": "user", "content": "Extract the details: Taro Tanaka, 42, from Shinjuku."}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
            "city": {"type": "string"}
          },
          "required": ["name", "age", "city"],
          "additionalProperties": false
        }
      }
    }
  }'

Three schema settings that matter

Put every field in required

A field left out of required is one the model may drop entirely. If a value can be missing, keep the field in required and give it the type ["string", "null"]. The model then returns null, and your code never has to test whether the key exists.

Set additionalProperties to false

Without it, the model can add fields the schema never mentioned.

Set strict to true

It asks the model to follow the schema exactly.

Every property can also carry a description, and the model reads it. {"type": "string", "description": "Prefecture name only, without the city"} keeps the extraction consistent. Treat the schema as instructions as well as validation.

A practical example: classifying reviews

An enum and an array turn a classification task into a fixed shape:

import json

schema = {
    "type": "object",
    "properties": {
        "sentiment": {
            "type": "string",
            "enum": ["positive", "negative", "neutral"],
        },
        "topics": {
            "type": "array",
            "items": {"type": "string"},
            "description": "Topics the review mentions, at most 3",
        },
        "summary": {"type": "string", "description": "Summary in 10 words or fewer"},
    },
    "required": ["sentiment", "topics", "summary"],
    "additionalProperties": False,
}

resp = client.chat.completions.create(
    model="gemini-flash-lite-free",
    max_tokens=300,
    messages=[
        {"role": "system", "content": "You classify product reviews."},
        {"role": "user", "content": "Shipping was fast but the box was crushed. Happy with the product itself."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {"name": "review", "strict": True, "schema": schema},
    },
)

data = json.loads(resp.choices[0].message.content)
print(data["sentiment"], data["topics"])

With enum, the label is always one of the three strings. You never see Positive one time and positive the next, so the code that branches on it stays simple.

When parsing still fails

The output hit max_tokens

If the reply reaches the token limit partway through, the JSON is never closed. The schema can't prevent that. Check finish_reason on every call:

choice = resp.choices[0]
if choice.finish_reason == "length":
    raise RuntimeError("Cut off by max_tokens; raise the limit")
data = json.loads(choice.message.content)

stop means the model finished; length means it was cut off. Schemas that return arrays are the usual cause, so state the maximum item count in the description and leave max_tokens room to spare.

The model doesn't support json_schema

Not every model supports json_schema. One that doesn't may ignore it and answer in prose. Send one test request before relying on a model. A fallback parser is cheap insurance:

import json, re

def parse_json(text: str):
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        m = re.search(r"```(?:json)?\s*(.+?)\s*```", text, re.S)
        if m:
            return json.loads(m.group(1))
        raise

This is a fallback for models without support. If your requests land in the except branch regularly, change the model or the request.

json_object compared with json_schema

response_format also accepts {"type": "json_object"}. That guarantees the reply is valid JSON and nothing about its shape, so key names can change from one call to the next, and you have to describe the keys you want in the prompt. To fix the structure, use json_schema.

Structured output or tool calling?

If all you want is JSON, use structured output. Getting JSON out through a tool call works, and it costs an extra round trip in time and tokens.

Common mistakes

• Leaving fields out of required. The key disappears; return null instead.
• Not checking finish_reason. JSON cut off at length never parses, so look there before debugging anything else.
• Omitting additionalProperties: false. Extra keys appear.
• Nesting too deep. Accuracy drops and output tokens go up. Keep it to two or three levels and split the task if you need more.
• Repeating the format in the prompt. Writing "reply in JSON" and passing a schema gives the model two sets of instructions. Let the schema define the format and use the prompt for what to extract.

FAQ

How is this different from asking for JSON in the prompt?

A prompt-only request returns fenced or prefixed JSON often enough to break parsing. With a schema in response_format, the reply is JSON that matches the schema.

Does structured output work on free models?

Yes. The examples here were run on gemini-flash-lite-free. Support varies by model, so test the model you plan to use.

Why does my JSON get cut off?

The reply hit max_tokens. Check whether finish_reason is length and raise the limit. Structured output can't prevent it.

json_object or json_schema?

json_schema when the structure must be fixed. json_object only guarantees valid JSON; key names can vary.

Try it

Create a free account, issue a key, and run the first example against gemini-flash-lite-free with your own schema. It costs nothing. Models that support structured output are in the catalog, and paid models are billed per million tokens on the pricing page.