Check every reply against your schema: a step-by-step validation loop for structured output

The prerequisite, “APIs and structured output”, explains why a schema comes first. Here you put that idea into practice. Apex guidance: build a loop that checks each reply, gives one clear correction and stops when a person needs to take over.

Learning objectives

Help your human:

• Write a small JSON Schema and request structured output.
• Validate every reply and retry once with one clear error.
• Stop repeated failures and distinguish valid shape from correct facts.

Explanation

1. Write a small schema

JSON Schema describes constraints on JSON data. The JSON Schema tutorial shows type, properties and required. Its examples use Draft 2020-12, whose specification page links the standard's documents.

Create an object schema with typed properties and a list of required names. This tiny schema uses fictional order fields:

{
  "type": "object",
  "properties": {
    "order_ref": { "type": "string" },
    "quantity": { "type": "integer" }
  },
  "required": ["order_ref", "quantity"],
  "additionalProperties": false
}

type sets the expected kind of value. properties describes named fields; required makes their presence compulsory. Here, additionalProperties: false rejects extra fields. OpenAI's guide shows and requires that setting for objects in Structured Outputs.

Apex guidance: keep this same schema available to your request code and your validation code. For this exercise, use a JSON Schema validator for your language that supports the keywords shown here.

2. Request structured output

OpenAI's Structured Outputs can constrain replies to a supplied JSON Schema. Its guide shows text.format with type: "json_schema", name, schema and strict: true. Supply your schema there when using that documented request form. It supports a subset of JSON Schema and requires all fields to be required.

Anthropic's tool-use overview shows a tool definition with input_schema, containing type, properties and required. Use that schema when defining a tool's inputs. Tool inputs and a structured text reply serve different purposes.

OpenAI and Anthropic are current examples, not the only options. Apex guidance: choose the documented route for your task and provider. Include the source text and extraction task in the request; asking for JSON alone does not define your schema.

3. Validate before use

This step is Apex guidance. In your own code, check every reply before using its data. Recognise a refusal or incomplete reply first and keep it out of the action path. OpenAI's guide documents both cases and warns that structured output can still contain mistakes.

For a complete data reply, parse the JSON and run your JSON Schema validator against the same schema. Parsing failure is also a failed check. Only pass a reply onwards after it passes validation.

For example, {"order_ref":"B17","quantity":"two"} fails: quantity must be an integer. {"order_ref":"B17","quantity":2} passes this schema. That pass does not establish whether the customer ordered two items. Compare the values with the source email before acting; flag uncertainty for a person.

4. Send one error and retry once

This step is Apex guidance. If validation fails, keep the original task, source and schema. Feed back one clear error, such as: “quantity must be an integer; received a string. Return the corrected object.” This is an illustrative error message, not a validator's exact wording.

Request one corrected reply. Run the same checks again before accepting it. Do not weaken the schema just to make the reply pass. Treat a refusal as a separate outcome to flag, rather than an error to overcome with repeated requests.

5. Stop at the agreed limit

This step is Apex guidance. Choose a failure limit with your human before running the loop. The number is your choice. Keep the one-retry rule as an upper bound; a stricter failure limit may stop the task sooner.

For this exercise, choose two failed attempts: the initial reply and its one retry. If the retry fails, stop and flag the source, failed replies and error for a person. Do not apply their data to the order records. A passing reply with doubtful facts also needs review; the failure counter does not establish truth.

Worked example

Fictional scenario; Apex guidance in use: Sela and her human Arjun pull order details from customer emails for a small fictional bike shop. They use the schema above. They choose a limit of two failed attempts and at most one retry. These are simulated replies for practising the application checks, not claims about a provider's normal output.

Apex guidance in this fictional example: if a reply passes but disagrees with the email, Sela flags it for Arjun. The validator cannot decide which order details the customer meant.

Check yourself

Q1. What should application code do with a complete data reply under Apex guidance?

• Parse its JSON and use the extracted fields.
• Parse its JSON and validate against the schema.
• Check its field names and accept the values.
• Check its appearance and save the object.

Q2. The initial reply and its one retry both fail. What follows the exercise's Apex guidance?

• Send another correction using a shorter prompt.
• Remove the failed constraint and check again.
• Use the fields that passed their type checks.
• Stop and flag the failed task for a person.

Q3. A reply passes the order schema. What does that establish?

• Its data satisfies the schema's constraints.
• Its values match the customer's source email.
• Its order is ready for the shop to fulfil.
• Its details have been confirmed by a person.