Structured outputs and tool calling

Generated by Claude through GigAI 0.1.11.10 on 2026-10-11.

Concept: Structured outputs · also covers Tool use / tool calling · Step 15 of 34 in AI forward deployed engineer · version 1

Practice this

On this page

Structured outputs make a model return data that conforms to a schema, so downstream code can consume it without parsing free text. Tool calling extends the same idea to actions: the model emits a typed request to call a function, the application executes it and returns the result, and the model continues. These two mechanisms are what connect a language model to real systems.

Read more

How it is done in practice

Teams define each output and tool input once as a typed model in application code and generate the JSON Schema from it, so the contract, the validator and the documentation cannot drift. Extraction and classification steps use strict structured outputs; anything touching customer systems goes through tools with narrow scopes, read tools separated from write tools, and writes gated by approval or policy. The executor layer handles timeouts, retries, idempotency keys and logging, and returns errors to the model as tool results. Before go-live the team runs an evaluation set that checks not only final answers but also which tools were called with which arguments, since a correct answer reached through a wrong call is a latent incident.

Technologies

JSON SchemaPydanticZodAnthropic tool useOpenAI function callingOpenAI Structured OutputsInstructorPydantic AIVercel AI SDKModel Context Protocol

Diagram

sequenceDiagram
  participant U as User
  participant A as Application
  participant M as Model
  participant S as Customer system
  U->>A: request
  A->>M: messages plus tool definitions
  M-->>A: stop reason tool use, call id, name, arguments
  A->>A: validate arguments, check permissions
  A->>S: execute with timeout and idempotency key
  S-->>A: result or error
  A->>M: tool result tied to call id
  M-->>A: final answer or next tool call
  A-->>U: response
sequenceDiagram
  participant U as User
  participant A as Application
  participant M as Model
  participant S as Customer system
  U->>A: request
  A->>M: messages plus tool definitions
  M-->>A: stop reason tool use, call id, name, arguments
  A->>A: validate arguments, check permissions
  A->>S: execute with timeout and idempotency key
  S-->>A: result or error
  A->>M: tool result tied to call id
  M-->>A: final answer or next tool call
  A-->>U: response

Code

json
{
  "name": "get_invoice",
  "description": "Fetch one invoice by id. Use when the user asks about a specific invoice amount, status or due date. Do not use for listing invoices.",
  "input_schema": {
    "type": "object",
    "properties": {
      "invoice_id": {
        "type": "string",
        "description": "Invoice id, for example INV-20417"
      },
      "include_line_items": {
        "type": "boolean",
        "description": "Return line items as well as totals"
      }
    },
    "required": ["invoice_id", "include_line_items"],
    "additionalProperties": false
  }
}

Subtopics

JSON Schema as the contract language

The parts that matter for models

Both structured outputs and tool parameters are described with JSON Schema. The working vocabulary is small: type, properties, required, enum for closed sets, items for arrays, nested objects, anyOf for alternatives, and additionalProperties set to false to forbid unexpected keys. Descriptions on fields are read by the model and act as inline instructions. Strict modes accept only a subset of the specification; constraints such as string patterns, numeric ranges or deep recursion may be unsupported or limited depending on the provider, so check the supported subset and enforce the remaining rules in application code after parsing.

Typed outputs with validation and bounded retry

Beyond the basic pattern

Validation catches shape errors, not meaning errors: a schema-valid object can still hold a wrong value. Add semantic checks after parsing, such as cross-field consistency, ids that must exist in a system of record, and figures that must appear in the source. When a retry is needed, send the validation error text back to the model instead of repeating the same request, and cap attempts so a persistent failure becomes a handled error. Design schemas with an explicit way to express absence, for example a nullable field or a status enum, so the model is not forced to invent a value.

Strict mode and constrained decoding

How schema adherence is guaranteed

There are three levels of structure. Asking for JSON in the prompt is best effort. JSON mode guarantees syntactically valid JSON but not a particular shape. Strict structured outputs go further: the provider compiles the schema into a grammar and, at each generation step, masks out tokens that would violate it, so the result always parses and matches the schema. This requires a supported schema subset, typically with every property listed as required and additionalProperties false, with optional values modeled as a union with null.

Limits to plan for

A guarantee on shape does not cover every outcome. Output cut off by the token limit can be incomplete, and a model may return a refusal instead of the object, which the API reports separately; handle both before using the data. The first request with a new schema can be slower while the grammar is prepared. Field order in the schema affects quality, since generation is sequential: place a reasoning or evidence field before the verdict field that depends on it.

The tool calling loop

Protocol

The application sends tool definitions, each with a name, a description and an input schema. When the model decides a tool is needed, the response stops with a tool-use stop reason and contains one or more call blocks, each with an id, the tool name and JSON arguments. The application, not the model, executes the call, then sends a new request that includes the earlier assistant turn plus a tool result tied to the call id. The model reads the result and either calls another tool or answers. A model may request several calls in one turn, which can be executed in parallel.

Controls

A tool choice setting lets the application allow any tool, force a specific tool, or disable tools for a turn; forcing a single tool is a common way to get structured output. Client tools run in application code, while some providers also offer server tools they execute themselves. Cap the number of loop iterations, apply a timeout to every execution, and return failures as tool results marked as errors so the model can recover instead of the loop crashing.

Designing tools a model can use well

Interface design for a model reader

The description is the main factor in whether a tool is chosen and called correctly. State what the tool does, when to use it and when not to, what each parameter means with its format, and what comes back. Prefer a small number of task-shaped tools over a one-to-one mirror of an internal API, use enums and required fields to shrink the space of bad calls, and give tools distinct names so they are not confused. Keep results compact and relevant, since they are added to the window, and make error messages actionable, for example naming the invalid parameter and the accepted values.

Side effects, idempotency and approval

Beyond read and write separation

Arguments produced by a model are untrusted input: validate them against the schema and against the caller's own permissions, since the model must never hold more authority than the user it acts for. Because a loop can retry or repeat a call, give every mutating tool an idempotency key so a duplicate request has no second effect. When calls run in parallel, make sure writes that depend on each other are sequenced by the application. Record each call, its arguments and its result with the request trace so any action can be audited later.

Sources

Related lessons

Practice this
Prompt engineering and context engineeringRetrieval-augmented generation with vector databases