Structured Outputs - Requesty Docs

Requesty makes every supported model speak structured JSON — from simple json_object mode to strict, schema-enforced json_schema mode. One API, consistent behavior, regardless of the underlying provider.

Get your API key in the Requesty Console.

Overview

Mode	Guarantee	Provider support
`json_object`	Model returns valid JSON (no schema enforcement)	All models
`json_schema`	Model output conforms exactly to your JSON Schema	OpenAI, Anthropic, Google, and more — check `supports_json_schema` in List Models

Use json_schema whenever you need type-safe, parseable output. It eliminates the need for retry loops and manual validation — the model is constrained at the decoding level to only produce tokens that satisfy your schema.

For reusable structured outputs, save json_object or json_schema as the prompt’s Response Format in the Prompt Library. Requests using that prompt_id inherit the response format automatically, and the prompt-level setting overrides any caller-provided response_format.

JSON Schema (Recommended)

JSON Schema mode gives you guaranteed structured output. You define the exact shape of the response, and the model is constrained to produce only valid output conforming to that schema.

Chat Completions API

Python (OpenAI SDK)
Python (raw schema)
TypeScript
cURL

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

# Define your schema as a Pydantic model
class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

# Use the SDK's built-in parsing — it sends the schema
# and parses the response in one step
completion = client.beta.chat.completions.parse(
    model="openai/gpt-4.1",
    messages=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ],
    response_format=CalendarEvent,
)

event = completion.choices[0].message.parsed
print(event.name)          # "Science Fair"
print(event.date)          # "Friday"
print(event.participants)  # ["Alice", "Bob"]

import json
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

# Pass the JSON Schema directly in response_format
completion = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-5",
    messages=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "CalendarEvent",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "date": {"type": "string"},
                    "participants": {
                        "type": "array",
                        "items": {"type": "string"}
                    }
                },
                "required": ["name", "date", "participants"],
                "additionalProperties": False
            }
        }
    },
)

event = json.loads(completion.choices[0].message.content)
print(event)
# {"name": "Science Fair", "date": "Friday", "participants": ["Alice", "Bob"]}

import OpenAI from "openai";
import { zodResponseFormat } from "openai/helpers/zod";
import { z } from "zod";

const client = new OpenAI({
  apiKey: "YOUR_REQUESTY_API_KEY",
  baseURL: "https://router.requesty.ai/v1",
});

const CalendarEvent = z.object({
  name: z.string(),
  date: z.string(),
  participants: z.array(z.string()),
});

const completion = await client.beta.chat.completions.parse({
  model: "openai/gpt-4.1",
  messages: [
    { role: "system", content: "Extract the event information." },
    { role: "user", content: "Alice and Bob are going to a science fair on Friday." },
  ],
  response_format: zodResponseFormat(CalendarEvent, "CalendarEvent"),
});

const event = completion.choices[0].message.parsed;
console.log(event);
// { name: "Science Fair", date: "Friday", participants: ["Alice", "Bob"] }

curl https://router.requesty.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_REQUESTY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4.1",
    "messages": [
      {"role": "system", "content": "Extract the event information."},
      {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "CalendarEvent",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "date": {"type": "string"},
            "participants": {
              "type": "array",
              "items": {"type": "string"}
            }
          },
          "required": ["name", "date", "participants"],
          "additionalProperties": false
        }
      }
    }
  }'

Responses API

Python (OpenAI SDK)
Python (raw schema)
TypeScript
cURL

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]

response = client.responses.create(
    model="openai/gpt-4.1",
    instructions="Extract the event information.",
    input="Alice and Bob are going to a science fair on Friday.",
    text={
        "format": {
            "type": "json_schema",
            "name": "CalendarEvent",
            "strict": True,
            "schema": CalendarEvent.model_json_schema(),
        }
    },
)

event = CalendarEvent.model_validate_json(response.output_text)
print(event.name)          # "Science Fair"
print(event.participants)  # ["Alice", "Bob"]

import json
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

response = client.responses.create(
    model="anthropic/claude-sonnet-4-5",
    instructions="Extract the event information.",
    input="Alice and Bob are going to a science fair on Friday.",
    text={
        "format": {
            "type": "json_schema",
            "name": "CalendarEvent",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "date": {"type": "string"},
                    "participants": {
                        "type": "array",
                        "items": {"type": "string"}
                    }
                },
                "required": ["name", "date", "participants"],
                "additionalProperties": False
            }
        }
    },
)

event = json.loads(response.output_text)
print(event)
# {"name": "Science Fair", "date": "Friday", "participants": ["Alice", "Bob"]}

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: "YOUR_REQUESTY_API_KEY",
  baseURL: "https://router.requesty.ai/v1",
});

const response = await client.responses.create({
  model: "openai/gpt-4.1",
  instructions: "Extract the event information.",
  input: "Alice and Bob are going to a science fair on Friday.",
  text: {
    format: {
      type: "json_schema",
      name: "CalendarEvent",
      strict: true,
      schema: {
        type: "object",
        properties: {
          name: { type: "string" },
          date: { type: "string" },
          participants: { type: "array", items: { type: "string" } },
        },
        required: ["name", "date", "participants"],
        additionalProperties: false,
      },
    },
  },
});

const event = JSON.parse(response.output_text);
console.log(event);
// { name: "Science Fair", date: "Friday", participants: ["Alice", "Bob"] }

curl https://router.requesty.ai/v1/responses \
  -H "Authorization: Bearer YOUR_REQUESTY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4-5",
    "instructions": "Extract the event information.",
    "input": "Alice and Bob are going to a science fair on Friday.",
    "text": {
      "format": {
        "type": "json_schema",
        "name": "CalendarEvent",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "date": {"type": "string"},
            "participants": {
              "type": "array",
              "items": {"type": "string"}
            }
          },
          "required": ["name", "date", "participants"],
          "additionalProperties": false
        }
      }
    }
  }'

Schema Requirements

When using json_schema mode with strict: true, your schema must follow these rules:

All fields must be required

Every property in your schema must be listed in the required array. Optional fields should use a union type with null instead.

{
  "type": "object",
  "properties": {
    "name": { "type": "string" },
    "nickname": { "type": ["string", "null"] }
  },
  "required": ["name", "nickname"],
  "additionalProperties": false
}

additionalProperties must be false

Set "additionalProperties": false on every object in your schema. This tells the model exactly which keys to produce.

Supported types

string, number, integer, boolean, null, array, object, and unions via anyOf. Enums are supported via the enum keyword.

Nested objects

Nested objects must also have required and additionalProperties: false:

{
  "type": "object",
  "properties": {
    "address": {
      "type": "object",
      "properties": {
        "street": { "type": "string" },
        "city": { "type": "string" }
      },
      "required": ["street", "city"],
      "additionalProperties": false
    }
  },
  "required": ["address"],
  "additionalProperties": false
}

JSON Object Mode

For models that don’t support json_schema, or when you just need valid JSON without strict schema enforcement, use json_object mode. The model will return valid JSON, but you’re responsible for instructing it on the desired structure via your prompt.

Chat Completions
Responses API

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

completion = client.chat.completions.create(
    model="openai/gpt-4.1",
    messages=[
        {
            "role": "system",
            "content": (
                "Extract entities from the text. Return JSON with this structure: "
                '{"people": [...], "places": [...], "dates": [...]}'
            ),
        },
        {
            "role": "user",
            "content": "John met Sarah in Paris on January 5th.",
        },
    ],
    response_format={"type": "json_object"},
)

import json
data = json.loads(completion.choices[0].message.content)
print(data)
# {"people": ["John", "Sarah"], "places": ["Paris"], "dates": ["January 5th"]}

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

response = client.responses.create(
    model="openai/gpt-4.1",
    instructions=(
        "Extract entities from the text. Return JSON with this structure: "
        '{"people": [...], "places": [...], "dates": [...]}'
    ),
    input="John met Sarah in Paris on January 5th.",
    text={"format": {"type": "json_object"}},
)

import json
data = json.loads(response.output_text)
print(data)
# {"people": ["John", "Sarah"], "places": ["Paris"], "dates": ["January 5th"]}

With json_object mode, you must instruct the model to produce JSON in your system or user message. The model is only guaranteed to return valid JSON — not any particular structure. Use json_schema mode for guaranteed structure.

Real-World Examples

Multi-step extraction pipeline

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

class SentimentAnalysis(BaseModel):
    sentiment: str  # "positive", "negative", "neutral"
    confidence: float
    key_phrases: list[str]
    summary: str

completion = client.beta.chat.completions.parse(
    model="anthropic/claude-sonnet-4-5",
    messages=[
        {
            "role": "system",
            "content": "Analyze the sentiment of the given text. Be precise with confidence scores.",
        },
        {
            "role": "user",
            "content": "The new product launch exceeded all expectations. Customer feedback has been overwhelmingly positive, though some users reported minor issues with the onboarding flow.",
        },
    ],
    response_format=SentimentAnalysis,
)

result = completion.choices[0].message.parsed
print(f"Sentiment: {result.sentiment} ({result.confidence:.0%})")
print(f"Key phrases: {result.key_phrases}")

Enums and complex types

from openai import OpenAI
from pydantic import BaseModel
from enum import Enum

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

class Priority(str, Enum):
    high = "high"
    medium = "medium"
    low = "low"

class Task(BaseModel):
    title: str
    priority: Priority
    due_date: str | None
    tags: list[str]

class TaskList(BaseModel):
    tasks: list[Task]

completion = client.beta.chat.completions.parse(
    model="vertex/gemini-2.5-flash",
    messages=[
        {"role": "system", "content": "Extract tasks from the meeting notes."},
        {
            "role": "user",
            "content": "We need to fix the login bug ASAP. Also, update the docs by next Friday. Low priority: clean up the test suite.",
        },
    ],
    response_format=TaskList,
)

for task in completion.choices[0].message.parsed.tasks:
    print(f"[{task.priority.value}] {task.title}")

Chain of thought with structured output

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

class MathSolution(BaseModel):
    steps: list[str]
    final_answer: float

completion = client.beta.chat.completions.parse(
    model="openai/gpt-4.1",
    messages=[
        {
            "role": "system",
            "content": "Solve the math problem step by step. Show your work in the steps array.",
        },
        {
            "role": "user",
            "content": "A store has a 25% off sale. If a jacket originally costs $80, and there's an additional 10% member discount applied after the sale price, what's the final price?",
        },
    ],
    response_format=MathSolution,
)

solution = completion.choices[0].message.parsed
for i, step in enumerate(solution.steps, 1):
    print(f"Step {i}: {step}")
print(f"Answer: ${solution.final_answer}")

Framework Integrations

Requesty’s structured outputs work seamlessly with popular frameworks. Since Requesty is OpenAI-compatible, just point the framework’s OpenAI client at https://router.requesty.ai/v1.

from langchain_openai import ChatOpenAI
from langchain_core.messages import SystemMessage, HumanMessage
from pydantic import BaseModel

class Answer(BaseModel):
    championships: int
    summary: str

llm = ChatOpenAI(
    model="anthropic/claude-sonnet-4-5",
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

# .with_structured_output() sends json_schema under the hood
structured_llm = llm.with_structured_output(Answer)

result = structured_llm.invoke([
    SystemMessage(content="Answer questions about NBA players."),
    HumanMessage(content="How many championships has LeBron James won?"),
])
print(f"{result.championships} championships")

import instructor
from openai import OpenAI
from pydantic import BaseModel

client = instructor.from_openai(
    OpenAI(
        api_key="YOUR_REQUESTY_API_KEY",
        base_url="https://router.requesty.ai/v1",
    )
)

class UserInfo(BaseModel):
    name: str
    age: int
    email: str

user = client.chat.completions.create(
    model="openai/gpt-4.1",
    messages=[
        {"role": "user", "content": "John Doe is 30 years old. His email is [email protected]"},
    ],
    response_model=UserInfo,
)
print(f"{user.name}, {user.age}, {user.email}")

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider
from pydantic import BaseModel

class CityInfo(BaseModel):
    name: str
    country: str
    population: int
    known_for: list[str]

provider = OpenAIProvider(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)
model = OpenAIChatModel("anthropic/claude-sonnet-4-5", provider=provider)

agent = Agent(
    model,
    system_prompt="Provide information about cities.",
    output_type=CityInfo,
)

result = agent.run_sync("Tell me about Tokyo")
print(f"{result.output.name}, {result.output.country}")
print(f"Known for: {', '.join(result.output.known_for)}")

import litellm
import json

response = litellm.completion(
    model="openai/anthropic/claude-sonnet-4-5",
    messages=[
        {"role": "system", "content": "Extract the key entities."},
        {"role": "user", "content": "Apple Inc. was founded by Steve Jobs in Cupertino."},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "Entities",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "companies": {"type": "array", "items": {"type": "string"}},
                    "people": {"type": "array", "items": {"type": "string"}},
                    "locations": {"type": "array", "items": {"type": "string"}}
                },
                "required": ["companies", "people", "locations"],
                "additionalProperties": False
            }
        }
    },
    api_key="YOUR_REQUESTY_API_KEY",
    api_base="https://router.requesty.ai/v1",
)

print(json.loads(response.choices[0].message.content))

import dspy

lm = dspy.LM(
    model="openai/anthropic/claude-sonnet-4-5",
    api_key="YOUR_REQUESTY_API_KEY",
    api_base="https://router.requesty.ai/v1",
)

class ExtractInfo(dspy.Signature):
    """Extract structured information from text."""
    text: str = dspy.InputField()
    companies: list[str] = dspy.OutputField(desc="Company names mentioned")
    people: list[str] = dspy.OutputField(desc="People mentioned")
    locations: list[str] = dspy.OutputField(desc="Locations mentioned")

with dspy.context(lm=lm):
    predict = dspy.Predict(ExtractInfo)
    result = predict(text="Sundar Pichai leads Google from their Mountain View office.")
    print(result.companies)  # ["Google"]
    print(result.people)     # ["Sundar Pichai"]

from haystack.components.generators.chat import OpenAIChatGenerator
from haystack.utils import Secret
from haystack.dataclasses import ChatMessage
import json

generator = OpenAIChatGenerator(
    model="vertex/gemini-2.5-flash",
    api_key=Secret.from_token("YOUR_REQUESTY_API_KEY"),
    api_base_url="https://router.requesty.ai/v1",
    generation_kwargs={
        "response_format": {
            "type": "json_schema",
            "json_schema": {
                "name": "Analysis",
                "strict": True,
                "schema": {
                    "type": "object",
                    "properties": {
                        "topic": {"type": "string"},
                        "sentiment": {"type": "string"},
                        "confidence": {"type": "number"}
                    },
                    "required": ["topic", "sentiment", "confidence"],
                    "additionalProperties": False
                }
            }
        }
    },
)

generator.warm_up()
result = generator.run(messages=[
    ChatMessage.from_system("Analyze the topic and sentiment of the text."),
    ChatMessage.from_user("I absolutely love the new Python 3.13 release!"),
])

analysis = json.loads(result["replies"][0].text)
print(f"Topic: {analysis['topic']}, Sentiment: {analysis['sentiment']}")

Streaming with Structured Outputs

Both json_object and json_schema work with streaming. Tokens are delivered incrementally, and you can parse the complete JSON once the stream finishes.

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

class Recipe(BaseModel):
    name: str
    ingredients: list[str]
    steps: list[str]

# Stream with json_schema — tokens arrive incrementally
stream = client.chat.completions.create(
    model="openai/gpt-4.1",
    messages=[
        {"role": "system", "content": "Generate a recipe."},
        {"role": "user", "content": "Quick pasta carbonara"},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "Recipe",
            "strict": True,
            "schema": Recipe.model_json_schema(),
        }
    },
    stream=True,
)

content = ""
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        content += delta
        print(delta, end="", flush=True)

# Parse the complete response
recipe = Recipe.model_validate_json(content)
print(f"\n\nRecipe: {recipe.name}")
print(f"Steps: {len(recipe.steps)}")

See the streaming documentation for more details.

Finding Compatible Models

Use the List Models endpoint to discover which models support structured outputs:

curl https://router.requesty.ai/v1/models \
  -H "Authorization: Bearer YOUR_REQUESTY_API_KEY" | \
  jq '.data[] | select(.supports_json_schema == true) | .id'

The response includes:

supports_json_schema: true — model supports strict json_schema mode
supports_json_object: true — model supports json_object mode (most models)

Comparison: Chat Completions vs Responses API

Both APIs support the same structured output modes. The difference is in how you pass the format:

	Chat Completions	Responses API
Parameter	`response_format` (top-level)	`text.format` (nested)
Schema location	`response_format.json_schema.schema`	`text.format.schema`
Name field	`response_format.json_schema.name`	`text.format.name`
Strict field	`response_format.json_schema.strict`	`text.format.strict`
SDK helper	`client.beta.chat.completions.parse()`	Manual schema passing

response_format={
    "type": "json_schema",
    "json_schema": {
        "name": "MySchema",
        "strict": True,
        "schema": { ... }
    }
}

text={
    "format": {
        "type": "json_schema",
        "name": "MySchema",
        "strict": True,
        "schema": { ... }
    }
}

Tips & Best Practices

Keep schemas simple

Simpler schemas produce more reliable outputs. Flatten deeply nested structures where possible and limit arrays to 10–20 items in your schema descriptions.

Use descriptions for guidance

Add description fields to your schema properties to guide the model on what to extract:

{
  "type": "object",
  "properties": {
    "confidence": {
      "type": "number",
      "description": "A confidence score between 0.0 and 1.0"
    }
  }
}

Handle optional fields with null unions

Since strict mode requires all properties in required, use null unions for optional values:

class UserProfile(BaseModel):
    name: str
    email: str
    phone: str | None  # Will be null if not found

Combine with system prompts

The schema enforces structure, but your system prompt controls the content quality. Be specific about what you want extracted and how.

Use fallback policies for reliability

Combine structured outputs with fallback policies to automatically retry with a different model if the primary one fails:

completion = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-5",
    messages=[...],
    response_format=MySchema,
    extra_body={
        "requesty": {
            "policy": "your-fallback-policy-id"
        }
    },
)

Error Handling

from openai import OpenAI
from pydantic import BaseModel, ValidationError

client = OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

class ExtractedData(BaseModel):
    items: list[str]
    count: int

try:
    completion = client.beta.chat.completions.parse(
        model="openai/gpt-4.1",
        messages=[
            {"role": "user", "content": "List three fruits."},
        ],
        response_format=ExtractedData,
    )
    data = completion.choices[0].message.parsed
    print(data)

except ValidationError as e:
    # Schema validation failed (unlikely with json_schema mode)
    print(f"Validation error: {e}")

except Exception as e:
    # API or network error
    print(f"API error: {e}")

Common issues:

400 error with json_schema → Your schema likely has an unsupported construct. Check the schema requirements above.
Model returns empty content → Ensure your prompt gives the model enough context to fill all required fields.
Model not supporting json_schema → Check supports_json_schema via List Models or fall back to json_object mode.

​Overview

​JSON Schema (Recommended)

​Chat Completions API

​Responses API

​Schema Requirements

​JSON Object Mode

​Real-World Examples

​Multi-step extraction pipeline

​Enums and complex types

​Chain of thought with structured output

​Framework Integrations

​Streaming with Structured Outputs

​Finding Compatible Models

​Comparison: Chat Completions vs Responses API

​Tips & Best Practices

​Error Handling

Overview

JSON Schema (Recommended)

Chat Completions API

Responses API

Schema Requirements

JSON Object Mode

Real-World Examples

Multi-step extraction pipeline

Enums and complex types

Chain of thought with structured output

Framework Integrations

Streaming with Structured Outputs

Finding Compatible Models

Comparison: Chat Completions vs Responses API

Tips & Best Practices

Error Handling