Auto Caching

Requesty’s auto caching automatically caches long system prompts and repeated content to reduce costs on any provider that supports prompt caching. Cache hits are billed at a fraction of the normal input token cost. Many providers (such as OpenAI, DeepSeek, and Google Gemini implicit caching) manage prompt caching automatically on their side — no flag or special request format needed. For providers that require explicit cache breakpoints (such as Anthropic), auto caching adds them for you. Auto caching works on both /v1/chat/completions and /v1/responses endpoints.

View cache analytics in the Requesty Console.

What Auto Cache Does

Auto caching is provider-level prompt caching. When enabled, the router automatically adds cache breakpoints to the largest content blocks in your request before forwarding it to the provider (for providers that require explicit breakpoints, such as Anthropic). The provider then caches those token prefixes on their side. For providers that cache automatically, no breakpoints are needed — caching just happens. This means:

You still send the full message history with every request. The payload size does not change.
The provider recognizes cached prefixes and charges reduced rates for tokens it has already seen.
Cache hits are billed at a fraction of the normal input token cost (up to 90% savings).

Auto caching is not server-side response storage. There is no way to skip sending your conversation history. See Auto Cache vs. Response IDs for details.

How Auto Cache Works

The auto_cache flag is a boolean parameter sent within the requesty field in your request payload.

Value	Behavior
`true`	Instructs the router to add cache breakpoints to the largest content blocks before forwarding to the provider
`false`	Bypasses caching for this request (useful when cache writes have extra costs)
Not provided	Falls back to default behavior based on request origin (e.g., Cline, Roo Code, Forge, Open WebUI, Hermes, and Pi default to caching)

Chat Completions API

Include the auto_cache flag within the requesty object in your request:

import openai

client = openai.OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

system_prompt = "YOUR ENTIRE KNOWLEDGEBASE"

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4-5",
    messages=[
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": "What is the capital of France?"}
    ],
    extra_body={
        "requesty": {
            "auto_cache": True
        }
    }
)
print(response.choices[0].message.content)

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: "YOUR_REQUESTY_API_KEY",
  baseURL: "https://router.requesty.ai/v1",
});

const response = await client.chat.completions.create({
  model: "anthropic/claude-sonnet-4-5",
  messages: [
    { role: "system", content: "YOUR ENTIRE KNOWLEDGEBASE" },
    { role: "user", content: "What is the capital of France?" }
  ],
  requesty: {
    auto_cache: true
  }
});

console.log(response.choices[0].message.content);

curl https://router.requesty.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_REQUESTY_API_KEY" \
  -d '{
    "model": "anthropic/claude-sonnet-4-5",
    "messages": [
      {"role": "system", "content": "YOUR ENTIRE KNOWLEDGEBASE"},
      {"role": "user", "content": "What is the capital of France?"}
    ],
    "requesty": {
      "auto_cache": true
    }
  }'

Responses API

Auto caching works identically on the /v1/responses endpoint. Include the same requesty.auto_cache flag:

import openai

client = openai.OpenAI(
    api_key="YOUR_REQUESTY_API_KEY",
    base_url="https://router.requesty.ai/v1",
)

system_prompt = "YOUR ENTIRE KNOWLEDGEBASE"

response = client.responses.create(
    model="anthropic/claude-sonnet-4-5",
    instructions=system_prompt,
    input="What is the capital of France?",
    extra_body={
        "requesty": {
            "auto_cache": True
        }
    }
)
print(response.output_text)

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: "YOUR_REQUESTY_API_KEY",
  baseURL: "https://router.requesty.ai/v1",
});

const response = await client.responses.create({
  model: "anthropic/claude-sonnet-4-5",
  instructions: "YOUR ENTIRE KNOWLEDGEBASE",
  input: "What is the capital of France?",
  requesty: {
    auto_cache: true
  }
});

console.log(response.output_text);

curl https://router.requesty.ai/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_REQUESTY_API_KEY" \
  -d '{
    "model": "anthropic/claude-sonnet-4-5",
    "instructions": "YOUR ENTIRE KNOWLEDGEBASE",
    "input": "What is the capital of France?",
    "requesty": {
      "auto_cache": true
    }
  }'

Multi-turn with Responses API

For multi-turn conversations, include the full conversation history in the input array along with the requesty.auto_cache flag. The router caches the largest content blocks so the provider charges reduced rates for the repeated prefix:

response = client.responses.create(
    model="anthropic/claude-sonnet-4-5",
    instructions="YOUR ENTIRE KNOWLEDGEBASE",
    input=[
        {"role": "user", "content": "What is the capital of France?"},
        {"role": "assistant", "content": "The capital of France is Paris."},
        {"role": "user", "content": "What about Germany?"}
    ],
    extra_body={
        "requesty": {
            "auto_cache": True
        }
    }
)

Auto Cache vs. Response IDs

OpenAI’s Responses API supports a previous_response_id parameter that lets OpenAI store conversation state server-side so you don’t have to resend the full history. This is an OpenAI-specific feature — it works when routing to OpenAI models through Requesty, because OpenAI handles the storage on their end. For non-OpenAI models, previous_response_id is not available because these providers don’t store responses server-side. Instead, send the full conversation history (with auto_cache for providers that need explicit breakpoints) to get cost savings:

Approach	How it works	Provider support
`auto_cache`	You send full history; provider caches token prefixes and charges less for cache hits	Any provider that supports prompt caching
`previous_response_id`	Provider stores conversation server-side; you send only the new message	OpenAI only

Important Notes

Provider Support: Prompt caching works with any provider that supports it. The auto_cache flag matters mainly for providers that require explicit cache breakpoints or charge extra for cache writes (such as Anthropic); many other providers cache automatically regardless of the flag.

Explicit Control: auto_cache provides explicit control. Set to true to attempt caching, false to prevent caching for providers where cache writes incur extra costs.
Default Behavior: If auto_cache is not specified, the caching behavior reverts to defaults based on request origin.
Cost Savings: Cache hits are billed at a fraction of the normal input token cost. This is especially effective for applications with large system prompts or knowledge bases.
Minimum Token Length: Providers enforce a minimum cacheable prefix length (for example, at least 1,024 tokens for Anthropic, 2,048 for Claude 3.5 Haiku). Content shorter than the provider’s minimum will not be cached.
Automatic Provider-Side Caching: Many providers (e.g., OpenAI, DeepSeek, Gemini implicit caching) apply prompt caching automatically on their side. You’ll see cache hits in your cache analytics even without setting auto_cache.
Vertex AI file attachments: For vertex/... Gemini models, auto caching is skipped when the cacheable prefix includes file attachments with URIs that are not on Google Cloud Storage (gs://). External file URLs (for example https://...) still work for inference, but explicit cache creation is not attempted for those requests. Inline/base64 attachments (file_data) and gs:// URIs are supported in cached content.

Managed Caching

If you want Requesty to manage caching on your behalf, including custom TTL, cache warming, or advanced caching strategies, reach out to [email protected].

Getting Started

LLM Gateway

Model Capabilities

Analytics & Monitoring

Access Control

Organization

MCP Gateway

What Auto Cache Does

How Auto Cache Works

Chat Completions API

Responses API

Multi-turn with Responses API

Auto Cache vs. Response IDs

Important Notes

Managed Caching

​What Auto Cache Does

​How Auto Cache Works

​Chat Completions API

​Responses API

​Multi-turn with Responses API

​Auto Cache vs. Response IDs

​Important Notes

​Managed Caching

What Auto Cache Does

How Auto Cache Works

Chat Completions API

Responses API

Multi-turn with Responses API

Auto Cache vs. Response IDs

Important Notes

Managed Caching