> ## Documentation Index
> Fetch the complete documentation index at: https://docs.requesty.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Auto Caching
> Reduce costs by up to 90% with automatic prompt caching. Works with Anthropic and Gemini models on both Chat Completions and Responses APIs
Requesty's auto caching automatically caches long system prompts and repeated content to reduce costs on providers that support prompt caching (Anthropic, Gemini). Cache hits are billed at a fraction of the normal input token cost.
Auto caching works on both `/v1/chat/completions` and `/v1/responses` endpoints.
**[View cache analytics](https://app.requesty.ai/analytics/cache)** in the Requesty Console.
## What Auto Cache Does
Auto caching is **provider-level prompt caching**. When enabled, the router automatically adds cache breakpoints to the largest content blocks in your request before forwarding it to the provider (Anthropic or Gemini). The provider then caches those token prefixes on their side.
This means:
* **You still send the full message history** with every request. The payload size does not change.
* **The provider recognizes cached prefixes** and charges reduced rates for tokens it has already seen.
* **Cache hits are billed at a fraction** of the normal input token cost (up to 90% savings).
Auto caching is not server-side response storage. There is no way to skip sending your conversation history. See [Auto Cache vs. Response IDs](#auto-cache-vs-response-ids) for details.
## How Auto Cache Works
The `auto_cache` flag is a boolean parameter sent within the `requesty` field in your request payload.
| Value | Behavior |
| ------------ | ------------------------------------------------------------------------------------------------------------------------------------ |
| `true` | Instructs the router to add cache breakpoints to the largest content blocks before forwarding to the provider |
| `false` | Bypasses caching for this request (useful when cache writes have extra costs) |
| Not provided | Falls back to default behavior based on request origin (e.g., Cline, Roo Code, Forge, Open WebUI, Hermes, and Pi default to caching) |
## Chat Completions API
Include the `auto_cache` flag within the `requesty` object in your request:
```python Python theme={"dark"}
import openai
client = openai.OpenAI(
api_key="YOUR_REQUESTY_API_KEY",
base_url="https://router.requesty.ai/v1",
)
system_prompt = "YOUR ENTIRE KNOWLEDGEBASE"
response = client.chat.completions.create(
model="anthropic/claude-sonnet-4-5",
messages=[
{"role": "system", "content": system_prompt},
{"role": "user", "content": "What is the capital of France?"}
],
extra_body={
"requesty": {
"auto_cache": True
}
}
)
print(response.choices[0].message.content)
```
```typescript TypeScript theme={"dark"}
import OpenAI from 'openai';
const client = new OpenAI({
apiKey: "YOUR_REQUESTY_API_KEY",
baseURL: "https://router.requesty.ai/v1",
});
const response = await client.chat.completions.create({
model: "anthropic/claude-sonnet-4-5",
messages: [
{ role: "system", content: "YOUR ENTIRE KNOWLEDGEBASE" },
{ role: "user", content: "What is the capital of France?" }
],
requesty: {
auto_cache: true
}
});
console.log(response.choices[0].message.content);
```
```bash cURL theme={"dark"}
curl https://router.requesty.ai/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_REQUESTY_API_KEY" \
-d '{
"model": "anthropic/claude-sonnet-4-5",
"messages": [
{"role": "system", "content": "YOUR ENTIRE KNOWLEDGEBASE"},
{"role": "user", "content": "What is the capital of France?"}
],
"requesty": {
"auto_cache": true
}
}'
```
## Responses API
Auto caching works identically on the `/v1/responses` endpoint. Include the same `requesty.auto_cache` flag:
```python Python theme={"dark"}
import openai
client = openai.OpenAI(
api_key="YOUR_REQUESTY_API_KEY",
base_url="https://router.requesty.ai/v1",
)
system_prompt = "YOUR ENTIRE KNOWLEDGEBASE"
response = client.responses.create(
model="anthropic/claude-sonnet-4-5",
instructions=system_prompt,
input="What is the capital of France?",
extra_body={
"requesty": {
"auto_cache": True
}
}
)
print(response.output_text)
```
```typescript TypeScript theme={"dark"}
import OpenAI from 'openai';
const client = new OpenAI({
apiKey: "YOUR_REQUESTY_API_KEY",
baseURL: "https://router.requesty.ai/v1",
});
const response = await client.responses.create({
model: "anthropic/claude-sonnet-4-5",
instructions: "YOUR ENTIRE KNOWLEDGEBASE",
input: "What is the capital of France?",
requesty: {
auto_cache: true
}
});
console.log(response.output_text);
```
```bash cURL theme={"dark"}
curl https://router.requesty.ai/v1/responses \
-H "Content-Type: application/json" \
-H "Authorization: Bearer YOUR_REQUESTY_API_KEY" \
-d '{
"model": "anthropic/claude-sonnet-4-5",
"instructions": "YOUR ENTIRE KNOWLEDGEBASE",
"input": "What is the capital of France?",
"requesty": {
"auto_cache": true
}
}'
```
### Multi-turn with Responses API
For multi-turn conversations, include the full conversation history in the `input` array along with the `requesty.auto_cache` flag. The router caches the largest content blocks so the provider charges reduced rates for the repeated prefix:
```python theme={"dark"}
response = client.responses.create(
model="anthropic/claude-sonnet-4-5",
instructions="YOUR ENTIRE KNOWLEDGEBASE",
input=[
{"role": "user", "content": "What is the capital of France?"},
{"role": "assistant", "content": "The capital of France is Paris."},
{"role": "user", "content": "What about Germany?"}
],
extra_body={
"requesty": {
"auto_cache": True
}
}
)
```
## Auto Cache vs. Response IDs
OpenAI's Responses API supports a `previous_response_id` parameter that lets OpenAI store conversation state server-side so you don't have to resend the full history. **This is an OpenAI-specific feature** — it works when routing to OpenAI models through Requesty, because OpenAI handles the storage on their end.
For **non-OpenAI models** (Anthropic, Gemini, etc.), `previous_response_id` is not available because these providers don't store responses server-side. Instead, use `auto_cache` with the full conversation history to get cost savings:
| Approach | How it works | Provider support |
| ---------------------- | ------------------------------------------------------------------------------------- | ----------------- |
| `auto_cache` | You send full history; provider caches token prefixes and charges less for cache hits | Anthropic, Gemini |
| `previous_response_id` | Provider stores conversation server-side; you send only the new message | OpenAI only |
## Important Notes
**Provider Support**: The `auto_cache` flag is respected by providers where cache writes incur extra costs, including Anthropic and Gemini.
1. **Explicit Control**: `auto_cache` provides explicit control. Set to `true` to attempt caching, `false` to prevent caching for providers where cache writes incur extra costs.
2. **Default Behavior**: If `auto_cache` is not specified, the caching behavior reverts to defaults based on request origin.
3. **Cost Savings**: Cache hits are billed at a fraction of the normal input token cost. This is especially effective for applications with large system prompts or knowledge bases.
4. **Minimum Token Length**: The cached prefix must be at least 1,024 tokens for Anthropic (2,048 for Claude 3.5 Haiku). Content shorter than that will not be cached.
## Managed Caching
If you want Requesty to manage caching on your behalf, including custom TTL, cache warming, or advanced caching strategies, reach out to [support@requesty.ai](mailto:support@requesty.ai).