Image Understanding

Requesty router supports sending images to vision-capable models for analysis, understanding, and description through the standard chat completions endpoint.

Browse vision models in the Requesty Console.

How It Works

Vision-capable models use the same /v1/chat/completions endpoint as text models, but accept images in the message content. You can include images using either data URLs or file URLs. Like PDF documents, images can also be passed via URL reference to the AI models for processing.

Request Format

Requesty supports two methods for sending images to vision models:

Method 1: Data URLs (Base64)

Include base64-encoded images directly in the request:

curl https://router.requesty.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_REQUESTY_API_KEY" \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "What is in this image?"
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
            }
          }
        ]
      }
    ]
  }'

Method 2: File URLs

Reference images hosted on the web by passing the image URL directly:

curl https://router.requesty.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_REQUESTY_API_KEY" \
  -d '{
    "model": "anthropic/claude-sonnet-4-5",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Describe this image in detail"
          },
          {
            "type": "image_url",
            "image_url": {
              "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a7/Atlantic_near_Faroe_Islands.jpg/1200px-Atlantic_near_Faroe_Islands.jpg"
            }
          }
        ]
      }
    ]
  }'

Vertex AI Gemini Models: When using file URLs with Vertex AI Gemini models, you must specify the MIME type in the request:

{
  "type": "image_url",
  "image_url": {
    "mime_type": "image/jpg",
    "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a7/Atlantic_near_Faroe_Islands.jpg/1200px-Atlantic_near_Faroe_Islands.jpg"
  }
}

Response Format

The response follows the standard chat completions format with the model’s analysis of the image:

{
  "model": "openai/gpt-4o",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The image shows a beautiful sunset over mountain ranges. The sky is painted with vibrant shades of orange, pink, and purple, with the sun just touching the horizon. The mountains are silhouetted against the colorful sky, creating dramatic layers of depth."
      }
    }
  ]
}

Code Examples

Python Example

Using File URLs

from openai import OpenAI

requesty_api_key = "YOUR_REQUESTY_API_KEY"

client = OpenAI(
    api_key=requesty_api_key,
    base_url="https://router.requesty.ai/v1",
)

response = client.chat.completions.create(
    model="anthropic/claude-3-5-sonnet-20241022",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What's in this image?"
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://code-basics.com/rails/active_storage/representations/proxy/eyJfcmFpb.png"
                    }
                }
            ]
        }
    ]
)

print(response.choices[0].message.content)

Using Base64 Data URLs

import base64
from openai import OpenAI

requesty_api_key = "YOUR_REQUESTY_API_KEY"

client = OpenAI(
    api_key=requesty_api_key,
    base_url="https://router.requesty.ai/v1",
)

# Read and encode local image
with open("path/to/image.png", "rb") as image_file:
    base64_image = base64.b64encode(image_file.read()).decode('utf-8')

response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Analyze this image and describe what you see"
                },
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/png;base64,{base64_image}"
                    }
                }
            ]
        }
    ]
)

print(response.choices[0].message.content)

JavaScript/TypeScript Example

Using File URLs

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'YOUR_REQUESTY_API_KEY',
  baseURL: 'https://router.requesty.ai/v1',
});

async function analyzeImage() {
  const response = await client.chat.completions.create({
    model: 'anthropic/claude-3-5-sonnet-20241022',
    messages: [
      {
        role: 'user',
        content: [
          {
            type: 'text',
            text: 'What objects can you identify in this image?'
          },
          {
            type: 'image_url',
            image_url: {
              url: 'https://code-basics.com/rails/active_storage/representations/proxy/eyJfcmFpb.png'
            }
          }
        ]
      }
    ]
  });

  console.log(response.choices[0].message.content);
}

analyzeImage();

Using Base64 Data URLs

import OpenAI from 'openai';
import fs from 'fs';

const client = new OpenAI({
  apiKey: 'YOUR_REQUESTY_API_KEY',
  baseURL: 'https://router.requesty.ai/v1',
});

async function analyzeLocalImage() {
  // Read and encode local image
  const imageBuffer = fs.readFileSync('path/to/image.png');
  const base64Image = imageBuffer.toString('base64');

  const response = await client.chat.completions.create({
    model: 'openai/gpt-4o',
    messages: [
      {
        role: 'user',
        content: [
          {
            type: 'text',
            text: 'Describe this image in detail'
          },
          {
            type: 'image_url',
            image_url: {
              url: `data:image/png;base64,${base64Image}`
            }
          }
        ]
      }
    ]
  });

  console.log(response.choices[0].message.content);
}

analyzeLocalImage();

Supported Models

Requesty supports vision capabilities across multiple AI providers. To see the complete list of models with vision support, navigate to the Model Library in the Requesty web application. Popular vision-capable models include:

OpenAI: GPT models, o3, o3-pro, and all o4 reasoning models.
Anthropic: Claude 4 and 4.5 models.
Google: Gemini 2.5 models.
xAI: Grok 4 models.

Provider-Specific Notes

Most providers support both data URLs and file URLs
Google AI Studio (Gemini) only supports data URLs (base64-encoded images)

Vertex AI Gemini requires the MIME type to be specified in the request.

{
  "type": "image_url",
  "image_url": {
    "mime_type": "image/jpg",
    "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/a/a7/Atlantic_near_Faroe_Islands.jpg/1200px-Atlantic_near_Faroe_Islands.jpg"
  }
}

Check the Model Library in the Requesty web application for the most up-to-date list of vision-capable models and their specific capabilities.

Limitations

Image size limits vary by provider and model
File URLs must be publicly accessible
Base64-encoded images increase request payload size
Response time may be longer for image analysis compared to text-only requests
Some providers have content filtering or safety restrictions for image analysis
For Gemini models using file URLs, MIME type must be specified as a query parameter

Getting Started

LLM Gateway

Model Capabilities

Analytics & Monitoring

Access Control

Organization

MCP Gateway

How It Works

Request Format

Method 1: Data URLs (Base64)

Method 2: File URLs

Response Format

Code Examples

Python Example

Using File URLs

Using Base64 Data URLs

JavaScript/TypeScript Example

Using File URLs

Using Base64 Data URLs

Supported Models

Provider-Specific Notes

Limitations

​How It Works

​Request Format

​Method 1: Data URLs (Base64)

​Method 2: File URLs

​Response Format

​Code Examples

​Python Example

​Using File URLs

​Using Base64 Data URLs

​JavaScript/TypeScript Example

​Using File URLs

​Using Base64 Data URLs

​Supported Models

​Provider-Specific Notes

​Limitations

How It Works

Request Format

Method 1: Data URLs (Base64)

Method 2: File URLs

Response Format

Code Examples

Python Example

Using File URLs

Using Base64 Data URLs

JavaScript/TypeScript Example

Using File URLs

Using Base64 Data URLs

Supported Models

Provider-Specific Notes

Limitations