Latency Tracking
The General tab shows a real-time latency chart with three views:| Metric | What it measures |
|---|---|
| Average | Mean response time across all requests |
| P50 | Median — 50% of requests are faster than this |
| P90 | 90th percentile — only 10% of requests are slower |
What Latency Includes
Total request latency measures the full round-trip: your request hitting Requesty → routed to the provider → model inference → response streamed back. This is the real end-to-end time your users experience.Advanced Performance Analysis
Use the Advanced tab for deeper analysis:Latency by Model
- Set Metric to
latency_ms - Set Group By to
model - Set Calculation to
P50,P90,P95, orP99
Latency Over Time
- Set Time Grouping to
hourorday - Watch for latency spikes that correlate with peak traffic or provider issues
Error Rate Analysis
- Set Metric to
requests - Filter by error status to see failure patterns
- Group by
modelorproviderto identify unreliable providers
Using Performance Data to Optimize
Set Up Latency-Based Routing
If you see that one provider is consistently faster, create a Latency Routing Policy to automatically use the fastest provider:Set Up Fallback for Reliability
If a provider has high error rates, create a Fallback Policy to automatically retry with another provider:Reduce Latency with Caching
Auto Caching can eliminate latency entirely for repeated requests. Check the Savings tab to see your cache hit rate — cached responses return in single-digit milliseconds.Use EU Routing for European Users
If your users are in Europe, route through the EU endpoint (https://router.eu.requesty.ai/v1) to reduce network latency by 30-50%.
Export Performance Data
From the Advanced tab:- Set Metric to
latency_ms, Calculation toP90, Group By tomodel - Set time range and grouping
- Click Export CSV to download the data
Integration
- Usage Analytics — Full dashboard with all metrics
- Cost Tracking — Correlate performance with cost
- Latency Routing — Automatically pick the fastest model
- Fallback Policies — Auto-retry on provider failures
- Spending Alerts — Get notified on anomalies