Current limits
| Limit | Current behavior |
|---|---|
Request body (/v1/chat/completions) | 4 MiB |
| Request body (dashboard APIs) | 256 KiB |
| Input | Up to the route’s input limit in estimated tokens (128,000 on current routes) |
| Messages | 1–64 |
Output (max_tokens) | 1 through the route’s output cap; default 4,096 or the cap if lower |
| Temperature | 0–2; default 0.2 |
| Rate | 30 requests per minute per production key; 10 per sandbox key |
| Sandbox volume | 1,000 requests per workspace per UTC day |
| Active keys | Up to 20 per workspace |
| Monthly key budget | $0.01–$10,000 (production keys) |
| Request duration | Up to 10 minutes; streams fail after 60 seconds without data |
Per-route input and output limits are listed in GET /v1/models capabilities and on the Models page.
Provider context is not your gateway limit
The gateway estimates input tokens before dispatch: about one token per 4 bytes of the serialized messages, tool definitions and schemas, plus 8 tokens per message and 16 per tool. A request whose estimate exceeds the route’s limit returns 400 input_too_large. The provider still enforces the model’s exact context window, so text that tokenizes unusually densely can be rejected upstream.
The credit reservation is calculated separately from the serialized byte count, which can only over-estimate tokens. Unused reservation is returned when the request settles at the provider-reported usage.
Control concurrency on your side
Budget reservations are atomic, so in-flight requests consume spend capacity. The current gateway has no public concurrency quota endpoint. Use bounded workers and client-side queues to avoid a burst exhausting your wallet or rate limit. There is no automatic request batching.
Unsupported request features
Check authenticated GET /v1/models for capability flags. Streaming, strict schema outputs, and customer-executed function tools are supported selectively. There is one completion choice, no built-in tool execution, no automatic retries, and no content replay. Schemas are limited to 160 nodes, depth 8, and 64 properties per object; at most 16 function tools or calls are accepted.