Endpoint & headers
POST /v1/chat/completions
Authorization: Bearer <GATEWAY_API_KEY>
Content-Type: application/json
Idempotency-Key: <unique-logical-request-id>Idempotency-Key is optional but strongly recommended. Streaming is available on supported routes; multipart requests are not supported. The whole request body must fit within 4 MiB.
Request body
Required: model and 1–64 messages. Optional: max_tokens, temperature, stream, stream_options, response_format, tools, tool_choice, parallel_tool_calls, data_classification, and use_case. Model bounds still apply.
Message roles are system, user, assistant, and tool. Text content is a string. An assistant tool-call message may have null content and a tool_calls array. Tool results include tool_call_id. Return exactly one result for every pending call before the next conversational message.
stream_options accepts only include_usage and requires stream:true. See streaming, structured outputs, and tools.
Example
{
"model": "care-small",
"messages": [
{
"role": "user",
"content": "Explain an AI gateway in one sentence."
}
],
"max_tokens": 64,
"data_classification": "synthetic",
"use_case": "administrative_support"
}The sample uses the local demo’s care-small model. With a sandbox key use sandbox-chat; with a production key, discover the catalog.
Input and output limits
Admission estimates the UTF-8 bytes of messages, tool definitions, tool arguments/results, and response schemas, plus conservative per-message and per-tool overhead. These all count toward the configured input bound and maximum credit reservation. This is deliberately stricter than the provider’s advertised context window.
Caller-supplied provider URLs, credentials, caching options, storage controls, and fallback routes remain rejected. The gateway owns store:false and provider routing.
Returns
On success: 200, the completion schema described in responses and usage, and X-Gateway-Request-ID. Every response also carries X-Request-ID for the HTTP request. Errors use the standard error object.