Start a stream
Use stream:true and stream_options:{"include_usage":true}. The response is text/event-stream. Every event uses an OpenAI-compatible chat.completion.chunk envelope. Content deltas arrive before the finish event; tool-call arguments are buffered until complete and validated.
{
"model": "gpt-6-luna",
"messages": [
{
"role": "user",
"content": "Describe a fictional clinic workflow."
}
],
"stream": true,
"stream_options": {
"include_usage": true
},
"max_tokens": 128,
"data_classification": "synthetic"
}Completion, usage, and errors
The final choice has a finish_reason and Gateway billing metadata. When requested, a separate chunk with choices:[] contains final usage and the same billing metadata. A successful stream ends with data: [DONE]. The gateway always requests provider usage internally, even when you omit client usage reporting.
Errors before streaming begins use normal HTTP status codes and JSON. After streaming begins, errors are sent as data: {"error":{"code":"...","message":"..."}}, followed by [DONE] when the connection permits. [DONE] alone is not evidence of success: check for errors and the successful finish event.
Disconnects and retries
Closing the connection requests cancellation. A provider may already have incurred cost. Reliable usage is charged at the saved model rates plus 15%; missing usage leaves a reservation for reconciliation. Partial text is not replayable. Do not infer a zero charge from a lost connection, and do not automatically repeat the request with a new idempotency key.
For synthetic-only requests, a bounded 512-character holdback preserves the existing identifier checks. Short outputs can arrive in one block. This is real upstream streaming with content checks, not a time-to-first-token guarantee.