Response structure
{
"id": "req_example",
"object": "chat.completion",
"created": 1793000000,
"model": "care-small",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Example synthetic response."
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 25,
"completion_tokens": 8,
"total_tokens": 33
},
"gateway": {
"mode": "demo",
"charged_usd": 3.2775e-05,
"markup_percent": 15,
"request_id": "req_example",
"usage_review": false,
"healthcare_policy": {
"version": "healthcare-v1",
"hash": "example_policy_hash",
"classification": "synthetic",
"use_case": "administrative_support"
}
}
}Illustrative values are shown above. Actual IDs, usage, policy hashes, timestamps, and charges vary. Demo usage is estimated; real routes use validated provider usage.
Fields to use in your application
| Field | Use |
|---|---|
choices[0].message.content | Assistant text. No tool calls or multimodal parts. |
choices[0].finish_reason | stop or length; length indicates the configured output limit was reached. |
usage | Prompt, completion, and total token counts. |
gateway.charged_usd | Settled customer charge in US dollars, including the platform markup. |
gateway.mode | demo, staging, or live. |
gateway.healthcare_policy | Policy version/hash, effective classification, and supported use case. |
gateway.usage_review | True if usage exceeded the reservation assumptions; the key is paused for review. |
Two different request IDs
X-Request-ID identifies the HTTP request, including an error response. The completion id, gateway.request_id, and successful X-Gateway-Request-ID identify the same gateway generation. Save these IDs with content-free operational metadata.
Store only what your application needs
The gateway does not retain completion text for replay. If your product needs conversation history, your application must implement it under its own privacy, access, and retention policies. An SDK or application logger can still persist prompts or answers; the gateway cannot remove copies made by your client.
Measure Gateway processing time
Successful completions include gateway.timing: admission_ms measures Gateway work before upstream dispatch, upstream_ms measures the upstream exchange, and settlement_ms measures local work after the provider finishes. Ordinary JSON responses also include the Server-Timing header. Streaming responses include these measurements in their final metadata frame.
Streaming text adds first_token_forward_ms, measured from the first upstream text delta to the first forwarded text. Synthetic-only routes retain an identifier-check look-behind, and tool arguments wait for validation. These safeguards can intentionally delay output.
These server-side measurements exclude client-to-Gateway network time and load-balancer transit. Upstream time includes request transmission, provider queue/generation, response reading, and streaming backpressure. Measure end-to-end latency in your client as well. No prompt, response, credential, or schema is included in timing data.