# Streaming

> Server-sent events from /v1/chat/completions, the final usage chunk, disconnects and mid-stream failures.

Source: https://cairva.xyz/docs/streaming

Set `"stream": true` on a chat completion and the response arrives as server-sent events. Usage and the `cairva` metering object come in the last chunk, so you don't need `stream_options`.

```endpoint
POST /v1/chat/completions
```

## Example

```bash curl
curl -N https://cairva.xyz/v1/chat/completions \
  -H "Authorization: Bearer $CAIRVA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "gemma4:e2b", "stream": true,
       "messages": [{"role": "user", "content": "Count to five."}]}'
```

```javascript JavaScript
const stream = await client.chat.completions.create({
  model: "gemma4:e2b",
  messages: [{ role: "user", content: "Count to five." }],
  stream: true,
});

for await (const chunk of stream) {
  const text = chunk.choices[0]?.delta?.content;
  if (text) process.stdout.write(text);
  // The last chunk has no choices. It carries usage and the cairva object.
  if (chunk.usage) console.log("\n", chunk.usage, chunk.cairva);
}
```

```python Python
stream = client.chat.completions.create(
    model="gemma4:e2b",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
    # The last chunk has no choices. It carries usage and the cairva object.
    if chunk.usage:
        print("\n", chunk.usage, chunk.model_extra["cairva"])
```

## Event sequence

The response has `Content-Type: text/event-stream`. Each event is one `data:` line holding a `chat.completion.chunk`:

1. Content deltas: `choices[0].delta.content` with the next piece of text and `finish_reason: null`.
2. One chunk with an empty `delta` and the `finish_reason`.
3. One chunk with an empty `choices` array, `usage`, and the `cairva` object.
4. `data: [DONE]`.

```text
data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1793520000,"model":"gemma4:e2b","choices":[{"index":0,"delta":{"content":"One"},"finish_reason":null}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1793520000,"model":"gemma4:e2b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-…","object":"chat.completion.chunk","created":1793520000,"model":"gemma4:e2b","choices":[],"usage":{"prompt_tokens":12,"completion_tokens":9,"total_tokens":21},"cairva":{"request_id":"…","allocation_id":"…","service_mode":"sandbox","usage_source":"provider_reported","remaining":{"input":999976,"output":499991}}}

data: [DONE]
```

Deltas carry `content` only. There's no separate first chunk with `role`. The `cairva` object has the same fields as on a [non-streaming response](https://cairva.xyz/docs/chat-completions#the-cairva-object). If the provider didn't report usage, `usage` is `null` and `usage_source` is `pending`.

## Errors before the stream starts

Authentication, validation and admission errors, such as `rate_limited` or `allocation_exhausted`, come back as a normal JSON error with an HTTP status. No stream is opened. See [Errors](https://cairva.xyz/docs/errors).

## Errors during the stream

Once admission passes, the response is already `200` with an open stream, and the provider is called from inside it. So a provider failure, even one before the first token, arrives as one event and then the stream closes, without `[DONE]`:

```json
{"error": {"code": "upstream_error", "message": "The inference provider stopped responding"}}
```

The request's hold is released and nothing is charged for it.

## Disconnecting early

If your client disconnects, Cairva keeps reading the provider's stream to the end so the provider's usage report still arrives. The tokens the model generated after you left are still metered. To spend less, set `max_tokens` lower instead of disconnecting.

> [!NOTE]
> Reasoning models can spend output tokens on reasoning that isn't streamed as content. A stream can end with few or no content deltas and `finish_reason: "length"`, and those tokens are still metered. See [Chat completions](https://cairva.xyz/docs/chat-completions#output-tokens-you-don-t-see).