# Streaming

> Server-sent events from the chat endpoint, and what is saved to the thread when a stream is cut short.

Set `"stream": true` on a [chat completion](https://app.alphaneural.io/docs/chat-completions) and the reply arrives as server-sent events, in the OpenAI streaming format, as the model produces it.

```bash
curl -N https://backend.alfnrl.io/v1/chat/completions \
  -H "Authorization: Bearer $ALPHANEURAL_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Thread-Id: $THREAD_ID" \
  -d '{
    "model": "openrouter/openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Write a haiku about latency."}],
    "stream": true
  }'
```

```python
# client and thread_id as set up in the Quickstart.
stream = client.chat.completions.create(
    model="openrouter/openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a haiku about latency."}],
    stream=True,
    extra_headers={"X-Thread-Id": thread_id},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

```ts
// client and threadId as set up in the Quickstart.
const stream = await client.chat.completions.create(
  {
    model: "openrouter/openai/gpt-4o-mini",
    messages: [{ role: "user", content: "Write a haiku about latency." }],
    stream: true,
  },
  { headers: { "X-Thread-Id": threadId } },
);
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
```

## What you receive

Response headers:

| Header | |
|---|---|
| `Content-Type` | `text/event-stream` |
| `Cache-Control` | `no-cache` |
| `X-Accel-Buffering` | `no`, so proxies pass events through as they arrive. |
| `X-Thread-Id` | The thread, as on a non-streamed response. Sent before the first event. |

Events are forwarded exactly as the provider sends them, including the closing `data: [DONE]`. Nothing is added to or rewritten in the stream.

## Errors

Checks that happen before the model is called, such as the key, the balance and the thread, fail with an ordinary JSON error. So does a model call that fails before its stream starts: the provider's error body and status are passed through as JSON rather than as an empty stream. Once the first event has been sent, the status is `200` and cannot change. See [Errors](https://app.alphaneural.io/docs/errors).

## What is saved to the thread

The API reassembles the reply as it forwards it, and adds it to the thread when the stream ends.

- A stream that ends with `data: [DONE]` or a `finish_reason` is saved as a complete reply.
- A stream that is cut short, because your client disconnected or the provider's stream broke off, still has what arrived saved, followed by a separate text part: `[reply truncated: the stream ended before the model finished]`. A later reader, or model, can tell a finished answer from a partial one.
- If no text arrived at all, nothing is added. Your question stays in the thread.

## Streaming and routing

`stream` cannot be combined with your own `escalation.ladder`. An answer has to be complete before it can be checked, and streaming a first attempt that is then discarded would show the reader an answer that was about to be replaced. The combination gets `400` `stream cannot be combined with escalation: the output must be complete before it can be verified`.

`alphaneural/auto` accepts `stream: true`, but no rung is streamed either. The ladder runs to completion first, and only then is the accepted answer sent as SSE: one frame whose `delta` carries the whole message, one closing frame with `finish_reason`, `usage` and the `escalation` record, then `data: [DONE]`. Nothing from a rejected rung is ever sent, and the first byte arrives when the whole answer is ready. The `X-Escalation-*` headers carry the receipt before the body does. A ladder that is exhausted answers `422` as JSON, not as a stream. See [Routing](https://app.alphaneural.io/docs/routing).