Streaming

Set "stream": true on a chat completion and the reply arrives as server-sent events, in the OpenAI streaming format, as the model produces it.

curl -N https://backend.alfnrl.io/v1/chat/completions \
  -H "Authorization: Bearer $ALPHANEURAL_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Thread-Id: $THREAD_ID" \
  -d '{
    "model": "openrouter/openai/gpt-4o-mini",
    "messages": [{"role": "user", "content": "Write a haiku about latency."}],
    "stream": true
  }'

# client and thread_id as set up in the Quickstart.
stream = client.chat.completions.create(
    model="openrouter/openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a haiku about latency."}],
    stream=True,
    extra_headers={"X-Thread-Id": thread_id},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

// client and threadId as set up in the Quickstart.
const stream = await client.chat.completions.create(
  {
    model: "openrouter/openai/gpt-4o-mini",
    messages: [{ role: "user", content: "Write a haiku about latency." }],
    stream: true,
  },
  { headers: { "X-Thread-Id": threadId } },
);
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

What you receive

Response headers:

Events are forwarded exactly as the provider sends them, including the closing data: [DONE]. Nothing is added to or rewritten in the stream.

Errors

Checks that happen before the model is called, such as the key, the balance and the thread, fail with an ordinary JSON error. So does a model call that fails before its stream starts: the provider's error body and status are passed through as JSON rather than as an empty stream. Once the first event has been sent, the status is 200 and cannot change. See Errors.

What is saved to the thread

The API reassembles the reply as it forwards it, and adds it to the thread when the stream ends.

• A stream that ends with data: [DONE] or a finish_reason is saved as a complete reply.
• A stream that is cut short, because your client disconnected or the provider's stream broke off, still has what arrived saved, followed by a separate text part: [reply truncated: the stream ended before the model finished]. A later reader, or model, can tell a finished answer from a partial one.
• If no text arrived at all, nothing is added. Your question stays in the thread.

Streaming and routing

stream cannot be combined with your own escalation.ladder. An answer has to be complete before it can be checked, and streaming a first attempt that is then discarded would show the reader an answer that was about to be replaced. The combination gets 400 stream cannot be combined with escalation: the output must be complete before it can be verified.

alphaneural/auto accepts stream: true, but no rung is streamed either. The ladder runs to completion first, and only then is the accepted answer sent as SSE: one frame whose delta carries the whole message, one closing frame with finish_reason, usage and the escalation record, then data: [DONE]. Nothing from a rejected rung is ever sent, and the first byte arrives when the whole answer is ready. The X-Escalation-* headers carry the receipt before the body does. A ladder that is exhausted answers 422 as JSON, not as a stream. See Routing.