Streaming

Keiro streams completion responses as server-sent events. Set stream to true, process events in order, and keep the connection open until the surface's terminal event or in-band error.

Copy Page https://docs.keirolabs.ai/streaming.md

Use curl -N when testing from a terminal so client-side buffering does not hide incremental delivery.

During a long stream, the gateway writes the static SSE comment : keepalive at least every 10 seconds (Messages uses its native ping event). SSE comments carry no model output, consume no event sequence number, and are not replayed; conforming parsers ignore them while the bytes keep transport-idle timers alive.

Chat Completions stream#

Chat Completions sends data: chunks with text in choices[0].delta.content. Tool-call fragments appear in choices[0].delta.tool_calls. Assemble by choice and tool-call index.

The successful stream ends with a terminal choice and data: [DONE].

Runnable streaming samples live on Chat Completions.

Responses stream#

Responses emits typed events. Read type and ignore event types your client does not handle.

Successful Responses streams also finish with data: [DONE] after the terminal event.

Runnable streaming samples live on Responses.

Messages stream#

Messages emits message_start, content-block start/delta/stop events, message_delta, and message_stop. The terminal message_delta carries the final stop reason and usage. Tool calls use tool_use content blocks and input JSON deltas. When extended thinking is enabled, unsigned thinking summary blocks stream first, through thinking_delta events, and close before the text block opens.

See Messages for supported request fields and content shapes.

Stream lifetime and time entitlements#

A stream that is actively working is never ended for taking long. While the model keeps producing activity, the stream stays open until it finishes or reaches the wall-clock entitlement for its requested reasoning effort. A live stream ends early in exactly two documented cases:

The requested reasoning effort — reasoning.effort on Responses, reasoning_effort on Chat Completions — selects both compute depth and the run's time entitlement:

• Idle timeout — no model activity for the idle window. Keepalive comments are transport liveness only; they do not count as model activity.
• Ceiling timeout — the run reaches the wall-clock entitlement for its effort tier, measured from stream accept.

Requests that do not set a reasoning effort use the model's default reasoning depth, never below the medium row. eb1-frontier-preview and models documented with extended runtimes default to the max row: long runs need no opt-in. Messages requests carry no effort field; an extended-thinking budget selects the entitlement instead — thinking.budget_tokens of 16,384 or more earns the high row, 131,072 or more the xhigh row, and smaller budgets keep the medium row.

When a bound fires, the stream ends with one honest terminal event, and tokens consumed before the stop are billed:

Configure client timeouts to fit the entitlement, not the other way around: use a read or idle timeout of at least 30 seconds (keepalives arrive at least every 10 seconds, so 30 seconds only trips on a dead connection) and a total timeout at least two minutes above the ceiling for the effort you request.

• Responses: response.incomplete with incomplete_details.reason of stream_idle_timeout or stream_ceiling_timeout.
• Chat Completions: a terminal choice with finish_reason length.
• Messages: a terminal message_delta with stop_reason max_tokens.

Handle in-band errors#

Once response headers and the first body byte are sent, the HTTP status cannot change. A later failure therefore arrives in the stream.

Treat an error frame as terminal. Do not wait for normal completion after it. See Errors for the shared code taxonomy.

• Chat Completions places type, code, message, and optional retry_after_seconds on the error frame.
• Responses and Messages place those fields under error.

Verify delivery#

1
2
3
4
5
6
7
8
9
10
11
12
13

printf 'Keiro API key: '
IFS= read -rs KEIRO_BEARER
printf '\n'

curl -sS -N https://api.keirolabs.ai/v1/responses \
  -w '\nTTFB=%{time_starttransfer}s total=%{time_total}s http=%{http_code}\n' \
  -H "Content-Type: application/json" \
  -d '{
    "model": "eb1-preview",
    "stream": true,
    "input": "Count from one to five, one number per line."
  }' \
  -H @- <<<"Authorization: Bearer $KEIRO_BEARER"

Time to first byte should normally be lower than total time. Similar values can mean the response was short or that a client, proxy, or network path buffered the stream.

Retry safely#

An interrupted stream can contain output that the user already saw. Retry only when repeating the logical operation is safe. Use an idempotency key for retry-sensitive requests, honor Retry-After, and avoid concatenating a retry onto partial output as if it were one response.

Related pages#

• API reference
• Tool calling
• Errors
• Idempotency