Streaming responses

Set stream: true and the response becomes a text/event-stream of chunks in the OpenAI streaming format. Existing SSE parsers work unchanged.

Chunk format#

Each event carries a partial delta. The stream ends with a literal [DONE].

data: {"id":"cmpl_9f2a1c","choices":[{"delta":{"role":"assistant"},"index":0}]}
 
data: {"id":"cmpl_9f2a1c","choices":[{"delta":{"content":"Reserved"},"index":0}]}
 
data: {"id":"cmpl_9f2a1c","choices":[{"delta":{"content":" capacity"},"index":0}]}
 
data: {"id":"cmpl_9f2a1c","choices":[{"delta":{},"index":0,"finish_reason":"stop"}]}
 
data: [DONE]

Consuming the stream#

const stream = await client.chat.completions.create({
  model: "stac-1",
  messages,
  stream: true,
});
 
for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (delta) process.stdout.write(delta);
}

stream = client.chat.completions.create(
    model="stac-1",
    messages=messages,
    stream=True,
)
 
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

curl -N "https://api.trystac.com/v1/chat/completions" \
  -H "Authorization: Bearer $STAC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "model": "your-stack-model", "messages": [...], "stream": true }'

The -N flag matters for curl: without it the output is buffered and the stream looks like a slow non-streaming response.

Cancelling#

Abort the HTTP request. Stac stops generation as soon as the connection closes, freeing that capacity for the next request.

const controller = new AbortController();
setTimeout(() => controller.abort(), 5000);
 
try {
  const stream = await client.chat.completions.create(
    { model: "stac-1", messages, stream: true },
    { signal: controller.signal }
  );
} catch (error) {
  if (error.name === "AbortError") return;
  throw error;
}

Errors mid-stream#

Once the response headers are sent the status is already 200, so a failure after that point can only arrive as a broken connection, not an HTTP error status. Treat a stream that ends without a [DONE] event as a failure and retry the request from scratch — there's no partial-resume.