Streaming

The platform supports SSE (Server-Sent Events) streaming. The first token arrives sooner, so your UI can render as generation happens.

Basic usage#

Client setup (base_url and API key) is covered in the Quickstart; each snippet below repeats only the minimum.

OpenAI SDK#

from openai import OpenAI

client = OpenAI(
    base_url="https://jiuye.zsopc.com/v1",
    api_key="sk-gpushare-xxx",
)

stream = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Stream a haiku"}],
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)

Anthropic SDK#

from anthropic import Anthropic

client = Anthropic(
    base_url="https://jiuye.zsopc.com",  # ⚠️ no /v1
    api_key="sk-gpushare-xxx",
)

with client.messages.stream(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Stream a haiku"}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Gemini SDK#

from google import genai

client = genai.Client(
    api_key="sk-gpushare-xxx",
    http_options={"base_url": "https://jiuye.zsopc.com"},
)

stream = client.models.generate_content_stream(
    model="gemini-2.5-pro",
    contents="Stream a haiku",
)
for chunk in stream:
    print(chunk.text, end="", flush=True)

When stream=true is required#

Otherwise the gateway returns 400 invalid_request_error with the message Tools `web_search` and `image_generation` require `stream: true`.

SSE wire format#

The gateway forwards the upstream SSE as-is: each data: line is one chunk.

OpenAI Chat stream chunks#

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Hello"}}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" world"}}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-xxx","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":10,"completion_tokens":2,"total_tokens":12}}

data: [DONE]

Two things hand-written parsers get wrong most often:

• The finish_reason chunk carries no usage — usage arrives as a separate trailing chunk right after it, which is the standard stream_options.include_usage convention
• The usage chunk's choices is an empty array, so chunk.choices[0].delta goes out of bounds — check chunk.choices is non-empty before reading delta

Anthropic stream events#

event: message_start
data: {"type":"message_start","message":{...}}

event: content_block_delta
data: {"type":"content_block_delta","delta":{"type":"text_delta","text":"Hello"}}

event: message_stop
data: {"type":"message_stop"}

Gemini streaming (SSE)#

data: {"candidates":[{"content":{"parts":[{"text":"Hello"}]}}]}

data: {"candidates":[{"content":{"parts":[{"text":" world"}]},"finishReason":"STOP"}]}

The Gemini protocol has no [DONE] sentinel — the stream closing is the end signal, so don't wait for [DONE].

Parsing by hand (curl, or a client with no SDK)#

curl https://jiuye.zsopc.com/v1/chat/completions \
  -H "Authorization: Bearer $PLATFORM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.4",
    "stream": true,
    "messages": [{"role":"user","content":"Hello"}]
  }' \
  --no-buffer | while read line; do
    echo "$line"
  done

Parsing with JavaScript fetch:

const resp = await fetch("https://jiuye.zsopc.com/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${process.env.PLATFORM_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "gpt-5.4",
    stream: true,
    messages: [{ role: "user", content: "Hello" }],
  }),
});

const reader = resp.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });

  const lines = buffer.split("\n");
  buffer = lines.pop() ?? "";

  for (const line of lines) {
    if (!line.startsWith("data: ")) continue;
    const data = line.slice(6);
    if (data === "[DONE]") return;
    const chunk = JSON.parse(data);
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  }
}

Getting token usage#

OpenAI Chat streaming does not return a trailing usage chunk by default. To get real token counts, add stream_options:

stream = client.chat.completions.create(
    model="gpt-5.4",
    messages=[...],
    stream=True,
    stream_options={"include_usage": True},  # the important bit
)

last_chunk = None
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="")
    last_chunk = chunk

# the final chunk carries usage
print(f"\nTokens: {last_chunk.usage}")

Internally the gateway auto-injects stream_options.include_usage=true for most upstream channels (except GLM, which sends a usage chunk anyway). So you'll get token counts without setting it — but setting it explicitly is more robust.

Interrupting and cancelling#

OpenAI SDK (Python)#

stream = client.chat.completions.create(..., stream=True)
try:
    for chunk in stream:
        if some_user_canceled():
            stream.close()  # close explicitly; the gateway sees a client disconnect
            break
        print(chunk.choices[0].delta.content or "", end="")
except KeyboardInterrupt:
    stream.close()

fetch (JavaScript)#

const controller = new AbortController();

const resp = await fetch("https://jiuye.zsopc.com/v1/chat/completions", {
  signal: controller.signal,
  // ...
});

// then, to interrupt:
controller.abort();

When the client disconnects, the gateway's upstream connection closes immediately too (on both HTTP and WebSocket channels) — the mechanism is a dropped connection rather than an explicit cancel instruction, and the effect is that the upstream stops generating. Tokens already generated are still billed; if the disconnect means the trailing usage chunk never arrived, the gateway books output tokens estimated from the characters already streamed.

Error handling#

Before the stream opens: an ordinary HTTP error, not SSE#

Auth failure (401), exhausted balance (402), a nonexistent model (400 model_not_found) and no available channel (503) all happen before the stream opens and come back as an ordinary HTTP JSON error body — there's no SSE yet, so handle them like non-streaming errors. Full list: Error handling.

After the stream opens (HTTP 200): it depends on the path#

Tool requests (web_search / image_generation, on the WS V2 channel): a mid-stream error is synthesised into an error chunk followed by [DONE]:

data: {"error":{"message":"<the specific reason>","type":"api_error","code":"upstream_error"}}

data: [DONE]

type and code are fixed values (api_error / upstream_error), so don't branch on code to classify the error — the actual reason is only in the message text. When parsing, check for an error field before handling delta.content, and treat error as the end-of-turn signal.

Plain text chat and every /v1/messages, /v1/responses and /v1beta path: when the upstream fails mid-stream the gateway synthesises nothing and the stream simply ends early — there's no guarantee of [DONE], finish_reason, message_stop or any error frame. A robust client must treat "stream ended without a terminator" as an interruption, not a normal completion. (Error events native to the upstream protocol, such as Anthropic's event: error, are forwarded verbatim — but those come from the upstream; the gateway never generates them.)

On either path, whatever content streamed before the error or interruption is partially valid and can be kept on screen.

The ~180-second ceiling#

A single request — streaming included, across /v1/chat/completions, /v1/messages, /v1/responses and every /v1beta chat endpoint — has a 180-second total ceiling. Streaming just delivers the first token sooner; it doesn't exempt you from the total. A timeout looks exactly like the "stream ends early" case above: if long answers always get cut off around the three-minute mark, this is almost certainly why. Long generations with a large max_tokens, slow models and long image_generation turns can all hit it. What to do:

• Lower max_tokens, or split long work across several turns
• Set your SDK timeout to ≥ 200s (slightly above the gateway ceiling, so the gateway is the one that decides it timed out)
• Tokens already produced are billed as usual

Per-endpoint timeouts: API reference.

Reverse proxies and Nginx#

The gateway sets these response headers:

Content-Type: text/event-stream
Cache-Control: no-cache, no-store, no-transform
X-Accel-Buffering: no

If you proxy this gateway behind your own reverse proxy, make sure buffering is off (proxy_buffering off; in Nginx) — otherwise the whole streamed response is buffered until it completes and delivered in one go, which defeats the point.

Debugging#

Watching the raw SSE with curl is the most direct approach:

curl https://jiuye.zsopc.com/v1/chat/completions \
  -H "Authorization: Bearer $PLATFORM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-5.4","stream":true,"messages":[{"role":"user","content":"hi"}]}' \
  --no-buffer 2>&1 | head -20

If you see the response arrive all at once instead of chunk by chunk, something in between is buffering.