LLM API streaming and SSE integration guide

Streaming output does not mean "the model is faster", but rather allows the client to continuously receive incremental content before the full result is generated. It can improve the first word waiting experience, but it also introduces SSE parsing, proxy buffering, connection interruption and usage statistics Wait for additional questions.

Check before you start

Complete three confirmations first:

1. Use non-streaming requests to verify that the API Key, model ID, and underlying API are working properly.
2. Confirm that the target model and API support streaming output.
3. Verify that clients, reverse proxies, and CDNs do not cache or merge event streams.

If non-streaming requests also fail, first check for 401, model not found or API paths. Do not start debugging directly from SSE.

Configuration steps

1. Use curl to view the original event stream

curl -N https://www.aifast.club/v1/chat/completions \
  -H "Authorization: Bearer $AIFAST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "YOUR_MODEL_ID",
    "stream": true,
    "messages": [
      {"role": "user", "content": "Use three points to explain what SSE。"}
    ]
  }'

-N Will turn off curl's output buffering. Under normal circumstances, the terminal will have multiple consecutive data: chunks of data instead of waiting for the entire content to be generated and displayed at once.

2. Python SDK streaming reading

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AIFAST_API_KEY"],
    base_url="https://www.aifast.club/v1",
)

stream = client.chat.completions.create(
    model="YOUR_MODEL_ID",
    stream=True,
    messages=[
        {"role": "user", "content": "give three API Online inspection items。"}
    ],
)

for chunk in stream:
    if not chunk.choices:
        continue
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

Don't assume that every chunk of data has text. Tool calls, role information, end reasons or compatibility layer events may appear in different fields. The formal code needs to first determine whether the fields exist.

3. Node.js SDK streaming reading

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AIFAST_API_KEY,
  baseURL: "https://www.aifast.club/v1",
});

const stream = await client.chat.completions.create({
  model: "YOUR_MODEL_ID",
  stream: true,
  messages: [
    { role: "user", content: "Explaining Two Common Failures of Streaming Output。" },
  ],
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

4. Record completeness rather than just display text

The production environment records at least:

FAQ

The server is streaming, but the web page is still displayed at once

Use first curl -N Verify the origin site. If the terminal can display chunks, the problem is usually with your backend framework, reverse proxy, or frontend reading logic. Check whether response buffering is turned on and whether it is actually read from the stream response.body。

The output suddenly stops halfway

The last data block, end reason, and request ID are logged. Common causes include read timeout, network outage, agent idle timeout, browser canceling the request, or upstream build failure. Do not write incomplete content to the database as a successful result.

Why there is no usage for streaming requests

Some protocols only return full usage in the last data block. If the connection is interrupted early, the client may not get the final statistics. The accounting reconciliation is based on the console and server records, and cannot only rely on the data blocks received by the front end.

Next step

Test short text, long text, Chinese, multi-byte characters, active cancellation, network disconnection and proxy timeout respectively. Before going online, the maximum task duration, disconnection status, and user-visible retry entry should also be set for streaming requests.