实现 ChatGPT 的流式输出 Server-Sent Events

临江仙455
1,052 阅读7分钟

引言

在现代网络应用中，实时性成为了用户体验的关键之一。而在构建聊天应用或实时通信工具时，实现流式输出效果是至关重要的。本文将探讨ChatGPT是如何利用 Server-Sent Events技术来实现流式输出效果，从而提升用户体验。

什么是Server-Sent Events (SSE)

Server-Sent Events 是一种 HTML5 技术，允许服务器向客户端推送事件流。与传统的客户端轮询或 WebSockets 相比，Server-Sent Events 提供了一种更简单、更轻量级的方式来实现服务器到客户端的单向通信。

通俗易懂一些理解就是，服务端与客户端建立了 长连接，服务端源源不断地向客户端推送消息。服务端就相当于河流的上游，客户端就相当于河流的下游，水往低处流，这就是 SSE 的流式传输。

ChatGPT 的流式输出需求

ChatGPT 是一个基于人工智能的聊天机器人，它可以与用户进行自然语言交互。在某些情况下，用户可能期望看到聊天内容的实时响应，而不是等待整个响应完全生成后再一次性展示。这就需要实现 ChatGPT 的流式输出效果，让用户能够看到逐步生成的响应。

SSE的工作原理

  • 建立连接：客户端通过发送一个普通的HTTP GET请求来初始化一个SSE连接。这个请求的响应是一个持久的数据流，而不是常见的一次性响应。
  • 发送消息：服务器可以随时通过这个持久的响应发送消息，消息格式是简单的文本数据。
  • 保持连接：连接会保持打开状态，直到客户端或服务器决定关闭。

SSE与WebSocket的比较

  • 单向 vs 双向：SSE仅支持服务器到客户端的单向数据流，而WebSocket支持全双工通信。
  • 简单性：SSE在实现上比WebSocket简单，尤其是对于只需要单向通信的场景。
  • 兼容性：SSE可以在任何支持HTTP的平台上使用，而WebSocket需要特定的服务器和客户端支持。

优势与注意事项

使用 Server-Sent Events 技术实现 ChatGPT 的流式输出效果具有以下优势：

  • 轻量级: 与 WebSockets 相比，Server-Sent Events 更简单、更轻量级，适用于不需要双向通信的场景。
  • 易于实现: Server-Sent Events 的实现相对简单，无需处理复杂的握手过程，适合快速构建原型或小规模应用。

然而，也需要注意以下事项：

  • 单向通信: Server-Sent Events 只支持服务器向客户端的单向通信，不适用于需要客户端向服务器发送数据的场景。
  • 浏览器兼容性: 虽然大多数现代浏览器都支持 Server-Sent Events，但某些旧版本的浏览器可能不支持或支持不完全。

效果图

dz9rb-2mnc6.gif

请求数据

{
    "messages":[
        {
            "role":"user",
            "content":"测试"
        }
    ],
    "stream":true,"model":"gpt-3.5-turbo",
    "temperature":0.6,
    "presence_penalty":0,
    "frequency_penalty":0,
    "top_p":1
}

响应数据

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"role":"assistant","content":""},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"你"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"好"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"，"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"有"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"什"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"么"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"可以"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"帮"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"助"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"你"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"的"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"吗"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{"content":"？"},"logprobs":null,"finish_reason":null}]}

data: {"id":"chatcmpl-9MUYfVVE1iwSfVzsLsniiMBiQ76dV","object":"chat.completion.chunk","created":1715147709,"model":"gpt-3.5-turbo-0125","system_fingerprint":null,"choices":[{"index":0,"delta":{},"logprobs":null,"finish_reason":"stop"}]}

data: [DONE]

代码案例

  • openai配置
const REQUEST_TIMEOUT_MS = 6000;

const OpenaiPath = {
  ChatPath: "v1/chat/completions", // chatgpt 聊天接口
};

const modelConfig = {
  model: "gpt-3.5-turbo",
  temperature: 0.6,
  top_p: 1,
  max_tokens: 1024,
  presence_penalty: 0,
  frequency_penalty: 0,
  token: "", // openai api key 
  openaiUrl: "https://api.openai.com",
  historyMessageCount: 1,
  compressMessageLengthThreshold: 1000,
};

function useAccessStore() {
  return modelConfig;
}
  • 获取 HTTP 请求头部信息
function getHeaders() {
  const headers = {
      "Content-Type": "application/json",
      "x-requested-with": "XMLHttpRequest",
      Authorization: `Bearer ${useAccessStore().token.trim()}`,
    };
    return headers;
}
  • 格式化对象并返回其 JSON 表示形式
function prettyObject(msg) {
  const obj = msg;
  if (typeof msg !== "string") {
    msg = JSON.stringify(msg, null, "  ");
  }
  if (msg === "{}") {
    return obj.toString();
  }
  if (msg.startsWith("```json")) {
    return msg;
  }
  return ["```json", msg, "```"].join("\n");
}
  • 封装ChatGptApi
import { EventStreamContentType, fetchEventSource } from "@microsoft/fetch-event-source";

class ChatGPTApi {
  path(path) {
    let openaiUrl = useAccessStore().openaiUrl;
    if (!openaiUrl) {
      openaiUrl = modelConfig.openaiUrl;
    }
    return openaiUrl + path;
  }
  extractMessage(res) {
    return res.choices?.at(0)?.message?.content ?? "";
  }
  generateRequestPayload(messages, modelConfig, options) {
    return {
      messages: messages.slice(-Number(modelConfig.historyMessageCount)), // 上下文
      stream: options.stream, // 流式传输
      model: modelConfig.model, // 模型
      temperature: Number(modelConfig.temperature), // 随机性
      presence_penalty: modelConfig.presence_penalty, //话题新鲜度
      frequency_penalty: modelConfig.frequency_penalty, // 频率惩罚度
      top_p: modelConfig.top_p, // 核采样
    };
  }
  // 生成聊天消息
  async chat(options) {
   const messages = options.messages.map(({ role, content }) => ({ role, content }));
   
    const modelConfig = {
      ...useAccessStore(),
      ...{
        model: options.config.model,
      },
    };
    const requestPayload = this.generateRequestPayload(messages, modelConfig, options.config);
    console.log("[Request] openai payload: ", requestPayload);
    const shouldStream = !!options.config.stream;
    const controller = new AbortController();
    options.onController?.(controller);
    try {
      const chatPath = this.path(OpenaiPath.ChatPath);
      const chatPayload = {
        method: "POST",
        body: JSON.stringify(requestPayload),
        signal: controller.signal,
        headers: getHeaders(),
      };
      const requestTimeoutId = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
      // 流式输出
      if (shouldStream) {
        let responseText = "";
        let remainText = "";
        let finished = false; // 是否已完成。
        // 通过逐步添加文本的方式，以一种动画效果显示响应文本
        function animateResponseText() {
          if (finished || controller.signal.aborted) {
            responseText += remainText;
            console.log("[Response Animation] finished");
            return;
          }
          if (remainText.length > 0) {
            const fetchCount = Math.max(1, Math.round(remainText.length / 60));
            const fetchText = remainText.slice(0, fetchCount);
            responseText += fetchText;
            remainText = remainText.slice(fetchCount);
            options.onUpdate?.(responseText, fetchText);
          }
          requestAnimationFrame(animateResponseText);
        }
        // start animaion
        animateResponseText();
        const finish = () => {
          if (!finished) {
            finished = true;
            options.onFinish(responseText + remainText);
          }
        };
        controller.signal.onabort = finish;
        fetchEventSource(chatPath, {
          ...chatPayload,
          // 建立连接的回调
          async onopen(res) {
            clearTimeout(requestTimeoutId);
            const contentType = res.headers.get("content-type");
            console.log("[OpenAI] request response content type: ", contentType);
            if (contentType?.startsWith("text/plain")) {
              responseText = await res.clone().text();
              return finish();
            }
            const stream = !contentType?.startsWith(EventStreamContentType);
            const isRequestError = !res.ok || stream || res.status !== 200;
            if (isRequestError) {
              const responseTexts = [responseText];
              let extraInfo = await res.clone().text();
              try {
                const resJson = await res.clone().json();
                extraInfo = prettyObject(resJson);
              } catch (e) {
                console.log("[resJson]", e);
              }
              if (res.status === 401) {
                options.onError?.(extraInfo);
              }
              if (extraInfo) {
                responseTexts.push(extraInfo);
              }
              responseText = responseTexts.join("\n\n");
              return finish();
            } else {
              console.log(res);
            }
          },
          // 接收一次数据段时回调流式返回
          onmessage(msg) {
            if (msg.data === "[DONE]" || finished) {
              return finish();
            }
            const text = msg.data;
            try {
              const json = JSON.parse(text);
              const delta = json.choices[0].delta.content;
              if (delta) {
                remainText += delta;
              }
            } catch (e) {
              console.error("[Request] parse error", text, msg);
            }
          },
          // 正常结束的回调
          onclose() {
            finish();
          },
          // 连接出现异常回调
          onerror(e) {
            options.onError?.(e);
            throw e;
          },
          openWhenHidden: true,
        });
      } else {
        const res = await fetch(chatPath, chatPayload);
        clearTimeout(requestTimeoutId);
        const resJson = await res.json();
        const message = this.extractMessage(resJson);
        options.onFinish(message);
      }
    } catch (e) {
      console.log("[Request] failed to make a chat reqeust", e);
      options.onError?.(e);
    }
  }
}
  • 调用函数
 const api = new ChatGPTApi();
 
 await api.chat({
    messages: [{role:'user',content:'你好'}],
    config: { model: 'gpt-3.5-turbo', stream: true },
    onUpdate(message) {
      console.log("[chat] onUpdate:", message);
    },
    onFinish(message) {
      console.log("[chat] onFinish:", message);
    },
    onError(error) {
      console.error("[chat] failed:", error);
    },
    onController(controller) {
      console.log("[chat] onController:", controller);
    },
 });

结语

通过利用 Server-Sent Events 技术，实现 ChatGPT 的流式输出效果，从而提升用户与聊天机器人的交互体验。然而，在选择技术时，还需根据具体应用场景权衡利弊，并注意兼容性和性能等方面的问题。

github：PureChat

文档：PureChat | docs

预览地址: purechat.cn

avatar
文章
阅读
粉丝