Streaming AI Responses with Server-Sent Events

Start with one small response

Streaming lets your application display an answer while the model is still generating it. Instead of waiting for a finished paragraph, the reader sees text arrive progressively. Server-sent events, or SSE, carry these updates over an HTTP response. OpenAI documents this approach for streaming model output. ([developers.openai.com](https://developers.openai.com/api/docs/guides/streaming-responses))

For your first walkthrough, use a deliberately small task: ask for three sentences explaining a familiar concept. Keep the same prompt throughout testing. Your first milestone is simple: one request, one answer that appears gradually, and a clear indication when it finishes.

Keep the relay key behind your server

Build the flow around your application server. The browser sends the prompt to your server, which makes the authenticated model request and forwards the response. Keep your relay key there rather than embedding it in browser code. This gives you a place to enforce your own request limits.

For a MyRelay setup, use its API base and your personal key with an OpenAI-compatible SDK. Create a MyRelay account when you are ready to test with your own budget. Keep this first integration narrow: choose one available model and avoid switching models while you debug the streaming flow.

Enable streaming and read incremental updates

Enable the SDK’s streaming option for your chat completion request. Consume the returned stream incrementally rather than waiting for a complete message. In OpenAI’s Chat Completions format, streamed chunks contain incremental changes in a delta field instead of a finished message field. Some chunks contain role information or no visible text, so append only actual text content. ([developers.openai.com](https://developers.openai.com/api/docs/guides/streaming-responses?utm_source=openai))

Maintain one accumulated answer for the request. Add each text fragment to that answer in arrival order, then update the display. Treat each fragment as a piece of the same response, not as a separate chat message.

Separate network chunks from complete events

If you use an SDK, prefer its streaming parser. If you handle SSE yourself, remember that the event stream is UTF-8 text, with blank lines separating events. Multiple data lines can belong to one event, and comment lines are not answer content. A network read is therefore not a reliable event boundary. ([developer.mozilla.org](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events?utm_source=openai))

Keep incomplete data in a buffer until an entire event is available. Test this parser separately from the model by feeding it deliberately split input. Include a split inside a word and another inside a multibyte character. The reconstructed answer should remain unchanged.

Distinguish completion from an interrupted connection

Handle completion according to the API format you actually use. OpenAI-style Chat Completions can send a final completion marker and, when requested, a separate usage chunk. Do not assume the final visible text fragment contains everything needed to close the request. ([developers.openai.com](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create?utm_source=openai))

Give the interface distinct states for generating, completed, stopped, and interrupted. If the connection disappears without a confirmed completion, retain the partial answer but mark it incomplete. Avoid silently restarting the whole generation; ask the reader to retry deliberately instead.

Make stopping a request an explicit action

Add a stop control before testing longer answers. Browser Fetch requests can be cancelled through an AbortController. Connect that cancellation to your server’s upstream request handling rather than merely hiding incoming text in the interface. ([developer.mozilla.org](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch?utm_source=openai))

Treat cancellation as a best-effort stop, not a promise of zero additional cost. Test what happens when the reader stops immediately, halfway through an answer, and just before completion. In every case, the interface should settle into a clear state.

Check usage before increasing your test budget

Where supported, request streaming usage information. OpenAI documents that its usage-enabled Chat Completions stream includes a final usage chunk, but an interrupted stream may never deliver it. Missing usage is not evidence that a request was free. ([developers.openai.com](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create?utm_source=openai))

After each small test, review the request’s tokens and cost in MyRelay. Record whether it completed, stopped, or failed. Once those cases behave predictably, increase answer length gradually. Make reliable completion and understandable spending your milestones, rather than a dramatic first demo.