Streaming and loading states

UI patterns for partial tokens, mid-stream failures, retries, and cancel — for products built on a streaming completion.

Streaming makes a response feel instant, but it also introduces states a non-streaming UI never has to handle: partial content, a stream that dies halfway, and a user who wants to stop generation early.

The four states to design for

• Empty → first token. There's a real gap between "request sent" and "first token arrives" — show a lightweight typing indicator here, not a blocking spinner. A spinner implies indeterminate wait; a typing indicator sets the expectation that content is about to stream in.
• Streaming. Render tokens as they arrive rather than buffering. If your content needs structure (markdown, code blocks), use a renderer that tolerates incomplete syntax — an unclosed code fence mid-stream shouldn't break formatting for the tokens already shown.
• Mid-stream failure. A stream can end early on a provider disconnect or a guardrail block. Don't silently truncate — show what was generated plus a clear "response was interrupted" affordance with a retry action. Losing this distinction is the single most common streaming UX bug: users read a truncated answer as complete.
• User-initiated cancel. Disconnecting the client cancels the upstream request immediately — wire your UI's cancel button to actually close the connection, not just hide the loading state, so you're not paying for tokens the user already dismissed.

Retries

• Retry the whole request, not a resumed stream — Relixr (like the providers behind it) doesn't support resuming a partial generation from a token offset.
• On a retry, keep the partial content visible until the retry's first new token arrives, rather than clearing the UI to a blank state — it reduces perceived latency on the retry.
• Cap automatic retries at one or two attempts with backoff. Beyond that, hand control back to the user — a request that fails twice in a row is more likely a real outage than a transient blip, and see Multi-provider failover for how routing rules can absorb this instead of your UI retry logic doing it.

Cost and metadata timing

Cost and routing metadata (see Headers & metadata) arrive as a final chunk after the last content token, not incrementally — don't build a UI that expects a running cost total mid-stream. Render it once the stream closes, consistent with the pattern described in Designing cost-aware UX.