Skip to content

Streaming responses are cut off after fifteen seconds #459

Description

@0xmc

What needs to happen

Responses that stay open longer than fifteen seconds are cut off at the edge, and there is no field anywhere in the proxy API to change that. A user measuring a deliberately slow twenty-five second response saw it closed at about sixteen seconds with bytes still outstanding. That rules out server-sent events, streaming APIs and long-lived agent tool calls behind our edge today, which is an awkward gap for a platform positioning itself at agent workloads.

Why this matters

We advertise streaming-shaped protocol support, so people reasonably build on it and find the ceiling in production rather than in the docs. The user in question had to discover the limit by probing, then redesign around it. Either the limit should be raised and configurable, or it should be stated plainly enough that nobody builds a streaming feature before finding out.

Desired outcome

  • A user can serve a response that stays open for minutes, through configuration they can reach from the proxy API
  • Server-sent events and streaming tool calls work end to end through the edge
  • Whatever ceiling remains is documented with its value, rather than found by experiment
  • The claim of streaming protocol support and the behaviour of the edge agree

Related to datum-cloud/enhancements#292

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

Priority

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions