Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

fix(ai-proxy): prevent status change after 200 on streaming body_reader failure#13655

Open
okaybase wants to merge 2 commits into
apache:masterapache/apisix:masterfrom
okaybase:fix-ai-proxyokaybase/apisix:fix-ai-proxyCopy head branch name to clipboard
Open

fix(ai-proxy): prevent status change after 200 on streaming body_reader failure#13655
okaybase wants to merge 2 commits into
apache:masterapache/apisix:masterfrom
okaybase:fix-ai-proxyokaybase/apisix:fix-ai-proxyCopy head branch name to clipboard

Conversation

@okaybase

@okaybase okaybase commented Jul 3, 2026

Copy link
Copy Markdown
Member

Description

When an AI provider's SSE stream stalls mid-response (e.g. the upstream sends a
few chunks then hangs), the body_reader() call in parse_streaming_response
times out. The error handler at ai-providers/base.lua:551 unconditionally
returns a transport-layer error code via transport_http.handle_error(err)
(504 for timeout, 500 for other errors).

However, before entering parse_streaming_response, the caller at
ai-proxy/base.lua:350 has already set the Content-Type response header.
If at least one SSE chunk has been dispatched to the downstream client, the
HTTP response headers — including status 200 — have already been flushed.
Returning a 504/500 after this point causes nginx to emit:
attempt to set status 504 via ngx.exit after sending out the response status 200
attempt to set ngx.status after sending out response headers

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. bug Something isn't working labels Jul 3, 2026
@okaybase okaybase changed the title fix(ai-proxy): prevent status-after-200 error on body_reader timeout in streaming response fix(ai-proxy): prevent status change after 200 on streaming body_reader failure Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Morty Proxy This is a proxified and sanitized view of the page, visit original site.