feat(ai-aws-content-moderation): moderate LLM responses, including streams#13735
Open
shreemaan-abhishek wants to merge 1 commit into
apache:masterapache/apisix:masterfrom
shreemaan-abhishek:feat/aws-content-moderation-responseshreemaan-abhishek/apisix:feat/aws-content-moderation-responseCopy head branch name to clipboard
Open
feat(ai-aws-content-moderation): moderate LLM responses, including streams#13735shreemaan-abhishek wants to merge 1 commit intoapache:masterapache/apisix:masterfrom shreemaan-abhishek:feat/aws-content-moderation-responseshreemaan-abhishek/apisix:feat/aws-content-moderation-responseCopy head branch name to clipboard
shreemaan-abhishek wants to merge 1 commit into
apache:masterapache/apisix:masterfrom
shreemaan-abhishek:feat/aws-content-moderation-responseshreemaan-abhishek/apisix:feat/aws-content-moderation-responseCopy head branch name to clipboard
Conversation
…reams The plugin only moderated requests, while ai-aliyun-content-moderation moderates both directions. AWS Comprehend `DetectToxicContent` scores response text just as well, so this was a plugin gap, not an API limit. Adds `check_response` and a `lua_body_filter`: - non-streaming: the assembled completion is moderated in one call - streaming `realtime`: batches are moderated as they arrive (`stream_check_cache_size` / `stream_check_interval`), and the rest of the stream is replaced with a provider-compatible refusal on a hit - streaming `final_packet`: the assembled response is moderated and the last chunk annotated with `risk_level` The verdict is reported on `ctx.var.llm_content_risk_level` (`high` or `none`) so logging matches the aliyun plugin. Comprehend failures on the response side are logged and let the content through: once bytes are on the wire there is no fail-closed option, and buffered responses behave the same way for consistency.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
ai-aws-content-moderationonly moderated requests, while its siblingai-aliyun-content-moderationmoderates both directions. AWS ComprehendDetectToxicContentscores response text just as well, so this was a plugin gap rather than an API limitation. It builds on #13647, which moved the plugin to theaccessphase with protocol-aware content extraction.What changed — a new
check_responseoption plus alua_body_filter:ai_chat)deny_codewith a provider-compatible refusal bodystream_check_mode: realtimestream_check_cache_size(chars) andstream_check_interval(seconds); a hit replaces the rest of the stream with the refusalstream_check_mode: final_packet(default)risk_levelNew schema fields:
check_response(defaultfalse),stream_check_mode(defaultfinal_packet),stream_check_cache_size(default128),stream_check_interval(default3). All default to the previous behavior, so existing configurations are unaffected.The verdict is published on
ctx.var.llm_content_risk_levelashighornone. Comprehend grades nothing, it only scores against the configured thresholds, so the verdict is binary; reusing the same ctx var the aliyun plugin sets keeps logging and downstream consumers uniform.Deny reasons now name the side that tripped (
request body exceeds .../response body exceeds ...).A Comprehend failure on the response side is logged and the content passes through: once bytes are on the wire there is no fail-closed option, and buffered responses behave the same way for consistency. This matches
ai-aliyun-content-moderation. The request side is unchanged and still fails closed with a 500.Tests —
t/plugin/ai-aws-content-moderation.tTEST 18-30 cover non-streaming deny and pass-through, the provider-compatible deny body with a customdeny_messageat the defaultdeny_code,check_responseoff leaving a toxic response untouched,final_packetannotation,realtimecutting the stream off mid-flight, the upstream-error skip, and schema validation of the new fields. New fixtures live undert/fixtures/aws/. The Comprehend mock gained a fallback for free-form response text, since response content is not a fixture key; exact-key lookup still runs first, so every existing test is unchanged.Docs —
docs/en/latest/plugins/ai-aws-content-moderation.mdupdated with the new attributes and the response-moderation description.Checklist