Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings
Discussion options

Kos-M
Sep 18, 2025

Hello all ,
Im trying to enable streaming responses on my agent that i have deployed in runtime.
But unfortunately is not streams the chunks , just responds the final text.

Locally the endpoint is working as expected.
For simplifying the streaming tests im running this curl command so i can see if streams chunks or not.

 curl --no-buffer --location 'http://127.0.0.1:8080/invocations 'Content-Type: application/json'   --data '{
   "input": {
       "prompt": "hello",
            "prompt_uuid": "...",
       "user_timezone": "...",
       "session_id": "...","strat": 2
   }
   
}'

my entrypoint looks like this :

app = FastAPI(title="--", version="1.0.0")
@app.post("/invocations")
async def invoke_agent(invocation_request:InvocationRequest, request: Request):
    user_message = invocation_request.input.get("prompt", "")
    async def generate():
        async for event in orchestrator.agent.stream_async(user_message):
                    if strat == 1 and "data" in event:
                        yield event["data"].encode('utf-8')       
                    if  strat == 2 and "data" in event and "delta" in event:
                        yield event['delta']['text'].encode('utf-8')
                    if strat == 3 and "data" in event:
                        yield event["data"]           
                    if strat == 4 and "event" in event and "contentBlockDelta" in event["event"] and "delta" in event["event"]["contentBlockDelta"] and "text" in event["event"]["contentBlockDelta"]["delta"]:
                        yield event["event"]["contentBlockDelta"]["delta"]["text"]                 
    return StreamingResponse(generate(), media_type="text/plain", headers={"transfer-encoding": "chunked"})

@app.get("/ping")
async def ping():
    return {"status": "healthy"}


uvicorn.run(app, host="0.0.0.0", port=8080)

At this point i should say that have tried media_type="text/plain" , media_type="text/stream"
with and without transfer-encoding header.

Locally is set by default the response header of transfer-encoding to chunked , but not in hosted requests.
Also added the strat param to rtest all different ways i can yield the data (locally works on all )

What im missing ?
any idea ?

If you reached so far , thanks anyway :)
You must be logged in to vote

Replies: 2 comments

Comment options

Unshure
Sep 19, 2025
Maintainer

Hi @Kos-M, thanks for reaching out. I was able to run a slightly altered version of your code locally, and was able to see the streamed response:

from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
from strands import Agent
import uvicorn

app = FastAPI(title="--", version="1.0.0")
agent = Agent()


@app.post("/invocations")
async def invoke_agent(request: Request):
    user_message = await request.json()
    prompt = user_message.get('prompt')

    async def generate():
        async for event in agent.stream_async(prompt):
            if "data" in event:
                yield event["data"].encode('utf-8')
            if "data" in event and "delta" in event:
                yield event['delta']['text'].encode('utf-8')
            if "data" in event:
                yield event["data"]
            if "event" in event and "contentBlockDelta" in event["event"] and "delta" in event["event"]["contentBlockDelta"] and "text" in event["event"]["contentBlockDelta"]["delta"]:
                yield event["event"]["contentBlockDelta"]["delta"]["text"]
    return StreamingResponse(generate(), media_type="text/plain", headers={"transfer-encoding": "chunked"})

@app.get("/ping")
async def ping():
    return {"status": "healthy"}

Then using this curl command:
curl --no-buffer http://127.0.0.1:1234/invocations -H 'Content-Type: application/json' --data '{"prompt": "hello"}'

Can you let me know if this works for you? And if not, can you give some more information about the environment you are running this code in?
You must be logged in to vote
0 replies
Comment options

Kos-M
Sep 19, 2025
Author

Thanks @Unshure for the feedback ,

It seems that i had enabled cors midleware in the app , and causes that behavior.
Wondering if all middlewares brokes the streaming.
There is a similar discussion in fastapi repo ,

fastapi/fastapi#10701
You must be logged in to vote
0 replies
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
🙏
Q&A
Labels
None yet
2 participants
@Kos-M @Unshure
Morty Proxy This is a proxified and sanitized view of the page, visit original site.