Name and Version

build_info: b8826-30dce2cf2

Operating systems

Linux

GGML backends

Vulkan, CUDA

Hardware

Tested on Vulkan with:
AMD Ryzen 7 3800XT 8-Core Processor + AMD Radeon VII

and tested on CUDA with:
AMD Ryzen 7 3800XT 8-Core Processor + NVIDIA GeForce RTX 3090

Models

Unsloth Gemma4 E4B-it-GGUF (https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF)

Problem description & steps to reproduce

The new OpenAI compatible transcription API (/v1/audio/transcriptions) doesn't seem to work in Router mode.

When I run a API request like this:

curl -v -F response_format=json -F model=Gemma4-E4B-vulkan -F prompt="Transcribe exactly." -F file=@kling-sample.mp3 http://gfxrig01:8081/v1/audio/transcriptions
* Host gfxrig01:8081 was resolved.
* IPv6: ******, ******
* IPv4: ******
*   Trying [******]:8081...
* Connected to gfxrig01 (******) port 8081
> POST /v1/audio/transcriptions HTTP/1.1
> Host: gfxrig01:8081
> User-Agent: curl/8.7.1
> Accept: */*
> Content-Length: 780341
> Content-Type: multipart/form-data; boundary=------------------------VpTOuUqa5y0gLulttzO9ne
> 
* upload completely sent off: 780341 bytes

then curl will never get a response back from llama.cpp and hangs infinitely waiting for a response. And in the llama.cpp log I only see the following:

srv  proxy_reques: proxying request to model Gemma4-E4B-vulkan on port 33655

And as soon as I cancel the curl request the llama.cpp logs shows:

 srv  log_server_r: done request: POST /v1/audio/transcriptions 172.20.0.1 500

When I start the model without using router mode, then it works as expected. Also the same file send via the built-in WebUI works flawlessly. So from my observation only router mode in combination with using the new API is affected.

First Bad Commit

As this is a quite new feature I suspect that this never worked before. So there is no "Bad Commit"

Relevant log output

Logs provided in "Problem Description"