llama.cpp OpenAI-compatible server on Vulkan for AMD Strix Halo (gfx1151), GGUF weights pinned to GTT not VRAM. Serves poolside Laguna S 2.1, Gemma 4 and Qwen3.6 GGUFs on the stock Vulkan image, plus an opt-in ROCmFP4 + MTP stack (Ubuntu 26.04 + TheRock ROCm 7.13). Docker Compose, with real measured benchmarks.
docker-compose amd vulkan mtp rocm laguna llama-cpp local-llm llm-inference qwen gguf openai-compatible strix-halo gfx1151 rocmfp4
-
Updated
Jul 24, 2026 - Python