Linked issue

None

Type of change

• Bug fix (non-breaking change that fixes an issue)
• Performance improvement (includes benchmark results below)
• Documentation update
• New feature or public API (requires prior proposal or issue approval)
• Refactor / internal cleanup (no user-visible change)
• Build, CI, or tooling change

Motivation

This PR adds a W4A16 quantized matmul path intended to let GPTQ/int4 Llama-style models run on RDNA3 GPUs through MAX.

The specific local validation target is hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4 on rdna3/3.5. Without the RDNA3 W4A16 execution path, RDNA gpus cannot serve quantized gptq/awq models .

What changed

• Added a new Mojo W4A16 quantization kernel implementation and builtin graph compiler entrypoint.
• Added graph op plumbing for the GPTQ/W4A16 dispatch path.
• Added attention-with-RoPE support for the quantized path.
• Added focused graph tests covering the new W4A16 builtin behavior.

Testing

Validated locally on branch rdna3-quantized-model:

./bazelw run //max/python/max/_entrypoints:pipelines -- serve --help

This built successfully and printed the serve CLI help.

Then launched MAX serve with:

./bazelw run //max/python/max/_entrypoints:pipelines -- serve \
  --model hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4 \
  --devices gpu:0 \
  --max-batch-size 8 \
  --device-memory-utilization 0.9 \
  --quantization-encoding gptq \
  --trust-remote-code \
  --pretty-print-config

Confirmed /v1/models returned the expected model ID and max_model_len: 131072.

Sent two OpenAI-compatible chat completion smoke tests:

• arithmetic prompt: What is 17 plus 25? returned The sum of 17 and 25 is 42.
• HTTP 200 explanation prompt returned coherent explanatory text, with finish_reason: "length" only because the smoke test capped max_tokens at 80

Not yet run:

• the focused graph test target directly
• mmlu benchmarks to evalulate model accuracy against a vllm baseline

Checklist

• The linked issue above has been reviewed by a maintainer and is agreed-upon, or this is a trivial fix that does not need prior approval
• PR is small and focused — I've split larger changes into a sequence of smaller PRs where possible
• I ran ./bazelw run format to format my changes
• I added or updated tests to cover my changes
• If AI tools assisted with this contribution, I have included an Assisted-by: trailer in my commit message or this PR description

AI assistance disclosure

Assisted-by: Codex