Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

FEAT: [model] Support Qwen3-VL - #4112

#4112
Merged
qinxuye merged 5 commits into
xorbitsai:mainxorbitsai/inference:mainfrom
Jun-Howie:Qwen3-VLJun-Howie/inference:Qwen3-VLCopy head branch name to clipboard
Sep 30, 2025
Merged

FEAT: [model] Support Qwen3-VL#4112
qinxuye merged 5 commits into
xorbitsai:mainxorbitsai/inference:mainfrom
Jun-Howie:Qwen3-VLJun-Howie/inference:Qwen3-VLCopy head branch name to clipboard

Conversation

@Jun-Howie

@Jun-Howie Jun-Howie commented Sep 29, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

What's New

  1. Model Support

    • Added support for Qwen3-VL Instruct/Thinking models.

  2. Quantized Models

    • Provided additional FP8 and AWQ quantized models.
    • Fully tested and supported.

  3. Tool Calling

    • Added support for tool_call, consistent with the Qwen3 series.

  4. Frontend UI

    • Introduced the --enable-expert-parallel parameter.
    • Aligned with vllm/core.py, enabling expert parallelism so that each GPU is allocated the same number of experts.

Dependencies (vLLM Backend)

pip install git+https://github.com/huggingface/transformers
pip install qwen-vl-utils==0.0.14

# Recommended (if the stable version is not working, use nightly)
# pip install 'vllm>0.10.2' 
uv pip install -U vllm \
    --torch-backend=auto \
    --extra-index-url https://wheels.vllm.ai/nightly

Usage Examples

2 × A100-80G (AWQ)

vllm serve \
    ./Qwen3-VL-235B-A22B-Instruct-AWQ \
    --enable-expert-parallel \
    --max-model-len 32768 \
    --tensor-parallel-size 2

4 × A100-80G (FP8)

vllm serve \
   ./Qwen3-VL-235B-A22B-Instruct-FP8 \
   --enable-expert-parallel \
   --max-model-len 32768 \
   --tensor-parallel-size 4

Test Results

544a39cc26a19af63cafcd550f2a9455 79eb8973fc6200a325f6b3a755ec03e2

@XprobeBot XprobeBot added the gpu label Sep 29, 2025
@XprobeBot XprobeBot added this to the v1.x milestone Sep 29, 2025
@qinxuye qinxuye changed the title Support Qwen3-VL FEAT: [model] Support Qwen3-VL Sep 29, 2025
qinxuye
qinxuye approved these changes Sep 30, 2025
View reviewed changes

@qinxuye qinxuye left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@qinxuye
qinxuye merged commit bc3b42c into xorbitsai:main Sep 30, 2025
4 of 13 checks passed
@Jun-Howie
Jun-Howie deleted the Qwen3-VL branch October 9, 2025 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@qinxuye qinxuye qinxuye approved these changes

Assignees

No one assigned

Labels

feature gpu

Projects

None yet

Milestone

v1.x

Development

Successfully merging this pull request may close these issues.

3 participants

@Jun-Howie @qinxuye @XprobeBot
Morty Proxy This is a proxified and sanitized view of the page, visit original site.