📄 Technical Report | 📖Project Page ｜🤗 Hugging Face｜ 🤖 ModelScope

🌐 Ming-UniVision is a groundbreaking multimodal large language model (MLLM) that unifies vision understanding, generation, and editing within a single autoregressive next-token prediction (NTP) framework, powered by MingTok — the first continuous, unified visual tokenizer. By eliminating discrete quantization and leveraging a shared continuous latent space, Ming-UniVision enables seamless, end-to-end multimodal reasoning across diverse tasks. Trained on high-fidelity continuous visual representations, Ming-UniVision supports multi-round, in-context vision-language interactions, such as iterative question answering, image generation, and semantic editing — all without needing to decode intermediate states into pixels. This enables efficient, coherent, and human-like multimodal dialogue with consistent feature dynamics throughout.

• 🌐 First NTP MLLM with Continuous Unified Vision Representations: Ming-UniVision unifies vision and language via next-token prediction using continuous visual tokens — no discrete quantization, full autoregressive generative paradigm, and support for both understanding and generation in a shared latent space.
• 🖼️ First Continuous Unified Visual Tokenizer: MingTok-Vision enables both understanding and generation in a single continuous space, preserving semantic and perceptual quality.
• ⚡ 3.5× Faster Training Convergence: Shared representation reduces conflict between tasks, enabling faster, more stable joint training.
• 🔄 Multi-Round In-Context Vision Tasks: Perform iterative reasoning, generation, and editing in one latent space — no image decoding needed mid-process.
• 🔗 Single Space, Unified Workflow: All modalities and tasks share one coherent feature space — simpler training, efficient inference, true autoregressive fusion.

• [2025.10.09] 📄 Technical Report Released!
  The full technical report is now available on arXiv:
  👉 arXiv:2510.06590 | Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizer
  Dive into the architecture, unified continuous tokenizer, and end-to-end autoregressive framework that power our system.

• [2025.10.02] 🔥 We’re live!
  We’re thrilled to announce the release of Ming-UniVision and MingTok-Vision — the first joint autoregressive vision-language system with unified continuous visual tokenization!

  ✨ Enable seamless multimodal reasoning, generation, and editing in a single latent space.
  🚀 Faster training, richer semantics, and true end-to-end autoregression — no quantization, no compromises.

  👉 Check out our blog post to learn how we’re redefining unified vision-language intelligence.

MingTok-Vision achieves strong image reconstruction capability and Ming-UniVision enables unified multimodal understanding and generation within a single continuous latent space.

MingTok-Vision achieves competitive reconstruction quality with high PSNR and low rFID, demonstrating its ability to preserve both perceptual fidelity and semantic structure in a continuous representation.

Ming-UniVision achieves competitive performance on multimodal understanding benchmarks, showing that continuous latent tokens can effectively support high-level vision-language reasoning without discrete quantization.

Ming-UniVision achieves top performance among unified representation models in text-to-image generation, demonstrating superior object composition and spatial reasoning capabilities.

🔗 Both models are publicly available for research. Visit the respective pages for model details, inference examples, and integration guides.

First, clone the repository and install the required dependencies:

git clone https://github.com/inclusionAI/Ming-UniVision.git
cd Ming-UniVision
conda create -n ming python=3.10 -y
conda activate ming
pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu128
pip install flash_attn
modelscope download --model inclusionAI/Ming-UniVision-16B-A3B --local_dir ./models/Ming-UniVision-16B-A3B
modelscope download --model inclusionAI/MingTok-Vision --local_dir ./models/MingTok-Vision

# >=44GB VRAM
python app.py

# >=22GB VRAM
python app.py --dtype int8

# >=14GB VRAM
python app.py --dtype int4

Here's a simple demo to reconstruct an image using the continuous latent space of MingTok-Vision:

# test_infer_recon_image.py
import torch
from PIL import Image
import torchvision.transforms as T
from omegaconf import OmegaConf
from mingtok.modeling_mingtok import MingTok
from mingtok.utils import CenterCropProcessor

if __name__ == "__main__":
    # Load model
    mingtok_model = MingTok.from_pretrained("./models/MingTok-Vision")
    mingtok_model = mingtok_model.cuda()

    img_path = "mingtok/asset/mingtok.png"
    save_path = "mingtok/asset/mingtok_recon.png"

    # Load and preprocess image
    image = Image.open(img_path).convert("RGB")
    processor = CenterCropProcessor(image_size=512, mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5])
    image = processor(image).cuda().unsqueeze(0)

    # Reconstruct
    out = mingtok_model.forward_enc_dec(image)

    # Denormalize and save
    output_mean = torch.Tensor([0.5, 0.5, 0.5]).view(1, -1, 1, 1).cuda()
    output_std = torch.Tensor([0.5, 0.5, 0.5]).view(1, -1, 1, 1).cuda()
    output_image = (out * output_std + output_mean)[0]
    output_image = T.ToPILImage()(output_image)
    output_image.save(save_path)
    print(f"Reconstructed image saved to {save_path}")

Use MingUniVisionInfer for unified tasks including image generation, understanding, as well as single- and multi-round editing.

from mingunivisioninfer import MingUniVisionInfer
model = MingUniVisionInfer("./models/Ming-UniVision-16B-A3B")

# single round generation
image_gen_prompt = "Please generate the corresponding image based on the description. A cute girl."
messages = [{
  "role": "HUMAN",
  "content": [{"type": "text", "text": image_gen_prompt},],
}]
output_text = model.generate(messages, max_new_tokens=512, output_image_prefix="a_cute_girl")
model.reset_inner_state()

# single ground understanding
messages = [{
  "role": "HUMAN",
  "content": [
    {"type": "image", "image": "a_cute_girl.png"},
    {"type": "text", "text": "Please describe the picture in detail."},
  ],
}]
output_text = model.generate(messages, max_new_tokens=512)
print(output_text)
model.reset_inner_state()

# multi-round editing
messages = [{
  "role": "HUMAN",
  "content": [
    {"type": "image", "image": "a_cute_girl.png"},
    {"type": "text", "text": "Given the edit instruction: Change the color of her cloth to red, please identify the editing region"},
  ],
}]
output_text = model.generate(messages, max_new_tokens=512, for_edit=True, output_image_prefix="edit_round_0")

messages = [{
  "role": "HUMAN",
  "content": [
    {"type": "text", "text": "Change the color of her cloth to red."},
  ],
}]
output_text = model.generate(messages, max_new_tokens=512, for_edit=True, output_image_prefix="edit_round_1")

messages = [{
  "role": "HUMAN",
  "content": [
    {"type": "text", "text": "Refine the image for better clarity."},
  ],
}]
output_text = model.generate(messages, max_new_tokens=512, for_edit=True, output_image_prefix="edit_round_2")

model.reset_inner_state()

# single round text-based conversation
messages = [{
  "role": "HUMAN",
  "content": [
    {"type": "text", "text": "请详细介绍鹦鹉的习性。"},
  ],
}]

output_text = model.generate(messages, max_new_tokens=512)
print(output_text)
model.reset_inner_state()

📌 Tips:

• Image generation: Use descriptive prompts + output_image_prefix to save output.
• Image understanding: Include "image" and "text" in the same message.
• Image editing: Chain multiple generate(..., for_edit=True) calls with unique output_image_prefix names.
• Multi-turn interactions are supported via internal state — call model.reset_inner_state() to reset.
• Input types: "text" and "image" — flexible order, mixed inputs allowed.

📝 Note (Model Limitations):

• The current model was trained with only two-turn conversations, and has not been optimized for alternating rounds of image understanding and generation, although it may generalize to more than two turns during inference. As a result, performance may be limited in complex, multi-modal dialogue scenarios requiring deep contextual reasoning across turns.
• This open-sourced version was trained using mixed-resolution strategies: high resolution for image understanding, but lower resolution for image editing and generation. Additionally, large-scale interleaved image-text data was not included during pretraining.
• Due to these factors, image editing quality and consistency may be suboptimal compared to fully end-to-end, high-resolution multimodal models. We are actively working on improved versions with unified resolution training and richer interleaved data.

Note: We test the examples on hardware of NVIDIA H800-80GB/H20-96G with CUDA 12.4.

If you find our work useful in your research or applications, please consider citing:

@article{huang2025mingunivision,
  title={Ming-UniVision: Joint Image Understanding and Generation with a Unified Continuous Tokenizer},
  author={Huang, Ziyuan and Zheng, DanDan and Zou, Cheng and Liu, Rui and Wang, Xiaolong and Ji, Kaixiang and Chai, Weilong and Sun, Jianxin and Wang, Libin and Lv, Yongjie and Huang, Taozhi and Liu, Jiajia and Guo, Qingpei and Yang, Ming and Chen, Jingdong and Zhou, Jun},
  journal={arXiv preprint arXiv:2510.06590},
  year={2025}
}