large-multimodal-models

Star

Here are 86 public repositories matching this topic...

VITA-MLLM / VITA

Star

✨✨[NeurIPS 2025] VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

multimodal-large-language-models large-multimodal-models omni-modal-video-understanding omni-language-model omni-model

Updated Mar 28, 2025
Python

OpenAdaptAI / OpenAdapt

Sponsor

Star

Open Source Generative Process Automation (i.e. Generative RPA). AI-First Process Automation with Large ([Language (LLMs) / Action (LAMs) / Multimodal (LMMs)] / Visual Language (VLMs)) Models

Updated Mar 4, 2026
Python

NVlabs / describe-anything

Star

[ICCV 2025] Implementation for Describe Anything: Detailed Localized Image and Video Captioning

vision-language-model large-multimodal-models describe-anything detailed-localized-captioning

Updated Jun 26, 2025
Python

ShareGPT4Omni / ShareGPT4Video

Star

[NeurIPS 2024] An official implementation of "ShareGPT4Video: Improving Video Understanding and Generation with Better Captions"

gpt sora text-to-video large-language-models chatgpt large-vision-language-models large-multimodal-models gpt-4v large-video-language-models

Updated Oct 9, 2024
Python

TinyLLaVA / TinyLLaVA_Factory

Star

A Framework of Small-scale Large Multimodal Models

nlp transformers llama vision-language llava large-multimodal-models tinyllama

Updated May 7, 2026
Python

richard-peng-xia / awesome-multimodal-in-medical-imaging

Star

A collection of resources on applications of multi-modal learning in medical imaging.

medical-imaging multimodal-learning visual-question-answering multimodal-deep-learning large-language-models medical-report-generation multimodal-large-language-models large-multimodal-models

Updated Feb 8, 2026

LLaVA-VL / LLaVA-Plus-Codebase

Star

LLaVA-Plus: Large Language and Vision Assistants that Plug and Learn to Use Skills

agent tool-use large-language-models multimodal-large-language-models large-multimodal-models

Updated Feb 1, 2024
Python

ictnlp / LLaVA-Mini

Star

LLaVA-Mini is a unified large multimodal model (LMM) that can support the understanding of images, high-resolution images, and videos in an efficient manner.

video efficient vision llama multimodal large-language-models vision-language-model llava visual-instruction-tuning multimodal-large-language-models gpt4v large-multimodal-models gpt4o

Updated Jun 29, 2025
Python

MMMU-Benchmark / MMMU

Star

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

machine-learning natural-language-processing deep-neural-networks computer-vision deep-learning evaluation question-answering stem multimodality multimodal-learning visual-question-answering multimodal multimodal-deep-learning foundation-models large-language-models llm llms large-multimodal-models

Updated Feb 12, 2026
Python

xiaoachen98 / Open-LLaVA-NeXT

Star

An open-source implementation for training LLaVA-NeXT.

chatbot llama multimodal multi-modality gpt-4 visual-language-learning chatgpt vision-language-model llava large-multimodal-models llama3 gpt4o llava-next

Updated Oct 23, 2024
Python

shikiw / OPERA

Star

[CVPR 2024 Highlight] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

chatbot llama multimodal gpt-4 chatgpt vision-language-model vision-language-learning large-multimodal-models

Updated Aug 24, 2024
Python

ictnlp / Stream-Omni

Star

Stream-Omni is a GPT-4o-like language-vision-speech chatbot that simultaneously supports interaction across various modality combinations.

chatbot speech tts speech-synthesis vision speech-recognition question-answering llama speech-to-text interaction asr large-language-models llm chatgpt vision-language-model large-multimodal-models gpt-4o mutlimodal

Updated Jun 17, 2025
Python

zjysteven / lmms-finetune

Star

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.