⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)
-
Updated
Dec 6, 2024 - Python
#
continual-pre-training
Here are 6 public repositories matching this topic...
[ICML 2025] Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
llama data-quality mistral pre-training continual neural-symbolic data-centric-ai llm continual-pre-training
-
Updated
May 14, 2025 - Python
Llama-3-SynE: A Significantly Enhanced Version of Llama-3 with Advanced Scientific Reasoning and Chinese Language Capabilities | 继续预训练提升 Llama-3 的科学推理和中文能力
-
Updated
Dec 17, 2024 - Python
Official repository of "CoMP: Continual Multimodal Pre-training for Vision Foundation Models"
-
Updated
Apr 3, 2025 - Python
A open-source framework designed to adapt pre-trained Language Models (LLMs), such as Llama, Mistral, and Mixtral, to a wide array of domains and languages.
-
Updated
May 27, 2024 - Python
Source code of our paper "Examining forgetting in continual pre-training of aligned large language models"
-
Updated
Nov 30, 2023 - Python
Improve this page
Add a description, image, and links to the continual-pre-training topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the continual-pre-training topic, visit your repo's landing page and select "manage topics."