Neural Magic
Neural Magic (Acquired by Red Hat) empowers developers to optimize & deploy LLMs at scale. Our model compression & acceleration enable top performance with vLLM
Pinned Loading
Repositories
Showing 10 of 99 repositories
- gorilla Public Forked from ShishirPatil/gorilla
Gorilla: Training and Evaluating LLMs for Function Calls (Tool Calls)
- vllm Public Forked from vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
- nm-vllm-omni-ent Public Forked from vllm-project/vllm-omni
A framework for efficient model inference with omni-modality models
- model-validation-configs Public
- DeepGEMM Public Forked from deepseek-ai/DeepGEMM
DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling
- tokenspeed Public Forked from lightseekorg/tokenspeed
TokenSpeed is a speed-of-light LLM inference engine.
- speculators Public Forked from vllm-project/speculators
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
Top languages
Loading…
Most used topics
Loading…