Search results

Model deployment error

Status: Open.

#242 In ServerlessLLM/ServerlessLLM;

· bipinbohara opened on Apr 22, 2025

Issue while saving and deploying a model

Status: Open.

#237 In ServerlessLLM/ServerlessLLM;

· bipinbohara opened on Apr 6, 2025

[BUG] Qwen/Qwen2.5-7B-Instruct generate nothing when using sllm-store

BugSomething isn't workingSomething isn't working

Status: Open.

#220 In ServerlessLLM/ServerlessLLM;

· dblate opened on Mar 3, 2025

[BUG] <code>lm_head</code> parameter is loaded as a weight when transformers doesn't

Status: Open.

#217 In ServerlessLLM/ServerlessLLM;

· dalongbao opened on Feb 17, 2025

[Feature Request] Add shared pinned memory pool for offloading enabled frameworks

Status: Open.

#210 In ServerlessLLM/ServerlessLLM;

· drunkcoding opened on Feb 12, 2025

[BUG] dtype of loaded model is <code>torch.bfloat16</code> instead of <code>torch.float16</code> like <code>transformers</code> for <code>Qwen/Qwen2.5-1.5B</code>

Status: Open.

#209 In ServerlessLLM/ServerlessLLM;

· dalongbao opened on Feb 11, 2025

[Feature Request] support stream chat

New feature

Priority 1Medium priorityMedium priority

Status: Open.

#176 In ServerlessLLM/ServerlessLLM;

· dblate opened on Dec 3, 2024

[Feature Request] [Enhancement] Merge <code>sllm-cli</code> and <code>sllm-serve</code> into a Unified Interface

New feature

Priority 1Medium priorityMedium priority

Project health

Status: Open.

#170 In ServerlessLLM/ServerlessLLM;

· future-xy opened on Nov 29, 2024

[BUG] Does not support using multiple GPUs in current vLLM patch

Status: Open.

#157 In ServerlessLLM/ServerlessLLM;

· attteegood opened on Nov 16, 2024

[BUG] Does not support multi-GPU in vLLM

Status: Open.

#156 In ServerlessLLM/ServerlessLLM;

· hoaaosnw opened on Nov 15, 2024

[Feature Request] AI-based spell check

Good first issueGood for newcomersGood for newcomers

Priority 2Low priorityLow priority

Project health

Status: Open.

#151 In ServerlessLLM/ServerlessLLM;

· future-xy opened on Nov 12, 2024

[Feature Request] Does sllm support multi-node inference?

QuestionFurther information is requestedFurther information is requested

Status: Open.

#133 In ServerlessLLM/ServerlessLLM;

· Timenumber opened on Oct 31, 2024