Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Llama.close() doesn't clean up chat_handler causing null pointer on second load of a vision model #2342

Copy link
Copy link

Description

@WarmneoN
Issue body actions

In ComfyUI's ToriiGate Llama.cpp Vision Generate node, when keep_model_alive is set to false, the first generation works, but the second inference crashes.

Root cause: when unloading the model, Llama.close() does not clean up the chat_handler. On second load, the C++ layer is polluted and mtmd_ctx becomes a null pointer.

Temporary fix: in nodes_api.py _drop_local_llama, before del llm:

ch = getattr(llm, "chat_handler", None)
if ch is not None and hasattr(ch, "_exit_stack"):
    try:
        ch._exit_stack.close()
    except Exception:
        pass

Environment: Python 3.13 · PyTorch 2.9.1+cu130 · ComfyUI v0.28.0 · llama-cpp-python 0.3.34

Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    Morty Proxy This is a proxified and sanitized view of the page, visit original site.