Not unload a model from VRAM anymore. #434

Closed

Closed

Not unload a model from VRAM anymore.#434

The last version 0.1.66, not unload model from VRAM.
I thought the problem was with oobabooga/text-generation-webui oobabooga/text-generation-webui#2920, but after digging into the code from projects, I think the problem is with this library.
I think there

llama-cpp-python/llama_cpp/llama.py

Line 1439 in 442213b

def __del__(self):

should be added a new llama_free_model function.

Sorry if I'm wrong, I don't work with python.

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests