Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit a05d904

Browse filesBrowse files
notwanotwa
authored
fix: Circular dependancy preventing early Llama object free (abetlen#1176)
commit 9018270 introduced a cyclic dependency within Llama objects. That change causes old models to linger in memory longer than necessary, thereby creating memory bloat in most applications attempting to switch between models at runtime. This patch simply removes the problematic line, allowing models to deallocate without relying on GC. One might also consider combining `weakref.ref` with a `@property` if the `llama` attribute is absolutely necessary to expose in the tokenizer class.
1 parent 918ff27 commit a05d904
Copy full SHA for a05d904

File tree

Expand file treeCollapse file tree

1 file changed

+0
-1
lines changed
Filter options
Expand file treeCollapse file tree

1 file changed

+0
-1
lines changed

‎llama_cpp/llama_tokenizer.py

Copy file name to clipboardExpand all lines: llama_cpp/llama_tokenizer.py
-1Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,6 @@ def detokenize(
2727

2828
class LlamaTokenizer(BaseLlamaTokenizer):
2929
def __init__(self, llama: llama_cpp.Llama):
30-
self.llama = llama
3130
self._model = llama._model # type: ignore
3231

3332
def tokenize(

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.