Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Conversation

@e-c-d
Copy link
Contributor

@e-c-d e-c-d commented Dec 18, 2023

Using the functions llama_kv_cache_view_update and llama_kv_cache_view_free causes a crash. The root cause is a discrepancy between the ctypes definition and the actual definition of the C functions. For example,

# LLAMA_API void llama_kv_cache_view_free(struct llama_kv_cache_view * view);         # <--- This is a pointer to struct
def llama_kv_cache_view_free(view: llama_kv_cache_view):
    """Free a KV cache view. (use only for debugging purposes)"""
    return _lib.llama_kv_cache_view_free(view)


_lib.llama_kv_cache_view_free.argtypes = [llama_kv_cache_view]         # <--- This is the struct itself

@abetlen abetlen merged commit 2993936 into abetlen:main Dec 18, 2023
@abetlen
Copy link
Owner

abetlen commented Dec 18, 2023

@e-c-d thank you!

@e-c-d e-c-d deleted the bugfix/kv-cache-view-pointer-types branch December 19, 2023 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.