You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Then when you run the server you'll need to also specify the `functionary` chat_format
81
+
Then when you run the server you'll need to also specify either `functionary-v1` or `functionary-v2` chat_format.
82
+
83
+
Note that since functionary requires a HF Tokenizer due to discrepancies between llama.cpp and HuggingFace's tokenizers as mentioned [here](https://github.com/abetlen/llama-cpp-python/blob/main?tab=readme-ov-file#function-calling), you will need to pass in the path to the tokenizer too. The tokenizer files are already included in the respective HF repositories hosting the gguf files.
Check out this [example notebook](https://github.com/abetlen/llama-cpp-python/blob/main/examples/notebooks/Functions.ipynb) for a walkthrough of some interesting use cases for function calling.
Copy file name to clipboardExpand all lines: examples/notebooks/Functions.ipynb
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -9,7 +9,7 @@
9
9
"The OpenAI compatbile web server in `llama-cpp-python` supports function calling.\n",
10
10
"\n",
11
11
"Function calling allows API clients to specify a schema that gives the model a format it should respond in.\n",
12
-
"Function calling in `llama-cpp-python` works by combining models pretrained for function calling such as [`functionary`](https://huggingface.co/abetlen/functionary-7b-v1-GGUF) with constrained sampling to produce a response that is compatible with the schema.\n",
12
+
"Function calling in `llama-cpp-python` works by combining models pretrained for function calling such as [`functionary`](https://huggingface.co/meetkai) with constrained sampling to produce a response that is compatible with the schema.\n",
13
13
"\n",
14
14
"Note however that this improves but does not guarantee that the response will be compatible with the schema.\n",
0 commit comments