Closed
Description
Name and Version
version: 4713 (a4f011e8)
built with x86_64-conda-linux-gnu-cc (Anaconda gcc) 11.2.0 for x86_64-conda-linux-gnu
I don't know whether it's a bug or not.
The latest Jinja chat template for the DeepSeek r1 model adds a <think>\n
postfix to force the model into thinking.
However, this makes all the responses losing the heading <think>
tag like this:
I suggest manually adding the <think>
prefix in response when add_generation_prompt = true
.
Operating systems
Linux
Which llama.cpp modules do you know to be affected?
libllama (core library)
Command line
numactl --interleave=0-1 ./llama-server -ngl 0 --mlock --no-mmap --numa numactl -t 62 --port 10000 --host 0.0.0.0 -m ../../../DeepSeek-R1-UD-IQ2_XXS/DeepSeek-R1-UD-IQ2_XXS-00001-of-00004.gguf --jinja --chat-template-file ../../models/templates/llama-cpp-deepseek-r1.jinja --reasoning-format deepseek
Problem description & steps to reproduce
- Running
llama-server
- Chatting with
DeepSeek R1
First Bad Commit
No response