Commit ab028cb
Migrate inference to llama_batch and llama_decode api (abetlen#795)
* Add low-level batching notebook
* fix: tokenization of special characters: (abetlen#850)
It should behave like llama.cpp, where most out of the box usages
treat special characters accordingly
* Update CHANGELOG
* Cleanup
* Fix runner label
* Update notebook
* Use llama_decode and batch api
* Support logits_all parameter
---------
Co-authored-by: Antoine Lizee <antoine.lizee@gmail.com>1 parent f436e0c commit ab028cbCopy full SHA for ab028cb
File tree
Expand file treeCollapse file tree
3 files changed
+753
-8
lines changedOpen diff view settings
Filter options
- examples/notebooks
- llama_cpp
- tests
Expand file treeCollapse file tree
3 files changed
+753
-8
lines changedOpen diff view settings
0 commit comments