Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit ab028cb

Browse filesBrowse files
abetlenantoine-lizee
abetlen
and
antoine-lizee
authored
Migrate inference to llama_batch and llama_decode api (abetlen#795)
* Add low-level batching notebook * fix: tokenization of special characters: (abetlen#850) It should behave like llama.cpp, where most out of the box usages treat special characters accordingly * Update CHANGELOG * Cleanup * Fix runner label * Update notebook * Use llama_decode and batch api * Support logits_all parameter --------- Co-authored-by: Antoine Lizee <antoine.lizee@gmail.com>
1 parent f436e0c commit ab028cb
Copy full SHA for ab028cb

File tree

Expand file treeCollapse file tree

3 files changed

+753
-8
lines changed
Filter options
Expand file treeCollapse file tree

3 files changed

+753
-8
lines changed

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.