add KV cache quantization options#1307

Merged

abetlen merged 5 commits intoabetlen:mainfrom

Limour-dev:main

Apr 1, 2024

Commits on Mar 28, 2024

add KV cache quantization options
Show description for 94a9519
Limour-dev
authored
94a9519
Copy full SHA for 94a9519

Commits on Mar 31, 2024

Merge branch 'main' into Limour-dev/main
abetlen
committed
c4a2f85
Copy full SHA for c4a2f85
Add ggml_type
abetlen
committed
1a6a9a3
Copy full SHA for 1a6a9a3
Use ggml_type instead of string for quantization
abetlen
committed
fcb8051
Copy full SHA for fcb8051

Commits on Apr 1, 2024

Add server support
abetlen
committed
7828382
Copy full SHA for 7828382