.\cmd_windows.bat
(F:\oobabooga_windows\installer_files\env) F:\oobabooga_windows>pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir
Collecting llama-cpp-python
  Downloading llama-cpp-python-0.1.60.tar.gz (1.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 516.0 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting typing-extensions>=4.5.0
  Downloading typing_extensions-4.6.3-py3-none-any.whl (31 kB)
Collecting numpy>=1.20.0
  Downloading numpy-1.24.3-cp310-cp310-win_amd64.whl (14.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.8/14.8 MB 1.2 MB/s eta 0:00:00
Collecting diskcache>=5.6.1
  Downloading diskcache-5.6.1-py3-none-any.whl (45 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.6/45.6 kB 2.2 MB/s eta 0:00:00
Building wheels for collected packages: llama-cpp-python
  Building wheel for llama-cpp-python (pyproject.toml) ... done
  Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.60-cp310-cp310-win_amd64.whl size=68815 sha256=3963c2f660e18df2a5e72cf2295cffd4d0c632accea654a06b31ee96cf6c5c52
  Stored in directory: F:\oobabooga_windows\installer_files\pip-ephem-wheel-cache-1a8td5kq\wheels\eb\a7\7e\e2f6aaef90347cd520e86d37bf5c613d1b96eeab4512dc080a
Successfully built llama-cpp-python
Installing collected packages: typing-extensions, numpy, diskcache, llama-cpp-python
  Attempting uninstall: typing-extensions
    Found existing installation: typing_extensions 4.5.0
    Uninstalling typing_extensions-4.5.0:
      Successfully uninstalled typing_extensions-4.5.0
  Attempting uninstall: numpy
    Found existing installation: numpy 1.24.3
    Uninstalling numpy-1.24.3:
      Successfully uninstalled numpy-1.24.3
  Attempting uninstall: diskcache
    Found existing installation: diskcache 5.6.1
    Uninstalling diskcache-5.6.1:
      Successfully uninstalled diskcache-5.6.1
  Attempting uninstall: llama-cpp-python
    Found existing installation: llama-cpp-python 0.1.57
    Uninstalling llama-cpp-python-0.1.57:
      Successfully uninstalled llama-cpp-python-0.1.57
Successfully installed diskcache-5.6.1 llama-cpp-python-0.1.60 numpy-1.24.3 typing-extensions-4.6.3

PS F:\oobabooga_windows> .\start_windows.bat
bin F:\oobabooga_windows\installer_files\env\lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll
INFO:Loading settings from settings.yaml...
The following models are available:

1. facebook_opt-350m
2. gpt4-x-alpaca-native-13B-ggml-q5_1.bin
3. guanaco-13B.ggmlv3.q5_1.bin
4. Manticore-13B-Chat-Pyg-Guanaco-GGML-q4_0.bin
5. Manticore-13B.ggmlv3.q5_1.bin
6. Manticore-13B.ggmlv3.q5_K_M.bin
7. pygmalion-13b-ggml-q5_1.bin
8. Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_1.bin
9. WizardLM-13B-1.0.ggmlv3.q5_1.bin
10. WizardLM-Uncensored-SuperCOT-Storytelling.ggmlv3.q4_0.bin

Which one do you want to load? 1-10

3

INFO:Loading guanaco-13B.ggmlv3.q5_1.bin...
Traceback (most recent call last):
  File "F:\oobabooga_windows\text-generation-webui\server.py", line 1079, in <module>
    shared.model, shared.tokenizer = load_model(shared.model_name)
  File "F:\oobabooga_windows\text-generation-webui\modules\models.py", line 94, in load_model
    output = load_func(model_name)
  File "F:\oobabooga_windows\text-generation-webui\modules\models.py", line 262, in llamacpp_loader
    from modules.llamacpp_model import LlamaCppModel
  File "F:\oobabooga_windows\text-generation-webui\modules\llamacpp_model.py", line 11, in <module>
    from llama_cpp import Llama, LlamaCache
  File "F:\oobabooga_windows\installer_files\env\lib\site-packages\llama_cpp\__init__.py", line 1, in <module>
    from .llama_cpp import *
  File "F:\oobabooga_windows\installer_files\env\lib\site-packages\llama_cpp\llama_cpp.py", line 77, in <module>
    _lib = _load_shared_library(_lib_base_name)
  File "F:\oobabooga_windows\installer_files\env\lib\site-packages\llama_cpp\llama_cpp.py", line 68, in _load_shared_library
    raise FileNotFoundError(
FileNotFoundError: Shared library with base name 'llama' not found

Done!
Press any key to continue . . .
PS F:\oobabooga_windows>