File tree Expand file tree Collapse file tree 1 file changed +25
-3
lines changed
Filter options
Expand file tree Collapse file tree 1 file changed +25
-3
lines changed
Original file line number Diff line number Diff line change @@ -15,7 +15,7 @@ This package provides:
15
15
- OpenAI-like API
16
16
- LangChain compatibility
17
17
18
- ## Installation
18
+ ## Installation from PyPI (recommended)
19
19
20
20
Install from PyPI (requires a c compiler):
21
21
@@ -26,8 +26,30 @@ pip install llama-cpp-python
26
26
The above command will attempt to install the package and build build ` llama.cpp ` from source.
27
27
This is the recommended installation method as it ensures that ` llama.cpp ` is built with the available optimizations for your system.
28
28
29
- This method defaults to using ` make ` to build ` llama.cpp ` on Linux / MacOS and ` cmake ` on Windows.
30
- You can force the use of ` cmake ` on Linux / MacOS setting the ` FORCE_CMAKE=1 ` environment variable before installing.
29
+
30
+ ### Installation with OpenBLAS / cuBLAS / CLBlast
31
+
32
+ ` llama.cpp ` supports multiple BLAS backends for faster processing.
33
+ Use the ` FORCE_CMAKE=1 ` environment variable to force the use of ` cmake ` and install the pip package for the desired BLAS backend.
34
+
35
+ To install with OpenBLAS, set the ` LLAMA_OPENBLAS=1 ` environment variable before installing:
36
+
37
+ ``` bash
38
+ LLAMA_OPENBLAS=1 FORCE_CMAKE=1 pip install llama-cpp-python
39
+ ```
40
+
41
+ To install with cuBLAS, set the ` LLAMA_CUBLAS=1 ` environment variable before installing:
42
+
43
+ ``` bash
44
+ LLAMA_CUBLAS=1 FORCE_CMAKE=1 pip install llama-cpp-python
45
+ ```
46
+
47
+ To install with CLBlast, set the ` LLAMA_CLBLAST=1 ` environment variable before installing:
48
+
49
+ ``` bash
50
+ LLAMA_CLBLAST=1 FORCE_CMAKE=1 pip install llama-cpp-python
51
+ ```
52
+
31
53
32
54
## High-level API
33
55
You can’t perform that action at this time.
0 commit comments