Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 1db3b58

Browse filesBrowse files
committed
docs: Add docs explaining how to install pre-built wheels.
1 parent c50309e commit 1db3b58
Copy full SHA for 1db3b58

File tree

Expand file treeCollapse file tree

1 file changed

+45
-0
lines changed
Filter options
Expand file treeCollapse file tree

1 file changed

+45
-0
lines changed

‎README.md

Copy file name to clipboardExpand all lines: README.md
+45Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,15 @@ This will also build `llama.cpp` from source and install it alongside this pytho
4444

4545
If this fails, add `--verbose` to the `pip install` see the full cmake build log.
4646

47+
**Pre-built Wheel (New)**
48+
49+
It is also possible to install a pre-built wheel with basic CPU support.
50+
51+
```bash
52+
pip install llama-cpp-python \
53+
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
54+
```
55+
4756
### Installation Configuration
4857

4958
`llama.cpp` supports a number of hardware acceleration backends to speed up inference as well as backend specific options. See the [llama.cpp README](https://github.com/ggerganov/llama.cpp#build) for a full list.
@@ -108,6 +117,30 @@ To install with cuBLAS, set the `LLAMA_CUDA=on` environment variable before inst
108117
CMAKE_ARGS="-DLLAMA_CUDA=on" pip install llama-cpp-python
109118
```
110119

120+
**Pre-built Wheel (New)**
121+
122+
It is also possible to install a pre-built wheel with CUDA support. As long as your system meets some requirements:
123+
124+
- CUDA Version is 12.1, 12.2 or 12.3
125+
- Python Version is 3.10, 3.11 or 3.12
126+
127+
```bash
128+
pip install llama-cpp-python \
129+
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/<cuda-version>
130+
```
131+
132+
Where `<cuda-version>` is one of the following:
133+
- `cu121`: CUDA 12.1
134+
- `cu122`: CUDA 12.2
135+
- `cu123`: CUDA 12.3
136+
137+
For example, to install the CUDA 12.1 wheel:
138+
139+
```bash
140+
pip install llama-cpp-python \
141+
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121
142+
```
143+
111144
</details>
112145

113146
<details>
@@ -119,6 +152,18 @@ To install with Metal (MPS), set the `LLAMA_METAL=on` environment variable befor
119152
CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python
120153
```
121154

155+
**Pre-built Wheel (New)**
156+
157+
It is also possible to install a pre-built wheel with Metal support. As long as your system meets some requirements:
158+
159+
- MacOS Version is 11.0 or later
160+
- Python Version is 3.10, 3.11 or 3.12
161+
162+
```bash
163+
pip install llama-cpp-python \
164+
--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal
165+
```
166+
122167
</details>
123168
<details>
124169

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.