Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 70f629a

Browse filesBrowse files
authored
Update README.md
1 parent eaff7a8 commit 70f629a
Copy full SHA for 70f629a

File tree

Expand file treeCollapse file tree

1 file changed

+13
-6
lines changed
Filter options
Expand file treeCollapse file tree

1 file changed

+13
-6
lines changed

‎docker/README.md

Copy file name to clipboardExpand all lines: docker/README.md
+13-6Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,13 @@
11
# Get model from Hugging Face
22
`python3 ./hug_model.py`
33

4-
You should now have a model in the current directory and model.bin symlinked to it for the subsequent Docker build and copy step. e.g.
4+
You should now have a model in the current directory and `model.bin` symlinked to it for the subsequent Docker build and copy step. e.g.
55
```
66
docker $ ls -lh *.bin
7-
-rw-rw-r-- 1 user user 4.8G May 23 18:30 llama-7b.ggmlv3.q5_1.bin
7+
-rw-rw-r-- 1 user user 4.8G May 23 18:30 <downloaded-model-file>.q5_1.bin
88
lrwxrwxrwx 1 user user 24 May 23 18:30 model.bin -> <downloaded-model-file>.q5_1.bin
99
```
10-
- Note #1: Make sure you have enough disk space to d/l the model. As the model is then copied into the image you will need at least
10+
**Note #1:** Make sure you have enough disk space to d/l the model. As the model is then copied into the image you will need at least
1111
**TWICE** as much disk space as the size of the model:
1212

1313
| Model | Quantized size |
@@ -17,16 +17,23 @@ lrwxrwxrwx 1 user user 24 May 23 18:30 model.bin -> <downloaded-model-file>.q5
1717
| 30B | 25 GB |
1818
| 65B | 50 GB |
1919

20-
- Note #2: If you want to pass or tune additional parameters, customise `./start_server.sh` before running `docker build ...`
20+
**Note #2:** If you want to pass or tune additional parameters, customise `./start_server.sh` before running `docker build ...`
2121

22-
# Use OpenBLAS (No NVidia GPU, defaults to `python:3-slim-bullseye` Docker base image)
22+
# Install Docker Server
23+
24+
**Note #3:** This was tested with Docker running on Linux. If you can get it working on Windows or MacOS, please update this README with a PR!
25+
26+
[Install Docker Engine](https://docs.docker.com/engine/install)
27+
28+
# Use OpenBLAS
29+
No NVidia GPU, defaults to `python:3-slim-bullseye` Docker base image and OpenBlAS:
2330
## Build:
2431
`docker build --build-arg -t openblas .`
2532
## Run:
2633
`docker run --cap-add SYS_RESOURCE -t openblas`
2734

2835
# Use CuBLAS
29-
Requires NVidia GPU and Docker NVidia support (see https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html)
36+
Requires NVidia GPU and Docker NVidia support (see [container-toolkit/install-guide](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html))
3037
## Build:
3138
`docker build --build-arg IMAGE=nvidia/cuda:12.1.1-devel-ubuntu22.04 -t opencuda .`
3239
## Run:

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.