We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
2 parents 98bbd1c + f712a04 commit 980903dCopy full SHA for 980903d
README.md
@@ -64,12 +64,20 @@ This allows you to use llama.cpp compatible models with any OpenAI compatible cl
64
65
To install the server package and get started:
66
67
+Linux/MacOS
68
```bash
69
pip install llama-cpp-python[server]
70
export MODEL=./models/7B/ggml-model.bin
71
python3 -m llama_cpp.server
72
```
73
74
+Windows
75
+```cmd
76
+pip install llama-cpp-python[server]
77
+SET MODEL=..\models\7B\ggml-model.bin
78
+python3 -m llama_cpp.server
79
+```
80
+
81
Navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to see the OpenAPI documentation.
82
83
## Docker image
0 commit comments