Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 25062ce

Browse filesBrowse files
abetlenabetlen
authored
Merge pull request abetlen#140 from abetlen/Niek/main
Add Dockerfile
2 parents e644e75 + 36c8148 commit 25062ce
Copy full SHA for 25062ce

File tree

Expand file treeCollapse file tree

5 files changed

+223
-1
lines changed
Filter options
Expand file treeCollapse file tree

5 files changed

+223
-1
lines changed

‎.dockerignore

Copy file name to clipboard
+166Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
_skbuild/
2+
3+
.envrc
4+
5+
models/
6+
7+
# Byte-compiled / optimized / DLL files
8+
__pycache__/
9+
*.py[cod]
10+
*$py.class
11+
12+
# C extensions
13+
*.so
14+
15+
# Distribution / packaging
16+
.Python
17+
build/
18+
develop-eggs/
19+
dist/
20+
downloads/
21+
eggs/
22+
.eggs/
23+
lib/
24+
lib64/
25+
parts/
26+
sdist/
27+
var/
28+
wheels/
29+
share/python-wheels/
30+
*.egg-info/
31+
.installed.cfg
32+
*.egg
33+
MANIFEST
34+
35+
# PyInstaller
36+
# Usually these files are written by a python script from a template
37+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
38+
*.manifest
39+
*.spec
40+
41+
# Installer logs
42+
pip-log.txt
43+
pip-delete-this-directory.txt
44+
45+
# Unit test / coverage reports
46+
htmlcov/
47+
.tox/
48+
.nox/
49+
.coverage
50+
.coverage.*
51+
.cache
52+
nosetests.xml
53+
coverage.xml
54+
*.cover
55+
*.py,cover
56+
.hypothesis/
57+
.pytest_cache/
58+
cover/
59+
60+
# Translations
61+
*.mo
62+
*.pot
63+
64+
# Django stuff:
65+
*.log
66+
local_settings.py
67+
db.sqlite3
68+
db.sqlite3-journal
69+
70+
# Flask stuff:
71+
instance/
72+
.webassets-cache
73+
74+
# Scrapy stuff:
75+
.scrapy
76+
77+
# Sphinx documentation
78+
docs/_build/
79+
80+
# PyBuilder
81+
.pybuilder/
82+
target/
83+
84+
# Jupyter Notebook
85+
.ipynb_checkpoints
86+
87+
# IPython
88+
profile_default/
89+
ipython_config.py
90+
91+
# pyenv
92+
# For a library or package, you might want to ignore these files since the code is
93+
# intended to run in multiple environments; otherwise, check them in:
94+
# .python-version
95+
96+
# pipenv
97+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
98+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
99+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
100+
# install all needed dependencies.
101+
#Pipfile.lock
102+
103+
# poetry
104+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
105+
# This is especially recommended for binary packages to ensure reproducibility, and is more
106+
# commonly ignored for libraries.
107+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
108+
#poetry.lock
109+
110+
# pdm
111+
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
112+
#pdm.lock
113+
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
114+
# in version control.
115+
# https://pdm.fming.dev/#use-with-ide
116+
.pdm.toml
117+
118+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
119+
__pypackages__/
120+
121+
# Celery stuff
122+
celerybeat-schedule
123+
celerybeat.pid
124+
125+
# SageMath parsed files
126+
*.sage.py
127+
128+
# Environments
129+
.env
130+
.venv
131+
env/
132+
venv/
133+
ENV/
134+
env.bak/
135+
venv.bak/
136+
137+
# Spyder project settings
138+
.spyderproject
139+
.spyproject
140+
141+
# Rope project settings
142+
.ropeproject
143+
144+
# mkdocs documentation
145+
/site
146+
147+
# mypy
148+
.mypy_cache/
149+
.dmypy.json
150+
dmypy.json
151+
152+
# Pyre type checker
153+
.pyre/
154+
155+
# pytype static type analyzer
156+
.pytype/
157+
158+
# Cython debug symbols
159+
cython_debug/
160+
161+
# PyCharm
162+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
163+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
164+
# and can be added to the global gitignore or merged into this file. For a more nuclear
165+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
166+
.idea/

‎.github/workflows/build-docker.yaml

Copy file name to clipboard
+33Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
name: Build Docker
2+
3+
on: workflow_dispatch
4+
5+
jobs:
6+
docker:
7+
name: Build and push Docker image
8+
runs-on: ubuntu-latest
9+
needs: build-n-publish
10+
steps:
11+
- name: Checkout
12+
uses: actions/checkout@v3
13+
14+
- name: Set up QEMU
15+
uses: docker/setup-qemu-action@v2
16+
17+
- name: Set up Docker Buildx
18+
uses: docker/setup-buildx-action@v2
19+
20+
- name: Login to GitHub Container Registry
21+
uses: docker/login-action@v2
22+
with:
23+
registry: ghcr.io
24+
username: ${{ github.repository_owner }}
25+
password: ${{ secrets.GITHUB_TOKEN }}
26+
27+
- name: Build and push
28+
uses: docker/build-push-action@v4
29+
with:
30+
push: true # push to registry
31+
pull: true # always fetch the latest base images
32+
platforms: linux/amd64,linux/arm64 # build for both amd64 and arm64
33+
tags: ghcr.io/abetlen/llama-cpp-python:latest

‎.github/workflows/publish.yaml

Copy file name to clipboardExpand all lines: .github/workflows/publish.yaml
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,4 @@ jobs:
2828
# if: startsWith(github.ref, 'refs/tags')
2929
uses: pypa/gh-action-pypi-publish@release/v1
3030
with:
31-
password: ${{ secrets.PYPI_API_TOKEN }}
31+
password: ${{ secrets.PYPI_API_TOKEN }}

‎Dockerfile

Copy file name to clipboard
+15Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
FROM python:3-bullseye
2+
3+
# We need to set the host to 0.0.0.0 to allow outside access
4+
ENV HOST 0.0.0.0
5+
6+
COPY . .
7+
8+
# Install the package
9+
RUN apt update && apt install -y libopenblas-dev
10+
RUN python -m pip install --upgrade pip pytest cmake scikit-build setuptools fastapi uvicorn sse-starlette
11+
12+
RUN LLAMA_OPENBLAS=1 python3 setup.py develop
13+
14+
# Run the server
15+
CMD python3 -m llama_cpp.server

‎README.md

Copy file name to clipboardExpand all lines: README.md
+8Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,14 @@ python3 -m llama_cpp.server
7272

7373
Navigate to [http://localhost:8000/docs](http://localhost:8000/docs) to see the OpenAPI documentation.
7474

75+
## Docker image
76+
77+
A Docker image is available on [GHCR](https://ghcr.io/abetlen/llama-cpp-python). To run the server:
78+
79+
```bash
80+
docker run --rm -it -p8000:8000 -v /path/to/models:/models -eMODEL=/models/ggml-model-name.bin ghcr.io/abetlen/llama-cpp-python:latest
81+
```
82+
7583
## Low-level API
7684

7785
The low-level API is a direct `ctypes` binding to the C API provided by `llama.cpp`.

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.