Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 5d756de

Browse filesBrowse files
abetlenabetlen
authored
Merge branch 'main' into add_unlimited_max_tokens
2 parents 90e1021 + 236c4cf commit 5d756de
Copy full SHA for 5d756de

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Dismiss banner
Expand file treeCollapse file tree

50 files changed

+5063
-1174
lines changed

‎.dockerignore

Copy file name to clipboard
+166Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
_skbuild/
2+
3+
.envrc
4+
5+
models/
6+
7+
# Byte-compiled / optimized / DLL files
8+
__pycache__/
9+
*.py[cod]
10+
*$py.class
11+
12+
# C extensions
13+
*.so
14+
15+
# Distribution / packaging
16+
.Python
17+
build/
18+
develop-eggs/
19+
dist/
20+
downloads/
21+
eggs/
22+
.eggs/
23+
lib/
24+
lib64/
25+
parts/
26+
sdist/
27+
var/
28+
wheels/
29+
share/python-wheels/
30+
*.egg-info/
31+
.installed.cfg
32+
*.egg
33+
MANIFEST
34+
35+
# PyInstaller
36+
# Usually these files are written by a python script from a template
37+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
38+
*.manifest
39+
*.spec
40+
41+
# Installer logs
42+
pip-log.txt
43+
pip-delete-this-directory.txt
44+
45+
# Unit test / coverage reports
46+
htmlcov/
47+
.tox/
48+
.nox/
49+
.coverage
50+
.coverage.*
51+
.cache
52+
nosetests.xml
53+
coverage.xml
54+
*.cover
55+
*.py,cover
56+
.hypothesis/
57+
.pytest_cache/
58+
cover/
59+
60+
# Translations
61+
*.mo
62+
*.pot
63+
64+
# Django stuff:
65+
*.log
66+
local_settings.py
67+
db.sqlite3
68+
db.sqlite3-journal
69+
70+
# Flask stuff:
71+
instance/
72+
.webassets-cache
73+
74+
# Scrapy stuff:
75+
.scrapy
76+
77+
# Sphinx documentation
78+
docs/_build/
79+
80+
# PyBuilder
81+
.pybuilder/
82+
target/
83+
84+
# Jupyter Notebook
85+
.ipynb_checkpoints
86+
87+
# IPython
88+
profile_default/
89+
ipython_config.py
90+
91+
# pyenv
92+
# For a library or package, you might want to ignore these files since the code is
93+
# intended to run in multiple environments; otherwise, check them in:
94+
# .python-version
95+
96+
# pipenv
97+
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
98+
# However, in case of collaboration, if having platform-specific dependencies or dependencies
99+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
100+
# install all needed dependencies.
101+
#Pipfile.lock
102+
103+
# poetry
104+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
105+
# This is especially recommended for binary packages to ensure reproducibility, and is more
106+
# commonly ignored for libraries.
107+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
108+
#poetry.lock
109+
110+
# pdm
111+
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
112+
#pdm.lock
113+
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
114+
# in version control.
115+
# https://pdm.fming.dev/#use-with-ide
116+
.pdm.toml
117+
118+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
119+
__pypackages__/
120+
121+
# Celery stuff
122+
celerybeat-schedule
123+
celerybeat.pid
124+
125+
# SageMath parsed files
126+
*.sage.py
127+
128+
# Environments
129+
.env
130+
.venv
131+
env/
132+
venv/
133+
ENV/
134+
env.bak/
135+
venv.bak/
136+
137+
# Spyder project settings
138+
.spyderproject
139+
.spyproject
140+
141+
# Rope project settings
142+
.ropeproject
143+
144+
# mkdocs documentation
145+
/site
146+
147+
# mypy
148+
.mypy_cache/
149+
.dmypy.json
150+
dmypy.json
151+
152+
# Pyre type checker
153+
.pyre/
154+
155+
# pytype static type analyzer
156+
.pytype/
157+
158+
# Cython debug symbols
159+
cython_debug/
160+
161+
# PyCharm
162+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
163+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
164+
# and can be added to the global gitignore or merged into this file. For a more nuclear
165+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
166+
.idea/

‎.github/ISSUE_TEMPLATE/bug_report.md

Copy file name to clipboard
+96Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
---
2+
name: Bug report
3+
about: Create a report to help us improve
4+
title: ''
5+
labels: ''
6+
assignees: ''
7+
8+
---
9+
10+
# Prerequisites
11+
12+
Please answer the following questions for yourself before submitting an issue.
13+
14+
- [ ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
15+
- [ ] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
16+
- [ ] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
17+
- [ ] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.
18+
19+
# Expected Behavior
20+
21+
Please provide a detailed written description of what you were trying to do, and what you expected `llama-cpp-python` to do.
22+
23+
# Current Behavior
24+
25+
Please provide a detailed written description of what `llama-cpp-python` did, instead.
26+
27+
# Environment and Context
28+
29+
Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.
30+
31+
* Physical (or virtual) hardware you are using, e.g. for Linux:
32+
33+
`$ lscpu`
34+
35+
* Operating System, e.g. for Linux:
36+
37+
`$ uname -a`
38+
39+
* SDK version, e.g. for Linux:
40+
41+
```
42+
$ python3 --version
43+
$ make --version
44+
$ g++ --version
45+
```
46+
47+
# Failure Information (for bugs)
48+
49+
Please help provide information about the failure if this is a bug. If it is not a bug, please remove the rest of this template.
50+
51+
# Steps to Reproduce
52+
53+
Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.
54+
55+
1. step 1
56+
2. step 2
57+
3. step 3
58+
4. etc.
59+
60+
**Note: Many issues seem to be regarding functional or performance issues / differences with `llama.cpp`. In these cases we need to confirm that you're comparing against the version of `llama.cpp` that was built with your python package, and which parameters you're passing to the context.**
61+
62+
Try the following:
63+
64+
1. `git clone https://github.com/abetlen/llama-cpp-python`
65+
2. `cd llama-cpp-python`
66+
3. `rm -rf _skbuild/` # delete any old builds
67+
4. `python setup.py develop`
68+
5. `cd ./vendor/llama.cpp`
69+
6. Follow [llama.cpp's instructions](https://github.com/ggerganov/llama.cpp#build) to `cmake` llama.cpp
70+
7. Run llama.cpp's `./main` with the same arguments you previously passed to llama-cpp-python and see if you can reproduce the issue. If you can, [log an issue with llama.cpp](https://github.com/ggerganov/llama.cpp/issues)
71+
72+
# Failure Logs
73+
74+
Please include any relevant log snippets or files. If it works under one configuration but not under another, please provide logs for both configurations and their corresponding outputs so it is easy to see where behavior changes.
75+
76+
Also, please try to **avoid using screenshots** if at all possible. Instead, copy/paste the console output and use [Github's markdown](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax) to cleanly format your logs for easy readability.
77+
78+
Example environment info:
79+
```
80+
llama-cpp-python$ git log | head -1
81+
commit 47b0aa6e957b93dbe2c29d53af16fbae2dd628f2
82+
83+
llama-cpp-python$ python3 --version
84+
Python 3.10.10
85+
86+
llama-cpp-python$ pip list | egrep "uvicorn|fastapi|sse-starlette|numpy"
87+
fastapi 0.95.0
88+
numpy 1.24.3
89+
sse-starlette 1.3.3
90+
uvicorn 0.21.1
91+
92+
llama-cpp-python/vendor/llama.cpp$ git log | head -3
93+
commit 66874d4fbcc7866377246efbcee938e8cc9c7d76
94+
Author: Kerfuffle <44031344+KerfuffleV2@users.noreply.github.com>
95+
Date: Thu May 25 20:18:01 2023 -0600
96+
```

‎.github/ISSUE_TEMPLATE/feature_request.md

Copy file name to clipboard
+20Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
---
2+
name: Feature request
3+
about: Suggest an idea for this project
4+
title: ''
5+
labels: ''
6+
assignees: ''
7+
8+
---
9+
10+
**Is your feature request related to a problem? Please describe.**
11+
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
12+
13+
**Describe the solution you'd like**
14+
A clear and concise description of what you want to happen.
15+
16+
**Describe alternatives you've considered**
17+
A clear and concise description of any alternative solutions or features you've considered.
18+
19+
**Additional context**
20+
Add any other context or screenshots about the feature request here.

‎.github/dependabot.yml

Copy file name to clipboard
+11Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# To get started with Dependabot version updates, you'll need to specify which
2+
# package ecosystems to update and where the package manifests are located.
3+
# Please see the documentation for all configuration options:
4+
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
5+
6+
version: 2
7+
updates:
8+
- package-ecosystem: "pip" # See documentation for possible values
9+
directory: "/" # Location of package manifests
10+
schedule:
11+
interval: "weekly"

‎.github/workflows/build-docker.yaml

Copy file name to clipboard
+39Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
name: Build Docker
2+
3+
on: workflow_dispatch
4+
5+
permissions:
6+
contents: write
7+
packages: write
8+
9+
jobs:
10+
docker:
11+
name: Build and push Docker image
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Checkout
15+
uses: actions/checkout@v3
16+
with:
17+
submodules: "true"
18+
19+
- name: Set up QEMU
20+
uses: docker/setup-qemu-action@v2
21+
22+
- name: Set up Docker Buildx
23+
uses: docker/setup-buildx-action@v2
24+
25+
- name: Login to GitHub Container Registry
26+
uses: docker/login-action@v2
27+
with:
28+
registry: ghcr.io
29+
username: ${{ github.repository_owner }}
30+
password: ${{ secrets.GITHUB_TOKEN }}
31+
32+
- name: Build and push
33+
uses: docker/build-push-action@v4
34+
with:
35+
context: .
36+
push: true # push to registry
37+
pull: true # always fetch the latest base images
38+
platforms: linux/amd64,linux/arm64 # build for both amd64 and arm64
39+
tags: ghcr.io/abetlen/llama-cpp-python:latest

‎.github/workflows/publish.yaml

Copy file name to clipboardExpand all lines: .github/workflows/publish.yaml
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,4 @@ jobs:
2828
# if: startsWith(github.ref, 'refs/tags')
2929
uses: pypa/gh-action-pypi-publish@release/v1
3030
with:
31-
password: ${{ secrets.PYPI_API_TOKEN }}
31+
password: ${{ secrets.PYPI_API_TOKEN }}

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.