Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit f8fcb3e

Browse filesBrowse files
abetlenabetlen
authored
feat: Update sampling API for llama.cpp (abetlen#1742)
* Initial samplng api update * Fix logger * Update tests * Update * Remove seed * Add sampling chain * Remove unnused test * Use Qwen2 0.5B for ci tests * Fix typo * Fix typo * Update cache version * Use real model for tests * Add huggingface-hub as a test dependency * Remove RUST_LOG=trace * Add actual logit processor test
1 parent a4e1451 commit f8fcb3e
Copy full SHA for f8fcb3e

File tree

Expand file treeCollapse file tree

10 files changed

+1035
-2083
lines changed
Filter options
Expand file treeCollapse file tree

10 files changed

+1035
-2083
lines changed

‎.github/workflows/test.yaml

Copy file name to clipboardExpand all lines: .github/workflows/test.yaml
+54-27Lines changed: 54 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
11
name: Tests
2-
32
on:
43
pull_request:
54
branches:
@@ -8,14 +7,34 @@ on:
87
branches:
98
- main
109

10+
env:
11+
REPO_ID: Qwen/Qwen2-0.5B-Instruct-GGUF
12+
MODEL_FILE: qwen2-0_5b-instruct-q8_0.gguf
13+
1114
jobs:
12-
build-linux:
15+
download-model:
16+
runs-on: ubuntu-latest
17+
steps:
18+
- name: Set up Python
19+
uses: actions/setup-python@v5
20+
with:
21+
python-version: "3.9"
22+
- name: Install huggingface-hub
23+
run: pip install huggingface-hub
24+
- name: Download model
25+
run: huggingface-cli download ${{ env.REPO_ID }} ${{ env.MODEL_FILE }}
26+
- name: Cache model
27+
uses: actions/cache@v4
28+
with:
29+
path: ~/.cache/huggingface/hub
30+
key: ${{ runner.os }}-model-${{ env.REPO_ID }}-${{ env.MODEL_FILE }}
1331

32+
build-linux:
33+
needs: download-model
1434
runs-on: ubuntu-latest
1535
strategy:
1636
matrix:
1737
python-version: ["3.9", "3.10", "3.11", "3.12"]
18-
1938
steps:
2039
- uses: actions/checkout@v4
2140
with:
@@ -26,36 +45,35 @@ jobs:
2645
with:
2746
python-version: ${{ matrix.python-version }}
2847
cache: 'pip'
29-
48+
- name: Restore model cache
49+
uses: actions/cache@v3
50+
with:
51+
path: ~/.cache/huggingface/hub
52+
key: ${{ runner.os }}-model-${{ env.REPO_ID }}-${{ env.MODEL_FILE }}
3053
- name: Install dependencies (Linux/MacOS)
3154
if: runner.os != 'Windows'
3255
run: |
3356
python -m pip install --upgrade pip
3457
python -m pip install uv
35-
RUST_LOG=trace python -m uv pip install -e .[all] --verbose
58+
python -m uv pip install -e .[all] --verbose
3659
shell: bash
37-
3860
- name: Install dependencies (Windows)
3961
if: runner.os == 'Windows'
40-
env:
41-
RUST_LOG: trace
4262
run: |
4363
python -m pip install --upgrade pip
4464
python -m pip install uv
4565
python -m uv pip install -e .[all] --verbose
46-
shell: cmd
47-
66+
shell: cmd
4867
- name: Test with pytest
4968
run: |
5069
python -m pytest
5170
5271
build-windows:
53-
72+
needs: download-model
5473
runs-on: windows-latest
5574
strategy:
5675
matrix:
5776
python-version: ["3.9", "3.10", "3.11", "3.12"]
58-
5977
steps:
6078
- uses: actions/checkout@v4
6179
with:
@@ -66,19 +84,23 @@ jobs:
6684
with:
6785
python-version: ${{ matrix.python-version }}
6886
cache: 'pip'
87+
88+
- name: Restore model cache
89+
uses: actions/cache@v3
90+
with:
91+
path: ~/.cache/huggingface/hub
92+
key: ${{ runner.os }}-model-${{ env.REPO_ID }}-${{ env.MODEL_FILE }}
6993

7094
- name: Install dependencies (Linux/MacOS)
7195
if: runner.os != 'Windows'
7296
run: |
7397
python -m pip install --upgrade pip
7498
python -m pip install uv
75-
RUST_LOG=trace python -m uv pip install -e .[all] --verbose
99+
python -m uv pip install -e .[all] --verbose
76100
shell: bash
77101

78102
- name: Install dependencies (Windows)
79103
if: runner.os == 'Windows'
80-
env:
81-
RUST_LOG: trace
82104
run: |
83105
python -m pip install --upgrade pip
84106
python -m pip install uv
@@ -90,12 +112,11 @@ jobs:
90112
python -m pytest
91113
92114
build-macos:
93-
115+
needs: download-model
94116
runs-on: macos-latest
95117
strategy:
96118
matrix:
97119
python-version: ["3.9", "3.10", "3.11", "3.12"]
98-
99120
steps:
100121
- uses: actions/checkout@v4
101122
with:
@@ -106,34 +127,36 @@ jobs:
106127
with:
107128
python-version: ${{ matrix.python-version }}
108129
cache: 'pip'
130+
131+
- name: Restore model cache
132+
uses: actions/cache@v3
133+
with:
134+
path: ~/.cache/huggingface/hub
135+
key: ${{ runner.os }}-model-${{ env.REPO_ID }}-${{ env.MODEL_FILE }}
109136

110137
- name: Install dependencies (Linux/MacOS)
111138
if: runner.os != 'Windows'
112139
run: |
113140
python -m pip install --upgrade pip
114141
python -m pip install uv
115-
RUST_LOG=trace python -m uv pip install -e .[all] --verbose
142+
python -m uv pip install -e .[all] --verbose
116143
shell: bash
117144

118145
- name: Install dependencies (Windows)
119146
if: runner.os == 'Windows'
120-
env:
121-
RUST_LOG: trace
122147
run: |
123148
python -m pip install --upgrade pip
124149
python -m pip install uv
125150
python -m uv pip install -e .[all] --verbose
126-
shell: cmd
151+
shell: cmd
127152

128153
- name: Test with pytest
129154
run: |
130155
python -m pytest
131156
132-
133157
build-macos-metal:
134-
158+
needs: download-model
135159
runs-on: macos-latest
136-
137160
steps:
138161
- uses: actions/checkout@v4
139162
with:
@@ -144,18 +167,22 @@ jobs:
144167
with:
145168
python-version: "3.9"
146169

170+
- name: Restore model cache
171+
uses: actions/cache@v3
172+
with:
173+
path: ~/.cache/huggingface/hub
174+
key: ${{ runner.os }}-model-${{ env.REPO_ID }}-${{ env.MODEL_FILE }}
175+
147176
- name: Install dependencies (Linux/MacOS)
148177
if: runner.os != 'Windows'
149178
run: |
150179
python -m pip install --upgrade pip
151180
python -m pip install uv
152-
RUST_LOG=trace CMAKE_ARGS="-DLLAMA_METAL=on" python -m uv pip install .[all] --verbose
181+
CMAKE_ARGS="-DLLAMA_METAL=on" python -m uv pip install .[all] --verbose
153182
shell: bash
154183

155184
- name: Install dependencies (Windows)
156185
if: runner.os == 'Windows'
157-
env:
158-
RUST_LOG: trace
159186
run: |
160187
python -m pip install --upgrade pip
161188
CMAKE_ARGS="-DGGML_METAL=on" python -m pip install .[all] --verbose

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.