Skip to content

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

simplify CI and install extras #698

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Jan 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
fda0f56
simplify CI, WIP
kushalkolar Jan 17, 2025
37d87db
tweak
kushalkolar Jan 17, 2025
909c8c0
jupyter rfb v0.5.0 not on pypi yet
kushalkolar Jan 17, 2025
4daf503
tweak
kushalkolar Jan 17, 2025
8dd07db
twaek
kushalkolar Jan 17, 2025
4933a32
ok let's make this look ugly
kushalkolar Jan 17, 2025
765e794
typical bash and sed bleh
kushalkolar Jan 17, 2025
e22dd0a
ok so github really likes it ugly
kushalkolar Jan 17, 2025
e2428b2
needs more sed
kushalkolar Jan 17, 2025
2e1002f
see if core dump fixed
kushalkolar Jan 17, 2025
389c911
imgui and no-imgui matrix for regen screenshots
kushalkolar Jan 17, 2025
c3b346f
update test utils
kushalkolar Jan 17, 2025
b2d6c3b
account for imagewidget which needs imgui
kushalkolar Jan 17, 2025
157bdc7
modify regen
kushalkolar Jan 17, 2025
5c4f0c5
tweak
kushalkolar Jan 17, 2025
4e24945
skip imgui required tests when not installed
kushalkolar Jan 17, 2025
6468150
cleanup
kushalkolar Jan 17, 2025
0facc8e
move check earlier
kushalkolar Jan 17, 2025
0425217
more informative artifact upload names
kushalkolar Jan 17, 2025
80072ae
typo
kushalkolar Jan 17, 2025
0caa458
use upload-artifact v4
kushalkolar Jan 17, 2025
d3f99a1
add no imgui ground truth imageS
kushalkolar Jan 17, 2025
defcbc3
increase jupyter_rfb version
kushalkolar Jan 17, 2025
65e99d7
use bash find to exclude/include imagewidget nbs depending on imgui e…
kushalkolar Jan 18, 2025
ad4143b
test imagewidget gc in nbs only if imgui installed
kushalkolar Jan 18, 2025
8679d89
regen screenshots nb with and w/o imgui
kushalkolar Jan 18, 2025
b295d12
add nb back to regen matrix
kushalkolar Jan 18, 2025
cdc100c
we actually only need 2 variants of regen
kushalkolar Jan 18, 2025
e27e3f3
prefix no-imgui for nb screenshots
kushalkolar Jan 18, 2025
de96a23
add no-imgui-nb screenshots
kushalkolar Jan 18, 2025
6482e27
python version
kushalkolar Jan 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 26 additions & 67 deletions 93 .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,28 +15,24 @@ on:

jobs:
test-build-full:
name: Test Linux, notebook + offscreen
name: Test Linux
runs-on: ubuntu-latest
timeout-minutes: 30
timeout-minutes: 10
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can go back to 10 mins now that we don't have python3.10 which required more time due to no imgui wheels existing for it.

if: ${{ !github.event.pull_request.draft }}
strategy:
fail-fast: false
matrix:
include:
- name: Test py311
pyversion: '3.11'
- name: Test py312
pyversion: '3.12'
- name: Test py313
pyversion: '3.13'
python: ["3.11", "3.12", "3.13"]
imgui_dep: ["imgui", ""]
notebook_dep: ["notebook", ""]
steps:
- uses: actions/checkout@v4
with:
lfs: true
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.pyversion }}
python-version: ${{ matrix.python }}
- name: Install llvmpipe and lavapipe for offscreen canvas
run: |
sudo apt-get update -y -qq
Expand All @@ -47,7 +43,12 @@ jobs:
# remove pygfx from install_requires, we install using pygfx@main
sed -i "/pygfx/d" ./setup.py
pip install git+https://github.com/pygfx/pygfx.git@main
pip install -e ".["tests"]"
- name: Install fastplotlib
run: |
# create string with one of: tests,imgui,notebook; test,imgui; test,notebook ; tests
# sed removes trailing comma
# install fastplotlib with given extras options from above
pip install -e ".[$(echo "tests,${{ matrix.imgui_dep }},${{ matrix.notebook_dep }}" | sed -e "s/,\+/,/g" -e "s/,$//")]"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copy pasted the sed from stackoverflow

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nifty

- name: Show wgpu backend
run:
python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"
Expand All @@ -60,69 +61,27 @@ jobs:
PYGFX_EXPECT_LAVAPIPE: true
run: |
WGPU_FORCE_OFFSCREEN=1 pytest -v tests/
pytest -v examples
FASTPLOTLIB_NB_TESTS=1 pytest --nbmake examples/notebooks/
- uses: actions/upload-artifact@v4
if: ${{ failure() }}
with:
name: screenshot-diffs
path: |
examples/diffs
examples/notebooks/diffs

test-build-offscreen:
name: Test Linux, only offscreen
runs-on: ubuntu-latest
timeout-minutes: 30
if: ${{ !github.event.pull_request.draft }}
strategy:
fail-fast: false
matrix:
include:
- name: Test py311
pyversion: '3.11'
- name: Test py312
pyversion: '3.12'
- name: Test py313
pyversion: '3.13'
steps:
- uses: actions/checkout@v4
with:
lfs: true
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.pyversion }}
- name: Install llvmpipe and lavapipe for offscreen canvas
run: |
sudo apt-get update -y -qq
sudo apt-get install --no-install-recommends -y libegl1-mesa-dev libgl1-mesa-dri libxcb-xfixes0-dev mesa-vulkan-drivers xorg-dev
- name: Install dev dependencies
run: |
python -m pip install --upgrade pip setuptools
# remove pygfx from install_requires, we install using pygfx@main
sed -i "/pygfx/d" ./setup.py
pip install git+https://github.com/pygfx/pygfx.git@main
pip install -e ".["tests-desktop"]"
- name: Show wgpu backend
run:
python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"
- name: fetch git lfs files
run: |
git lfs fetch --all
git lfs pull
- name: Test examples
Comment on lines -63 to -114
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

git is stupid, this entire block is actually the previous non-notebooks test pipeline that came after afterwards

WGPU_FORCE_OFFSCREEN=1 pytest -v examples/
- name: Test examples notebooks, exclude ImageWidget notebook
if: ${{ matrix.notebook_dep == 'notebook' }}
env:
PYGFX_EXPECT_LAVAPIPE: true
run: |
WGPU_FORCE_OFFSCREEN=1 pytest -v tests/
pytest -v examples
# test notebooks, exclude ImageWidget notebooks
run: FASTPLOTLIB_NB_TESTS=1 pytest --nbmake $(find ./examples/notebooks/ -maxdepth 1 -type f -name "*.ipynb" ! -name "image_widget*.ipynb" -print | xargs)
- name: Test ImageWidget notebooks
# test image widget notebooks only if imgui is installed
if: ${{ matrix.notebook_dep == 'notebook' && matrix.imgui_dep == 'imgui' }}
env:
PYGFX_EXPECT_LAVAPIPE: true
run: FASTPLOTLIB_NB_TESTS=1 pytest --nbmake $(find ./examples/notebooks/ -maxdepth 1 -type f -name "image_widget*.ipynb" -print | xargs)
- uses: actions/upload-artifact@v4
if: ${{ failure() }}
with:
name: screenshot-diffs
name: screenshot-diffs-${{ matrix.pyversion }}-${{ matrix.imgui_dep }}-${{ matrix.notebook_dep }}
path: |
examples/diffs
examples/notebooks/diffs


# test-build-full-mac:
# name: Test Mac, notebook + glfw
Expand Down
2 changes: 1 addition & 1 deletion 2 .github/workflows/docs-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
# remove pygfx from install_requires, we install using pygfx@main
sed -i "/pygfx/d" ./setup.py
pip install git+https://github.com/pygfx/pygfx.git@main
pip install -e ".["docs"]"
pip install -e ".[docs,notebook,imgui]"
- name: Show wgpu backend
run:
python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"
Expand Down
29 changes: 23 additions & 6 deletions 29 .github/workflows/screenshots.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 10
if: ${{ !github.event.pull_request.draft }}
strategy:
fail-fast: false
matrix:
imgui_dep: ["imgui", ""]
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -34,21 +38,34 @@ jobs:
# remove pygfx from install_requires, we install using pygfx@main
sed -i "/pygfx/d" ./setup.py
pip install git+https://github.com/pygfx/pygfx.git@main
pip install -e ".["tests"]"
- name: Install fastplotlib
run: |
# create string with one of: tests,imgui,notebook; test,imgui; test,notebook ; tests
# sed removes trailing comma
# install fastplotlib with with or without imgui depending on build matrix
pip install -e ".[$(echo "tests,notebook,${{ matrix.imgui_dep }}" | sed -e "s/,\+/,/g" -e "s/,$//")]"
- name: Show wgpu backend
run:
python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"
- name: Test examples
- name: Generate screenshots
env:
PYGFX_EXPECT_LAVAPIPE: true
run: |
# regenerate screenshots
REGENERATE_SCREENSHOTS=1 pytest -v examples
FASTPLOTLIB_NB_TESTS=1 REGENERATE_SCREENSHOTS=1 pytest --nbmake examples/notebooks/
- uses: actions/upload-artifact@v3
WGPU_FORCE_OFFSCREEN=1 REGENERATE_SCREENSHOTS=1 pytest -v examples
- name: Generate screenshots notebook, exclude image widget
env:
PYGFX_EXPECT_LAVAPIPE: true
run: FASTPLOTLIB_NB_TESTS=1 REGENERATE_SCREENSHOTS=1 pytest --nbmake $(find ./examples/notebooks/ -maxdepth 1 -type f -name "*.ipynb" ! -name "image_widget*.ipynb" -print | xargs)
- name: Generate screenshots notebook, include image widget
if: ${{ matrix.imgui_dep == 'imgui' }}
env:
PYGFX_EXPECT_LAVAPIPE: true
run: FASTPLOTLIB_NB_TESTS=1 REGENERATE_SCREENSHOTS=1 pytest --nbmake $(find ./examples/notebooks/ -maxdepth 1 -type f -name "image_widget*.ipynb" -print | xargs)
- uses: actions/upload-artifact@v4
if: always()
with:
name: screenshots
name: screenshots-${{ matrix.imgui_dep }}
path: |
examples/screenshots/
examples/notebooks/screenshots/
4 changes: 2 additions & 2 deletions 4 CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ cd fastplotlib

```bash
# install all extras in place
pip install -e ".[notebook,docs,tests]"
pip install -e ".[imgui, notebook, docs, tests]"
```

5. Add the upstream remote branch:
Expand Down Expand Up @@ -280,7 +280,7 @@ to open source software packages.
# after cloning
cd fastplotlib
# install dev dependencies
pip install -e ".[tests, docs, notebook]"
pip install -e ".[imgui, tests, docs, notebook]"
```

3) Check out a feature branch from `main`
Expand Down
21 changes: 18 additions & 3 deletions 21 examples/notebooks/nb_test_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,21 @@ def plot_test(name, fig: fpl.Figure):


def regenerate_screenshot(name, data):
iio.imwrite(SCREENSHOTS_DIR.joinpath(f"nb-{name}.png"), data)
if fpl.IMGUI:
prefix = ""
else:
prefix = "no-imgui-"

iio.imwrite(SCREENSHOTS_DIR.joinpath(f"{prefix}nb-{name}.png"), data)


def assert_screenshot_equal(name, data):
ground_truth = iio.imread(SCREENSHOTS_DIR.joinpath(f"nb-{name}.png"))
if fpl.IMGUI:
prefix = ""
else:
prefix = "no-imgui-"

ground_truth = iio.imread(SCREENSHOTS_DIR.joinpath(f"{prefix}nb-{name}.png"))

img = normalize_image(data)
ref_img = normalize_image(ground_truth)
Expand Down Expand Up @@ -140,9 +150,14 @@ def get_diffs_rgba(slicer):
diffs_rgba = diffs_rgba.astype("u1")
return diffs_rgba[..., slicer]

if fpl.IMGUI:
prefix = ""
else:
prefix = "no-imgui-"

# split into an rgb and an alpha diff
diffs = {
DIFFS_DIR.joinpath(f"nb-diff-{name}-rgb.png"): slice(0, 3),
DIFFS_DIR.joinpath(f"{prefix}nb-diff-{name}-rgb.png"): slice(0, 3),
}

for path, slicer in diffs.items():
Expand Down
3 changes: 3 additions & 0 deletions 3 examples/notebooks/screenshots/no-imgui-nb-astronaut.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/notebooks/screenshots/no-imgui-nb-camera.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/notebooks/screenshots/no-imgui-nb-lines-3d.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/notebooks/screenshots/no-imgui-nb-lines-colors.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/notebooks/screenshots/no-imgui-nb-lines-data.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/notebooks/screenshots/no-imgui-nb-lines.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
70 changes: 20 additions & 50 deletions 70 examples/notebooks/test_gc.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -202,57 +202,27 @@
"metadata": {},
"outputs": [],
"source": [
"movies = [np.random.rand(100, 100, 100) for i in range(6)]\n",
"\n",
"iw = fpl.ImageWidget(movies)\n",
"\n",
"# add some events onto all the image graphics\n",
"for g in iw.managed_graphics:\n",
" for f in g._features:\n",
" g.add_event_handler(feature_changed_handler, f)\n",
"\n",
"iw.show()"
]
},
{
"cell_type": "markdown",
"id": "189bcd7a-40a2-4e84-abcf-c334e50f5544",
"metadata": {},
"source": [
"# Test that setting new data with different dims clears old ImageGraphics"
"if fpl.IMGUI:\n",
" # do image widget tests only if imgui is installed\n",
" movies = [np.random.rand(100, 100, 100) for i in range(6)]\n",
" \n",
" iw = fpl.ImageWidget(movies)\n",
" \n",
" # add some events onto all the image graphics\n",
" for g in iw.managed_graphics:\n",
" for f in g._features:\n",
" g.add_event_handler(feature_changed_handler, f)\n",
" \n",
" iw.show()\n",
" \n",
" old_graphics = [weakref.proxy(g) for g in iw.managed_graphics]\n",
" \n",
" # Test that setting new data with different dims clears old ImageGraphics\n",
" new_movies = [np.random.rand(100, 200, 200) for i in range(6)]\n",
" \n",
" iw.set_data(new_movies)\n",
" test_references(old_graphics)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7e855043-91c1-4f6c-bed3-b69cf4a87f84",
"metadata": {},
"outputs": [],
"source": [
"old_graphics = [weakref.proxy(g) for g in iw.managed_graphics]\n",
"\n",
"new_movies = [np.random.rand(100, 200, 200) for i in range(6)]\n",
"\n",
"iw.set_data(new_movies)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "59e3c193-5672-4a66-bdca-12f1dd675d32",
"metadata": {},
"outputs": [],
"source": [
"test_references(old_graphics)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ad3d2a24-88b3-4071-a49c-49667d5a7813",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down
3 changes: 3 additions & 0 deletions 3 examples/screenshots/no-imgui-gridplot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/screenshots/no-imgui-gridplot_non_square.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/screenshots/no-imgui-heatmap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/screenshots/no-imgui-image_cmap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/screenshots/no-imgui-image_rgb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/screenshots/no-imgui-image_rgbvminvmax.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/screenshots/no-imgui-image_simple.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/screenshots/no-imgui-image_small.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions 3 examples/screenshots/no-imgui-image_vminvmax.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading
Morty Proxy This is a proxified and sanitized view of the page, visit original site.