Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 1f69a60

Browse filesBrowse files
kushalkolarkushalkolar
authored
notebook screenshot tests (#277)
* started nb screenshot tests, add reset_vmin_vmax() on ImageCmapFeature * nb test screenshots to CI * add simple.ipynb test screenshots * imageio note in simple.ipynb * try to make failure detection work * faster scatter nb, less points * lines_cmap nb screenshot tests * simplify notebook_finished() * use real image for lines underlay * fix arg * fix test * update nb screenshots * type annotation, small change to try and get git lfs to pull in PR action
1 parent c7265d3 commit 1f69a60
Copy full SHA for 1f69a60

22 files changed

+714-251Lines changed: 714 additions & 251 deletions
Expand file treeCollapse file tree
Open diff view settings
Collapse file

‎.github/workflows/ci.yml‎

Copy file name to clipboardExpand all lines: .github/workflows/ci.yml
+3-1Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,4 +97,6 @@ jobs:
9797
if: ${{ failure() }}
9898
with:
9999
name: screenshot-diffs
100-
path: examples/desktop/diffs
100+
path: |
101+
examples/desktop/diffs
102+
examples/notebooks/diffs
Collapse file

‎.github/workflows/screenshots.yml‎

Copy file name to clipboardExpand all lines: .github/workflows/screenshots.yml
+4-1Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,11 @@ jobs:
4444
run: |
4545
# regenerate screenshots
4646
REGENERATE_SCREENSHOTS=1 pytest -v examples
47+
REGENERATE_SCREENSHOTS=1 pytest --nbmake examples/notebooks/
4748
- uses: actions/upload-artifact@v3
4849
if: always()
4950
with:
5051
name: screenshots
51-
path: examples/desktop/screenshots/
52+
path: |
53+
examples/desktop/screenshots/
54+
examples/notebooks/screenshots/
Collapse file

‎examples/notebooks/lines_cmap.ipynb‎

Copy file name to clipboardExpand all lines: examples/notebooks/lines_cmap.ipynb
+114Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,19 @@
1313
"import fastplotlib as fpl"
1414
]
1515
},
16+
{
17+
"cell_type": "code",
18+
"execution_count": null,
19+
"id": "5d2ef4aa-0e4c-4694-ae2e-05da1153a413",
20+
"metadata": {
21+
"tags": []
22+
},
23+
"outputs": [],
24+
"source": [
25+
"# this is only for testing, you do not need this to use fastplotlib\n",
26+
"from nb_test_utils import plot_test, notebook_finished"
27+
]
28+
},
1629
{
1730
"cell_type": "code",
1831
"execution_count": null,
@@ -49,6 +62,18 @@
4962
"plot.show()"
5063
]
5164
},
65+
{
66+
"cell_type": "code",
67+
"execution_count": null,
68+
"id": "727282c3-aadf-420f-a88e-9dd4d4e91263",
69+
"metadata": {
70+
"tags": []
71+
},
72+
"outputs": [],
73+
"source": [
74+
"plot_test(\"lines-cmap-white\", plot)"
75+
]
76+
},
5277
{
5378
"cell_type": "markdown",
5479
"id": "889b1858-ed64-4d6b-96ad-3883fbe4d38e",
@@ -69,6 +94,19 @@
6994
"plot.graphics[0].cmap = \"jet\""
7095
]
7196
},
97+
{
98+
"cell_type": "code",
99+
"execution_count": null,
100+
"id": "3c9b0bc8-b176-425c-8036-63dc55ab7466",
101+
"metadata": {
102+
"tags": []
103+
},
104+
"outputs": [],
105+
"source": [
106+
"# for testing, ignore\n",
107+
"plot_test(\"lines-cmap-jet\", plot)"
108+
]
109+
},
72110
{
73111
"cell_type": "code",
74112
"execution_count": null,
@@ -81,6 +119,19 @@
81119
"plot.graphics[0].cmap.values = sine[:, 1]"
82120
]
83121
},
122+
{
123+
"cell_type": "code",
124+
"execution_count": null,
125+
"id": "6b19d2d4-90e7-40ed-afb9-13abe5474ace",
126+
"metadata": {
127+
"tags": []
128+
},
129+
"outputs": [],
130+
"source": [
131+
"# for testing, ignore\n",
132+
"plot_test(\"lines-cmap-jet-values\", plot)"
133+
]
134+
},
84135
{
85136
"cell_type": "code",
86137
"execution_count": null,
@@ -93,6 +144,19 @@
93144
"plot.graphics[0].cmap.values = cosine[:, 1]"
94145
]
95146
},
147+
{
148+
"cell_type": "code",
149+
"execution_count": null,
150+
"id": "0a6c4739-fa61-4532-865e-21107eab76f9",
151+
"metadata": {
152+
"tags": []
153+
},
154+
"outputs": [],
155+
"source": [
156+
"# for testing, ignore\n",
157+
"plot_test(\"lines-cmap-jet-values-cosine\", plot)"
158+
]
159+
},
96160
{
97161
"cell_type": "code",
98162
"execution_count": null,
@@ -105,6 +169,19 @@
105169
"plot.graphics[0].cmap = \"viridis\""
106170
]
107171
},
172+
{
173+
"cell_type": "code",
174+
"execution_count": null,
175+
"id": "45acfd2f-09f5-418c-bca5-3e574348b7d5",
176+
"metadata": {
177+
"tags": []
178+
},
179+
"outputs": [],
180+
"source": [
181+
"# for testing, ignore\n",
182+
"plot_test(\"lines-cmap-viridis\", plot)"
183+
]
184+
},
108185
{
109186
"cell_type": "code",
110187
"execution_count": null,
@@ -129,6 +206,19 @@
129206
"plot.graphics[0].cmap.values = cmap_values"
130207
]
131208
},
209+
{
210+
"cell_type": "code",
211+
"execution_count": null,
212+
"id": "7548407f-05ed-4c47-93cc-131c61f8e242",
213+
"metadata": {
214+
"tags": []
215+
},
216+
"outputs": [],
217+
"source": [
218+
"# for testing, ignore\n",
219+
"plot_test(\"lines-cmap-viridis-values\", plot)"
220+
]
221+
},
132222
{
133223
"cell_type": "code",
134224
"execution_count": null,
@@ -147,6 +237,30 @@
147237
"id": "c290c642-ba5f-4a46-9a17-c434cb39de26",
148238
"metadata": {},
149239
"outputs": [],
240+
"source": [
241+
"# for testing, ignore\n",
242+
"plot_test(\"lines-cmap-tab-10\", plot)"
243+
]
244+
},
245+
{
246+
"cell_type": "code",
247+
"execution_count": null,
248+
"id": "c4b9e735-72e9-4f0e-aa3e-43db57e65c99",
249+
"metadata": {
250+
"tags": []
251+
},
252+
"outputs": [],
253+
"source": [
254+
"# for testing, ignore\n",
255+
"notebook_finished()"
256+
]
257+
},
258+
{
259+
"cell_type": "code",
260+
"execution_count": null,
261+
"id": "f6735cc0-910c-4854-ac50-8ee553a6475e",
262+
"metadata": {},
263+
"outputs": [],
150264
"source": []
151265
}
152266
],
Collapse file

‎examples/notebooks/nb_test_utils.py‎

Copy file name to clipboard
+87Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
from typing import *
2+
import os
3+
from pathlib import Path
4+
5+
import imageio.v3 as iio
6+
import numpy as np
7+
8+
from fastplotlib import Plot, GridPlot
9+
10+
# make dirs for screenshots and diffs
11+
current_dir = Path(__file__).parent
12+
13+
SCREENSHOTS_DIR = current_dir.joinpath("screenshots")
14+
DIFFS_DIR = current_dir.joinpath("diffs")
15+
16+
os.makedirs(SCREENSHOTS_DIR, exist_ok=True)
17+
os.makedirs(DIFFS_DIR, exist_ok=True)
18+
19+
20+
# store all the failures to allow the nb to proceed to test other examples
21+
FAILURES = list()
22+
23+
24+
def plot_test(name, plot: Union[Plot, GridPlot]):
25+
snapshot = plot.canvas.snapshot()
26+
27+
if "REGENERATE_SCREENSHOTS" in os.environ.keys():
28+
if os.environ["REGENERATE_SCREENSHOTS"] == "1":
29+
regenerate_screenshot(name, snapshot.data)
30+
31+
try:
32+
assert_screenshot_equal(name, snapshot.data)
33+
except AssertionError:
34+
FAILURES.append(name)
35+
36+
37+
def regenerate_screenshot(name, data):
38+
iio.imwrite(SCREENSHOTS_DIR.joinpath(f"nb-{name}.png"), data)
39+
40+
41+
def assert_screenshot_equal(name, data):
42+
ground_truth = iio.imread(SCREENSHOTS_DIR.joinpath(f"nb-{name}.png"))
43+
44+
is_similar = np.allclose(data, ground_truth)
45+
46+
update_diffs(name, is_similar, data, ground_truth)
47+
48+
assert is_similar, (
49+
f"notebook snapshot for {name} has changed"
50+
)
51+
52+
53+
def update_diffs(name, is_similar, img, ground_truth):
54+
diffs_rgba = None
55+
56+
def get_diffs_rgba(slicer):
57+
# lazily get and cache the diff computation
58+
nonlocal diffs_rgba
59+
if diffs_rgba is None:
60+
# cast to float32 to avoid overflow
61+
# compute absolute per-pixel difference
62+
diffs_rgba = np.abs(ground_truth.astype("f4") - img)
63+
# magnify small values, making it easier to spot small errors
64+
diffs_rgba = ((diffs_rgba / 255) ** 0.25) * 255
65+
# cast back to uint8
66+
diffs_rgba = diffs_rgba.astype("u1")
67+
return diffs_rgba[..., slicer]
68+
69+
# split into an rgb and an alpha diff
70+
diffs = {
71+
DIFFS_DIR.joinpath(f"nb-diff-{name}-rgb.png"): slice(0, 3),
72+
DIFFS_DIR.joinpath(f"nb-diff-{name}-alpha.png"): 3,
73+
}
74+
75+
for path, slicer in diffs.items():
76+
if not is_similar:
77+
diff = get_diffs_rgba(slicer)
78+
iio.imwrite(path, diff)
79+
elif path.exists():
80+
path.unlink()
81+
82+
83+
def notebook_finished():
84+
if len(FAILURES) > 0:
85+
raise AssertionError(
86+
f"Failures for plots:\n{FAILURES}"
87+
)
Collapse file

‎examples/notebooks/scatter.ipynb‎

Copy file name to clipboardExpand all lines: examples/notebooks/scatter.ipynb
+44-72Lines changed: 44 additions & 72 deletions
Large diffs are not rendered by default.
Collapse file

‎examples/notebooks/screenshots/nb-astronaut.png‎

Copy file name to clipboard
+3Lines changed: 3 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Loading
Collapse file

‎examples/notebooks/screenshots/nb-astronaut_RGB.png‎

Copy file name to clipboard
+3Lines changed: 3 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Loading
Collapse file

‎examples/notebooks/screenshots/nb-camera.png‎

Copy file name to clipboard
+3Lines changed: 3 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Loading
Collapse file

‎examples/notebooks/screenshots/nb-lines-3d.png‎

Copy file name to clipboard
+3Lines changed: 3 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Loading
Collapse file

‎examples/notebooks/screenshots/nb-lines-cmap-jet-values-cosine.png‎

Copy file name to clipboard
+3Lines changed: 3 additions & 0 deletions
  • Display the source diff
  • Display the rich diff
Loading

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.