Description
Going through the logs from @mattip 's logs in #19154 I think the failures we are seeing can be classified as below.
finalizer / weakrefs different in pypy
We probably should just detect and skip pypy on these.
- FAILED lib/matplotlib/tests/test_animation.py::test_animation_delete[anim0]
- FAILED lib/matplotlib/tests/test_animation.py::test_animation_repr_html[anim0-none-None-imagemagick]
- FAILED lib/matplotlib/tests/test_animation.py::test_animation_repr_html[anim0-none-None-ffmpeg]
- FAILED lib/matplotlib/tests/test_animation.py::test_funcanimation_cache_frame_data[False]
- FAILED lib/matplotlib/tests/test_cbook.py::Test_callback_registry::test_callback_complete
pypy does not implement a working sys.getsizeof
Skip this test on pypy, we are only using it to make sure that the output of rendering makes sense.
- FAILED lib/matplotlib/tests/test_backend_pdf.py::test_multipage_properfinalize
stack walking different?
The failures due to raising an unexpected warning are a bit more concerning. We are doing some frame walking / inspection to sort out if we should warn or not in
matplotlib/lib/matplotlib/backend_bases.py
Lines 1624 to 1665 in 09ab2c2
If someone who understands pypy can fix that to work on both that would be best, but if not we should detect if we are on pypy and fall back to never warning. It is operating in a slightly degraded mode (users may not get warnings they should see), but the status quo is going to warn for things the user can not do anything about (I think every time we use tightlayout of bbox_inches='tight'
).
This is probably the only failures I would hold 3.4 over.
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight[png]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight[svg]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_suptile_legend[pdf]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_clipping[png]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_clipping[svg]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_raster[pdf]
- FAILED lib/matplotlib/tests/test_backend_pgf.py::test_bbox_inches_tight - mat...
- FAILED lib/matplotlib/tests/test_backend_pgf.py::test_mixedmode[pdf] - matplo...
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight[pdf]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_suptile_legend[png]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_suptile_legend[svg]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_clipping[pdf]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_raster[png]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_raster[svg]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_tight_pcolorfast - matpl...
- FAILED lib/matplotlib/tests/test_figure.py::test_tightbbox_box_aspect[svg] - ...
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_only_on_non_finite_bbox
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_noop_tight_bbox - matplo...
- FAILED lib/matplotlib/tests/test_offsetbox.py::test_annotationbbox_extents - ...
- FAILED lib/mpl_toolkits/tests/test_axes_grid1.py::test_image_grid[png] - matp...
segfaults in tk
We should either track this down or just skip the tests.
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_backend[toolbar2-tkagg]
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_backend[toolmanager-gtk3agg]
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_backend[toolmanager-tkagg]
subprocess hung
We should either track this down or just skip the tests.
- FAILED lib/matplotlib/tests/test_backend_tk.py::test_never_update - Failed: S...
- FAILED lib/matplotlib/tests/test_backend_tk.py::test_figuremanager_cleans_own_mainloop
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_backend[toolbar2-gtk3agg]
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_thread_safety[gtk3agg]
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_thread_safety[tkagg]
not matching numpy?
These both look like they are upstream bugs in pypy? We should probably just skip these.
- FAILED lib/matplotlib/tests/test_cbook.py::test_step_fails[args1] - SystemErr...
- FAILED lib/matplotlib/tests/test_matplotlib.py::test_importable_with__OO - As...
checking refcounts
These tests do not make sense for pypy, we should skip them.
- FAILED lib/matplotlib/tests/test_quiver.py::test_quiver_memory_leak - Attribu...
- FAILED lib/matplotlib/tests/test_quiver.py::test_quiver_key_memory_leak - Att...
sockets not working
The comments around this suggest we never got this working on azure (which I assume is what is hosting GH actions) so we should sort out why the code to not run these tests in that case is not being triggered here.
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_webagg - Faile...