Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings
Discussion options

I'm trying to build a C extension package for some exotic architectures, and run its test suite within cibuildwheel.
The test suite in turn has binary extension dependencies (pillow and numpy), which are not available on PyPI for the archs in question.

However, I think they are provided by the container image, and could be installed with the system package manager.
Now, I tried doing that with CIBW_BEFORE_TEST, but found that cibuildwheel sets up a venv for testing, where system python packages are not recognized.

Could an option be added to cibuildwheel to pass --system-site-packages when creating the venv?
Or should I manually set the PYTHONPATH or something?

You must be logged in to vote

You would need a custom testing image for each Python version, I believe. We don't have a way to set custom testing images at all currently; I think at that point you could just do the testing in a separate step after cibuildwheel runs.

Replies: 1 comment · 5 replies

Comment options

I think I'd probably try setting PYTHONPATH in test-environment. I'm guessing you're using a custom container image, so you probably know the exact paths where the system packages are installed.

If you're building multiple versions of python, you'll need to use TOML overrides to set a different path for each python version.

You must be logged in to vote
5 replies
@mara004
Comment options

Thanks for the pointers, I'll give that a try.

It's not actually a custom image, just cibuildwheel-managed ppc64le and s390x so far, but I should be able to figure out the paths.
Also tried riscv64, but didn't work because one of our build dependencies (gn) is not available in the riscv64 container.
For loongarch64 I plan to use https://github.com/loong64/cibuildwheel/tree/loong64-v3.1.3 which probably uses a custom image.

On the other hand, I'm already wondering if we wouldn't be better off doing something custom with cross-compilation, as emulation is so slow...

@mara004
Comment options

OK, I can confirm export PYTHONPATH="${PYTHONPATH}:/usr/lib64/python3.6/site-packages" does the trick with a current manylinux_2_28 container.

@mara004
Comment options

Oh no, in fact it doesn't work. I just forgot to add the && so commands got lost and the test suite wasn't actually executed.
The failure is logical: system packages are built against python 3.6, whereas cibuildwheel only supports python 3.8 and above, but the python version must match up for C extension modules from numpy and PIL to work.
For the record, here is the failure:
https://github.com/pypdfium2-team/pypdfium2/actions/runs/16995896979/job/48186393271

@henryiii
Comment options

You would need a custom testing image for each Python version, I believe. We don't have a way to set custom testing images at all currently; I think at that point you could just do the testing in a separate step after cibuildwheel runs.

Answer selected by mara004
@mara004
Comment options

I think at that point you could just do the testing in a separate step after cibuildwheel runs.

True, but then we'll have to set up test container & emulation on our own, which defeats some of cibuildwheel's convenience...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
🙏
Q&A
Labels
None yet
3 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.