Closed
Description
Describe the issue:
I am trying to migrate away from using custom cython imports in order to have access to the Iterator API declarations at numpy/ndarrayobject.h by using the numpy provided ones added in v2.0.0. However if I try to build the project using the file here I get a build failure. Cc @seberg
Reproduce the code example:
pip install -r requirements-dev.txt
cythonize polyagamma/*.pyx
pip install -e .Error message:
gcc -pthread -B /home/zoj/.local/share/micromamba/envs/polyagamma-dev/compiler_compat -fno-s
trict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /home/zoj/.local/share/micromamba/envs/polyag
amma-dev/include -fPIC -O2 -isystem /home/zoj/.local/share/micromamba/envs/polyagamma-dev/include
-fPIC -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/tmp/pip-build-env-0bay5vae/overlay/lib/python
3.12/site-packages/numpy/_core/include -I./include -I/home/zoj/.local/share/micromamba/envs/polyag
amma-dev/include/python3.12 -c polyagamma/_polyagamma.c -o /tmp/tmpsbownh73.build-temp/polyagamma/
_polyagamma.o -O2 -std=c99
polyagamma/_polyagamma.c: In function ‘__pyx_pf_10polyagamma_11_polyagamma_rvs’:
polyagamma/_polyagamma.c:6265:16: error: assignment to ‘int (**)(NpyIter *)’ {aka ‘int (**)(
struct NpyIter_InternalOnly *)’} from incompatible pointer type ‘int (*)(NpyIter *)’ {aka ‘int (*)
(struct NpyIter_InternalOnly *)’} [-Wincompatible-pointer-types]
6265 | __pyx_t_20 = NpyIter_GetIterNext(__pyx_v_it, NULL); if (unlikely(__pyx_t_20 == (
(__pyx_t_5numpy_NpyIter_IterNextFunc *)NULL))) __PYX_ERR(0, 243, __pyx_L57_error)
| ^
polyagamma/_polyagamma.c: In function ‘__pyx_f_10polyagamma_11_polyagamma_dispatch’:
polyagamma/_polyagamma.c:7507:16: error: assignment to ‘int (**)(NpyIter *)’ {aka ‘int (**)(
struct NpyIter_InternalOnly *)’} from incompatible pointer type ‘int (*)(NpyIter *)’ {aka ‘int (*)
(struct NpyIter_InternalOnly *)’} [-Wincompatible-pointer-types]
7507 | __pyx_t_10 = NpyIter_GetIterNext(__pyx_v_it, NULL); if (unlikely(__pyx_t_10 == (
(__pyx_t_5numpy_NpyIter_IterNextFunc *)NULL))) __PYX_ERR(0, 462, __pyx_L11_error)
| ^
Traceback (most recent call last):
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/spawn.py", line 70, in spawn
subprocess.check_call(cmd, env=_inject_macos_ver(env))
File "/home/zoj/.local/share/micromamba/envs/polyagamma-dev/lib/python3.12/subprocess.py",
line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/usr/bin/gcc', '-pthread', '-B', '/home/zoj/.local
/share/micromamba/envs/polyagamma-dev/compiler_compat', '-fno-strict-overflow', '-DNDEBUG', '-O2',
'-Wall', '-fPIC', '-O2', '-isystem', '/home/zoj/.local/share/micromamba/envs/polyagamma-dev/inclu
de', '-fPIC', '-O2', '-isystem', '/home/zoj/.local/share/micromamba/envs/polyagamma-dev/include',
'-fPIC', '-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION', '-I/tmp/pip-build-env-0bay5vae/overlay/lib
/python3.12/site-packages/numpy/_core/include', '-I./include', '-I/home/zoj/.local/share/micromamb
a/envs/polyagamma-dev/include/python3.12', '-c', 'polyagamma/_polyagamma.c', '-o', '/tmp/tmpsbownh
73.build-temp/polyagamma/_polyagamma.o', '-O2', '-std=c99']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/unixccompiler.py", line 200, in _compile
self.spawn(compiler_so + cc_args + [src, '-o', obj] + extra_postargs)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/ccompiler.py", line 1052, in spawn
spawn(cmd, dry_run=self.dry_run, **kwargs)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/spawn.py", line 76, in spawn
raise DistutilsExecError(
distutils.errors.DistutilsExecError: command '/usr/bin/gcc' failed with exit code 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/command/
editable_wheel.py", line 139, in run
self._create_wheel_file(bdist_wheel)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/command/
editable_wheel.py", line 340, in _create_wheel_file
files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/command/
editable_wheel.py", line 263, in _run_build_commands
self._run_build_subcommands()
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/command/
editable_wheel.py", line 290, in _run_build_subcommands
self.run_command(name)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/cmd.py", line 339, in run_command
self.distribution.run_command(command)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/dist.py"
, line 999, in run_command
super().run_command(command)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/dist.py", line 1002, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/command/
build_ext.py", line 99, in run
_build_ext.run(self)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/command/build_ext.py", line 365, in run
self.build_extensions()
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/command/build_ext.py", line 481, in build_extensions
self._build_extensions_serial()
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/command/build_ext.py", line 507, in _build_extensions_serial
self.build_extension(ext)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/command/
build_ext.py", line 264, in build_extension
_build_ext.build_extension(self, ext)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/command/build_ext.py", line 562, in build_extension
objects = self.compiler.compile(
^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/ccompiler.py", line 607, in compile
self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
File "/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distuti
ls/unixccompiler.py", line 202, in _compile
raise CompileError(msg)
distutils.errors.CompileError: command '/usr/bin/gcc' failed with exit code 1
/tmp/pip-build-env-0bay5vae/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.
py:1002: _DebuggingTips: Problem in editable installation.
!!
********************************************************************************
An error happened while installing `polyagamma` in editable mode.
The following steps are recommended to help debug this problem:
- Try to install the project normally, without using the editable mode.
Does the error still persist?
(If it does, try fixing the problem before attempting the editable mode).
- If you are using binary extensions, make sure you have all OS-level
dependencies installed (e.g. compilers, toolchains, binary libraries, ...).
- Try the latest version of setuptools (maybe the error was already fixed).
- If you (or your project dependencies) are using any setuptools extension
or customization, make sure they support the editable mode.
After following the steps above, if the problem still persists and
you think this is related to how setuptools handles editable installations,
please submit a reproducible example
(see https://stackoverflow.com/help/minimal-reproducible-example) to:
https://github.com/pypa/setuptools/issues
See https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details
.
********************************************************************************
!!
cmd_obj.run()
error: command '/usr/bin/gcc' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building editable for polyagamma
Failed to build polyagamma
ERROR: Could not build wheels for polyagamma, which is required to install pyproject.toml-based pr
ojects
make: *** [Makefile:14: dev] Error 1Python and NumPy Versions:
3.12.0 | packaged by conda-forge | (main, Oct 3 2023, 08:43:22) [GCC 12.3.0]
Runtime Environment:
[{'numpy_version': '2.2.3',
'python': '3.12.0 | packaged by conda-forge | (main, Oct 3 2023, 08:43:22) '
'[GCC 12.3.0]',
'uname': uname_result(system='Linux', node='zoj-desktop', release='6.12.17-1-lts', version='#1 S
MP PREEMPT_DYNAMIC Thu, 27 Feb 2025 14:12:30 +0000', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2',
'AVX512F',
'AVX512CD',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL'],
'not_found': ['AVX512_KNL', 'AVX512_KNM']}},
{'architecture': 'SkylakeX',
'filepath': '/home/zoj/.local/share/micromamba/envs/polyagamma-dev/lib/python3.12/site-packages/
numpy.libs/libscipy_openblas64_-6bb31eeb.so',
'internal_api': 'openblas',
'num_threads': 12,
'prefix': 'libscipy_openblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.28'}]
Context for the issue:
Previously, I was manually importing the cython declarations and defined them like so:
ctypedef int (*NpyIter_IterNextFunc)(NpyIter* it) noexcept nogil
cdef extern from "numpy/ndarrayobject.h":
NpyIter_IterNextFunc NpyIter_GetIterNext(NpyIter* it, char** errmsg) except NULL
...as shown here. This worked well and I was able to successfully build the project. Now if I use the declarions added in numpy v2.0.0
ctypedef int (*NpyIter_IterNextFunc)(NpyIter* it) noexcept nogil
NpyIter_IterNextFunc* NpyIter_GetIterNext(NpyIter* it, char** errmsg) except NULL
I get this build error. I believe this is due to NpyIter_GetIterNext returning a pointer to NpyIter_IterNextFunc instead of just the type.