Description
Describe the issue:
For some data arrays, np.histogram tries to allocate absurd amount of memory and crashes. This happens with bins = "auto" and supposedly is a result of miscalculation of required number of bins.
Reproduce the code example:
import numpy as np
Z = np.array(
[
1.99999999999995173450e-11,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
9.99999999999810431126e-12,
2.00000000000028260674e-11,
9.99999999999810431126e-12,
1.99999999999995173450e-11,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
9.99999999999810431126e-12,
2.00000000000028260674e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
1.99999999999995173450e-11,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
9.99999999999810431126e-12,
2.00000000000028260674e-11,
9.99999999999810431126e-12,
1.99999999999995173450e-11,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.99999999999995173450e-11,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
2.00000000000028260674e-11,
0.00000000000000000000e00,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
0.00000000000000000000e00,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
9.99999999999810431126e-12,
0.00000000000000000000e00,
1.00000000000014130337e-11,
1.99999999999995173450e-11,
0.00000000000000000000e00,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
0.00000000000000000000e00,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
1.00000000000014130337e-11,
9.99999999999810431126e-12,
]
)
np.histogram(Z, bins="auto")Error message:
Traceback (most recent call last):
File "/home/censored/bug.py", line 137, in <module>
np.histogram(Z, bins="auto")
File "/home/censored/.venv/lib/python3.12/site-packages/numpy/lib/histograms.py", line 780, in histogram
bin_edges, uniform_bins = _get_bin_edges(a, bins, range, weights)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/censored/.venv/lib/python3.12/site-packages/numpy/lib/histograms.py", line 446, in _get_bin_edges
bin_edges = np.linspace(
^^^^^^^^^^^^
File "/home/censored/.venv/lib/python3.12/site-packages/numpy/core/function_base.py", line 140, in linspace
y = _nx.arange(0, num, dtype=dt).reshape((-1,) + (1,) * ndim(delta))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
numpy.core._exceptions._ArrayMemoryError: Unable to allocate 98.2 TiB for an array with shape (13493864060128,) and data type float64Python and NumPy Versions:
1.26.4
3.12.7 (main, Feb 4 2025, 14:46:03) [GCC 14.2.0]
Runtime Environment:
[{'numpy_version': '1.26.4',
'python': '3.12.7 (main, Feb 4 2025, 14:46:03) [GCC 14.2.0]',
'uname': uname_result(system='Linux', node='censored', release='6.11.0-14-generic', version='#15-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 10 23:48:25 UTC 2025', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_KNM',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Zen',
'filepath': '/home/censored/.venv/lib/python3.12/site-packages/numpy.libs/libopenblas64_p-r0-0cf96a72.3.23.dev.so',
'internal_api': 'openblas',
'num_threads': 16,
'prefix': 'libopenblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.23.dev'}]
Context for the issue:
No response