Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

BUG: Make np.nonzero threading safe #28385

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Feb 23, 2025
Prev Previous commit
Next Next commit
review comments
  • Loading branch information
eendebakpt authored and charris committed Feb 23, 2025
commit ef3c7d08a64fd5be559859818f319739c2144c52
2 changes: 1 addition & 1 deletion 2 .github/workflows/compiler_sanitizers.yml
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ jobs:
- name: Test
run: |
# These tests are slow, so only run tests in files that do "import threading" to make them count
TSAN_OPTIONS="allocator_may_return_null=1:halt_on_error=1:suppressions=tools\ci\tsan_suppressions.txt" \
TSAN_OPTIONS="allocator_may_return_null=1:suppressions=/Users/runner/work/numpy/numpy/tools/ci/tsan_suppressions.txt" \
python -m spin test \
`find numpy -name "test*.py" | xargs grep -l "import threading" | tr '\n' ' '` \
-- -v -s --timeout=600 --durations=10
3 changes: 1 addition & 2 deletions 3 numpy/_core/src/multiarray/item_selection.c
Original file line number Diff line number Diff line change
Expand Up @@ -2893,10 +2893,10 @@ PyArray_Nonzero(PyArrayObject *self)
* the fast bool count is followed by this sparse path is faster
* than combining the two loops, even for larger arrays
*/
npy_intp * multi_index_end = multi_index + nonzero_count;
if (((double)nonzero_count / count) <= 0.1) {
npy_intp subsize;
npy_intp j = 0;
npy_intp * multi_index_end = multi_index + nonzero_count;
while (multi_index < multi_index_end) {
npy_memchr(data + j * stride, 0, stride, count - j,
&subsize, 1);
Expand All @@ -2912,7 +2912,6 @@ PyArray_Nonzero(PyArrayObject *self)
* stalls that are very expensive on most modern processors.
*/
else {
npy_intp *multi_index_end = multi_index + nonzero_count;
npy_intp j = 0;

/* Manually unroll for GCC and maybe other compilers */
Expand Down
9 changes: 7 additions & 2 deletions 9 numpy/_core/tests/test_multithreading.py
Original file line number Diff line number Diff line change
Expand Up @@ -274,16 +274,21 @@ def closure(b):

@pytest.mark.parametrize("dtype", [bool, int, float])
def test_nonzero_bool(dtype):
# See: gh-28361
#
# np.nonzero uses np.count_nonzero to determine the size of the output array
# In a second pass the indices of the non-zero elements are determined, but they can have changed
#
# This test triggers a data race which is suppressed in the TSAN CI. The test is to ensure
# np.nonzero does not generate a segmentation fault
x = np.random.randint(4, size=10_000).astype(dtype)

def func(seed):
def func():
x[::2] = np.random.randint(2)
try:
_ = np.nonzero(x)
except RuntimeError as ex:
assert 'number of non-zero array elements changed during function execution' in str(ex)

run_threaded(func, max_workers=10, pass_count=True, outer_iterations=50)
run_threaded(func, max_workers=10, pass_count=False, outer_iterations=50)

Morty Proxy This is a proxified and sanitized view of the page, visit original site.