Description
Describe the issue:
Prior to numpy 2.3.0, load
(and its relatives) could read a *.npy file with shape[0]=-1
, and correctly discover the number of rows in the file from its length. (The documentation is silent on the issue.) In 2.3.0, due to commit d8cba36, there is a check that the product of the entries in shape
is equal to the number of entries in the file, which fails for this case.
I think this bug/regression can be fixed by only checking that the count matches the actual length when shape[0]!=-1
(and possibly adding a check that all entries in the shape must be positive except for that). (I have never contributed, but am happy to try to raise a PR for this.)
Reproduce the code example:
import numpy as np
arr = np.load("badshape.npy")
## I cannot attach a *.npy file as an example
Error message:
File [~/.venv/lib/python3.13/site-packages/numpy/lib/_format_impl.py:874](http://localhost:8888/~/.venv/lib/python3.13/site-packages/numpy/lib/_format_impl.py#line=873), in read_array(fp, allow_pickle, pickle_kwargs, max_header_size)
870 array[i:i + read_count] = numpy.frombuffer(data, dtype=dtype,
871 count=read_count)
873 if array.size != count:
--> 874 raise ValueError(
875 "Failed to read all data for array. "
876 f"Expected {shape} = {count} elements, "
877 f"could only read {array.size} elements. "
878 "(file seems not fully written?)"
879 )
881 if fortran_order:
882 array.shape = shape[::-1]
ValueError: Failed to read all data for array. Expected (-1, 1) = -1 elements, could only read 1000 elements. (file seems not fully written?)
Python and NumPy Versions:
2.3.0
3.13.5 (main, Jun 11 2025, 15:36:57) [Clang 17.0.0 (clang-1700.0.13.3)]
Runtime Environment:
No response
Context for the issue:
I think that numpy itself never uses this "feature" but we have some home-grown npy-file writing code which does (and is useful so that we can append rows to the file without updating the header).