Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

BUG: load no longer reads files with shape=(-1,...) #29217

Copy link
Copy link
Open
@defjaf

Description

@defjaf
Issue body actions

Describe the issue:

Prior to numpy 2.3.0, load (and its relatives) could read a *.npy file with shape[0]=-1, and correctly discover the number of rows in the file from its length. (The documentation is silent on the issue.) In 2.3.0, due to commit d8cba36, there is a check that the product of the entries in shape is equal to the number of entries in the file, which fails for this case.

I think this bug/regression can be fixed by only checking that the count matches the actual length when shape[0]!=-1 (and possibly adding a check that all entries in the shape must be positive except for that). (I have never contributed, but am happy to try to raise a PR for this.)

Reproduce the code example:

import numpy as np
arr = np.load("badshape.npy")

## I cannot attach a *.npy file as an example

Error message:

File [~/.venv/lib/python3.13/site-packages/numpy/lib/_format_impl.py:874](http://localhost:8888/~/.venv/lib/python3.13/site-packages/numpy/lib/_format_impl.py#line=873), in read_array(fp, allow_pickle, pickle_kwargs, max_header_size)
    870             array[i:i + read_count] = numpy.frombuffer(data, dtype=dtype,
    871                                                      count=read_count)
    873 if array.size != count:
--> 874     raise ValueError(
    875         "Failed to read all data for array. "
    876         f"Expected {shape} = {count} elements, "
    877         f"could only read {array.size} elements. "
    878         "(file seems not fully written?)"
    879     )
    881 if fortran_order:
    882     array.shape = shape[::-1]

ValueError: Failed to read all data for array. Expected (-1, 1) = -1 elements, could only read 1000 elements. (file seems not fully written?)

Python and NumPy Versions:

2.3.0
3.13.5 (main, Jun 11 2025, 15:36:57) [Clang 17.0.0 (clang-1700.0.13.3)]

Runtime Environment:

No response

Context for the issue:

I think that numpy itself never uses this "feature" but we have some home-grown npy-file writing code which does (and is useful so that we can append rows to the file without updating the header).

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.