Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

TYP: fix ndarray.tolist() and .item() for unknown dtype #28644

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 4, 2025

Conversation

jorenham
Copy link
Member

@jorenham jorenham commented Apr 4, 2025

This fixes #27944 for both mypy and pyright.

I used the new generic generic type as a more direct alternative to match the .item() type of the underlying scalar-type. This way, we don't have to rely on on pyright's constraint set solving algorithm (recently introduced so not yet in pylance, and slightly incorrect in case of overlap). Mypy doesn't support constraint-sets, and it instead always chooses the first matching overload, causing it to behave incorrectly.

@jorenham jorenham added 09 - Backport-Candidate PRs tagged should be backported 41 - Static typing labels Apr 4, 2025
Copy link

github-actions bot commented Apr 4, 2025

Diff from mypy_primer, showing the effect of this PR on type check results on a corpus of open source code:

spark (https://github.com/apache/spark)
- python/pyspark/ml/connect/serialize.py:73: error: Argument 1 to "build_int_list" has incompatible type "int | list[int] | list[list[int]] | list[list[list[Any]]]"; expected "list[int]"  [arg-type]
- python/pyspark/ml/connect/serialize.py:75: error: Argument 1 to "build_float_list" has incompatible type "float | list[float] | list[list[float]] | list[list[list[Any]]]"; expected "list[float]"  [arg-type]
- python/pyspark/ml/connect/serialize.py:88: error: Argument 1 to "build_float_list" has incompatible type "str"; expected "list[float]"  [arg-type]
- python/pyspark/ml/connect/serialize.py:101: error: Argument 1 to "build_int_list" has incompatible type "str"; expected "list[int]"  [arg-type]
- python/pyspark/ml/connect/serialize.py:103: error: Argument 1 to "build_int_list" has incompatible type "str"; expected "list[int]"  [arg-type]
- python/pyspark/ml/connect/serialize.py:105: error: Argument 1 to "build_float_list" has incompatible type "str"; expected "list[float]"  [arg-type]
- python/pyspark/ml/connect/serialize.py:124: error: Argument 1 to "build_float_list" has incompatible type "str"; expected "list[float]"  [arg-type]

pandas (https://github.com/pandas-dev/pandas)
- pandas/core/arrays/masked.py:518: error: Incompatible return value type (got "str", expected "list[Any]")  [return-value]
- pandas/core/base.py:878: error: Incompatible return value type (got "list[Any] | str", expected "list[Any]")  [return-value]
- pandas/io/formats/format.py:1498: error: Incompatible return value type (got "Any | list[Any] | list[list[Any]] | list[list[list[Any]]]", expected "list[str]")  [return-value]
- pandas/core/groupby/generic.py:2145: error: Incompatible types in assignment (expression has type "str | list[str] | list[list[str]] | list[list[list[Any]]]", variable has type "ndarray[tuple[int, ...], dtype[Any]]")  [assignment]
- pandas/plotting/_matplotlib/style.py:276: error: Incompatible return value type (got "float | list[float] | list[list[float]] | list[list[list[Any]]]", expected "list[float]")  [return-value]

static-frame (https://github.com/static-frame/static-frame)
- static_frame/core/index.py:1536: error: Incompatible types in assignment (expression has type "str", variable has type "list[TLabel]")  [assignment]
- static_frame/core/pivot.py:757: error: Incompatible types in assignment (expression has type "str", variable has type "list[type[IndexBase]]")  [assignment]
- static_frame/core/frame.py:4281: error: Incompatible types in assignment (expression has type "str", variable has type "list[Callable[..., IndexBase] | type[Index[Any]] | None]")  [assignment]

xarray (https://github.com/pydata/xarray)
- xarray/core/utils.py: note: In function "decode_numpy_dict_values":
- xarray/core/utils.py:754: error: Incompatible types in assignment (expression has type "str", target has type "V")  [assignment]
- xarray/plot/utils.py: note: In function "_add_legend":
- xarray/plot/utils.py:1747: error: Incompatible types in assignment (expression has type "str | list[str] | list[list[str]] | list[list[list[Any]]]", variable has type "list[Any]")  [assignment]
- xarray/plot/utils.py:1748: error: Incompatible types in assignment (expression has type "str | list[str] | list[list[str]] | list[list[list[Any]]]", variable has type "list[Any]")  [assignment]
- xarray/plot/utils.py: note: At top level:
- xarray/tests/test_dataarray.py: note: In member "test_to_and_from_dict" of class "TestDataArray":
- xarray/tests/test_dataarray.py:3611: error: Incompatible types in assignment (expression has type "str", variable has type "ndarray[Any, Any]")  [assignment]
- xarray/tests/test_dataarray.py:3612: error: Incompatible types in assignment (expression has type "str | list[str] | list[list[str]] | list[list[list[Any]]]", variable has type "ndarray[tuple[int, ...], dtype[Any]]")  [assignment]

@jorenham jorenham requested a review from Copilot April 4, 2025 02:20
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (3)

numpy/init.pyi:2179

  • There appears to be a naming inconsistency with 'NDArray' here; if it is intended to refer to the current class 'ndarray', consider using consistent naming to avoid confusion.
def item(self: NDArray[generic[_T]], i0: SupportsIndex | tuple[SupportsIndex, ...] = ..., /, *args: SupportsIndex) -> _T:

numpy/init.pyi:5340

  • Using 'pyright: ignore[reportIncompatibleMethodOverride]' may suppress underlying type issues; if possible, consider refining the override signature to avoid relying on ignores.
def tolist(self: matrix[Any, dtype[generic[_T]]]) -> list[list[_T]]: ...  # pyright: ignore[reportIncompatibleMethodOverride]

numpy/init.pyi:2197

  • The general 'tolist' overload that returns 'Any' might mask type information for arrays with known shapes; verify that this behavior is intentional and does not regress type inference for well-defined cases.
def tolist(self, /) -> Any: ...

@charris charris merged commit a6ebba8 into numpy:main Apr 4, 2025
73 checks passed
@charris
Copy link
Member

charris commented Apr 4, 2025

Thanks Joren.

@jorenham jorenham deleted the typing/fix-27944 branch April 4, 2025 18:24
@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Apr 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

TYP: np.ndarray.tolist return type seems broken in numpy 2.2.0
2 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.