The Wayback Machine - https://web.archive.org/web/20201103164508/https://github.com/numpy/numpy/issues/16490
Skip to content
Sign up
Sign in Sign up

/ numpy

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't pickle UFuncTypeError #16490

Open
crusaderky opened this issue Jun 3, 2020 · 10 comments
Open

Can't pickle UFuncTypeError #16490

crusaderky opened this issue Jun 3, 2020 · 10 comments
Labels
00 - Bug good first issue
Milestone
1.19.5 release

Comments

@crusaderky
Copy link
Contributor

@crusaderky crusaderky commented Jun 3, 2020

numpy 1.18.4-py38h8854b6b_0 (conda-forge)
Linux x64

import pickle
import numpy


try:
    numpy.ones(1) * "foo"
except Exception as e:
    pickle.dumps(e)

Output:

Traceback (most recent call last):
  File "ufunctypeerror.py", line 6, in <module>
    numpy.ones(1) * "foo"
numpy.core._exceptions.UFuncTypeError: ufunc 'multiply' did not contain a loop with signature matching types (dtype('<U32'), dtype('<U32')) -> dtype('<U32')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "ufunctypeerror.py", line 8, in <module>
    pickle.dumps(e)
_pickle.PicklingError: Can't pickle <class 'numpy.core._exceptions.UFuncTypeError'>: it's not the same object as numpy.core._exceptions.UFuncTypeError
@eric-wieser eric-wieser added the 00 - Bug label Jun 3, 2020
@eric-wieser
Copy link
Member

@eric-wieser eric-wieser commented Jun 3, 2020

The error seems to come from:

https://github.com/python/cpython/blob/df773f8c5454acebe08c31e7308597fa5a8bf5df/Modules/_pickle.c#L3581-L3650

I can't help feeling there's a bug in the error message formatting here by python itself.
@eric-wieser
Copy link
Member

@eric-wieser eric-wieser commented Jun 3, 2020

Ah, I see the problem now. It's @_display_as_base causing the issue, and the simplest repro is:

import pickle
from numpy.core._exceptions import _UFuncNoLoopError
pickle.dumps(_UFuncNoLoopError)
@seberg
Copy link
Member

@seberg seberg commented Jun 3, 2020

Hmmm, do we have to set the qualname as well to make printing nice?
@eric-wieser
Copy link
Member

@eric-wieser eric-wieser commented Jun 3, 2020

If we want to keep digging, we could attach a metaclass to override pickling...
@seberg
Copy link
Member

@seberg seberg commented Jun 3, 2020

I would prefer if we can avoid it if it has no big issues (not sure if we can set a pickler externally, but I guess that is also type based). Printing seems fine without qualname, since it seems to rely on __name__. I suppose there may be subtleties where the qualname shines through?
@eric-wieser
Copy link
Member

@eric-wieser eric-wieser commented Jun 3, 2020 •
edited

It sounds like we want to drop both of these lines then:

numpy/numpy/core/_exceptions.py

Lines 29 to 30 in fb314a3

cls.__qualname__ = cls.__base__.__qualname__
set_module(cls.__base__.__module__)(cls)

@seberg
Copy link
Member

@seberg seberg commented Jun 3, 2020

I tried, and it pickles if I just drop the qualname, so maybe its fine?
@eric-wieser
Copy link
Member

@eric-wieser eric-wieser commented Jun 3, 2020

The module setting is currently a no-op anyway, but in all the cases when it isn't it would also break pickling.
@eric-wieser eric-wieser added this to the 1.20.0 release milestone Jun 17, 2020
@rossbar rossbar added NumPy Sprint and removed NumPy Sprint labels Jul 11, 2020
@skinny121
Copy link

@skinny121 skinny121 commented Sep 20, 2020

This also happens with MemoryError arising from allocating an array.
@seberg
Copy link
Member

@seberg seberg commented Sep 20, 2020

Hmmm, is this urgent enough to be fixed for 1.19.3? It seems like this may be problematic in some schemes of multiprocessing!
@seberg seberg modified the milestones: 1.20.0 release, 1.19.3 release Sep 20, 2020
bfgray3 added a commit to bfgray3/numpy that referenced this issue Sep 26, 2020
@bfgray3
BUG: ensure _UFuncNoLoopError can be pickled; closes numpy#16490
e92b12d
@bfgray3 bfgray3 mentioned this issue Sep 26, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug good first issue
Projects
None yet
Milestone
1.19.5 release
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
@seberg @charris @eric-wieser @rossbar @crusaderky @skinny121
You can’t perform that action at this time.
Morty Proxy This is a proxified and sanitized view of the page, visit original site.