Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Use fork to fix PicklingError#660

Open
sudipm-mukherjee wants to merge 1 commit intotomopy:mastertomopy/tomopy:masterfrom
sudipm-mukherjee:sudip-errorsudipm-mukherjee/tomopy:sudip-errorCopy head branch name to clipboard
Open

Use fork to fix PicklingError#660
sudipm-mukherjee wants to merge 1 commit intotomopy:mastertomopy/tomopy:masterfrom
sudipm-mukherjee:sudip-errorsudipm-mukherjee/tomopy:sudip-errorCopy head branch name to clipboard

Conversation

@sudipm-mukherjee
Copy link
Copy Markdown

@sudipm-mukherjee sudipm-mukherjee commented Nov 22, 2025

The tests are failing for s390x with Python3.14 and the error is:

=================================== FAILURES ===================================
_________ DistortionCorrectionTestCase.test_distortion_correction_proj _________

self = <test_tomopy.test_prep.test_alignment.DistortionCorrectionTestCase testMethod=test_distortion_correction_proj>

    def test_distortion_correction_proj(self):
        test_dir = os.path.dirname(os.path.realpath(__file__))
        file_path = os.path.join(
            os.path.dirname(test_dir),'test_data','discoef.txt')
        (xc, yc, list_fact) = load_distortion_coefs(file_path)
        assert_allclose(
>           distortion_correction_proj(
                read_file('distortion_3d.npy'), xc, yc, list_fact)[0],
            read_file('distortion_proj.npy'), rtol=1e-2)

test/test_tomopy/test_prep/test_alignment.py:69: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tomopy/prep/alignment.py:947: in distortion_correction_proj
    arr = mproc.distribute_jobs(
tomopy/util/mproc.py:295: in distribute_jobs
    with closing(mp.Pool(processes=ncore,
/usr/lib/python3.14/multiprocessing/context.py:119: in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
/usr/lib/python3.14/multiprocessing/pool.py:215: in __init__
    self._repopulate_pool()
/usr/lib/python3.14/multiprocessing/pool.py:306: in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
/usr/lib/python3.14/multiprocessing/pool.py:329: in _repopulate_pool_static
    w.start()
/usr/lib/python3.14/multiprocessing/process.py:121: in start
    self._popen = self._Popen(self)
                  ^^^^^^^^^^^^^^^^^
/usr/lib/python3.14/multiprocessing/context.py:300: in _Popen
    return Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^
/usr/lib/python3.14/multiprocessing/popen_forkserver.py:35: in __init__
    super().__init__(process_obj)
/usr/lib/python3.14/multiprocessing/popen_fork.py:20: in __init__
    self._launch(process_obj)
/usr/lib/python3.14/multiprocessing/popen_forkserver.py:47: in _launch
    reduction.dump(process_obj, buf)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <ForkServerProcess name='ForkServerPoolWorker-2' parent=12317 initial daemon>
file = <_io.BytesIO object at 0x3ffa4eb31a0>, protocol = None

    def dump(obj, file, protocol=None):
        '''Replacement for pickle.dump() using ForkingPickler.'''
>       ForkingPickler(file, protocol).dump(obj)
E       _pickle.PicklingError: Can't pickle <class 'ctypes.c_float'>: it's not the same object as ctypes.c_float
E       when serializing tuple item 0
E       when serializing multiprocessing.sharedctypes.c_float_le_Array_4096 reconstructor arguments
E       when serializing multiprocessing.sharedctypes.c_float_le_Array_4096 object
E       when serializing list item 0
E       when serializing tuple item 0
E       when serializing tuple item 3
E       when serializing dict item '_args'
E       when serializing multiprocessing.context.ForkServerProcess state
E       when serializing multiprocessing.context.ForkServerProcess object

/usr/lib/python3.14/multiprocessing/reduction.py:60: PicklingError
____________________ TestDistributeJobs.test_retrieve_phase ____________________

self = <test_tomopy.test_prep.test_phase.TestDistributeJobs testMethod=test_retrieve_phase>

    def test_retrieve_phase(self):
        assert_allclose(
>           retrieve_phase(read_file('proj.npy')),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            read_file('retrieve_phase.npy'), rtol=1e-6)

test/test_tomopy/test_prep/test_phase.py:62: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tomopy/prep/phase.py:123: in retrieve_phase
    arr = mproc.distribute_jobs(
tomopy/util/mproc.py:295: in distribute_jobs
    with closing(mp.Pool(processes=ncore,
/usr/lib/python3.14/multiprocessing/context.py:119: in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
/usr/lib/python3.14/multiprocessing/pool.py:215: in __init__
    self._repopulate_pool()
/usr/lib/python3.14/multiprocessing/pool.py:306: in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
/usr/lib/python3.14/multiprocessing/pool.py:329: in _repopulate_pool_static
    w.start()
/usr/lib/python3.14/multiprocessing/process.py:121: in start
    self._popen = self._Popen(self)
                  ^^^^^^^^^^^^^^^^^
/usr/lib/python3.14/multiprocessing/context.py:300: in _Popen
    return Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^
/usr/lib/python3.14/multiprocessing/popen_forkserver.py:35: in __init__
    super().__init__(process_obj)
/usr/lib/python3.14/multiprocessing/popen_fork.py:20: in __init__
    self._launch(process_obj)
/usr/lib/python3.14/multiprocessing/popen_forkserver.py:47: in _launch
    reduction.dump(process_obj, buf)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <ForkServerProcess name='ForkServerPoolWorker-13' parent=12317 initial daemon>
file = <_io.BytesIO object at 0x3ff9ce13a10>, protocol = None

    def dump(obj, file, protocol=None):
        '''Replacement for pickle.dump() using ForkingPickler.'''
>       ForkingPickler(file, protocol).dump(obj)
E       _pickle.PicklingError: Can't pickle <class 'ctypes.c_float'>: it's not the same object as ctypes.c_float
E       when serializing tuple item 0
E       when serializing multiprocessing.sharedctypes.c_float_le_Array_6144 reconstructor arguments
E       when serializing multiprocessing.sharedctypes.c_float_le_Array_6144 object
E       when serializing list item 0
E       when serializing tuple item 0
E       when serializing tuple item 3
E       when serializing dict item '_args'
E       when serializing multiprocessing.context.ForkServerProcess state
E       when serializing multiprocessing.context.ForkServerProcess object

/usr/lib/python3.14/multiprocessing/reduction.py:60: PicklingError
_________________ StripeRemovalTestCase.test_remove_stripe_fw __________________

self = <test_tomopy.test_prep.test_stripe.StripeRemovalTestCase testMethod=test_remove_stripe_fw>

    def test_remove_stripe_fw(self):
        assert_allclose(
>           srm.remove_stripe_fw(read_file('proj.npy')),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            read_file('remove_stripe_fw.npy'), rtol=1e-2)

test/test_tomopy/test_prep/test_stripe.py:71: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tomopy/prep/stripe.py:125: in remove_stripe_fw
    arr = mproc.distribute_jobs(tomo,
tomopy/util/mproc.py:295: in distribute_jobs
    with closing(mp.Pool(processes=ncore,
/usr/lib/python3.14/multiprocessing/context.py:119: in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
/usr/lib/python3.14/multiprocessing/pool.py:215: in __init__
    self._repopulate_pool()
/usr/lib/python3.14/multiprocessing/pool.py:306: in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
/usr/lib/python3.14/multiprocessing/pool.py:329: in _repopulate_pool_static
    w.start()
/usr/lib/python3.14/multiprocessing/process.py:121: in start
    self._popen = self._Popen(self)
                  ^^^^^^^^^^^^^^^^^
/usr/lib/python3.14/multiprocessing/context.py:300: in _Popen
    return Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^
/usr/lib/python3.14/multiprocessing/popen_forkserver.py:35: in __init__
    super().__init__(process_obj)
/usr/lib/python3.14/multiprocessing/popen_fork.py:20: in __init__
    self._launch(process_obj)
/usr/lib/python3.14/multiprocessing/popen_forkserver.py:47: in _launch
    reduction.dump(process_obj, buf)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <ForkServerProcess name='ForkServerPoolWorker-22' parent=12317 initial daemon>
file = <_io.BytesIO object at 0x3ff9cf38090>, protocol = None

    def dump(obj, file, protocol=None):
        '''Replacement for pickle.dump() using ForkingPickler.'''
>       ForkingPickler(file, protocol).dump(obj)
E       _pickle.PicklingError: Can't pickle <class 'ctypes.c_float'>: it's not the same object as ctypes.c_float
E       when serializing tuple item 0
E       when serializing multiprocessing.sharedctypes.c_float_le_Array_6144 reconstructor arguments
E       when serializing multiprocessing.sharedctypes.c_float_le_Array_6144 object
E       when serializing list item 0
E       when serializing tuple item 0
E       when serializing tuple item 3
E       when serializing dict item '_args'
E       when serializing multiprocessing.context.ForkServerProcess state
E       when serializing multiprocessing.context.ForkServerProcess object

/usr/lib/python3.14/multiprocessing/reduction.py:60: PicklingError
_________________ StripeRemovalTestCase.test_remove_stripe_ti __________________

self = <test_tomopy.test_prep.test_stripe.StripeRemovalTestCase testMethod=test_remove_stripe_ti>

    def test_remove_stripe_ti(self):
        assert_allclose(
>           srm.remove_stripe_ti(read_file('proj.npy')),
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            read_file('remove_stripe_ti.npy'), rtol=1e-2)

test/test_tomopy/test_prep/test_stripe.py:76: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tomopy/prep/stripe.py:202: in remove_stripe_ti
    arr = mproc.distribute_jobs(tomo,
tomopy/util/mproc.py:295: in distribute_jobs
    with closing(mp.Pool(processes=ncore,
/usr/lib/python3.14/multiprocessing/context.py:119: in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
/usr/lib/python3.14/multiprocessing/pool.py:215: in __init__
    self._repopulate_pool()
/usr/lib/python3.14/multiprocessing/pool.py:306: in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
/usr/lib/python3.14/multiprocessing/pool.py:329: in _repopulate_pool_static
    w.start()
/usr/lib/python3.14/multiprocessing/process.py:121: in start
    self._popen = self._Popen(self)
                  ^^^^^^^^^^^^^^^^^
/usr/lib/python3.14/multiprocessing/context.py:300: in _Popen
    return Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^
/usr/lib/python3.14/multiprocessing/popen_forkserver.py:35: in __init__
    super().__init__(process_obj)
/usr/lib/python3.14/multiprocessing/popen_fork.py:20: in __init__
    self._launch(process_obj)
/usr/lib/python3.14/multiprocessing/popen_forkserver.py:47: in _launch
    reduction.dump(process_obj, buf)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = <ForkServerProcess name='ForkServerPoolWorker-24' parent=12317 initial daemon>
file = <_io.BytesIO object at 0x3ff9cfe0c70>, protocol = None

    def dump(obj, file, protocol=None):
        '''Replacement for pickle.dump() using ForkingPickler.'''
>       ForkingPickler(file, protocol).dump(obj)
E       _pickle.PicklingError: Can't pickle <class 'ctypes.c_float'>: it's not the same object as ctypes.c_float
E       when serializing tuple item 0
E       when serializing multiprocessing.sharedctypes.c_float_le_Array_6144 reconstructor arguments
E       when serializing multiprocessing.sharedctypes.c_float_le_Array_6144 object
E       when serializing list item 0
E       when serializing tuple item 0
E       when serializing tuple item 3
E       when serializing dict item '_args'
E       when serializing multiprocessing.context.ForkServerProcess state
E       when serializing multiprocessing.context.ForkServerProcess object

/usr/lib/python3.14/multiprocessing/reduction.py:60: PicklingError
=========================== short test summary info ============================
FAILED test/test_tomopy/test_prep/test_alignment.py::DistortionCorrectionTestCase::test_distortion_correction_proj
FAILED test/test_tomopy/test_prep/test_phase.py::TestDistributeJobs::test_retrieve_phase
FAILED test/test_tomopy/test_prep/test_stripe.py::StripeRemovalTestCase::test_remove_stripe_fw
FAILED test/test_tomopy/test_prep/test_stripe.py::StripeRemovalTestCase::test_remove_stripe_ti

The full log can be seen at https://launchpadlibrarian.net/833008584/buildlog_ubuntu-resolute-s390x.tomopy_1.15.2+ds1-3build1_BUILDING.txt.gz

@sudipm-mukherjee
Use fork to fix PicklingError
05549ef 
Signed-off-by: Sudip Mukherjee <sudipm.mukherjee@gmail.com>
@sudipm-mukherjee sudipm-mukherjee marked this pull request as ready for review November 22, 2025 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@sudipm-mukherjee
Morty Proxy This is a proxified and sanitized view of the page, visit original site.