On Windows, the multiprocessing DupHandle.detach() method has race condition on DuplicateHandle(DUPLICATE_CLOSE_SOURCE).

Error on duplicate():

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 107, in spawn_main
    new_handle = reduction.duplicate(pipe_handle,
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\reduction.py", line 79, in duplicate
    return _winapi.DuplicateHandle(
PermissionError: [WinError 5] Access is denied

Example: bpo-34714


Error on detach():

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 117, in spawn_main
    exitcode = _main(fd)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\spawn.py", line 127, in _main
    self = reduction.pickle.load(from_parent)
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\connection.py", line 951, in rebuild_pipe_connection
    handle = dh.detach()
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\reduction.py", line 133, in detach
    self._access, False, _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied

Example: bpo-34513

Any possible workaround for this issue?
I seem to be consistingly hitting this issue.

> I seem to be consistingly hitting this issue.

It will help with test development if you can provide a minimal example that reliably reproduces the problem. 

In msg353064 I see DuplicateHandle calls failing with ERROR_ACCESS_DENIED (5). Assuming the process handles have the required PROCESS_DUP_HANDLE access, it's most likely the case that the underlying NtDuplicateObject system call is failing with STATUS_PROCESS_IS_TERMINATING (0xC000010A). For example:

    import ctypes
    ntdll = ctypes.WinDLL('ntdll')
    from subprocess import Popen, PIPE
    from _winapi import GetCurrentProcess, TerminateProcess
    from _winapi import DuplicateHandle, DUPLICATE_SAME_ACCESS

    p = Popen('cmd.exe', stdin=PIPE, stdout=PIPE, stderr=PIPE)
    TerminateProcess(p._handle, 0)

Try to duplicate the process handle into the terminated process:

    >>> source = GetCurrentProcess()
    >>> target = handle = p._handle
    >>> try:
    ...     DuplicateHandle(source, handle, target,
    ...         0, False, DUPLICATE_SAME_ACCESS)
    ... except:
    ...     status = ntdll.RtlGetLastNtStatus() 
    ...     print(f'NTSTATUS: {status & (2**32-1):#010x}')
    ...     raise
    ...
    NTSTATUS: 0xc000010a
    Traceback (most recent call last):
      File "<stdin>", line 2, in <module>
    PermissionError: [WinError 5] Access is denied

What I'm able to copy from the console(though I doubt I'm getting all of
the traceback):

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\spawn.py",
line 106, in spawn_main

    exitcode = _main(fd)

  File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\spawn.py",
line 116, in _main

    self = pickle.load(from_parent)

  File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\connection.py",
line 942, in rebuild_pipe_connection

    handle = dh.detach()

  File
"c:\hostedtoolcache\windows\python\3.5.4\x64\lib\multiprocessing\reduction.py",
line 128, in detach

    self._access, False, _winapi.DUPLICATE_CLOSE_SOURCE)

PermissionError: [WinError 5] Access is denied

Pedro Algarvio @ Phone

A sábado, 22/02/2020, 13:44, Eryk Sun <report@bugs.python.org> escreveu:

>
> Eryk Sun <eryksun@gmail.com> added the comment:
>
> > I seem to be consistingly hitting this issue.
>
> It will help with test development if you can provide a minimal example
> that reliably reproduces the problem.
>
> In msg353064 I see DuplicateHandle calls failing with ERROR_ACCESS_DENIED
> (5). Assuming the process handles have the required PROCESS_DUP_HANDLE
> access, it's most likely the case that the underlying NtDuplicateObject
> system call is failing with STATUS_PROCESS_IS_TERMINATING (0xC000010A). For
> example:
>
>     import ctypes
>     ntdll = ctypes.WinDLL('ntdll')
>     from subprocess import Popen, PIPE
>     from _winapi import GetCurrentProcess, TerminateProcess
>     from _winapi import DuplicateHandle, DUPLICATE_SAME_ACCESS
>
>     p = Popen('cmd.exe', stdin=PIPE, stdout=PIPE, stderr=PIPE)
>     TerminateProcess(p._handle, 0)
>
> Try to duplicate the process handle into the terminated process:
>
>     >>> source = GetCurrentProcess()
>     >>> target = handle = p._handle
>     >>> try:
>     ...     DuplicateHandle(source, handle, target,
>     ...         0, False, DUPLICATE_SAME_ACCESS)
>     ... except:
>     ...     status = ntdll.RtlGetLastNtStatus()
>     ...     print(f'NTSTATUS: {status & (2**32-1):#010x}')
>     ...     raise
>     ...
>     NTSTATUS: 0xc000010a
>     Traceback (most recent call last):
>       File "<stdin>", line 2, in <module>
>     PermissionError: [WinError 5] Access is denied
>
> ----------
> nosy: +eryksun
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <https://bugs.python.org/issue38263>
> _______________________________________
>

append Nosy List.

spawn_main() could handle a PermissionError by checking the exit code. If the parent has terminated already, then simply exit quietly. _winapi.PROCESS_QUERY_LIMITED_INFORMATION would need to be defined. For example:

    def spawn_main(pipe_handle, parent_pid=None, tracker_fd=None):
        """Run code specified by data received over a pipe."""
        assert is_forking(sys.argv), "Not forking"

        if sys.platform == 'win32':
            import msvcrt
            import _winapi

            if parent_pid is not None:
                source_process = _winapi.OpenProcess(
                    _winapi.SYNCHRONIZE |
                    _winapi.PROCESS_DUP_HANDLE |
                    _winapi.PROCESS_QUERY_LIMITED_INFORMATION,
                    False, parent_pid)
            else:
                source_process = None
            try:
                new_handle = reduction.duplicate(
                    pipe_handle, source_process=source_process)
                fd = msvcrt.open_osfhandle(new_handle, os.O_RDONLY)
                exitcode = _main(fd, source_process)
            except PermissionError:
                if (source_process is None or
                      _winapi.GetExitCodeProcess(source_process) ==
                      _winapi.STILL_ACTIVE):
                    raise
                exitcode = 1
        else:
            from . import resource_tracker
            resource_tracker._resource_tracker._fd = tracker_fd
            exitcode = _main(pipe_handle, os.dup(pipe_handle))

        sys.exit(exitcode)

File "c:/Users/jesvi/Documents/GitHub/Jesvi-Bot-Telegram/scripts/main.py", line 144, in thread_test
    p.start()
  File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.lock' object
Traceback (most recent call last):
  File "<string>", line 1, in <module>
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 107, in spawn_main
  File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py", line 107, in spawn_main
    new_handle = reduction.duplicate(pipe_handle,
  File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 79, in duplicate
    new_handle = reduction.duplicate(pipe_handle,
  File "C:\Users\jesvi\AppData\Local\Programs\Python\Python38\lib\multiprocessing\reduction.py", line 79, in duplicate
    return _winapi.DuplicateHandle(
OSError: [WinError 6] The handle is invalid
    return _winapi.DuplicateHandle(
OSError: [WinError 6] The handle is invalid

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Program Files\Python39\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Program Files\Python39\lib\multiprocessing\spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "C:\Program Files\Python39\lib\multiprocessing\connection.py", line 967, in rebuild_pipe_connection
    handle = dh.detach()
  File "C:\Program Files\Python39\lib\multiprocessing\reduction.py", line 131, in detach
    return _winapi.DuplicateHandle(
PermissionError: [WinError 5] Acesso negado