Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

ThreadSanitizer: data race in _struct module initialization with InterpreterPoolExecutor (free-threading build) #140260

Copy link
Copy link
@ashm-dev

Description

@ashm-dev
Issue body actions

Bug report

Bug description:

ThreadSanitizer detects a data race when using InterpreterPoolExecutor with the free-threading build. Multiple threads simultaneously write to the global variable lilendian_table in the _struct module during module initialization.

The race occurs at:

  • Thread T1 and T2: Both write to lilendian_table at _structmodule_exec (lines 2745:31 and 2746:33)
  • Location: global 'lilendian_table' of size 840 at 0x7ffff4513ca0 in _struct.cpython-315td-x86_64-linux-gnu.so

Both threads are executing through InterpreterPoolExecutor, importing modules in separate subinterpreters, which triggers concurrent initialization of the _struct extension module.

How to Reproduce

Build configuration:

CC=clang CXX=clang++ ./configure --with-thread-sanitizer --with-pydebug --enable-experimental-jit=yes --with-lto=full --with-tail-call-interp --disable-gil
make -j

Run command:

TSAN_OPTIONS=handle_segv=0 ./python -X dev -X showrefcount bug.py

Test script (bug.py):

from concurrent.futures import InterpreterPoolExecutor
from functools import lru_cache

@lru_cache(maxsize=512)
def fib(n):
    """Function that JIT will compile"""
    if n < 2:
        return n
    return fib(n-1) + fib(n-2)

if __name__ == '__main__':
    with InterpreterPoolExecutor() as t:
        for i in range(2):
            t.submit(fib, i)

Note: The race occurs with any loop iteration count (range(1), range(2), etc.)

TSAN Output

log.txt

Environment

  • CPython version: main branch
  • Operating System: Linux
  • Compiler: Clang
  • Build flags: --with-thread-sanitizer --with-pydebug --enable-experimental-jit=yes --with-lto=full --with-tail-call-interp --disable-gil

Additional Information

A second similar data race is also reported for another write in _structmodule_exec at line 2746:33 to the same global variable.

Both races involve InterpreterPoolExecutor threads executing module initialization code concurrently in different subinterpreters.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

Reactions are currently unavailable

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    Status

    Done
    Show more project fields

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      Morty Proxy This is a proxified and sanitized view of the page, visit original site.