Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

gh-140551: Fix dict crash if clear is called at lookup stage#140558

Merged
kumaraditya303 merged 11 commits into
python:mainpython/cpython:mainfrom
efimov-mikhail:issue-140551-dict_clear_at_lookupefimov-mikhail/cpython:issue-140551-dict_clear_at_lookupCopy head branch name to clipboard
Oct 29, 2025
Merged

gh-140551: Fix dict crash if clear is called at lookup stage#140558
kumaraditya303 merged 11 commits into
python:mainpython/cpython:mainfrom
efimov-mikhail:issue-140551-dict_clear_at_lookupefimov-mikhail/cpython:issue-140551-dict_clear_at_lookupCopy head branch name to clipboard

Conversation

@efimov-mikhail

@efimov-mikhail efimov-mikhail commented Oct 24, 2025

Copy link
Copy Markdown
Member

Comment thread Lib/test/test_dict.py
Comment thread Objects/dictobject.c Outdated
Comment thread Misc/NEWS.d/next/Core_and_Builtins/2025-10-24-20-42-33.gh-issue-140551.-9swrl.rst Outdated
@sergey-miryanov

Copy link
Copy Markdown
Contributor

For the following repro:

d = {}

class X(object):
    def __hash__(self):
        return 1
    def __eq__(self, other):
        d.clear()

d.setdefault(X(), None)
d.setdefault(X(), None)
d.setdefault(X(), None)

assert len(d) == 1
print(d)

It crashes inside dict_setdefault_ref_lock_held -> insertion_resize -> dictresize -> unicode_get_hash

@sergey-miryanov

Copy link
Copy Markdown
Contributor

For the following repro:

d = {}

class X(object):
    def __hash__(self):
        return 1
    def __eq__(self, other):
        d.clear()

d.setdefault(X(), None)
d.setdefault(X(), None)

assert len(d) == 1
print(d)

It crashes also inside unicode_get_hash but for line print(d).

Both repro tests on Windows 11 on main branch.

…e-140551.-9swrl.rst

Co-authored-by: Sergey Miryanov <sergey.miryanov@gmail.com>
@sergey-miryanov

Copy link
Copy Markdown
Contributor

Another one I found on main:

d = {}

class X(object):
    def __hash__(self):
        return 1
    def __eq__(self, other):
        d.clear()

d[X()] = None
d[X()] = None
for _ in range(10):
    d.pop(X(), None)

It is aborted with Assertion failed: PyUnicode_CheckExact(ep->me_key), file X:\Work\cpython\main\Objects\dictobject.c, line 1048

@efimov-mikhail

efimov-mikhail commented Oct 24, 2025

Copy link
Copy Markdown
Member Author

Another one I found on main:

d[X()] = None
d[X()] = None
for _ in range(10):
    d.pop(X(), None)

Yes, it's expected that d is in incorrect state after two assignments.
There're many ways to trigger assertion at that state.
All of them have been fixed with this PR, I suppose.

@sergey-miryanov sergey-miryanov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@methane

methane commented Oct 26, 2025

Copy link
Copy Markdown
Member

I'm not sure if this is the correct solution.

If the keys of a dictionary change during a dict lookup, can we trust the result of the lookup (e.g. ix)?

Shouldn't it raise an error if the keys of a dict change during a lookup?

@vstinner

Copy link
Copy Markdown
Member

I like this error:

            PyErr_SetString(PyExc_RuntimeError,
                    "dict mutated during update");

@colesbury

Copy link
Copy Markdown
Contributor

If the keys of a dictionary change during a dict lookup, can we trust the result of the lookup

CPython behavior has been to retry the lookup if the key changes (since Python 2.2, 453163d). I don't think we should change it.

Comment thread Lib/test/test_dict.py Outdated
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
@efimov-mikhail

Copy link
Copy Markdown
Member Author

I created #140702 for test_UnixDatagramServer() timeout.

IIUC, problem is fixed and CI is green.

@kumaraditya303 kumaraditya303 merged commit 02202c1 into python:main Oct 29, 2025
46 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @efimov-mikhail for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Sorry, @efimov-mikhail and @kumaraditya303, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 02202c117b5702f3325e62b07ccdeaa125fc8722 3.14

@miss-islington-app

Copy link
Copy Markdown

Sorry, @efimov-mikhail and @kumaraditya303, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 02202c117b5702f3325e62b07ccdeaa125fc8722 3.13

@efimov-mikhail

Copy link
Copy Markdown
Member Author

Thanks for the merge, @kumaraditya303.
I'll make backports manually.

efimov-mikhail added a commit to efimov-mikhail/cpython that referenced this pull request Oct 29, 2025
…age (python#140558)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
@bedevere-app

bedevere-app Bot commented Oct 29, 2025

Copy link
Copy Markdown

GH-140743 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Oct 29, 2025
efimov-mikhail added a commit to efimov-mikhail/cpython that referenced this pull request Oct 29, 2025
…age (python#140558)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
@bedevere-app

bedevere-app Bot commented Oct 29, 2025

Copy link
Copy Markdown

GH-140744 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Oct 29, 2025
@efimov-mikhail

Copy link
Copy Markdown
Member Author

FYI, @kumaraditya303
The backports are ready and have green CI.

kumaraditya303 pushed a commit that referenced this pull request Oct 30, 2025
…H-140558) (#140743)

gh-140551: Fix `dict` crash if `clear` is called at `lookup` stage (#140558)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
kumaraditya303 pushed a commit that referenced this pull request Oct 30, 2025
…H-140558) (#140744)

* gh-140551: Fix `dict` crash if `clear` is called at `lookup` stage (#140558)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
colesbury added a commit to colesbury/cpython that referenced this pull request Dec 3, 2025
This fixes a regression introduced in pythongh-140558. The interpreter would
crash if we inserted a non `str` key into a split table that matches an
existing key.
colesbury added a commit that referenced this pull request Dec 3, 2025
This fixes a regression introduced in gh-140558. The interpreter would
crash if we inserted a non `str` key into a split table that matches an
existing key.
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Dec 3, 2025
This fixes a regression introduced in pythongh-140558. The interpreter would
crash if we inserted a non `str` key into a split table that matches an
existing key.
(cherry picked from commit 547d8da)

Co-authored-by: Sam Gross <colesbury@gmail.com>
colesbury added a commit to colesbury/cpython that referenced this pull request Dec 3, 2025
…2229)

This fixes a regression introduced in pythongh-140558. The interpreter would
crash if we inserted a non `str` key into a split table that matches an
existing key.
(cherry picked from commit 547d8da)

Co-authored-by: Sam Gross <colesbury@gmail.com>
colesbury added a commit that referenced this pull request Dec 4, 2025
…2244)

This fixes a regression introduced in gh-140558. The interpreter would
crash if we inserted a non `str` key into a split table that matches an
existing key.
(cherry picked from commit 547d8da)

Co-authored-by: Sam Gross <colesbury@gmail.com>
colesbury added a commit that referenced this pull request Dec 4, 2025
…2245)

This fixes a regression introduced in gh-140558. The interpreter would
crash if we inserted a non `str` key into a split table that matches an
existing key.
(cherry picked from commit 547d8da)
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
…age (python#140558)

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
StanFromIreland pushed a commit to StanFromIreland/cpython that referenced this pull request Dec 6, 2025
This fixes a regression introduced in pythongh-140558. The interpreter would
crash if we inserted a non `str` key into a split table that matches an
existing key.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.