Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit aea0c58

Browse filesBrowse files
markshannonmarkshannon
authored
GH-127010: Don't lazily track and untrack dicts (GH-127027)
1 parent 7191b76 commit aea0c58
Copy full SHA for aea0c58

File tree

12 files changed

+44
-283
lines changed
Filter options

12 files changed

+44
-283
lines changed

‎Doc/library/gc.rst

Copy file name to clipboardExpand all lines: Doc/library/gc.rst
-2Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -204,8 +204,6 @@ The :mod:`gc` module provides the following functions:
204204
>>> gc.is_tracked({})
205205
False
206206
>>> gc.is_tracked({"a": 1})
207-
False
208-
>>> gc.is_tracked({"a": []})
209207
True
210208

211209
.. versionadded:: 3.1

‎Include/internal/pycore_dict.h

Copy file name to clipboardExpand all lines: Include/internal/pycore_dict.h
-2Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -43,8 +43,6 @@ extern int _PyDict_Next(
4343

4444
extern int _PyDict_HasOnlyStringKeys(PyObject *mp);
4545

46-
extern void _PyDict_MaybeUntrack(PyObject *mp);
47-
4846
// Export for '_ctypes' shared extension
4947
PyAPI_FUNC(Py_ssize_t) _PyDict_SizeOf(PyDictObject *);
5048

‎InternalDocs/garbage_collector.md

Copy file name to clipboardExpand all lines: InternalDocs/garbage_collector.md
+16-25Lines changed: 16 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -532,8 +532,8 @@ of `PyGC_Head` discussed in the `Memory layout and object structure`_ section:
532532
currently in. Instead, when that's needed, ad hoc tricks (like the
533533
`NEXT_MASK_UNREACHABLE` flag) are employed.
534534

535-
Optimization: delay tracking containers
536-
=======================================
535+
Optimization: delayed untracking containers
536+
===========================================
537537

538538
Certain types of containers cannot participate in a reference cycle, and so do
539539
not need to be tracked by the garbage collector. Untracking these objects
@@ -546,26 +546,17 @@ a container:
546546
2. When the container is examined by the garbage collector.
547547

548548
As a general rule, instances of atomic types aren't tracked and instances of
549-
non-atomic types (containers, user-defined objects...) are. However, some
550-
type-specific optimizations can be present in order to suppress the garbage
551-
collector footprint of simple instances. Some examples of native types that
552-
benefit from delayed tracking:
553-
554-
- Tuples containing only immutable objects (integers, strings etc,
555-
and recursively, tuples of immutable objects) do not need to be tracked. The
556-
interpreter creates a large number of tuples, many of which will not survive
557-
until garbage collection. It is therefore not worthwhile to untrack eligible
558-
tuples at creation time. Instead, all tuples except the empty tuple are tracked
559-
when created. During garbage collection it is determined whether any surviving
560-
tuples can be untracked. A tuple can be untracked if all of its contents are
561-
already not tracked. Tuples are examined for untracking in all garbage collection
562-
cycles. It may take more than one cycle to untrack a tuple.
563-
564-
- Dictionaries containing only immutable objects also do not need to be tracked.
565-
Dictionaries are untracked when created. If a tracked item is inserted into a
566-
dictionary (either as a key or value), the dictionary becomes tracked. During a
567-
full garbage collection (all generations), the collector will untrack any dictionaries
568-
whose contents are not tracked.
549+
non-atomic types (containers, user-defined objects...) are.
550+
551+
Tuples containing only immutable objects (integers, strings etc,
552+
and recursively, tuples of immutable objects) do not need to be tracked. The
553+
interpreter creates a large number of tuples, many of which will not survive
554+
until garbage collection. It is therefore not worthwhile to untrack eligible
555+
tuples at creation time. Instead, all tuples except the empty tuple are tracked
556+
when created. During garbage collection it is determined whether any surviving
557+
tuples can be untracked. A tuple can be untracked if all of its contents are
558+
already not tracked. Tuples are examined for untracking in all garbage collection
559+
cycles.
569560

570561
The garbage collector module provides the Python function `is_tracked(obj)`, which returns
571562
the current tracking status of the object. Subsequent garbage collections may change the
@@ -578,11 +569,11 @@ tracking status of the object.
578569
False
579570
>>> gc.is_tracked([])
580571
True
581-
>>> gc.is_tracked({})
572+
>>> gc.is_tracked(())
582573
False
574+
>>> gc.is_tracked({})
575+
True
583576
>>> gc.is_tracked({"a": 1})
584-
False
585-
>>> gc.is_tracked({"a": []})
586577
True
587578
```
588579

‎Lib/test/test_dict.py

Copy file name to clipboardExpand all lines: Lib/test/test_dict.py
-109Lines changed: 0 additions & 109 deletions
Original file line numberDiff line numberDiff line change
@@ -880,115 +880,6 @@ class C(object):
880880
gc.collect()
881881
self.assertIs(ref(), None, "Cycle was not collected")
882882

883-
def _not_tracked(self, t):
884-
# Nested containers can take several collections to untrack
885-
gc.collect()
886-
gc.collect()
887-
self.assertFalse(gc.is_tracked(t), t)
888-
889-
def _tracked(self, t):
890-
self.assertTrue(gc.is_tracked(t), t)
891-
gc.collect()
892-
gc.collect()
893-
self.assertTrue(gc.is_tracked(t), t)
894-
895-
def test_string_keys_can_track_values(self):
896-
# Test that this doesn't leak.
897-
for i in range(10):
898-
d = {}
899-
for j in range(10):
900-
d[str(j)] = j
901-
d["foo"] = d
902-
903-
@support.cpython_only
904-
def test_track_literals(self):
905-
# Test GC-optimization of dict literals
906-
x, y, z, w = 1.5, "a", (1, None), []
907-
908-
self._not_tracked({})
909-
self._not_tracked({x:(), y:x, z:1})
910-
self._not_tracked({1: "a", "b": 2})
911-
self._not_tracked({1: 2, (None, True, False, ()): int})
912-
self._not_tracked({1: object()})
913-
914-
# Dicts with mutable elements are always tracked, even if those
915-
# elements are not tracked right now.
916-
self._tracked({1: []})
917-
self._tracked({1: ([],)})
918-
self._tracked({1: {}})
919-
self._tracked({1: set()})
920-
921-
@support.cpython_only
922-
def test_track_dynamic(self):
923-
# Test GC-optimization of dynamically-created dicts
924-
class MyObject(object):
925-
pass
926-
x, y, z, w, o = 1.5, "a", (1, object()), [], MyObject()
927-
928-
d = dict()
929-
self._not_tracked(d)
930-
d[1] = "a"
931-
self._not_tracked(d)
932-
d[y] = 2
933-
self._not_tracked(d)
934-
d[z] = 3
935-
self._not_tracked(d)
936-
self._not_tracked(d.copy())
937-
d[4] = w
938-
self._tracked(d)
939-
self._tracked(d.copy())
940-
d[4] = None
941-
self._not_tracked(d)
942-
self._not_tracked(d.copy())
943-
944-
# dd isn't tracked right now, but it may mutate and therefore d
945-
# which contains it must be tracked.
946-
d = dict()
947-
dd = dict()
948-
d[1] = dd
949-
self._not_tracked(dd)
950-
self._tracked(d)
951-
dd[1] = d
952-
self._tracked(dd)
953-
954-
d = dict.fromkeys([x, y, z])
955-
self._not_tracked(d)
956-
dd = dict()
957-
dd.update(d)
958-
self._not_tracked(dd)
959-
d = dict.fromkeys([x, y, z, o])
960-
self._tracked(d)
961-
dd = dict()
962-
dd.update(d)
963-
self._tracked(dd)
964-
965-
d = dict(x=x, y=y, z=z)
966-
self._not_tracked(d)
967-
d = dict(x=x, y=y, z=z, w=w)
968-
self._tracked(d)
969-
d = dict()
970-
d.update(x=x, y=y, z=z)
971-
self._not_tracked(d)
972-
d.update(w=w)
973-
self._tracked(d)
974-
975-
d = dict([(x, y), (z, 1)])
976-
self._not_tracked(d)
977-
d = dict([(x, y), (z, w)])
978-
self._tracked(d)
979-
d = dict()
980-
d.update([(x, y), (z, 1)])
981-
self._not_tracked(d)
982-
d.update([(x, y), (z, w)])
983-
self._tracked(d)
984-
985-
@support.cpython_only
986-
def test_track_subtypes(self):
987-
# Dict subtypes are always tracked
988-
class MyDict(dict):
989-
pass
990-
self._tracked(MyDict())
991-
992883
def make_shared_key_dict(self, n):
993884
class C:
994885
pass

‎Misc/NEWS.d/next/Core_and_Builtins/2024-11-19-17-17-32.gh-issue-127010.9Cl4bb.rst

Copy file name to clipboard
+4Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
Simplify GC tracking of dictionaries. All dictionaries are tracked when
2+
created, rather than being lazily tracked when a trackable object was added
3+
to them. This simplifies the code considerably and results in a slight
4+
speedup.

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.