Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 985679f

Browse filesBrowse files
encukouhugovkericsnowcurrently
authored
gh-110812: Isolating Extension Modules HOWTO: List GC-related gotchas (GH-111504)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
1 parent a646560 commit 985679f
Copy full SHA for 985679f

File tree

1 file changed

+97
-6
lines changed
Filter options

1 file changed

+97
-6
lines changed

‎Doc/howto/isolating-extensions.rst

Copy file name to clipboardExpand all lines: Doc/howto/isolating-extensions.rst
+97-6Lines changed: 97 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -339,12 +339,44 @@ That is, heap types should:
339339
- Define a traverse function using ``Py_tp_traverse``, which
340340
visits the type (e.g. using :c:expr:`Py_VISIT(Py_TYPE(self))`).
341341

342-
Please refer to the :ref:`the documentation <type-structs>` of
342+
Please refer to the the documentation of
343343
:c:macro:`Py_TPFLAGS_HAVE_GC` and :c:member:`~PyTypeObject.tp_traverse`
344344
for additional considerations.
345345

346-
If your traverse function delegates to the ``tp_traverse`` of its base class
347-
(or another type), ensure that ``Py_TYPE(self)`` is visited only once.
346+
The API for defining heap types grew organically, leaving it
347+
somewhat awkward to use in its current state.
348+
The following sections will guide you through common issues.
349+
350+
351+
``tp_traverse`` in Python 3.8 and lower
352+
.......................................
353+
354+
The requirement to visit the type from ``tp_traverse`` was added in Python 3.9.
355+
If you support Python 3.8 and lower, the traverse function must *not*
356+
visit the type, so it must be more complicated::
357+
358+
static int my_traverse(PyObject *self, visitproc visit, void *arg)
359+
{
360+
if (Py_Version >= 0x03090000) {
361+
Py_VISIT(Py_TYPE(self));
362+
}
363+
return 0;
364+
}
365+
366+
Unfortunately, :c:data:`Py_Version` was only added in Python 3.11.
367+
As a replacement, use:
368+
369+
* :c:macro:`PY_VERSION_HEX`, if not using the stable ABI, or
370+
* :py:data:`sys.version_info` (via :c:func:`PySys_GetObject` and
371+
:c:func:`PyArg_ParseTuple`).
372+
373+
374+
Delegating ``tp_traverse``
375+
..........................
376+
377+
If your traverse function delegates to the :c:member:`~PyTypeObject.tp_traverse`
378+
of its base class (or another type), ensure that ``Py_TYPE(self)`` is visited
379+
only once.
348380
Note that only heap type are expected to visit the type in ``tp_traverse``.
349381

350382
For example, if your traverse function includes::
@@ -356,11 +388,70 @@ For example, if your traverse function includes::
356388
if (base->tp_flags & Py_TPFLAGS_HEAPTYPE) {
357389
// a heap type's tp_traverse already visited Py_TYPE(self)
358390
} else {
359-
Py_VISIT(Py_TYPE(self));
391+
if (Py_Version >= 0x03090000) {
392+
Py_VISIT(Py_TYPE(self));
393+
}
360394
}
361395

362-
It is not necessary to handle the type's reference count in ``tp_new``
363-
and ``tp_clear``.
396+
It is not necessary to handle the type's reference count in
397+
:c:member:`~PyTypeObject.tp_new` and :c:member:`~PyTypeObject.tp_clear`.
398+
399+
400+
Defining ``tp_dealloc``
401+
.......................
402+
403+
If your type has a custom :c:member:`~PyTypeObject.tp_dealloc` function,
404+
it needs to:
405+
406+
- call :c:func:`PyObject_GC_UnTrack` before any fields are invalidated, and
407+
- decrement the reference count of the type.
408+
409+
To keep the type valid while ``tp_free`` is called, the type's refcount needs
410+
to be decremented *after* the instance is deallocated. For example::
411+
412+
static void my_dealloc(PyObject *self)
413+
{
414+
PyObject_GC_UnTrack(self);
415+
...
416+
PyTypeObject *type = Py_TYPE(self);
417+
type->tp_free(self);
418+
Py_DECREF(type);
419+
}
420+
421+
The default ``tp_dealloc`` function does this, so
422+
if your type does *not* override
423+
``tp_dealloc`` you don't need to add it.
424+
425+
426+
Not overriding ``tp_free``
427+
..........................
428+
429+
The :c:member:`~PyTypeObject.tp_free` slot of a heap type must be set to
430+
:c:func:`PyObject_GC_Del`.
431+
This is the default; do not override it.
432+
433+
434+
Avoiding ``PyObject_New``
435+
.........................
436+
437+
GC-tracked objects need to be allocated using GC-aware functions.
438+
439+
If you use use :c:func:`PyObject_New` or :c:func:`PyObject_NewVar`:
440+
441+
- Get and call type's :c:member:`~PyTypeObject.tp_alloc` slot, if possible.
442+
That is, replace ``TYPE *o = PyObject_New(TYPE, typeobj)`` with::
443+
444+
TYPE *o = typeobj->tp_alloc(typeobj, 0);
445+
446+
Replace ``o = PyObject_NewVar(TYPE, typeobj, size)`` with the same,
447+
but use size instead of the 0.
448+
449+
- If the above is not possible (e.g. inside a custom ``tp_alloc``),
450+
call :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar`::
451+
452+
TYPE *o = PyObject_GC_New(TYPE, typeobj);
453+
454+
TYPE *o = PyObject_GC_NewVar(TYPE, typeobj, size);
364455

365456

366457
Module State Access from Classes

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.