Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 524a7f7

Browse filesBrowse files
encukoubarneygale
encukou
and
barneygale
authored
gh-103968: Deprecate creating heap types whose metaclass has custom tp_new. (GH-103972)
(That's a mouthful of an edge case!) Co-authored-by: Barney Gale <barney.gale@gmail.com>
1 parent 423d7fa commit 524a7f7
Copy full SHA for 524a7f7

File tree

Expand file treeCollapse file tree

6 files changed

+106
-12
lines changed
Filter options
Expand file treeCollapse file tree

6 files changed

+106
-12
lines changed

‎Doc/c-api/type.rst

Copy file name to clipboardExpand all lines: Doc/c-api/type.rst
+24-2Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -256,8 +256,13 @@ The following functions and structs are used to create
256256
The metaclass *metaclass* is used to construct the resulting type object.
257257
When *metaclass* is ``NULL``, the metaclass is derived from *bases*
258258
(or *Py_tp_base[s]* slots if *bases* is ``NULL``, see below).
259-
Note that metaclasses that override
260-
:c:member:`~PyTypeObject.tp_new` are not supported.
259+
260+
Metaclasses that override :c:member:`~PyTypeObject.tp_new` are not
261+
supported.
262+
(For backwards compatibility, other ``PyType_From*`` functions allow
263+
such metaclasses. They ignore ``tp_new``, which may result in incomplete
264+
initialization. This is deprecated and in Python 3.14+ such metaclasses will
265+
not be supported.)
261266
262267
The *bases* argument can be used to specify base classes; it can either
263268
be only one class or a tuple of classes.
@@ -305,6 +310,11 @@ The following functions and structs are used to create
305310
The function now finds and uses a metaclass corresponding to the provided
306311
base classes. Previously, only :class:`type` instances were returned.
307312
313+
The :c:member:`~PyTypeObject.tp_new` of the metaclass is *ignored*.
314+
which may result in incomplete initialization.
315+
Creating classes whose metaclass overrides
316+
:c:member:`~PyTypeObject.tp_new` is deprecated and in Python 3.14+ it
317+
will be no longer allowed.
308318
309319
.. c:function:: PyObject* PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases)
310320
@@ -317,6 +327,12 @@ The following functions and structs are used to create
317327
The function now finds and uses a metaclass corresponding to the provided
318328
base classes. Previously, only :class:`type` instances were returned.
319329
330+
The :c:member:`~PyTypeObject.tp_new` of the metaclass is *ignored*.
331+
which may result in incomplete initialization.
332+
Creating classes whose metaclass overrides
333+
:c:member:`~PyTypeObject.tp_new` is deprecated and in Python 3.14+ it
334+
will be no longer allowed.
335+
320336
.. c:function:: PyObject* PyType_FromSpec(PyType_Spec *spec)
321337
322338
Equivalent to ``PyType_FromMetaclass(NULL, NULL, spec, NULL)``.
@@ -327,6 +343,12 @@ The following functions and structs are used to create
327343
base classes provided in *Py_tp_base[s]* slots.
328344
Previously, only :class:`type` instances were returned.
329345
346+
The :c:member:`~PyTypeObject.tp_new` of the metaclass is *ignored*.
347+
which may result in incomplete initialization.
348+
Creating classes whose metaclass overrides
349+
:c:member:`~PyTypeObject.tp_new` is deprecated and in Python 3.14+ it
350+
will be no longer allowed.
351+
330352
.. c:type:: PyType_Spec
331353
332354
Structure defining a type's behavior.

‎Doc/whatsnew/3.12.rst

Copy file name to clipboardExpand all lines: Doc/whatsnew/3.12.rst
+20Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1320,6 +1320,21 @@ Porting to Python 3.12
13201320
available on debug builds. If you happen to be using it then you'll
13211321
need to start using ``_Py_GetGlobalRefTotal()``.
13221322

1323+
* The following functions now select an appropriate metaclass for the newly
1324+
created type:
1325+
1326+
* :c:func:`PyType_FromSpec`
1327+
* :c:func:`PyType_FromSpecWithBases`
1328+
* :c:func:`PyType_FromModuleAndSpec`
1329+
1330+
Creating classes whose metaclass overrides :c:member:`~PyTypeObject.tp_new`
1331+
is deprecated, and in Python 3.14+ it will be disallowed.
1332+
Note that these functions ignore ``tp_new`` of the metaclass, possibly
1333+
allowing incomplete initialization.
1334+
1335+
Note that :c:func:`PyType_FromMetaclass` (added in Python 3.12)
1336+
already disallows creating classes whose metaclass overrides ``tp_new``.
1337+
13231338
Deprecated
13241339
----------
13251340

@@ -1396,6 +1411,11 @@ Deprecated
13961411
* ``_PyErr_ChainExceptions`` is deprecated. Use ``_PyErr_ChainExceptions1``
13971412
instead. (Contributed by Irit Katriel in :gh:`102192`.)
13981413

1414+
* Using :c:func:`PyType_FromSpec`, :c:func:`PyType_FromSpecWithBases`
1415+
or :c:func:`PyType_FromModuleAndSpec` to create a class whose metaclass
1416+
overrides :c:member:`~PyTypeObject.tp_new` is deprecated.
1417+
Call the metaclass instead.
1418+
13991419
Removed
14001420
-------
14011421

‎Lib/test/test_capi/test_misc.py

Copy file name to clipboardExpand all lines: Lib/test/test_capi/test_misc.py
+14Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -681,6 +681,20 @@ def test_heaptype_with_custom_metaclass(self):
681681
with self.assertRaisesRegex(TypeError, msg):
682682
t = _testcapi.pytype_fromspec_meta(_testcapi.HeapCTypeMetaclassCustomNew)
683683

684+
def test_heaptype_with_custom_metaclass_deprecation(self):
685+
# gh-103968: a metaclass with custom tp_new is deprecated, but still
686+
# allowed for functions that existed in 3.11
687+
# (PyType_FromSpecWithBases is used here).
688+
class Base(metaclass=_testcapi.HeapCTypeMetaclassCustomNew):
689+
pass
690+
691+
with warnings_helper.check_warnings(
692+
('.*custom tp_new.*in Python 3.14.*', DeprecationWarning),
693+
):
694+
sub = _testcapi.make_type_with_base(Base)
695+
self.assertTrue(issubclass(sub, Base))
696+
self.assertIsInstance(sub, _testcapi.HeapCTypeMetaclassCustomNew)
697+
684698
def test_multiple_inheritance_ctypes_with_weakref_or_dict(self):
685699

686700
with self.assertRaises(TypeError):

‎Misc/NEWS.d/next/C API/2023-04-28-18-04-38.gh-issue-103968.EnVvOx.rst

Copy file name to clipboard
+4Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
:c:func:`PyType_FromSpec` and its variants now allow creating classes whose
2+
metaclass overrides :c:member:`~PyTypeObject.tp_new`. The ``tp_new`` is
3+
ignored. This behavior is deprecated and will be disallowed in 3.14+. The
4+
new :c:func:`PyType_FromMetaclass` already disallows it.

‎Modules/_testcapi/heaptype.c

Copy file name to clipboardExpand all lines: Modules/_testcapi/heaptype.c
+15-1Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ static PyObject *pytype_fromspec_meta(PyObject* self, PyObject *meta)
2222
"_testcapi.HeapCTypeViaMetaclass",
2323
sizeof(PyObject),
2424
0,
25-
Py_TPFLAGS_DEFAULT,
25+
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
2626
HeapCTypeViaMetaclass_slots
2727
};
2828

@@ -385,6 +385,19 @@ make_immutable_type_with_base(PyObject *self, PyObject *base)
385385
return PyType_FromSpecWithBases(&ImmutableSubclass_spec, base);
386386
}
387387

388+
static PyObject *
389+
make_type_with_base(PyObject *self, PyObject *base)
390+
{
391+
assert(PyType_Check(base));
392+
PyType_Spec ImmutableSubclass_spec = {
393+
.name = "_testcapi.Subclass",
394+
.basicsize = (int)((PyTypeObject*)base)->tp_basicsize,
395+
.slots = empty_type_slots,
396+
.flags = Py_TPFLAGS_DEFAULT,
397+
};
398+
return PyType_FromSpecWithBases(&ImmutableSubclass_spec, base);
399+
}
400+
388401

389402
static PyMethodDef TestMethods[] = {
390403
{"pytype_fromspec_meta", pytype_fromspec_meta, METH_O},
@@ -397,6 +410,7 @@ static PyMethodDef TestMethods[] = {
397410
test_from_spec_invalid_metatype_inheritance,
398411
METH_NOARGS},
399412
{"make_immutable_type_with_base", make_immutable_type_with_base, METH_O},
413+
{"make_type_with_base", make_type_with_base, METH_O},
400414
{NULL},
401415
};
402416

‎Objects/typeobject.c

Copy file name to clipboardExpand all lines: Objects/typeobject.c
+29-9Lines changed: 29 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3950,9 +3950,10 @@ check_basicsize_includes_size_and_offsets(PyTypeObject* type)
39503950
return 1;
39513951
}
39523952

3953-
PyObject *
3954-
PyType_FromMetaclass(PyTypeObject *metaclass, PyObject *module,
3955-
PyType_Spec *spec, PyObject *bases_in)
3953+
static PyObject *
3954+
_PyType_FromMetaclass_impl(
3955+
PyTypeObject *metaclass, PyObject *module,
3956+
PyType_Spec *spec, PyObject *bases_in, int _allow_tp_new)
39563957
{
39573958
/* Invariant: A non-NULL value in one of these means this function holds
39583959
* a strong reference or owns allocated memory.
@@ -4127,9 +4128,21 @@ PyType_FromMetaclass(PyTypeObject *metaclass, PyObject *module,
41274128
goto finally;
41284129
}
41294130
if (metaclass->tp_new != PyType_Type.tp_new) {
4130-
PyErr_SetString(PyExc_TypeError,
4131-
"Metaclasses with custom tp_new are not supported.");
4132-
goto finally;
4131+
if (_allow_tp_new) {
4132+
if (PyErr_WarnFormat(
4133+
PyExc_DeprecationWarning, 1,
4134+
"Using PyType_Spec with metaclasses that have custom "
4135+
"tp_new is deprecated and will no longer be allowed in "
4136+
"Python 3.14.") < 0) {
4137+
goto finally;
4138+
}
4139+
}
4140+
else {
4141+
PyErr_SetString(
4142+
PyExc_TypeError,
4143+
"Metaclasses with custom tp_new are not supported.");
4144+
goto finally;
4145+
}
41334146
}
41344147

41354148
/* Calculate best base, and check that all bases are type objects */
@@ -4316,22 +4329,29 @@ PyType_FromMetaclass(PyTypeObject *metaclass, PyObject *module,
43164329
return (PyObject*)res;
43174330
}
43184331

4332+
PyObject *
4333+
PyType_FromMetaclass(PyTypeObject *metaclass, PyObject *module,
4334+
PyType_Spec *spec, PyObject *bases_in)
4335+
{
4336+
return _PyType_FromMetaclass_impl(metaclass, module, spec, bases_in, 0);
4337+
}
4338+
43194339
PyObject *
43204340
PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases)
43214341
{
4322-
return PyType_FromMetaclass(NULL, module, spec, bases);
4342+
return _PyType_FromMetaclass_impl(NULL, module, spec, bases, 1);
43234343
}
43244344

43254345
PyObject *
43264346
PyType_FromSpecWithBases(PyType_Spec *spec, PyObject *bases)
43274347
{
4328-
return PyType_FromMetaclass(NULL, NULL, spec, bases);
4348+
return _PyType_FromMetaclass_impl(NULL, NULL, spec, bases, 1);
43294349
}
43304350

43314351
PyObject *
43324352
PyType_FromSpec(PyType_Spec *spec)
43334353
{
4334-
return PyType_FromMetaclass(NULL, NULL, spec, NULL);
4354+
return _PyType_FromMetaclass_impl(NULL, NULL, spec, NULL, 1);
43354355
}
43364356

43374357
PyObject *

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.