Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

gh-114314: ctypes: remove stgdict and switch to heap types #116458

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 60 commits into from
Mar 20, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
6155211
Move paramfunc into a new struct, PyStgInfo
encukou Feb 16, 2024
0171c30
Move the StgInfo out of StgDictObject and into the classes themselves
encukou Feb 16, 2024
acd91e9
Move StgInfo init/retrieval right after the corresponding StgDict one
encukou Feb 19, 2024
55af504
Move `size` from StgDict to StgInfo
encukou Feb 19, 2024
27e5c50
Move `align` from StgDict to StgInfo
encukou Feb 19, 2024
01bc637
Move `length` from StgDict to StgInfo, except for sizeof calculation
encukou Feb 19, 2024
c8017e9
Move part of the sizeof calculation
encukou Feb 19, 2024
40b012f
Move `ffi_type_pointer` from StgDict to StgInfo
encukou Feb 19, 2024
537defb
Move `proto` from StgDict to StgInfo. Doesn't work just yet.
encukou Feb 20, 2024
ace6623
Hack: Reserve space for StgInfo in static type objects
encukou Feb 20, 2024
6f775b9
If StgInfo is not initialized, act as if it's not there
encukou Feb 20, 2024
4aaa06a
Move `getfunc` & `setfunc`
encukou Feb 20, 2024
6808524
Move `argtypes`
encukou Feb 20, 2024
f031c00
Move `converters`
encukou Feb 20, 2024
535176c
Move `restype`
encukou Feb 20, 2024
5260003
Move `checker`
encukou Feb 20, 2024
0d3017e
Move `flags`
encukou Feb 20, 2024
269b825
Move `format`
encukou Feb 20, 2024
946f474
Move `ndim`
encukou Feb 20, 2024
607ed23
Move `shape`
encukou Feb 21, 2024
cb126b7
Replace more uses of StgDict
encukou Feb 21, 2024
fd04c91
Don't pass dicts to PyCStgDict_clone
encukou Feb 21, 2024
8dce549
Don't set StgDict as type dict; use name `attrdict` for the type __di…
encukou Feb 21, 2024
6705e0e
Remove more uses of StgDict
encukou Feb 21, 2024
8f99377
Remove StgDict
encukou Feb 21, 2024
daea3e5
Remove StgDict from comments, names, and platform-specific code
encukou Feb 21, 2024
f67211f
Switch PyCSimpleType_new to _init
encukou Feb 23, 2024
02aa9e1
Switch _SimpleCData to a heap type
encukou Feb 23, 2024
13feac7
Convert Struct and Union type _new to _init
encukou Feb 23, 2024
a68417b
Switch PyCPointerType_new to _init
encukou Feb 23, 2024
ad5d7b4
Switch PyCFuncPtrType_new to _init
encukou Feb 23, 2024
8d5e230
Switch CFuncPtr to heap type
encukou Feb 23, 2024
cdfae13
Switch Array to heap type
encukou Feb 23, 2024
2d80960
Switch _Pointer to a heap type
encukou Feb 23, 2024
3337ba4
Switch _ctypes.Union to heap type
encukou Feb 23, 2024
596cb89
Switch Structure to heap type
encukou Feb 23, 2024
91feb64
Make the type-adding macros more regular
encukou Feb 23, 2024
77c64fe
Switch _CData to a heap type
encukou Feb 23, 2024
0be46a1
Visit/decref type in PyCData visit/dealloc
encukou Feb 23, 2024
9be3520
Remove _HackyHeapType
encukou Feb 23, 2024
e523c2f
Decref attrdicts
encukou Mar 1, 2024
fb70e43
Delegate CType_Type deallocation to PyType_Type.tp_dealloc
encukou Mar 5, 2024
cc59dea
Fix Windows-specific crashes
neonene Mar 5, 2024
9bd9eb1
Update comments
encukou Mar 6, 2024
946a0b1
Add tests for metaclass details
encukou Mar 6, 2024
970b41d
Fix getting the module state
encukou Mar 7, 2024
2a450ad
Add a blurb, even though this *should* not affect usage
encukou Mar 7, 2024
9878809
Apply suggestions from code review
encukou Mar 11, 2024
b2ba89a
Inline PyStgInfo_From* & PyStgInfo_Init for performance
encukou Mar 11, 2024
605c30c
Make CType_Type's clear & dealloc safer
encukou Mar 11, 2024
41dea1b
Remove self decrefs in error cases in init functions
encukou Mar 11, 2024
be0888f
Add `static` to PyType_Spec
encukou Mar 11, 2024
ac6eb38
Remove redundant comment
encukou Mar 11, 2024
33be37f
Apply suggestions from code review
encukou Mar 11, 2024
0b9d0a7
Fix comment
encukou Mar 12, 2024
f02bad6
PyErr_WriteUnraisable clears the exception
encukou Mar 12, 2024
16faac7
Call tp_init for argument validation
encukou Mar 20, 2024
2461e50
Add test for swapped type creation
neonene Mar 20, 2024
ab90768
Merge branch 'main' into ctypes-no-stgdict
encukou Mar 20, 2024
de054af
Fix test_swapped_type_creation
encukou Mar 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Move flags
  • Loading branch information
encukou committed Mar 7, 2024
commit 0d3017eee53dfe40a84ab2a9bae2ec1f58ed670c
67 changes: 33 additions & 34 deletions 67 Modules/_ctypes/_ctypes.c
Original file line number Diff line number Diff line change
Expand Up @@ -655,14 +655,14 @@ StructUnionType_new(PyTypeObject *type, PyObject *args, PyObject *kwds, int isSt
Py_DECREF(result);
return NULL;
}
if (!isStruct) {
dict->flags |= TYPEFLAG_HASUNION;
}
StgInfo *info = PyStgInfo_Init(st, result);
if (!info) {
Py_DECREF(result);
return NULL;
}
if (!isStruct) {
info->flags |= TYPEFLAG_HASUNION;
}

/* replace the class dict by our updated stgdict, which holds info
about storage requirements of the instances */
Expand Down Expand Up @@ -713,8 +713,8 @@ StructUnionType_new(PyTypeObject *type, PyObject *args, PyObject *kwds, int isSt
Py_DECREF(result);
return NULL;
}
dict->flags &= ~DICTFLAG_FINAL; /* clear the 'final' flag in the subclass dict */
basedict->flags |= DICTFLAG_FINAL; /* set the 'final' flag in the baseclass dict */
info->flags &= ~DICTFLAG_FINAL; /* clear the 'final' flag in the subclass info */
baseinfo->flags |= DICTFLAG_FINAL; /* set the 'final' flag in the baseclass info */
return (PyObject *)result;
}
}
Expand Down Expand Up @@ -1218,7 +1218,7 @@ PyCPointerType_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
stginfo->length = 1;
stginfo->ffi_type_pointer = ffi_type_pointer;
stginfo->paramfunc = PyCPointerType_paramfunc;
stgdict->flags |= TYPEFLAG_ISPOINTER;
stginfo->flags |= TYPEFLAG_ISPOINTER;

if (PyDict_GetItemRef(typedict, &_Py_ID(_type_), &proto) < 0) {
Py_DECREF((PyObject *)result);
Expand Down Expand Up @@ -1683,8 +1683,8 @@ PyCArrayType_new(PyTypeObject *type, PyObject *args, PyObject *kwds)

itemalign = iteminfo->align;

if (itemdict->flags & (TYPEFLAG_ISPOINTER | TYPEFLAG_HASPOINTER))
stgdict->flags |= TYPEFLAG_HASPOINTER;
if (iteminfo->flags & (TYPEFLAG_ISPOINTER | TYPEFLAG_HASPOINTER))
stginfo->flags |= TYPEFLAG_HASPOINTER;

stginfo->size = itemsize * length;
stginfo->align = itemalign;
Expand Down Expand Up @@ -2271,21 +2271,21 @@ PyCSimpleType_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
switch (*proto_str) {
case 'z': /* c_char_p */
ml = &c_char_p_method;
stgdict->flags |= TYPEFLAG_ISPOINTER;
stginfo->flags |= TYPEFLAG_ISPOINTER;
break;
case 'Z': /* c_wchar_p */
ml = &c_wchar_p_method;
stgdict->flags |= TYPEFLAG_ISPOINTER;
stginfo->flags |= TYPEFLAG_ISPOINTER;
break;
case 'P': /* c_void_p */
ml = &c_void_p_method;
stgdict->flags |= TYPEFLAG_ISPOINTER;
stginfo->flags |= TYPEFLAG_ISPOINTER;
break;
case 's':
case 'X':
case 'O':
ml = NULL;
stgdict->flags |= TYPEFLAG_ISPOINTER;
stginfo->flags |= TYPEFLAG_ISPOINTER;
break;
default:
ml = NULL;
Expand Down Expand Up @@ -2578,7 +2578,7 @@ make_funcptrtype_dict(StgDictObject *stgdict, StgInfo *stginfo)
Py_XDECREF(ob);
return -1;
}
stgdict->flags = PyLong_AsUnsignedLongMask(ob) | TYPEFLAG_ISPOINTER;
stginfo->flags = PyLong_AsUnsignedLongMask(ob) | TYPEFLAG_ISPOINTER;
Py_DECREF(ob);

/* _argtypes_ is optional... */
Expand Down Expand Up @@ -2686,7 +2686,7 @@ PyCFuncPtrType_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
Py_DECREF((PyObject *)stgdict);
return NULL;
}
stgdict->flags |= TYPEFLAG_ISPOINTER;
stginfo->flags |= TYPEFLAG_ISPOINTER;

/* replace the class dict by our updated storage dict */
if (-1 == PyDict_Update((PyObject *)stgdict, result->tp_dict)) {
Expand Down Expand Up @@ -2949,7 +2949,14 @@ PyCData_reduce(PyObject *myself, PyObject *args)
{
CDataObject *self = (CDataObject *)myself;

if (PyObject_stgdict(myself)->flags & (TYPEFLAG_ISPOINTER|TYPEFLAG_HASPOINTER)) {
ctypes_state *st = GLOBAL_STATE();
StgInfo *info;
if (PyStgInfo_FromObject(st, myself, &info) < 0) {
return NULL;
}
assert(info);

if (info->flags & (TYPEFLAG_ISPOINTER|TYPEFLAG_HASPOINTER)) {
PyErr_SetString(PyExc_ValueError,
"ctypes objects containing pointers cannot be pickled");
return NULL;
Expand Down Expand Up @@ -3108,7 +3115,7 @@ PyCData_FromBaseObj(PyObject *type, PyObject *base, Py_ssize_t index, char *adr)
}
assert(info);

dict->flags |= DICTFLAG_FINAL;
info->flags |= DICTFLAG_FINAL;
cmem = (CDataObject *)((PyTypeObject *)type)->tp_alloc((PyTypeObject *)type, 0);
if (cmem == NULL) {
return NULL;
Expand Down Expand Up @@ -3140,28 +3147,25 @@ PyObject *
PyCData_AtAddress(PyObject *type, void *buf)
{
CDataObject *pd;
StgDictObject *dict;

if (PySys_Audit("ctypes.cdata", "n", (Py_ssize_t)buf) < 0) {
return NULL;
}

assert(PyType_Check(type));
dict = PyType_stgdict(type);
if (!dict) {
PyErr_SetString(PyExc_TypeError,
"abstract class");
return NULL;
}

ctypes_state *st = GLOBAL_STATE();
StgInfo *info;
if (PyStgInfo_FromType(st, type, &info) < 0) {
return NULL;
}
assert(info);
if (!info) {
PyErr_SetString(PyExc_TypeError,
"abstract class");
return NULL;
}

dict->flags |= DICTFLAG_FINAL;
info->flags |= DICTFLAG_FINAL;

pd = (CDataObject *)((PyTypeObject *)type)->tp_alloc((PyTypeObject *)type, 0);
if (!pd) {
Expand Down Expand Up @@ -3373,7 +3377,7 @@ GenericPyCData_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
}
assert(info);

dict->flags |= DICTFLAG_FINAL;
info->flags |= DICTFLAG_FINAL;

obj = (CDataObject *)type->tp_alloc(type, 0);
if (!obj)
Expand Down Expand Up @@ -3858,7 +3862,6 @@ PyCFuncPtr_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
PyCFuncPtrObject *self;
PyObject *callable;
StgDictObject *dict;
CThunkObject *thunk;

if (PyTuple_GET_SIZE(args) == 0)
Expand Down Expand Up @@ -3910,7 +3913,6 @@ PyCFuncPtr_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
*/

ctypes_state *st = GLOBAL_STATE();
dict = PyType_stgdict((PyObject *)type);
StgInfo *info;
if (PyStgInfo_FromType(st, (PyObject *)type, &info) < 0) {
return NULL;
Expand All @@ -3922,12 +3924,11 @@ PyCFuncPtr_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
" no argtypes");
return NULL;
}
assert(dict);

thunk = _ctypes_alloc_callback(callable,
info->argtypes,
info->restype,
dict->flags);
info->flags);
if (!thunk)
return NULL;

Expand Down Expand Up @@ -4267,7 +4268,6 @@ PyCFuncPtr_call(PyCFuncPtrObject *self, PyObject *inargs, PyObject *kwds)
PyObject *converters;
PyObject *checker;
PyObject *argtypes;
StgDictObject *dict = PyObject_stgdict((PyObject *)self);
PyObject *result;
PyObject *callargs;
PyObject *errcheck;
Expand All @@ -4287,7 +4287,6 @@ PyCFuncPtr_call(PyCFuncPtrObject *self, PyObject *inargs, PyObject *kwds)
}
assert(info); /* Cannot be NULL for PyCFuncPtrObject instances */

assert(dict); /* Cannot be NULL for PyCFuncPtrObject instances */
restype = self->restype ? self->restype : info->restype;
converters = self->converters ? self->converters : info->converters;
checker = self->checker ? self->checker : info->checker;
Expand Down Expand Up @@ -4341,7 +4340,7 @@ PyCFuncPtr_call(PyCFuncPtrObject *self, PyObject *inargs, PyObject *kwds)
int actual = Py_SAFE_DOWNCAST(PyTuple_GET_SIZE(callargs),
Py_ssize_t, int);

if ((dict->flags & FUNCFLAG_CDECL) == FUNCFLAG_CDECL) {
if ((info->flags & FUNCFLAG_CDECL) == FUNCFLAG_CDECL) {
/* For cdecl functions, we allow more actual arguments
than the length of the argtypes tuple.
*/
Expand Down Expand Up @@ -4371,7 +4370,7 @@ PyCFuncPtr_call(PyCFuncPtrObject *self, PyObject *inargs, PyObject *kwds)
piunk,
self->iid,
#endif
dict->flags,
info->flags,
converters,
restype,
checker);
Expand Down
3 changes: 2 additions & 1 deletion 3 Modules/_ctypes/ctypes.h
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@ typedef struct {
PyObject *converters; /* tuple([t.from_param for t in argtypes]) */
PyObject *restype; /* CDataObject or NULL */
PyObject *checker;
int flags; /* calling convention and such */
} StgInfo;

// Get a PyCTypeDataObject. These Return -1 on error, 0 if "not found", 1 on OK.
Expand Down Expand Up @@ -282,7 +283,7 @@ typedef struct {
//PyObject *converters; /* tuple([t.from_param for t in argtypes]) */
//PyObject *restype; /* CDataObject or NULL */
//PyObject *checker;
int flags; /* calling convention and such */
//int flags; /* calling convention and such */

/* pep3118 fields, pointers need PyMem_Free */
char *format;
Expand Down
35 changes: 16 additions & 19 deletions 35 Modules/_ctypes/stgdict.c
Original file line number Diff line number Diff line change
Expand Up @@ -435,7 +435,7 @@ _ctypes_alloc_format_padding(const char *prefix, Py_ssize_t padding)
int
PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct)
{
StgDictObject *stgdict, *basedict;
StgDictObject *stgdict;
Py_ssize_t len, offset, size, align, i;
Py_ssize_t union_size, total_align, aligned_size;
Py_ssize_t field_size = 0;
Expand Down Expand Up @@ -531,7 +531,7 @@ PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct
/* If this structure/union is already marked final we cannot assign
_fields_ anymore. */

if (stgdict->flags & DICTFLAG_FINAL) {/* is final ? */
if (stginfo->flags & DICTFLAG_FINAL) {/* is final ? */
PyErr_SetString(PyExc_AttributeError,
"_fields_ is final");
return -1;
Expand All @@ -545,22 +545,19 @@ PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct
if (stginfo->ffi_type_pointer.elements)
PyMem_Free(stginfo->ffi_type_pointer.elements);

basedict = PyType_stgdict((PyObject *)((PyTypeObject *)type)->tp_base);
if (basedict) {
stgdict->flags |= (basedict->flags &
(TYPEFLAG_HASUNION | TYPEFLAG_HASBITFIELD));
}

StgInfo *baseinfo;
if (PyStgInfo_FromType(st, (PyObject *)((PyTypeObject *)type)->tp_base,
&baseinfo) < 0) {
return -1;
}

if (baseinfo) {
stginfo->flags |= (baseinfo->flags &
(TYPEFLAG_HASUNION | TYPEFLAG_HASBITFIELD));
}
if (!isStruct) {
stgdict->flags |= TYPEFLAG_HASUNION;
stginfo->flags |= TYPEFLAG_HASUNION;
}
if (basedict) {
if (baseinfo) {
size = offset = baseinfo->size;
align = baseinfo->align;
union_size = 0;
Expand Down Expand Up @@ -640,12 +637,12 @@ PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct
assert(info);

stginfo->ffi_type_pointer.elements[ffi_ofs + i] = &info->ffi_type_pointer;
if (dict->flags & (TYPEFLAG_ISPOINTER | TYPEFLAG_HASPOINTER))
stgdict->flags |= TYPEFLAG_HASPOINTER;
stgdict->flags |= dict->flags & (TYPEFLAG_HASUNION | TYPEFLAG_HASBITFIELD);
dict->flags |= DICTFLAG_FINAL; /* mark field type final */
if (info->flags & (TYPEFLAG_ISPOINTER | TYPEFLAG_HASPOINTER))
stginfo->flags |= TYPEFLAG_HASPOINTER;
stginfo->flags |= info->flags & (TYPEFLAG_HASUNION | TYPEFLAG_HASBITFIELD);
info->flags |= DICTFLAG_FINAL; /* mark field type final */
if (PyTuple_Size(pair) == 3) { /* bits specified */
stgdict->flags |= TYPEFLAG_HASBITFIELD;
stginfo->flags |= TYPEFLAG_HASBITFIELD;
switch(info->ffi_type_pointer.type) {
case FFI_TYPE_UINT8:
case FFI_TYPE_UINT16:
Expand Down Expand Up @@ -983,7 +980,7 @@ PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct
if (num_ffi_types > 0) {
memset(structs, 0, num_ffi_types * sizeof(ffi_type));
}
if (ffi_ofs && (basedict != NULL)) {
if (ffi_ofs && (baseinfo != NULL)) {
memcpy(element_types,
baseinfo->ffi_type_pointer.elements,
ffi_ofs * sizeof(ffi_type *));
Expand Down Expand Up @@ -1082,12 +1079,12 @@ PyCStructUnionType_update_stgdict(PyObject *type, PyObject *fields, int isStruct

/* We did check that this flag was NOT set above, it must not
have been set until now. */
if (stgdict->flags & DICTFLAG_FINAL) {
if (stginfo->flags & DICTFLAG_FINAL) {
PyErr_SetString(PyExc_AttributeError,
"Structure or union cannot contain itself");
return -1;
}
stgdict->flags |= DICTFLAG_FINAL;
stginfo->flags |= DICTFLAG_FINAL;

return MakeAnonFields(type);
}
Morty Proxy This is a proxified and sanitized view of the page, visit original site.