Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

gh-74695: Add support for ctypes.c_bool on opposite endian systems #127280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
Loading
from

Conversation

zbarnett
Copy link

@zbarnett zbarnett commented Nov 26, 2024

@ghost
Copy link

ghost commented Nov 26, 2024

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

bedevere-app bot commented Nov 26, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@bedevere-app
Copy link

bedevere-app bot commented Nov 26, 2024

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@zbarnett
Copy link
Author

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

I'm thinking this probably won't need a news entry since it's just fixing a small bug that's been outstanding for quite a while.

But I'm sure it will affect someone's workflow.

@skirpichev
Copy link
Member

If it's a user-visible change - it worth a news entry.

Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementation looks pretty good, apart from my nitpicks. This needs a test though, could you add one somewhere in test_ctypes? (You should be able to just retrofit the reproducer from the issue as a test.)

Comment on lines 2 to 3

Previously, on a little endian system, attempting to use a ``c_bool`` field in a ``BigEndianStructure`` would result in an exception (TypeError: This type does not support other endian: <class 'ctypes.c_bool'>). Now, it works as expected.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love the effort on the blurb entry, but it doesn't need to be nearly this long. I think just the first sentence is fine.

Suggested change
Previously, on a little endian system, attempting to use a ``c_bool`` field in a ``BigEndianStructure`` would result in an exception (TypeError: This type does not support other endian: <class 'ctypes.c_bool'>). Now, it works as expected.

@@ -0,0 +1,3 @@
Added support for ``ctypes.c_bool`` for structs/unions that have an endianness opposite of the current system.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's use a reference:

Suggested change
Added support for ``ctypes.c_bool`` for structs/unions that have an endianness opposite of the current system.
Added support for :class:`ctypes.c_bool` for structs/unions that have an endianness opposite of the current system.

@@ -263,6 +263,8 @@ class c_void_p(_SimpleCData):

class c_bool(_SimpleCData):
_type_ = "?"
c_bool.__ctype_le__ = c_bool.__ctype_be__ = c_bool
_check_size(c_bool)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not disagreeing that this should be here, I'd prefer to keep unrelated changes out of a PR. Could you open a seperate PR that adds this? (No need for an issue or blurb entry on it, just the one line change should be fine; tag me and I'll add skip issue and skip news.)

Suggested change
_check_size(c_bool)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate the input but it's not really worth my time to split this single line out into a separate PR. Any real reason it can't ride along with this one?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't really like to accept changes that are unrelated to the actual fix. You can leave it in, but expect pushback from other reviewers.

Lib/ctypes/__init__.py Show resolved Hide resolved
@zbarnett
Copy link
Author

zbarnett commented Dec 1, 2024

The implementation looks pretty good, apart from my nitpicks. This needs a test though, could you add one somewhere in test_ctypes? (You should be able to just retrofit the reproducer from the issue as a test.)

Hey @ZeroIntensity, thanks for the review! This is my first time contributing to Python so I'm still getting a feel for how this project works. I just added some tests and cleaned up the news blurb.

@encukou
Copy link
Member

encukou commented Dec 3, 2024

As I noted on the issue, c_bool not being swappable is intended behaviour -- like pointers, it's a platform-specific type.

@ZeroIntensity
Copy link
Member

As far as I know, basically all systems interpret a bool as 1 byte, but I guess it's not always true. Why not just apply the __ctype_le__ and __ctype_be__ when sizeof(c_bool) == 1?

@encukou
Copy link
Member

encukou commented Dec 4, 2024

Then, using c_bool would break on some platforms.

If you know your bools are single bytes, you should use c_uint8 (or its synonym c_byte) -- that behaves the same everywhere.
Is there a use case where you can't do this?

@ZeroIntensity
Copy link
Member

It would break, but I don't think c_int8 is much better--if you put that in a struct, that would crash on all the systems that don't have single-byte bools. Opposite endian support here is sort of a nice convenience thing for when you know that it works.

@encukou
Copy link
Member

encukou commented Dec 6, 2024

that would crash

What would crash?

You can't really use the struct in C, where working with it would crash. Non-native endianness is mostly useful for data sent from another system. (Arguably it would fit better in struct rather than ctypes, but let's not go there today.)

For data interchange, you really should be using fixed-width types, e.g. int32_t, rather than platform-specific ones like int. The latter would “crash” on other systems.
In ctypes, mainly for historical reasons AFAIK, c_int32 is a synonym for c_int (or similar). If they were separate types, __ctype_le__ and __ctype_ge__ would, IMO, only make sense for c_int32.

@ZeroIntensity
Copy link
Member

What would crash?

Using a c_int8 in place of a c_bool somewhere would segfault if bool isn't a single byte, wouldn't it?

I can't say much about use cases: I haven't ever used ctypes for data exchange, so I'd be mostly speculating. I'd like to hear from @zbarnett before closing this.

@encukou
Copy link
Member

encukou commented Dec 12, 2024

Sorry for dropping the ball here.

Using a c_int8 in place of a c_bool somewhere would segfault if bool isn't a single byte, wouldn't it?

Generally yes, but here, “somewhere” is inside a struct with a non-native byte order. That is, a struct that's not meant for the current architecture. You can't quite use it from C.

I'd like to hear from @zbarnett before closing this.

Same here. @zbarnett, do you have a use case for this?

@zbarnett
Copy link
Author

I appreciate the time you both have taken to think through this.

I'm using ctypes to read/write data across the network to other systems that have opposite endianness. I am currently using c_uint8 as a workaround but it's a bit clunky having to deal with 0 and 1 instead of False and True. In almost all places this is being used, I'm also setting the bit length to 1 and it's often not aligned on a byte boundary anyway.

Regarding portability, aren't all the builtin types in ctypes provided assuming some things are constant (like bit endianness) that aren't actually 100% constant across all systems? I had, probably wrongly, assumed that a bool would almost always be represented by 8 bits. But I do see the value in explicitly defining the size in bits of all the types when interfacing with other systems. In that case, would we want something like c_bool1, c_bool8, c_bool32, etc. to allow a precise size to be defined but still allow usage of True/False in the Python code?

Either way, I would assume that if c_int supports other endian, c_bool should too. If anything, c_bool would be less problematic than c_int in most cases.

@encukou
Copy link
Member

encukou commented Dec 17, 2024

Thanks for the input!

Yes, I think c_bool1, c_bool8, c_bool32, etc. would fit better than adding c_bool.__ctype_le__, but we can do even better (eventually).
The feature request is similar to wanting to convert a C integer to a Python IntEnum. Both could be served by custom getter/setter conversion functions (here, ideally something as simple as getter=bool), so we decouple the representation of the C integer from how it's mapped to a Python object.

In almost all places this is being used, I'm also setting the bit length to 1 and it's often not aligned on a byte boundary anyway.

You also want c_uint8 for that. In the ms_struct layout, the offsets depend on the size of the type, not just the bit length of the field.

I'm sorry for complicating your simple use case, but if we add something that's hard to generalize, we'll need to deal with the edge cases piling up later :(

If anything, c_bool would be less problematic than c_int in most cases.

Unfortunately, bool is a can of worms -- for cross-platform data exchange at least.

@Yhg1s Yhg1s removed the needs backport to 3.12 only security fixes label Apr 8, 2025
@serhiy-storchaka serhiy-storchaka added the needs backport to 3.14 bugs and security fixes label May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants
Morty Proxy This is a proxified and sanitized view of the page, visit original site.