Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

gh-104223: Fix issues with inheriting from buffer classes #104227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
May 8, 2023
Merged

gh-104223: Fix issues with inheriting from buffer classes #104227

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
b7fb7a4
Fix buffer classes using super()
JelleZijlstra May 6, 2023
c717784
Fix bf_releasebuffer in child classes
JelleZijlstra May 6, 2023
9fcb214
Merge branch 'main' into pep688fix
JelleZijlstra May 6, 2023
88ce24d
Add a test
JelleZijlstra May 6, 2023
80a4b0b
Deal with __release_buffer__ saving references to the buffer
JelleZijlstra May 7, 2023
6e31664
Add another test
JelleZijlstra May 7, 2023
b73e29c
More paranoid bufferwrapper_releasebuf
JelleZijlstra May 7, 2023
15ad02d
that did not work
JelleZijlstra May 7, 2023
9ded873
Update Objects/typeobject.c
JelleZijlstra May 7, 2023
3b50138
Update Objects/typeobject.c
JelleZijlstra May 7, 2023
056efee
Merge remote-tracking branch 'upstream/main' into pep688fix
JelleZijlstra May 7, 2023
9ee8eb5
Better message for new flag
JelleZijlstra May 7, 2023
5536853
More tests
JelleZijlstra May 7, 2023
02d7b9c
Merge branch 'main' into pep688fix
JelleZijlstra May 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add another test
  • Loading branch information
@JelleZijlstra
JelleZijlstra committed May 7, 2023
commit 6e316646293f6e1344470ea50089ff45c48a3d43
20 changes: 20 additions & 0 deletions 20 Lib/test/test_buffer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4670,6 +4670,26 @@ def __release_buffer__(s, buffer: memoryview):
with self.assertRaises(ValueError):
smuggled_buffer.tobytes()

def test_release_saves_reference_no_subclassing(self):
ba = bytearray(b"hello")

class C:
def __buffer__(self, flags):
return memoryview(ba)

def __release_buffer__(self, buffer):
self.buffer = buffer

c = C()
with memoryview(c) as mv:
self.assertEqual(mv.tobytes(), b"hello")
self.assertEqual(c.buffer.tobytes(), b"hello")

with self.assertRaises(BufferError):
ba.clear()
c.buffer.release()
ba.clear()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a test in which the class has multiple inheritance? Also tests for when there are two or classes in mro which support buffer protocol?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding some tests. It's hard to come up with a case where multiple bases support the C buffer protocol, because that will almost inevitably lead to:

Traceback (most recent call last):
  File "/Users/jelle/py/cpython/Lib/test/test_buffer.py", line 4707, in test_two_buffer_bases
    class A(bytearray, bytes):
TypeError: multiple bases have instance lay-out conflict

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, maybe try one var length and one pure python type which implements buffer protocol.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, just pushed. Thanks for the review!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding some tests. It's hard to come up with a case where multiple bases support the C buffer protocol, because that will almost inevitably lead to:

I see, you would have to write a custom type in C whose layout doesn't conflicts for that. Probably an object which has just object header and supports buffer protocol.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I agree this is an edge case but I try to be extra careful when touching typeobject, hope you don't mind the extra work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to catch this now then right before 3.12 final!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed the bug I outlined above, I'll have to step out for a few hours and fix it after that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about it more, that crash isn't actually due to PEP 688: it reproduces without any Python __buffer__ method. So I don't think there's any more interesting cases to cover for PEP 688.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #104297 for that case.


if __name__ == "__main__":
unittest.main()
Morty Proxy This is a proxified and sanitized view of the page, visit original site.