GH-133711: Enable UTF-8 mode by default (PEP 686) #133712
Conversation
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
Doc/whatsnew/3.15.rst
Outdated
| * Python UTF-8 mode is now enabled by default. | ||
| It may be disabled with by setting :envvar:`PYTHONUTF8=0 <PYTHONUTF8>` as | ||
| an environment variable or by using the :option:`-X utf8=0 <-X>` flag. | ||
| See :pep:`686` for further details. |
There was a problem hiding this comment.
I feel like we can probably put some more explanation in here, such as that it affects TextIOWrapper and hence open(). The current description doesn't sound as scary as it needs to, in my opinion.
Along the lines of: "Python UTF-8 mode is now enabled by default. This means that (files/console/etc.) will now use UTF-8 regardless of system settings, unless specifically overridden in code (typically with an encoding= argument)."
There was a problem hiding this comment.
To be clear, it's nothing new. But we shouldn't assume that everyone already knows what UTF-8 mode implies. There are many more people out there who haven't ever thought about it than those who are waiting for it to be the default.
There was a problem hiding this comment.
Another effect of the UTF-8 Mode is that Python ignores the locale encoding.
There was a problem hiding this comment.
I've expanded the What's New entry, please see https://cpython-previews--133712.org.readthedocs.build/en/133712/whatsnew/3.15.html#other-language-changes
| @@ -75,7 +75,30 @@ New features | ||
| Other language changes | ||
| ====================== | ||
|
|
||
|
|
||
| * Python now uses UTF-8_ as the default encoding, independent of the system's |
There was a problem hiding this comment.
You might mention the UTF-8 Mode earlier since it has other side effects documented in the UTF-8 Mode section, such as changing sys.stdout error handler and ignoring the locale encoding.
There was a problem hiding this comment.
"default encoding" is UTF-8 since Python 3.0. (e.g. sys.getdefaultencoding())
# Conflicts: # Doc/whatsnew/3.15.rst
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as resolved.
This comment was marked as resolved.
I wrote #134839 to fix test_regrtest when PYTHONUTF8=1 env var is set. |
I wrote #134841 to fix test_readline for the Python UTF-8 Mode. |
This comment was marked as duplicate.
This comment was marked as duplicate.
Configure host Python fails with: It seems to be unrelated to this PR. |
cc @brettcannon, this builder seems to be consistently failing since ~15/05. |
|
Ping @AA-Turner, I made two suggestions. |
In fact, it would be safe to run this PR on buildbots before merging it. I will wait until GitHub CIs complete first. |
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit 3130318 🤖 Results will be shown at: https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F133712%2Fmerge If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again. |
|
Thanks @vstinner! A |
📚 Documentation preview 📚: https://cpython-previews--133712.org.readthedocs.build/