gh-151316: Prefer UTF-8 for glibc UTF-8-only locale aliases - #155549
gh-151316: Prefer UTF-8 for glibc UTF-8-only locale aliases#155549sankalpsthakur wants to merge 3 commits into
Conversation
|
The following commit authors need to sign the Contributor License Agreement: |
|
You'll have to sign the CLA before we can review. |
e0bc095 to
ffd9b41
Compare
X11 locale.alias mapped en_IN to ISO8859-1, so getlocale() invented a codeset that does not exist on modern UTF-8-only systems and broke setlocale(getlocale()) round-trips. Prefer en_IN.UTF-8 (mirroring other modernized aliases) and cover the round-trip in tests. Signed-off-by: Sankalp Thakur <sankalphimself@gmail.com>
ffd9b41 to
b959df9
Compare
picnixz
left a comment
There was a problem hiding this comment.
Is it the only locale that could have this issue? I would prefer we sweep them all to check if this is the case first instead of having commits for each locale. Also, I think we should instead have a test class that checks for hardcoded replacements thatwe introduce instead of just the case of en_IN.
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
Sweep bare locales that are UTF-8-only in glibc SUPPORTED but still mapped to obsolete X11 codesets, restore those defaults in makelocalealias, and test the hardcoded replacements as a class. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Swept the glibc UTF-8-only locale aliases (not just en_IN), added a test class for the hardcoded replacements, and dropped the irrelevant comment. Still need to finish CLA signing for sankalphimself@gmail.com before this can leave draft. |
|
Thanks for the review. I swept glibc
I have made the requested changes; please review again. |
|
Thanks for making the requested changes! @picnixz: please review the changes made to this pull request. |
Use support.subTests for the hardcoded replacements, shorten the test docstring with an issue link, and extract apply_x11_locales_patch() so the X11 override of glibc UTF-8 defaults is explicit. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Addressed the three new notes: shorter docstring with the issue link, @support.subTests for the replacements, and apply_x11_locales_patch() with a short comment on why X11 gets overridden. CLA for sankalphimself@gmail.com is still outstanding on my side. |
|
A fully-agentic PR is not accepted per our policy. AI tools can be used, but you're not meant to let them run in automode. If this happens again, we will restrict your access to our repositories. |
Summary
SUPPORTEDwere still mapped to obsolete X11 codesets becausemakelocalealiasapplies X11locale.aliasafter glibc.getlocale()invented the X11 codeset andsetlocale(getlocale())failed (reported foren_IN).Tools/i18n/makelocalealias.pyafter the X11 merge (plus ade_licarve-out, which has no bare SUPPORTED line), and cover the replacements withTestHardcodedLocaleReplacements.Validation
locale_aliasentries andnormalize/_parse_localenameexpectations for the 19 restored bare aliases.hi_IN.ISCII-DEV,az_AZ.ISO8859-9E) still normalize unchanged../configure && make+./python.exe -m test test_localewas not run here (sparse/partial clone); relying on CI for the built interpreter suite.Fixes #151316
AI/LLM disclosure