Description
ALT_DIGITS is a glibc specific item. It is supported by Python (there is an explicit list of supported items), but the result is not correct.
From The GNU C Library Reference Manual:
ALT_DIGITS
The return value is a representation of up to 100 values used to represent the values 0 to 99. As for ERA this value is not intended to be used directly, but instead indirectly through the strftime function. When the modifier O is used in a format which would otherwise use numerals to represent hours, minutes, seconds, weekdays, months, or weeks, the appropriate value for the locale is used instead.
This value is only defined in few locales: az_IR, fa_IR, ja_JP, lzh_TW, my_MM, or_IN, shn_MM.
But Python returns only one digit.
>>> import locale
>>> locale.setlocale(locale.LC_TIME, 'ja_JP')
'ja_JP'
>>> locale.setlocale(locale.LC_CTYPE, 'ja_JP')
'ja_JP'
>>> locale.nl_langinfo(locale.ALT_DIGITS)
'〇'This is because nl_langinfo(ALT_DIGITS) in C returns a string with embedded null characters.
How should we fix it?
- return a single string with 99 embedded null characters
- return a 100-tuple of strings
What should we return if the value is not defined (in most locales) -- empty string (current behavior), empty tuple or None?
cc @methane
Linked PRs
- gh-124969: Fix locale.nl_langinfo(locale.ALT_DIGITS) #124974
- gh-124969: Skip the test for ALT_DIGITS also on iOS #125177
- [3.13] gh-124969: Fix locale.nl_langinfo(locale.ALT_DIGITS) (GH-124974) #125232
- [3.12] [3.13] gh-124969: Fix locale.nl_langinfo(locale.ALT_DIGITS) (GH-124974) (GH-125232) #125284
- gh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning a string again #125774
- [3.13] gh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning a string again (GH-125774) #125804
- [3.12] gh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning a string again (GH-125774) #125805
Metadata
Metadata
Assignees
Labels
Projects
- StatusShow more project fieldsDone
Activity
pythongh-124969: Fix locale.nl_langinfo(locale.ALT_DIGITS)
pythongh-124969: Fix locale.nl_langinfo(locale.ALT_DIGITS)
gh-124969: Fix locale.nl_langinfo(locale.ALT_DIGITS) (GH-124974)
encukou commentedon Oct 9, 2024
@freakboy3742: this fix broke the iOS buildbot. You might have opinions on how to handle this on non-glibc platforms :)
pythongh-124969: Fix locale.nl_langinfo(locale.ALT_DIGITS) (pythonGH-…
pythongh-124969: Skip the test for ALT_DIGITS also on iOS
serhiy-storchaka commentedon Oct 9, 2024
It is interesting that ALT_DIGITS is a FreeBSD extension. And was inherited on Apple platforms. I wonder whether it is not set only for Japanese or for other locales (not installed on buildbots) too? fa_IR, lzh_TW, my_MM, or_IN, shn_MM?
gh-124969: Skip the test for ALT_DIGITS also on iOS (#125177)
freakboy3742 commentedon Oct 9, 2024
@serhiy-storchaka AFAICT,
jp_JPis the only locale in test test set that is defined, and it isn't defining a value for ALT_DIGITS. The others fall back as an unrecognised locale. Whether that can be fixed by installing something else on the buildbot/test machine, I can't say - but I haven't explicitly installed a JP locale on my machine.[3.13] pythongh-124969: Fix locale.nl_langinfo(locale.ALT_DIGITS) (py…
[3.13] pythongh-124969: Fix locale.nl_langinfo(locale.ALT_DIGITS) (py…
15 remaining items
serhiy-storchaka commentedon Oct 15, 2024
Thank you for pointing on this @kulikjak. It was used in Python sources as optionally defined, and it was not even documented in Linux manpages, so I thought that it is a unofficial extension.
Does it mean that
locale.nl_langinfo(locale.ALT_DIGITS)now crashes on Solaris? What was the output of the following script on still working Python?On Linux I now get
kulikjak commentedon Oct 16, 2024
Oh I get that - I didn't even know it existed before the test started failing :).
It doesn't crash, only the test fails with:
I ran your script (with the
'.'check removed as most of our locales have.UTF-8or other suffix) on latest 3.13 and the output is:Before, it looked like this (3.13.0):
serhiy-storchaka commentedon Oct 16, 2024
Thank you. Indeed, this looks like semicolon separated correct values.
locale.nl_langinfo(locale.ALT_DIGITS)always returned an empty result on *BSD and macOS, and on Windows it just is not implemented, so Linux was the only tier 1 platform on which it worked, but was broken. But since the POSIX specification says that values should be semicolon separated, it seems, that glibc is broken. I understand why they did this, it is more convenient for the C code. On Python side, it would be more convenient to return a tuple, but this is breaking change on Solaris, the only platform that implemented it right. So I think that we should fix the glibc implementation and return a semicolon separated string. I hope that no locale uses semicolon as an alternative digit or in the ERA items.kulikjak commentedon Oct 16, 2024
Makes sense, although I think that if we want the tuple (which is indeed nicer on the Python side), we can implement alternative parsing (for Solaris and possibly other more compliant platforms) that uses
;rather than\0as a separator as well.serhiy-storchaka commentedon Oct 20, 2024
It is interesting that a similar issue was fixed in Perl at the beginning of this year (Perl/perl5#21833). They return a single string, replacing null characters with semicolons. Actually, they support also other separators, but I do not think there is a reason to do this.
pythongh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning…
gh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning a str…
pythongh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning…
pythongh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning…
[3.12] gh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returnin…
[3.13] gh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returnin…
pythongh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning…
aalshrawy043-ops commentedon Oct 17, 2026