previously, LC_MESSAGES was treated specially as the only category
which could be set to a locale name without a definition file, in
order to facilitate gettext message translations when no libc locale
was available. LC_NUMERIC was completely un-settable, and LC_CTYPE
stored a flag intended to be used for a possible future byte-based C
locale, instead of storing a __locale_map pointer like the other
categories use.
this patch changes all categories to be represented by pointers to
__locale_map structures, and allows locale names without definition
files to be treated as valid locales with trivial definition when used
in any category. outwardly visible functional changes should be minor,
limited mainly to the strings read back from setlocale and the way
gettext handles translations in categories other than LC_MESSAGES.
various internal refactoring has also been performed, and improvements
in const correctness have been made.
if setlocale has not been called, the current locale's messages_name
may be a null pointer. the code path where it's assumed to be non-null
was only reachable if bindtextdomain had already been called, which is
normally not done in programs which do not call setlocale, so the
omitted check went unnoticed.
patch from Void Linux, with description rewritten.
the memory model we use internally for atomics permits plain loads of
values which may be subject to concurrent modification without
requiring that a special load function be used. since a compiler is
free to make transformations that alter the number of loads or the way
in which loads are performed, the compiler is theoretically free to
break this usage. the most obvious concern is with atomic cas
constructs: something of the form tmp=*p;a_cas(p,tmp,f(tmp)); could be
transformed to a_cas(p,*p,f(*p)); where the latter is intended to show
multiple loads of *p whose resulting values might fail to be equal;
this would break the atomicity of the whole operation. but even more
fundamental breakage is possible.
with the changes being made now, objects that may be modified by
atomics are modeled as volatile, and the atomic operations performed
on them by other threads are modeled as asynchronous stores by
hardware which happens to be acting on the request of another thread.
such modeling of course does not itself address memory synchronization
between cores/cpus, but that aspect was already handled. this all
seems less than ideal, but it's the best we can do without mandating a
C11 compiler and using the C11 model for atomics.
in the case of pthread_once_t, the ABI type of the underlying object
is not volatile-qualified. so we are assuming that accessing the
object through a volatile-qualified lvalue via casts yields volatile
access semantics. the language of the C standard is somewhat unclear
on this matter, but this is an assumption the linux kernel also makes,
and seems to be the correct interpretation of the standard.
while the __mo_lookup backend can verify that the translated message
ends with a null terminator, is has no way to know nplurals and thus
no way to verify that sufficiently many null terminators are present
in the string to satisfy all plural forms. the code in dcngettext was
already attempting to avoid reading past the end of the mo file
mapping, but failed to do so because the strlen call itself could
over-read. using strnlen instead allows us to avoid the problem.
this commit replaces the stub implementations with working message
translation functions. translation units are factored so as to prevent
pulling in the legacy, non-library-safe functions which use a global
textdomain in modern code which is using the versions with an explicit
domain argument. bind_textdomain_codeset is also placed in its own
file since it should not be needed by most programs.
this implementation is still missing some features: the LANGUAGE
environment variable (for multiple fallback languages) is not honored,
and non-default plural-form rules are not supported. these issues will
be addressed in a later commit.
one notable difference from the GNU implementation is that there is no
default path for loading translation files. in principle one could be
added, but since the documented correct usage is to call the
bindtextdomain function, a default path is probably unnecessary.
Rich Felker