this is necessary for MT-fork correctness now that the code runs under
locale lock. it would not be hard to avoid, but __get_locale is
already using libc-internal malloc anyway. this can be reconsidered
during locale overhaul later if needed.
in general, pthread_once is not compatible with MT-fork constraints
(commit 167390f055). here it actually no
longer matters, because it's now called with a lock held, but since
the lock is held it's pointless to use pthread_once.
this allows the lock to be shared with setlocale, eliminates repeated
per-category lock/unlock in newlocale, and will allow the use of
pthread_once in newlocale to be dropped (to be done separately).
the intent here is just to scan at least l bytes forward for the end
of the haystack and at least some decent minimum to avoid doing it
over and over if the needle is short, with no need to be precise. the
comment erroneously stated this as an estimate for MIN when it's
actually an estimate for MAX.
pthread_once is not compatible with MT-fork constraints (commit
167390f055) and is not needed here
anyway; we already have a lock suitable for initialization.
while changing this, fix a corner case where AT_MINSIGSTKSZ gives a
value that's more than MINSIGSTKSZ but by a margin of less than
2048, thereby causing the size to be reduced. it shouldn't matter but
the intent was to be the larger of a 2048-byte margin over the legacy
fixed minimum stack requirement or a 512-byte margin over the minimum
the kernel reports at runtime.
this change should have been made when priority inheritance mutex
support was added. if priority protection is also added at some point
the implementation will need to change and will probably no longer be
a simple bit shuffling.
both __clone and __syscall_cp_asm failed to restore the original value
of r6 after using it as a syscall argument register. the extent of
breakage is not known, and in some cases may be mitigated by the only
callers being internal to libc; if they used r6 but no longer needed
its value after the call, they may not have noticed the problem.
however at least posix_spawn (which uses __clone) was observed
returning to the application with the wrong value in r6, leading to
crash.
since the call frame ABI already provides a place to spill registers,
fixing this is just a matter of using it. in __clone, we also
spuriously restore r6 in the child, since the parent branch directly
returns to the caller. this takes the value from an uninitialized slot
of the child's stack, but is harmless since there is no caller to
return to in the child.
float_t should represent the type that is used to evaluate float
expressions internally. On s390x, float_t is currently set to double.
In contrast, the isa supports single-precision float operations and
compilers by default evaluate float in single precision, which
violates the C standard (sections 5.2.4.2.2 and 7.12 in C11/C17, to be
precise). With -fexcess-precision=standard, gcc evaluates float in
double precision, which aligns with the standard yet at the cost of
added conversion instructions.
gcc-11 will drop the special case to retrofit double precision
behavior for -fexcess-precision=standard so that __FLT_EVAL_METHOD__
will be 0 on s390x in any scenario.
To improve standards compliance and compatibility with future compiler
direction, this patch changes the definition of float_t to be derived
from the compiler's __FLT_EVAL_METHOD__.
reallocarray is an extension introduced by OpenBSD, which introduces
calloc overflow checking to realloc.
glibc 2.28 introduced support for this function behind _GNU_SOURCE,
while glibc 2.29 allows its usage in _DEFAULT_SOURCE.
inability to use realpath in chroot/container without procfs access
and at early boot prior to mount of /proc has been an ongoing issue,
and it turns out realpath was one of the last remaining interfaces
that needed procfs for its core functionality. during investigation
while reimplementing, it was determined that there were also serious
problems with the procfs-based implementation. most seriously it was
unsafe on pre-O_PATH kernels, and unlike other places where O_PATH was
used, the unsafety was hard or impossible to fix because O_NOFOLLOW
can't be used (since the whole purpose was to follow symlinks).
the new implementation is a direct one, performing readlink on each
path component to resolve it. an explicit stack, as opposed to
recursion, is used to represent the remaining components to be
processed. the stack starts out holding just the input string, and
reading a link pushes the link contents onto the stack.
unlike many other implementations, this one does not call getcwd
initially for relative pathnames. instead it accumulates initial ..
components to be applied to the working directory if the result is
still a relative path. this avoids calling getcwd (which may fail) at
all when symlink traversal will eventually yield an absolute path. it
also doesn't use any form of stat operation; instead it arranges for
readlink to tell it when a non-directory is used in a context where a
directory is needed. this minimizes the number of syscalls needed,
avoids accessing inodes when the directory table suffices, and reduces
the amount of code pulled in for static linking.
calling lutimes with tv=0 is valid if the application wants to set the
timestamps to the current time. this commit makes it so the timespec
struct is populated with values from tv only if tv != 0 and calls
utimensat with times=0 if tv == 0.
Update fanotify.h, see
linux commit 929943b38daf817f2e6d303ea04401651fc3bc05
fanotify: add support for FAN_REPORT_NAME
linux commit 83b7a59896dd24015a34b7f00027f0ff3747972f
fanotify: add basic support for FAN_REPORT_DIR_FID
linux commit 08b95c338e0c5a96e47f4ca314ea1e7580ecb5d7
fanotify: remove event FAN_DIR_MODIFY
FAN_DIR_MODIFY that was new in v5.7 is now removed from linux uapi,
but kept in musl, so we don't break api, linux cannot reuse the
value anyway.
see
linux commit 9b4feb630e8e9801603f3cab3a36369e3c1cf88d
arch: wire-up close_range()
linux commit 278a5fbaed89dacd04e9d052f4594ffd0e0585de
open: add close_range()
linux fails with EINVAL when a zero buffer size is passed to the
syscall. this is non-conforming because POSIX already defines EINVAL
with a significantly different meaning: the target is not a symlink.
since the request is semantically valid, patch it up by using a dummy
buffer of length one, and truncating the return value to zero if it
succeeds.
the v1 zoneinfo format with 32-bit time is deprecated. previously, the
v2 parsing code was only used if an exact match for '2' was found in
the version field of the header. this was already incorrect for v3
files (trivial differences from v2 that arguably didn't merit a new
version number anyway) but also failed to be future-proof.
since commit 3814333964, the condition
sizeof(time_t) > 4 is always true, so there is no functional change
being made here. but semantically, the 64-bit tables should always be
preferred now, because upstream zic (zoneinfo compiler) has quietly
switched to emitting empty 32-bit tables by default, and the resulting
backwards-incompatible zoneinfo files will be encountered in the wild.
commit d26e0774a5 moved the detach state
transition at exit before the thread list lock was taken. this
inadvertently allowed pthread_join to race to take the thread list
lock first, and proceed with unmapping of the exiting thread's memory.
we could fix this by just revering the offending commit and instead
performing __vm_wait unconditionally before taking the thread list
lock, but that may be costly. instead, bring back the old DT_EXITING
vs DT_EXITED state distinction that was removed in commit
8f11e6127f, and don't transition to
DT_EXITED (a value of 0, which is what pthread_join waits for) until
after the lock has been taken.
the original wcsnrtombs implementation, which has been largely
untouched since 0.5.0, attempted to build input-length-limiting
conversion on top of wcsrtombs, which only limits output length. as
best I recall, this choice was made out of a mix of disdain over
having yet another variant function to implement (added in POSIX 2008;
not standard C) and preference not to switch things around and
implement the wcsrtombs in terms of the more general new function,
probably over namespace issues. the strategy employed was to impose
output limits that would ensure the input limit wasn't exceeded, then
finish up the tail character-at-a-time. unfortunately, none of that
worked correctly.
first, the logic in the wcsrtombs loop was wrong in that it could
easily get stuck making no forward progress, by imposing an output
limit too small to convert even one character.
the character-at-a-time loop that followed was even worse. it made no
effort to ensure that the converted multibyte character would fit in
the remaining output space, only that there was a nonzero amount of
output space remaining. it also employed an incorrect interpretation
of wcrtomb's interface contract for converting the null character,
thereby failing to act on end of input, and remaining space accounting
was subject to unsigned wrap-around. together these errors allow
unbounded overflow of the destination buffer, controlled by input
length limit and input wchar_t string contents.
given the extent to which this function was broken, it's plausible
that most applications that would have been rendered exploitable were
sufficiently broken not to be usable in the first place. however, it's
also plausible that common (especially ASCII-only) inputs succeeded in
the wcsrtombs loop, which mostly worked, while leaving the wildly
erroneous code in the second loop exposed to particular non-ASCII
inputs.
CVE-2020-28928 has been assigned for this issue.
after a non-normal-type process-shared mutex is unlocked, it's
immediately available to another thread to lock, unlock, and destroy,
but the first unlocking thread may still have a pointer to it in its
robust_list pending slot. this means, on async process termination,
the kernel may attempt to access and modify the memory that used to
contain the mutex -- memory that may have been reused for some other
purpose after the mutex was destroyed.
setting up for this kind of race to occur is difficult to begin with,
requiring dynamic use of shared memory maps, and actually hitting the
race is very difficult even with a suitable setup. so this is mostly a
theoretical fix, but in any case the cost is very low.
the __vm_wait operation can delay forward progress arbitrarily long if
a thread holding the lock is interrupted by a signal. in a worst case
this can deadlock. any critical section holding the thread list lock
must respect lock ordering contracts and must not take any lock which
is not AS-safe.
to fix, move the determination of thread joinable/detached state to
take place before the killlock and thread list lock are taken. this
requires reverting the atomic state transition if we determine that
the exiting thread is the last thread and must call exit, but that's
easy to do since it's a single-threaded context with application
signals blocked.
as the outcome of Austin Group tracker issue #62, future editions of
POSIX have dropped the requirement that fork be AS-safe. this allows
but does not require implementations to synchronize fork with internal
locks and give forked children of multithreaded parents a partly or
fully unrestricted execution environment where they can continue to
use the standard library (per POSIX, they can only portably use
AS-safe functions).
up until recently, taking this allowance did not seem desirable.
however, commit 8ed2bd8bfc exposed the
extent to which applications and libraries are depending on the
ability to use malloc and other non-AS-safe interfaces in MT-forked
children, by converting latent very-low-probability catastrophic state
corruption into predictable deadlock. dealing with the fallout has
been a huge burden for users/distros.
while it looks like most of the non-portable usage in applications
could be fixed given sufficient effort, at least some of it seems to
occur in language runtimes which are exposing the ability to run
unrestricted code in the child as part of the contract with the
programmer. any attempt at fixing such contracts is not just a
technical problem but a social one, and is probably not tractable.
this patch extends the fork function to take locks for all libc
singletons in the parent, and release or reset those locks in the
child, so that when the underlying fork operation takes place, the
state protected by these locks is consistent and ready for the child
to use. locking is skipped in the case where the parent is
single-threaded so as not to interfere with legacy AS-safety property
of fork in single-threaded programs. lock order is mostly arbitrary,
but the malloc locks (including bump allocator in case it's used) must
be taken after the locks on any subsystems that might use malloc, and
non-AS-safe locks cannot be taken while the thread list lock is held,
imposing a requirement that it be taken last.
this change lifts undocumented restrictions on calls by replacement
mallocs to libc functions that might take these locks, and sets the
stage for lifting restrictions on the child execution environment
after multithreaded fork.
care is taken to #define macros to replace all four functions (malloc,
calloc, realloc, free) even if not all of them will be used, using an
undefined symbol name for the ones intended not to be used so that any
inadvertent future use will be caught at compile time rather than
directed to the wrong implementation.
allowing the application to replace malloc (since commit
c9f415d7ea) has brought multiple
headaches where it's used from various critical sections in libc
components. for example:
- the thread-local message buffers allocated for dlerror can't be
freed at thread exit time because application code would then run in
the context of a non-existant thread. this was handled in commit
aa5a9d15e0 by queuing them for free
later.
- the dynamic linker has to be careful not to pass memory allocated at
early startup time (necessarily using its own malloc) to realloc or
free after redoing relocations with the application and all
libraries present. bugs in this area were fixed several times, at
least in commits 0c5c8f5da6 and
2f1f51ae7b and possibly others.
- by calling the allocator from contexts where libc-internal locks are
held, we impose undocumented requirements on alternate malloc
implementations not to call into any libc function that might
attempt to take these locks; if they do, deadlock results.
- work to make fork of a multithreaded parent give the child an
unrestricted execution environment is blocked by lock order issues
as long as the application-provided allocator can be called with
libc-internal locks held.
these problems are all fixed by giving libc internals access to the
original, non-replaced allocator, for use where needed. it can't be
used everywhere, as some interfaces like str[n]dup, open_[w]memstream,
getline/getdelim, etc. are required to provide the called memory
obtained as if by (the public) malloc. and there are a number of libc
interfaces that are "pure library" code, not part of some internal
singleton, and where using the application's choice of malloc
implementation is preferable -- things like glob, regex, etc.
one might expect there to be significant cost to static-linked
programs, pulling in two malloc implementations, one of them
mostly-unused, if malloc is replaced. however, in almost all of the
places where malloc is used internally, care has been taken already
not to pull in realloc/free (i.e. to link with just the bump
allocator). this size optimization carries over automatically.
the newly-exposed internal allocator functions are obtained by
renaming the actual definitions, then adding new wrappers around them
with the public names. technically __libc_realloc and __libc_free
could be aliases rather than needing a layer of wrapper, but this
would almost surely break certain instrumentation (valgrind) and the
size and performance difference is negligible. __libc_calloc needs to
be handled specially since calloc is designed to work with either the
internal or the replaced malloc.
as a bonus, this change also eliminates the longstanding ugly
dependency of the static bump allocator on order of object files in
libc.a, by making it so there's only one definition of the malloc
function and having it in the same source file as the bump allocator.
the only place stdio was used here was for reading the ldso path file,
taking advantage of getdelim to automatically allocate and resize the
buffer. the motivation for use here was that, with shared libraries,
stdio is already available anyway and free to use. this has long been
a nuisance to users because getdelim's use of realloc here triggered a
valgrind bug, but removing it doesn't really fix that; on some archs
even calling the valgrind-interposed malloc at this point will crash.
the actual motivation for this change is moving towards getting rid of
use of application-provided malloc in parts of libc where it would be
called with libc-internal locks held, leading to the possibility of
deadlock if the malloc implementation doesn't follow unwritten rules
about which libc functions are safe for it to call. since getdelim is
required to produce a pointer as if by malloc (i.e. that can be passed
to reallor or free), it necessarily must use the public malloc.
instead of performing a realloc loop as the path file is read, first
query its size with fstat and allocate only once. this produces
slightly different truncation behavior when racing with writes to a
file, but neither behavior is or could be made safe anyway; on a live
system, ldso path files should be replaced by atomic rename only. the
change should also reduce memory waste.
thread-local buffers allocated for dlerror need to be queued for free
at a later time when the owning thread exits, since malloc may be
replaced by application code and the exiting context is not valid to
call application code from. the code to process queue of pending
frees, introduced in commit aa5a9d15e0,
gratuitously held the lock for the entire duration of queue
processing, updating the global queue pointer after each free, despite
there being no logical requirement that all frees finish before
another thread can access the queue.
instead, immediately claim the whole queue for freeing and release the
lock, then walk the list and perform frees without the lock held. the
change is unlikely to make any meaningful difference to performance,
but it eliminates one point where the allocator is called under an
internal lock. since the allocator may be application-provided, such
calls are undesirable because they allow application code to impede
forward progress of libc functions in other threads arbitrarily long,
and to induce deadlock if it calls a libc function that requires the
same lock.
the change also eliminates a lock ordering consideration that's an
impediment upcoming work with multithreaded fork.
the ABI type for the vector registers in fpregset_t, struct
fpsimd_context, and struct user_fpsimd_struct is __uint128_t, which
was presumably originally not used because it's a nonstandard type,
but its existence is mandated by the aarch64 psABI. use of the wrong
type here broke software using these structures, and encouraged
incorrect fixes with casts rather than reinterpretation of
representation.
the reasoning in commit 2d0bbe6c78 was
not entirely correct. while it's true that setting the waiters flag
ensures that the next unlock will perform a wake, it's possible that
the wake is consumed by a mutex waiter that has no relationship with
the condvar wait queue being processed, which then takes the mutex.
when that thread subsequently unlocks, it sees no waiters, and leaves
the rest of the condvar queue stuck.
bring back the waiter count adjustment, but skip it for PI mutexes,
for which a successful lock-after-waiting always sets the waiters bit.
if future changes are made to bring this same waiters-bit contract to
all lock types, this can be reverted.
sem_open is required to return the same sem_t pointer for all
references to the same named semaphore when it's opened more than once
in the same process. thus we keep a table of all the mapped semaphores
and their reference counts. the code path for sem_close checked the
reference count, but then proceeded to unmap the semaphore regardless
of whether the count had reached zero.
add an immediate unlock-and-return for the nonzero refcnt case so the
property of performing the munmap syscall after releasing the lock can
be preserved.
resource limits have been process-wide since linux 2.6.10, and the
prlimit syscall was added in 2.6.36, so prlimit can be assumed to set
the resource limits correctly for the whole process.
commit bd153422f2 reintroduced the bug
fixed in c21051e90c by refactoring the
__syscall_ret into _Fork where it once again runs before the atfork
handlers are called. since _Fork is a public interface that sets
errno, this can't be fixed the way it was fixed last time without
making new internal interfaces. instead, just save errno, and restore
it only on error to ensure that a value of 0 is never restored.
pthread_cond_wait arranged for requeued waiters to wake when the mutex
is unlocked by temporarily adjusting the mutex's waiter count. commit
54ca677983 broke this when introducing
PI mutexes by repurposing the waiter count field of the mutex
structure. since then, for PI mutexes, the waiter count adjustment was
misinterpreted by the mutex locking code as indicating that the mutex
is non a non-recoverable state.
it would be possible to special-case PI mutexes here, but instead just
drop all adjustment of the waiters count, and instead use the lock
word waiters bit for all mutex types. since the mutex is either held
by the caller or in unrecoverable state at the time the bit is set, it
will necessarily still be set at the time of any subsequent valid
unlock operation, and this will produce the desired effect of waking
the next waiter.
if waiter counts are entirely dropped at some point in the future this
code should still work without modification.
commit 25ea9f712c introduced a deadlock
to the posix_spawn child whereby, if abort was called in the parent
and ended up taking the abort lock to terminate the process, the
__libc_sigaction calls in the child would wait forever to obtain a
lock that would not be released. this could be fixed by having abort
set the abort lock as the exit futex address, but it's cleaner to just
remove the SIGABRT special handling from the internal __libc_sigaction
and lift it to the public sigaction function.
nothing but the posix_spawn child calls __libc_sigaction on SIGABRT,
and since commit b7bc966522 the abort
lock is held at the time of __clone, which precludes the child
inheriting a kernel-level signal disposition inconsistent with the
disposition on the abstract machine. this means it's fine to inspect
and modify the disposition in the child without a lock.
Merge changes from Solar Designer's crypt_blowfish v1.3. This makes
crypt_blowfish fully compatible with OpenBSD's bcrypt by adding
support for the $2b$ prefix (which behaves the same as
crypt_blowfish's $2y$).
this makes the code slightly smaller and eliminates timer_create from
relevance to possible future changes to multithreaded fork.
the barrier of a_store isn't technically needed here, but a_store is
used anyway for internal consistency of the memory model.
Ariadne Conill
Rich Felker
Drew DeVault
Marius Hillenbrand
Dominic Chen
Issam E. Maghni
Érico Rolim
Szabolcs Nagy
William Woodruff
Jinliang Li
James Y Knight
rcombs
Julien Ramseier