loop condition was incorrect and confusing and caused an infinite loop
when (broken) applications reaped the pid from a signal handler or
another thread before wordexp's call to waitpid could do so.
when WRDE_NOSPACE is returned, the we_wordv and we_wordc members must
be valid, because the interface contract allows them to return partial
results.
in the case of zero results (due either to resource exhaustion or a
zero-word input) the we_wordv array still should contain a terminating
null pointer and the initial we_offs null pointers. this is impossible
on resource exhaustion, so a correct application must presumably check
for a null pointer in we_wordv; POSIX however seems to ignore the
issue. the previous code may have crashed under this situation.
avoid using exit status to determine if a shell error occurred, since
broken programs may install SIGCHLD handlers which reap all zombies,
including ones that don't belong to them. using clone and __WCLONE
does not seem to work for avoiding this problem since exec resets the
exit signal to SIGCHLD.
instead, the new code uses a dummy word at the beginning of the
shell's output, which is ignored, to determine whether the command was
executed successfully. this also fixes a corner case where a word
string containing zero words was interpreted as a single zero-length
word rather than no words at all. POSIX does not seem to require this
case to be supported anyway, though.
in addition, the new code uses the correct retry idiom for waitpid to
ensure that spurious STOP/CONT signals in the child and/or EINTR in
the parent do not prevent successful wait for the child, and blocks
signals in the child.
rather than allocating a PATH_MAX-sized buffer when the caller does
not provide an output buffer, work first with a PATH_MAX-sized temp
buffer with automatic storage, and either copy it to the caller's
buffer or strdup it on success. this not only avoids massive memory
waste, but also avoids pulling in free (and thus the full malloc
implementation) unnecessarily in static programs.
this avoids failure if the file is not readable and avoids odd
behavior for device nodes, etc. on old kernels that lack O_PATH, the
old behavior (O_RDONLY) will naturally happen as the fallback.
I intend to add more Linux workarounds that depend on using these
pathnames, and some of them will be in "syscall" functions that, from
an anti-bloat standpoint, should not depend on the whole snprintf
framework.
GNU used several extensions that were incompatible with C99 and POSIX,
so they used alternate names for the standard functions.
The result is that we need these to run standards-conformant programs
that were linked with glibc.
1. as reported by William Haddon, the value returned by snprintf was
wrongly used as a length passed to sendto, despite it possibly
exceeding the buffer length. this could lead to invalid reads and
leaking additional data to syslog.
2. openlog was storing a pointer to the ident string passed by the
caller, rather than copying it. this bug is shared with (and even
documented in) other implementations like glibc, but such behavior
does not seem to meet the requirements of the standard.
3. extremely long ident provided to openlog, or corrupt ident due to
the above issue, could possibly have resulted in buffer overflows.
despite having the potential for smashing the stack, i believe the
impact is low since ident points to a short string literal in typical
application usage (and per the above bug, other usages will break
horribly on other implementations).
4. when used with LOG_NDELAY, openlog was not connecting the
newly-opened socket; sendto was being used instead. this defeated the
main purpose of LOG_NDELAY: preparing for chroot.
5. the default facility was not being used at all, so all messages
without an explicit facility passed to syslog were getting logged at
the kernel facility.
6. setlogmask was not thread-safe; no synchronization was performed
updating the mask. the fix uses atomics rather than locking to avoid
introducing a lock in the fast path for messages whose priority is not
in the mask.
7. in some code paths, the syslog lock was being unlocked twice; this
could result in releasing a lock that was actually held by a different
thread.
some additional enhancements to syslog such as a default identifier
based on argv[0] or similar may still be desired; at this time, only
the above-listed bugs have been fixed.
based on proposed patches by Daniel Cegiełka, with minor changes:
- use a weak symbol for optreset so it doesn't clash with namespace
- also reset optpos (position in multi-option arg like -lR)
- also make getopt_long support reset
also update syslog to use SOCK_CLOEXEC rather than separate fcntl
step, to make it safe in multithreaded programs that run external
programs.
emulation is not atomic; it could be made atomic by holding a lock on
forking during the operation, but this seems like overkill. my goal is
not to achieve perfect behavior on old kernels (which have plenty of
other imperfect behavior already) but to avoid catastrophic breakage
in (1) syslog, which would give no output on old kernels with the
change to use SOCK_CLOEXEC, and (2) programs built on a new kernel
where configure scripts detected a working SOCK_CLOEXEC, which later
get run on older kernels (they may otherwise fail to work completely).
previously, it was pretty much random which one of these trees a given
function appeared in. they have now been organized into:
src/linux: non-POSIX linux syscalls (possibly shard with other nixen)
src/legacy: various obsolete/legacy functions, mostly wrappers
src/misc: still mostly uncategorized; some misc POSIX, some nonstd
src/crypt: crypt hash functions
further cleanup will be done later.
to deal with the fact that the public headers may be used with pre-c99
compilers, __restrict is used in place of restrict, and defined
appropriately for any supported compiler. we also avoid the form
[restrict] since older versions of gcc rejected it due to a bug in the
original c99 standard, and instead use the form *restrict.
these limits could definitely use review, but for now, i feel
consistency and erring on the side of preventing servers from getting
bogged down by excessively-slow user-provided settings (think
.htpasswd) are the best policy. blowfish should be updated to match.
based on versions sent to the list by nsz, with some simplification
and debloating. i'd still like to get them a bit smaller, or ideally
merge them into a single file with most of the code being shared, but
that can be done later.
there are still some discussions going on about tweaking the code, but
at least thing brings us to the point of having something working in
the repository. hopefully the remaining major hashes (md5,sha) will
follow soon.
unfortunately, a large portion of programs which call crypt are not
prepared for its failure and do not check that the return value is
non-null before using it. thus, always "succeeding" but giving an
unmatchable hash is reportedly a better behavior than failing on
error.
it was suggested that we could do this the same way as other
implementations and put the null-to-unmatchable translation in the
wrapper rather than the individual crypt modules like crypt_des, but
when i tried to do it, i found it was making the logic in __crypt_r
for keeping track of which hash type we're working with and whether it
succeeded or failed much more complex, and potentially error-prone.
the way i'm doing it now seems to have essentially zero cost, anyway.
the new version is largely the work of Solar Designer, with minor
changes for integration with musl. compared to the old code, text size
is reduced by about 7k, stack space usage by about 70k, and
performance is greatly improved by avoiding expensive calculation of
constant tables on each run.
this version also adds support for extended des-based password hashes,
which allow for unlimited key (password) length and configurable
iteration counts.
i've also published the interface for crypt_r in a new crypt.h header.
especially since this is not a standard interface, i did not feel
compelled to match the glibc abi for the crypt_data structure. the
glibc structure is way too big to allocate on the stack; in fact it's
so big that the first usage may cause the main thread to exceed its
pre-committed stack size of 128k and thus could cause the program to
crash even on systems with overcommit disabled. the only legitimate
use of crypt_data for crypt_r is to store the hash string to return,
so i've reserved 256 bytes, which should be more than sufficient
(longest known password hashes are ~60 characters, and beyond that is
possibly even exceeding some implementations' passwd file field size
limit).
apparently some packages see stropts.h and want to be able to use
this. the implementation checks that the file descriptor is valid by
using fcntl/F_GETFD so it can report an error if not (as specified).
i did some testing trying to switch malloc to use the new internal
lock with priority inheritance, and my malloc contention test got
20-100 times slower. if priority inheritance futexes are this slow,
it's simply too high a price to pay for avoiding priority inversion.
maybe we can consider them somewhere down the road once the kernel
folks get their act together on this (and perferably don't link it to
glibc's inefficient lock API)...
as such, i've switch __lock to use malloc's implementation of
lightweight locks, and updated all the users of the code to use an
array with a waiter count for their locks. this should give optimal
performance in the vast majority of cases, and it's simple.
malloc is still using its own internal copy of the lock code because
it seems to yield measurably better performance with -O3 when it's
inlined (20% or more difference in the contention stress test).
I'm not sure if it's legal for wordexp to modify this field, but this
is the only easy/straightforward fix, and applications should not
care. if it's an issue, i can work out a different (but more complex)
solution later.
note that regardless of the name used, basename is always conformant.
it never takes on the bogus gnu behavior, unlike glibc where basename
is nonconformant when declared manually without including libgen.h.
Szabolcs Nagy
Rich Felker
Isaac Dunham
rofl0r