for conversion specifiers, alloc is always set when the specifier is
parsed. however, if scanf stops due to mismatching literal text,
either an uninitialized (if no conversions have been performed yet) or
stale (from the previous conversion) of the flag will be used,
possibly causing an invalid pointer to be passed to free when the
function returns.
the concept here is that %s and %c are essentially special-cases of
%[, with some minimal additional special-casing.
aside from simplifying the code and reducing the number of complex
code-paths that would need changing to make optimizations later, the
main purpose of this change is to simplify addition of the 'm'
modifier which causes scanf to allocate storage for the string being
read.
GNU used several extensions that were incompatible with C99 and POSIX,
so they used alternate names for the standard functions.
The result is that we need these to run standards-conformant programs
that were linked with glibc.
per interpretation for austin group issue #626, fflush(0) and exit()
must block waiting for a lock if another thread has locked a memory
stream with flockfile. this adds some otherwise-unnecessary
synchronization cost to use of memory streams, but there was already a
synchronization cost calling malloc anyway.
previously the stream was only added to the open file list in
single-threaded programs, so that upon subsequent call to
pthread_create, locking could be turned on for the stream.
this mirrors the stdio_impl.h cleanup. one header which is not
strictly needed, errno.h, is left in pthread_impl.h, because since
pthread functions return their error codes rather than using errno,
nearly every single pthread function needs the errno constants.
in a few places, rather than bringing in string.h to use memset, the
memset was replaced by direct assignment. this seems to generate much
better code anyway, and makes many functions which were previously
non-leaf functions into leaf functions (possibly eliminating a great
deal of bloat on some platforms where non-leaf functions require ugly
prologue and/or epilogue).
this header evolved to facilitate the extremely lazy practice of
omitting explicit includes of the necessary headers in individual
stdio source files; not only was this sloppy, but it also increased
build time.
now, stdio_impl.h is only including the headers it needs for its own
use; any further headers needed by source files are included directly
where needed.
some of these were coming from stdio functions locking files without
unlocking them. I believe it's useful for this to throw a warning, so
I added a new macro that's self-documenting that the file will never
be unlocked to avoid the warning in the few places where it's wrong.
for conformance, two functions should not have the same address. a
conforming program could use the addresses of getc and fgetc in ways
that assume they are distinct. normally i would just use a wrapper,
but these functions are so small and performance-critical that an
extra layer of function call could make the one that's a wrapper
nearly twice as slow, so I'm just duplicating the code instead.
these functions must behave as if they obtain the lock via flockfile
to satisfy POSIX requirements. since another thread can provably hold
the lock when they are called, they must wait to obtain the lock
before they can return, even if the correct return value could be
obtained without locking. in the case of fclose and freopen, failure
to do so could cause correct (albeit obscure) programs to crash or
otherwise misbehave; in the case of feof, ferror, and fwide, failure
to obtain the lock could sometimes return incorrect results. in any
case, having these functions proceed and return while another thread
held the lock was wrong.
1. don't open /dev/null just as a basis to copy flags; use shared
__fmodeflags function to get the right file flags for the mode.
2. handle the case (probably invalid, but whatever) case where the
original stream's file descriptor was closed; previously, the logic
re-closed it.
3. accept the "e" mode flag for close-on-exec; update dup3 to fallback
to using dup2 so we can simply call __dup3 instead of putting fallback
logic in freopen itself.
__release_ptc() is only valid in the parent; if it's performed in the
child, the lock will be unlocked early then double-unlocked later,
corrupting the lock state.
since we target systems without overcommit, special care should be
taken that system() and popen(), like posix_spawn(), do not fail in
processes whose commit charges are too high to allow ordinary forking.
this in turn requires special precautions to ensure that the parent
process's signal handlers do not end up running in the shared-memory
child, where they could corrupt the state of the parent process.
popen has also been updated to use pipe2, so it does not have a
fd-leak race in multi-threaded programs. since pipe2 is missing on
older kernels, (non-atomic) emulation has been added.
some silly bugs in the old code should be gone too.
this feature will be in the next version of POSIX, and can be used
internally immediately. there are many internal uses of fopen where
close-on-exec is needed to fix bugs.
to deal with the fact that the public headers may be used with pre-c99
compilers, __restrict is used in place of restrict, and defined
appropriately for any supported compiler. we also avoid the form
[restrict] since older versions of gcc rejected it due to a bug in the
original c99 standard, and instead use the form *restrict.
based on Gregor's patch sent to the list. includes:
- stdalign.h
- removing gets in C11 mode
- adding aligned_alloc and adjusting other functions to use it
- adding 'x' flag to fopen for exclusive mode
the strspn call was made for every format specifier and end-of-string,
even though the expected return value was 1-2 for normal usage.
replace with simple loop.
amusingly, this cuts more than 10% off the run time of printf("a"); on
the machine i tested it on.
sadly the same optimization is not possible for snprintf without
duplicating all the pseudo-FILE setup code, which is not worth it.
at the point pclose might receive and act on cancellation, it has
already invalidated the FILE passed to it. thus, per musl's QOI
guarantees about cancellation and resource allocation/deallocation,
it's not a candidate for cancellation.
if it were required to be a cancellation point by posix, we would have
to switch the order of deallocation, but somehow still close the pipe
in order to trigger the child process to exit. i looked into doing
this, but the logic gets ugly, and i'm not sure the semantics are
conformant, so i'd rather just leave it alone unless there's a need to
change it.
close was the only cancellation point called from popen, but it left
popen with major resource leaks if any call to close got cancelled.
the easiest, cheapest fix is just to use a non-cancellable close
function.
this one could never cause any problems unless the compiler/machine
goes to extra trouble to break oob pointer arithmetic, but it's best
to fix it anyway.
large precision values could cause out-of-bounds pointer arithmetic in
computing the precision cutoff (used to avoid expensive long-precision
arithmetic when the result will be discarded). per the C standard,
this is undefined behavior. one would expect that it works anyway, and
in fact it did in most real-world cases, but it was randomly
(depending on aslr) crashing in i386 binaries running on x86_64
kernels. this is because linux puts the userspace stack near 4GB
(instead of near 3GB) when the kernel is 64-bit, leading to the
out-of-bounds pointer arithmetic overflowing past the end of address
space and giving a very low pointer value, which then compared lower
than a pointer it should have been higher than.
the new code rearranges the arithmetic so that no overflow can occur.
while this bug could crash printf with memory corruption, it's
unlikely to have security impact in real-world applications since the
ability to provide an extremely large field precision value under
attacker-control is required to trigger the bug.
for seekable files, posix imposed requirements on the offset of the
underlying open file description after a stream is closed. this was
correctly handled (as a side effect of the unconditional fflush call)
when streams were explicitly closed by fclose, but was not handled
correctly at program exit time, where fflush(0) was being used.
the weak symbol hackery is to pull in __stdio_exit if either of
__toread or __towrite is used, but avoid calling it twice so we don't
have to keep extra state. the new __stdio_exit is a streamlined fflush
variant that avoids performing any unnecessary operations and which
never unlocks the files or open file list, so we can be sure no other
threads write new data to a stream's buffer after it's already
flushed.
there is no need/use for a flush hook. the write function serves this
purpose already. i originally created the hook for implementing mem
streams based on a mistaken reading of posix, and later realized it
wasn't useful but never removed it until now.
the old behavior was to only consider a stream to be "reading" or
"writing" if it had buffered, unread/unwritten data. this reportedly
differs from the traditional behavior of these functions, which is
essentially to return true as much as possible without creating the
possibility that both __freading and __fwriting could return true.
gnulib expects __fwriting to return true as soon as a file is opened
write-only, and possibly expects other cases that depend on the
traditional behavior. and since these functions exist mostly for
gnulib (does anything else use them??), they should match the expected
behavior to avoid even more ugly hacks and workarounds...
printf was not printing too many characters, but it was reading one
too many wchar_t elements from the input. this could lead to crashes
if running off the page, or spurious failure if the conversion of the
extra wchar_t resulted in EILSEQ.
Rich Felker
Isaac Dunham