also move all legacy inet_* functions into a single file to avoid
wasting object file and compile time overhead on them.
the added functions are legacy interfaces for working with classful
ipv4 network addresses. they have no modern usefulness whatsoever, but
some programs unconditionally use them anyway, and they're tiny.
based on patch by Strake with minor stylistic changes, and combined
into a single file. this patch remained open for a long time due to
some question as to whether ether_aton would be better implemented in
terms of sscanf, and it's time something was committed, so here it is.
the shgetc api, used internally in scanf and int/float scanning code
to handle field width limiting and pushback, was designed assuming
that pushback could be achieved via a simple decrement on the file
buffer pointer. this only worked by chance for regular FILE streams,
due to the linux readv bug workaround in __stdio_read which moves the
last requested byte through the buffer rather than directly back to
the caller. for unbuffered streams and streams not using __stdio_read
but some other underlying read function, the first character read
could be completely lost, and replaced by whatever junk happened to be
in the unget buffer.
to fix this, simply have shgetc, when it performs an underlying read
operation on the stream, store the character read at the -1 offset
from the read buffer pointer. this is valid even for unbuffered
streams, as they have an unget buffer located just below the start of
the zero-length buffer. the check to avoid storing the character when
it is already there is to handle the possibility of read-only buffers.
no application-exposed FILE types are allowed to use read-only
buffers, but sscanf and strto* may use them internally when calling
functions which use the shgetc api.
issue found and patch provided by Jens Gustedt. after the atomic store
to the error code field of the aiocb, the application is permitted to
free or reuse the storage, so further access is invalid. instead, use
the local copy that was already made.
due to the interface requirement of having the full state contained in
a single object of type unsigned int, it is difficult to provide a
reasonable-quality implementation; most good PRNGs are immediately
ruled out because they need larger state. the old rand_r gave very
poor output (very short period) in its lower bits; normally, it's
desirable to throw away the low bits (as in rand()) when using a LCG,
but this is not possible since the state is only 32 bits and we need
31 bits of output.
glibc's rand_r uses the same LCG as musl's, but runs it for 3
iterations and only takes 10-11 bits from each iteration to construct
the output value. this partially fixes the period issue, but
introduces bias: not all outputs have the same frequency, and many do
not appear at all. with such a low period, the bias is likely to be
observable.
I tried many approaches to "fix" rand_r, and the simplest I found
which made it pass the "dieharder" tests was applying this
transformation to the output. the "temper" function is taken from
mersenne twister, where it seems to have been chosen for some rigorous
properties; here, the only formal property I'm using is that it's
one-to-one and thus avoids introducing bias.
should further deficiencies in rand_r be reported, the obvious "best"
solution is applying a 32-bit cryptographic block cipher in CTR mode.
I identified several possible ciphers that could be used directly or
adapted, but as they would be a lot slower and larger, I do not see a
justification for using them unless the current rand_r proves
deficient for some real-world use.
apparently these features have been in Linux for a while now, so it
makes sense to support them. the bit twiddling seems utterly illogical
and wasteful, especially the negation, but that's how the kernel folks
chose to encode pids/tids into the clock id.
some applications rely on the low bits of rand() to be reasonably good
quality prng, so now it fixed by using the top bits of a 64 bit LCG,
this is simple, has small state and passes statistical tests.
D.E. Knuth attributes the multiplier to C.E. Haynes in TAOCP Vol2 3.3.4
the concept here is that %s and %c are essentially special-cases of
%[, with some minimal additional special-casing.
aside from simplifying the code and reducing the number of complex
code-paths that would need changing to make optimizations later, the
main purpose of this change is to simplify addition of the 'm'
modifier which causes scanf to allocate storage for the string being
read.
per Austin Group interpretation for issue #686, which cites the
requirements of ISO C, clock() cannot wrap. if the result is not
representable, it must return (clock_t)-1. in addition, the old code
was performing wrapping via signed overflow and thus invoking
undefined behavior.
since it seems impossible to accurately check for overflow with the
old times()-based fallback code, I have simply dropped the fallback
code for now, thus always returning -1 on ancient systems. if there's
a demand for making it work and somebody comes up with a way, it could
be reinstated, but the clock() function is essentially useless on
32-bit system anyway (it overflows in less than an hour).
it should be noted that I used LONG_MAX rather than ULONG_MAX, despite
32-bit archs using an unsigned type for clock_t. this discrepency with
the glibc/LSB type definitions will be fixed now; since wrapping of
clock_t is no longer supported, there's no use in it being unsigned.
The underflow exception is not raised correctly in some
cornercases (see previous fma commit), added comments
with examples for fmaf, fmal and non-x86 fma.
In fmaf store the result before returning so it has the
correct precision when FLT_EVAL_METHOD!=0
1) in downward rounding fma(1,1,-1) should be -0 but it was 0 with
gcc, the code was correct but gcc does not support FENV_ACCESS ON
so it used common subexpression elimination where it shouldn't have.
now volatile memory access is used as a barrier after fesetround.
2) in directed rounding modes there is no double rounding issue
so the complicated adjustments done for nearest rounding mode are
not needed. the only exception to this rule is raising the underflow
flag: assume "small" is an exactly representible subnormal value in
double precision and "verysmall" is a much smaller value so that
(long double)(small plus verysmall) == small
then
(double)(small plus verysmall)
raises underflow because the result is an inexact subnormal, but
(double)(long double)(small plus verysmall)
does not because small is not a subnormal in long double precision
and it is exact in double precision.
now this problem is fixed by checking inexact using fenv when the
result is subnormal
* use unsigned arithmetics
* use unsigned to store arg reduction quotient (so n&3 is understood)
* remove z=0.0 variables, use literal 0
* raise underflow and inexact exceptions properly when x is small
* fix spurious underflow in tanl
patch by Strake. previously is was not feasible to duplicate this
functionality of the functions these were modeled on, since argv[0]
was not saved at program startup, but now that it's available it's
easy to use.
* use unsigned arithmetics on the representation
* store arg reduction quotient in unsigned (so n%2 would work like n&1)
* use different convention to pass the arg reduction bit to __tan
(this argument used to be 1 for even and -1 for odd reduction
which meant obscure bithacks, the new n&1 is cleaner)
* raise inexact and underflow flags correctly for small x
(tanl(x) may still raise spurious underflow for small but normal x)
(this exception raising code increases codesize a bit, similar fixes
are needed in many other places, it may worth investigating at some
point if the inexact and underflow flags are worth raising correctly
as this is not strictly required by the standard)
* tanf manual reduction optimization is kept for now
* tanl code path is cleaned up to follow similar logic to tan and tanf
support for these was recently added to sysmacros.h. note that the
syscall argument is a long, despite dev_t being 64-bit, so on 32-bit
archs the high bits will be lost. it appears the high bits are just
glibc silliness and not part of the kernel api, anyway, but it's nice
that we have them there for future expansion if needed.
When FLT_EVAL_METHOD!=0 (only i386 with x87 fp) the excess
precision of an expression must be removed in an assignment.
(gcc needs -fexcess-precision=standard or -std=c99 for this)
This is done by extra load/store instructions which adds code
bloat when lot of temporaries are used and it makes the result
less precise in many cases.
Using double_t and float_t avoids these issues on i386 and
it makes no difference on other archs.
For now only a few functions are modified where the excess
precision is clearly beneficial (mostly polynomial evaluations
with temporaries).
object size differences on i386, gcc-4.8:
old new
__cosdf.o 123 95
__cos.o 199 169
__sindf.o 131 95
__sin.o 225 203
__tandf.o 207 151
__tan.o 605 499
erff.o 1470 1416
erf.o 1703 1649
j0f.o 1779 1745
j0.o 2308 2274
j1f.o 1602 1568
j1.o 2286 2252
tgamma.o 1431 1424
math/*.o 64164 63635
since CLOCKS_PER_SEC is 1000000 (required by XSI) and the times
syscall reports values in 1/100 second units (Linux), the correct
scaling factor is 10000, not 100. note that only ancient kernels which
lack clock_gettime are affected.
all return values are valid, and on 32-bit systems, values that look
like errors can and will occur. since the only actual error this
function could return is EFAULT, and it is only returnable when the
application has invoked undefined behavior, simply ignore the
possibility that the return value is actually an error code.
there are several reasons for this change. one is getting rid of the
repetition of the syscall signature all over the place. another is
sharing the constant masks without costly GOT accesses in PIC.
the main motivation, however, is accurately representing whether we
want to block signals that might be handled by the application, or all
signals.
they have already blocked signals before decrementing the thread
count, so the code being removed is unreachable in the case where the
thread is no longer counted.
now that blocking signals prevents any application code from running
while the last thread is exiting, the cas logic is no longer needed to
prevent decrementing below zero.
the thread count (1+libc.threads_minus_1) must always be greater than
or equal to the number of threads which could have application code
running, even in an async-signal-safe sense. there is at least one
dangerous race condition if this invariant fails to hold: dlopen could
allocate too little TLS for existing threads, and a signal handler
running in the exiting thread could claim the allocated TLS for itself
(via __tls_get_addr), leaving too little for the other threads it was
allocated for and thereby causing out-of-bounds access.
there may be other situations where it's dangerous for the thread
count to be too low, particularly in the case where only one thread
should be left, in which case locking may be omitted. however, all
such code paths seem to arise from undefined behavior, since
async-signal-unsafe functions are not permitted to be called from a
signal handler that interrupts pthread_exit (which is itself
async-signal-unsafe).
this change may also simplify logic in __synccall and improve the
chances of making __synccall async-signal-safe.
for the duration of the vm-sharing clone used by posix_spawn, all
signals are blocked in the parent process, including
implementation-internal signals. since __synccall cannot do anything
until successfully signaling all threads, the fact that signals are
blocked automatically yields the necessary safety.
aside from debloating and general simplification, part of the
motivation for removing the explicit lock is to simplify the
synchronization logic of __synccall in hopes that it can be made
async-signal-safe, which is needed to make setuid and setgid, which
depend on __synccall, conform to the standard. whether this will be
possible remains to be seen.
this allows /etc/ld-musl-$(ARCH).path to contain one path per line,
which is much more convenient for users than the :-delimited format,
which was a source of repeated and unnecessary confusion. for
simplicity, \n is also accepted in environment variables, though it
should probably not be used there.
at the same time, issues with overly long paths invoking UB or getting
truncated have been fixed. such issues should not have arisen with the
environment (which is size-limited) but could have been generated by a
path file larger than 2**31 bytes in length.
the standard is clear that the old behavior is conforming: "In this
case, [EILSEQ] shall be stored in errno and the conversion state is
undefined."
however, the specification of mbrtowc has one peculiarity when the
source argument is a null pointer: in this case, it's required to
behave as mbrtowc(NULL, "", 1, ps). no motivation is provided for this
requirement, but the natural one that comes to mind is that the intent
is to reset the mbstate_t object. for stateful encodings, such
behavior is actually specified: "If the corresponding wide character
is the null wide character, the resulting state described shall be the
initial conversion state." but in the case of UTF-8 where the
mbstate_t object contains a partially-decoded character rather than a
shift state, a subsequent '\0' byte indicates that the previous
partial character is incomplete and thus an illegal sequence.
naturally, applications using their own mbstate_t object should clear
it themselves after an error, but the standard presently provides no
way to clear the builtin mbstate_t object used when the ps argument is
a null pointer. I suspect this issue may be addressed in the future by
specifying that a null source argument resets the state, as this seems
to have been the intent all along.
for what it's worth, this change also slightly reduces code size.
the interface contract for mbtowc admits a much faster implementation
than mbrtowc can achieve; wrapping mbrtowc with an extra call frame
only made the situation worse.
since the regex implementation uses mbtowc already, this change should
improve regex performance too. it may be possible to improve
performance in other places internally by switching from mbrtowc to
mbtowc.
this simple change, in my measurements, makes about a 7% performance
improvement. at first glance this change would seem like a
compiler-specific hack, since the modified code is not even used.
however, I suspect the reason is that I'm eliminating a second path
into the main body of the code, allowing the compiler more flexibility
to optimize the normal (hot) path into the main body. so even if it
weren't for the measurable (and quite notable) difference in
performance, I think the change makes sense.
SA and SB are used as the lowest and highest valid starter bytes, but
the value of SB was one-past the last valid starter. this caused
access past the end of the state table when the illegal byte '\xf5'
was encountered in a starter position. the error did not show up in
full-character decoding tests, since the bogus state read from just
past the table was unlikely to admit any continuation bytes as valid,
but would have shown up had we tested feeding '\xf5' to the
byte-at-a-time decoding in mbrtowc: it would cause the funtion to
wrongly return -2 rather than -1.
I may eventually go back and remove all references to SA and SB,
replacing them with the values; this would make the code more
transparent, I think. the original motivation for using macros was to
allow misguided users of the code to redefine them for the purpose of
enlarging the set of accepted sequences past the end of Unicode...
also include fallback code for broken kernels that don't support the
flags. as usual, the fallback has a race condition that can leak file
descriptors.
this is a bit ugly, and the motivation for supporting it is
questionable. however the main factors were:
1. it will be useful to have this for certain internal purposes
anyway -- things like syslog.
2. applications can just save argv[0] in main, but it's hard to fix
non-portable library code that's depending on being able to get the
invocation name without the main application's help.
GNU used several extensions that were incompatible with C99 and POSIX,
so they used alternate names for the standard functions.
The result is that we need these to run standards-conformant programs
that were linked with glibc.
Rich Felker
Szabolcs Nagy
rofl0r
Isaac Dunham