general policy is that all source files defining a public API or an
ABI mechanism referenced by a public header should include the public
header that declares the interface, so that the compiler or analysis
tools can check the consistency of the declarations. Alexander Monakov
pointed out a number of violations of this principle a few years back.
fix them now.
add a member of appropriate type to the fpos_t union so that accesses
are well-defined. use long long instead of off_t since off_t is not
always exposed in stdio.h and there's no namespace-clean alias for it.
access is still performed using pointer casts rather than by naming
the union member as a matter of style; to the extent possible, the
naming of fields in opaque types defined in the public headers is not
treated as an API contract with the implementation. access via the
pointer cast is valid as long as the union has a member of matching
type.
previously this macro used an odd if/else form instead of the more
idiomatic do/while(0), making it unsafe against omission of trailing
semicolon. the omission would make the following statement conditional
instead of producing an error.
formally, calling readv with a zero-length first iov component should
behave identically to calling read on just the second component, but
presence of a zero-length iov component has triggered bugs in some
kernels and performs significantly worse than a simple read on some
file types.
the stdio FILE read backend's return type is size_t, not ssize_t, and
all of the special (non-fd-backed) FILE types already return the
number of bytes read (zero) on error or eof. only __stdio_read leaked
a syscall error return into its return value.
fread had a workaround for this behavior going all the way back to the
original check-in. remove the workaround since it's no longer needed.
the ':' in optstring has special meaning as a flag applying to the
previous option character, or to getopt's error handling behavior when
it appears at the beginning. don't also accept a "-:" option based on
its presence.
based loosely on patch by Hauke Mehrtens; converted to wrap the public
API of the underlying getrandom function rather than direct syscalls,
so that if/when a fallback implementation of getrandom is added it
will automatically get picked up by getentropy too.
this is more extensible if we need to consider additional errors, and
more efficient as long as the compiler does not know it can cache the
result of __errno_location (a surprisingly complex issue detailed in
commit a603a75a72).
It's better to make execvp continue PATH search on ENOTDIR rather than
issuing an error. Bogus entries should not render rest of PATH invalid.
Maintainer's note: POSIX seems to require the search to continue like
this as part of XBD 8.3 Other Environment Variables. Only errors that
conclusively determine non-existence are candidates for continuing;
otherwise for consistency we have to report the error.
when a null buffer pointer is passed to fmemopen, requesting it
allocate its own memory buffer, extremely large size arguments near
SIZE_MAX could overflow and result in underallocation. this results
from omission of the size of the cookie structure in the overflow
check but inclusion of it in the calloc call.
instead of accounting for individual small contributions to the total
allocation size needed, simply reject sizes larger than PTRDIFF_MAX,
which will necessarily fail anyway. then adding arbitrary fixed-size
structures is safe without matching up the expressions in the
comparison and the allocation.
Currently getcwd(3) can succeed without returning an absolute path
because the underlying getcwd syscall, starting with linux commit
v2.6.36-rc1~96^2~2, may succeed without returning an absolute path.
This is a conformance issue because "The getcwd() function shall
place an absolute pathname of the current working directory
in the array pointed to by buf, and return buf".
Fix this by checking the path returned by syscall and failing with
ENOENT if the path is not absolute. The error code is chosen for
consistency with the case when the current directory is unlinked.
Similar issue was fixed in glibc recently, see
https://sourceware.org/bugzilla/show_bug.cgi?id=22679
it's unclear from the specification whether the word "consumes" in
"consumes more than four bytes to represent a year" refers just to
significant places or includes leading zeros due to field width
padding. however the examples in the rationale indicate that the
latter was the intent. in particular, the year 270 is shown being
formatted by %+5Y as +0270 rather than 00270.
previously '+' prefixing was implemented just by comparing the year
against 10000. instead, count the number of significant digits and
padding bytes to be added, and use the total to determine whether to
apply the '+' prefix.
based on testing by Dennis Wölfing.
the code to strip initial sign and leading zeros inadvertently
stripped all the zeros and the subsequent '-' separating the month.
instead, only strip sign characters from the very first position, and
only strip zeros when they are followed by another digit.
based on testing by Dennis Wölfing.
in the original submission of the patch that became commit
7c709f2d4f, and in subsequent reading of
it by others, it was not clear that the new member had to be inserted
before canary_at_end, or that inserting it at that location was safe.
add comments to document.
Do not retry waitpid if the child was terminated by a signal. Do not
examine status: since we are not passing any flags, we will not receive
stop or continue notifications.
commit f9fb20b42d switched from using a
pipe for the result to conveying it via the child process exit status.
Alexander Monakov pointed out that the latter could fail if the
application is not expecting faccessat to produce a child and performs
a wait operation with __WCLONE or __WALL, and that it is not clear
whether it's guaranteed to work when SIGCHLD's disposition has been
set to SIG_IGN.
in addition, that commit introduced a bug that caused EACCES to be
produced instead of EBUSY due to an exit path that was overlooked when
the error channel was changed, and introduced a spurious retry loop
around the wait operation.
Consider the first equals sign found in the option to be the delimiter
between it and its argument, even if it matches an equals sign in the
option name. This avoids consuming the equals sign, which would prevent
finding the argument. Instead, it forces a partial match of the part of
the option name before the equals sign.
Maintainer's note: GNU getopt_long does not explicitly document this
behavior, but it can be seen as a consequence of how partial matches
are specified, and at least GNU (bfd) ld is known to make use of it.
If we find a partial option name match, we need to keep looking for
ambiguous/conflicting options. However, we need to remember the position
in the candidate argument to find its option-argument later, if there is
one. This fixes e.g. option "foobar" being given as "--fooba=baz".
commit 78897b0dc0 wrongly simplified
Dmitry Levin's original submitted patch fixing alt-form octal with the
zero flag and field width present, omitting the special case where the
value is zero. as a result, printf("%#o",0) wrongly prints "00" rather
than "0".
the logic prior to this commit was actually better, in that it was
aligned with how the alt-form flag (#) for printf is specified ("it
shall increase the precision"). at the time there was no good way to
avoid the zero flag issue with the old logic, but commit
167dfe9672 added tracking of whether an
explicit precision was provided.
revert commit 78897b0dc0 and switch to
using the explicit precision indicator for suppressing the zero flag.
In some places there has been a direct usage of the functions. Use the
macros consistently everywhere, such that it might be easier later on to
capture the fast path directly inside the macro and only have the call
overhead on the slow path.
A variant of this new lock algorithm has been presented at SAC'16, see
https://hal.inria.fr/hal-01304108. A full version of that paper is
available at https://hal.inria.fr/hal-01236734.
The main motivation of this is to improve on the safety of the basic lock
implementation in musl. This is achieved by squeezing a lock flag and a
congestion count (= threads inside the critical section) into a single
int. Thereby an unlock operation does exactly one memory
transfer (a_fetch_add) and never touches the value again, but still
detects if a waiter has to be woken up.
This is a fix of a use-after-free bug in pthread_detach that had
temporarily been patched. Therefore this patch also reverts
c1e27367a9
This is also the only place where internal knowledge of the lock
algorithm is used.
The main price for the improved safety is a little bit larger code.
Under high congestion, the scheduling behavior will be different
compared to the previous algorithm. In that case, a successful
put-to-sleep may appear out of order compared to the arrival in the
critical section.
previously, the charset names without endianness specified were always
interpreted as big endian. unicode specifies that UTF-16 and UTF-32
have BOM-determined endianness if BOM is present, and are otherwise
big endian. since commit 5b546faa67
added support for stateful encodings, it is now possible to implement
BOM support via the conversion descriptor state.
for conversions to these charsets, the output is always big endian and
does not have a BOM.
the mapping tables and code are not automatically generated; they were
produced by comparing the output of towupper/towlower against the
mappings in the UCD, ignoring characters that were previously excluded
from case mappings or from alphabetic status (micro sign and circled
letters), and adding table entries or code for everything else
missing.
based very loosely on a patch by Reini Urban.
the new version of the code used to generate these tables forces a
newline every 256 entries, whereas at the time these files were
originally generated and committed, it only wrapped them at 80
columns. the new behavior ensures that localized changes to the
tables, if they are ever needed, will produce localized diffs.
commit d060edf6c5 made the corresponding
changes to the iconv tables.
notes by maintainer:
both C and POSIX use the term UTC to specify related functionality,
despite POSIX defining it as something more like UT1 or historical
(pre-UTC) GMT without leap seconds. neither specifies the associated
string for %Z. old choice of "GMT" violated principle of least
surprise for users and some applications/tests. use "UTC" instead.
aside from theoretical arbitrary results due to UB, this could
practically cause unbounded overflow of static array if hit, but
hitting it depends on having more than 32 calls to at_quick_exit and
having them sufficiently often.
notes added by maintainer:
the '-' specifier allows default padding to be suppressed, and '_'
allows padding with spaces instead of the default (zeros).
these extensions seem to be included in several other implementations
including FreeBSD and derivatives, and Solaris. while portable
software should not depend on them, time format strings are often
exposed to the user for configurable time display. reportedly some
python programs also use and depend on them.
notes added by maintainer:
this function is a GNU extension. it was chosen over the similar BSD
function funopen because the latter depends on fpos_t being an
arithmetic type as part of its public API, conflicting with our
definition of fpos_t and with the intent that it be an opaque type. it
was accepted for inclusion because, despite not being widely used, it
is usually very difficult to extricate software using it from the
dependency on it.
calling pattern for the read and write callbacks is not likely to
match glibc or other implementations, but should work with any
reasonable callbacks. in particular the read function is never called
without at least one byte being needed to satisfy its caller, so that
spurious blocking is not introduced.
contracts for what callbacks called from inside libc/stdio can do are
always complicated, and at some point still need to be specified
explicitly. at the very least, the callbacks must return or block
indefinitely (they cannot perform nonlocal exits) and they should not
make calls to stdio using their own FILE as an argument.
previously, fgetwc left all but the first byte of an illegal sequence
unread (available for subsequent calls) when reading out of the FILE
buffer, but dropped all bytes contibuting to the error when falling
back to reading a byte at a time. neither behavior was ideal. in the
buffered case, each malformed character produced one error per byte,
rather than one per character. in the unbuffered case, consuming the
last byte that caused the transition from "incomplete" to "invalid"
state potentially dropped (and produced additional spurious encoding
errors for) the next valid character.
to handle both cases uniformly without duplicate code, revise the
buffered case to only cover situations where a complete and valid
character is present in the buffer, and fall back to byte-at-a-time
for all other cases. this allows using mbtowc (stateless) instead of
mbrtowc, which may slightly improve performance too.
when an encoding error has been hit in the byte-at-a-time case, leave
the final byte that produced the error unread (via ungetc) except in
the case of single-byte errors (for UTF-8, bytes c0, c1, f5-ff, and
continuation bytes with no lead byte). single-byte errors are fully
consumed so as not to leave the caller in an infinite loop repeating
the same error.
none of these changes are distinguished from a conformance standpoint,
since the file position is unspecified after encoding errors. they are
intended merely as QoI/consistency improvements.
fgetwc does not set the stream's error indicator on encoding errors,
making ferror insufficient to distinguish between error and eof
conditions. feof is also insufficient, since it will return true if
the file ended with a partial character encoding error.
whether fgetwc should be setting the error indicator itself is a
question with conflicting answers. the POSIX text for the function
states it as a requirement, but the ISO C text seems to require that
it not. this may be revisited in the future based on the outcome of
Austin Group issue #1170.
these encodings are still commonly used in messaging protocols and
such. the reverse mapping is implemented as a binary search of a list
of the jis 0208 characters in unicode order; the existing forward
table is used to perform the comparison in the search.
previously, 8-bit codepages could only remap the high 128 bytes; the
low range was assumed/forced to agree with ascii. interpretation of
codepage table headers has been changed so that it's possible to
represent mappings for up to 256 slots (fewer if the initial portion
of the map is elided because it coincides with unicode codepoints).
this requires consuming a bit more of the 10-bit space of characters
that can be represented in 8-bit codepages, but there's still a plenty
left. the size of the legacy_chars table is actually reduced now by
eliding the first 256 entries and considering them to map implicitly
via the identity map.
before these changes, there seem to have been minor bugs/omissions in
codepage table generation, so it's likely that some actual bug fixes
are silently included in this commit. round-trip testing of a few
codepages was performed on the new version of the code, but no
differential testing against the old version was done.
the new version of the code used to generate these tables forces a
newline every 256 entries, whereas at the time these files were
originally generated and committed, it only wrapped them at 80
columns. the new behavior ensures that localized changes to the
tables, if they are ever needed, will produce localized diffs. other
tables including hkscs were already committed in the new format.
binary comparison of the generated object files was performed to
confirm that no spurious changes slipped in.
If the syscall fails, errno must be set correctly for the caller.
There's no guarantee that the handlers registered with pthread_atfork
won't clobber errno, so we need to ensure it gets set after they are
called.
this implementation aims to match the baseline defined by rfc1468 (the
original mime charset definition) plus the halfwidth katakana
extension included in the whatwg definition of the charset. rejection
of si/so controls and newlines in doublebyte state are not currently
enforced. the jis x 0201 mode is currently interpreted as having the
yen sign and overline character in place of backslash and tilde; ascii
mode has the standard ascii characters in those slots.
assuming pointers obtained from malloc have some nonzero alignment,
repurpose the low bit of iconv_t as an indicator that the descriptor
is a stateless value representing the source and destination character
encodings.
Rich Felker
Hauke Mehrtens
Przemyslaw Pawelczyk
Dmitry V. Levin
Alexander Monakov
William Pitcock
Samuel Holland
Jens Gustedt
Natanael Copa
Timo Teräs
Szabolcs Nagy
Bobby Bingham