the man page for this nonstandardized function has historically
documented it as scanning for a substring; however, this is
functionally incorrect (matches the substring "atime" in the "noatime"
option, for example) and differs from other existing implementations.
with the change made here, it should match glibc and other
implementations, only matching whole options delimited by commas or
separated from a value by an equals sign.
When the pattern was changed from matching any whitespace to just
matching spaces and tabs, a newline started being appended to the
value of the matched field, if that field was a string. For example,
in a 4-field line, the mnt_opts field would have a newline on the end.
This happened because a newline is not a space or a tab, and so was
matched as part of the value before the end of the string was reached.
\n should therefore be added as a character that terminates a value.
This shouldn't interfere with the intention of the change to space and
tab only, as it was trying to make sure that other whitespace like
carriage returns, that should have been part of parsed values, were.
Fixes: f314e133
commit 895736d49b made these changes
along with fixing a real bug in LOG_MAKEPRI. based on further
information, they do not seem to be well-motivated or in line with
policy.
the result of LOG_FAC is not a meaningful facility value if we shift
it down like before, but apparently the way it is used by applications
is as an index into an array of facility names. moreover, all
historical systems which define it do so with the shift. as it is a
nonstandard interface, there is no justification for providing a macro
by the same name that is incompatible with historical practice.
the value of LOG_FACMASK likewise is 0x3f8 on all historical systems
checked. while only 5 bits are used for existing facility codes, the
convention seems to be that all 7 bits belong to the facility field
and theoretically could be used to expand to having more facilities.
that seems unlikely to happen, but there is no reason to make a
gratuitously incompatible change here.
these are nonstandard and unnecessary for using the associated
functionality, but resulted in applications that used them
malfunctioning.
patch based on proposed fix by erny hombre.
historically linux limited the number of supplementary groups a
process could be in to 32, but this limit was raised to 65536 in linux
2.6.4. proposals to support the new limit, change NGROUPS_MAX, or make
it dynamic have been stalled due to the impact it would have on
initgroups where the groups array exists in automatic storage.
the changes here decouple initgroups from the value of NGROUPS_MAX and
allow it to fall back to allocating a buffer in the case where
getgrouplist indicates the user has more supplementary groups than
could be reported in the buffer. getgrouplist already involves
allocation, so this does not pull in any new link dependency.
likewise, getgrouplist is already using the public malloc (vs internal
libc one), so initgroups does the same. if this turns out not to be
the best choice, both can be changed together later.
the initial buffer size is left at 32, but now as the literal value,
so that any potential future change to NGROUPS_MAX will not affect
initgroups.
depending on contents of the LC_TIME locale, log messages could be
malformatted (especially if the ABMON strings contain non-alphabetic
characters) or the subsequent code could invoke undefined behavior,
via passing a timebuf[] with unspecified contents to snprintf, if
the translated ABMON string did not fit in the 16-byte timebuf.
this does not appear to be a security-relevant bug, as locale loading
functionality is intentionally not available to set*id programs -- the
MUSL_LOCPATH environment variable is ignored when libc.secure is true,
and custom locales are not loadable without it.
this matters because the kernel-provided mtab only escapes tabs,
spaces, newlines, and backslashes. it leaves carriage returns, form
feeds, and vertical tabs literal.
As entries in mtab are delimited by spaces, whitespace characters
are escaped as octal sequences. When reading them out, we have to
unescape these sequences to get the proper string.
When an option that requires an argument is the last character of
argv[argc-1], getopt computes argv[argc] + optpos. While optpos
is always zero in this case, adding it to null pointer is still
undefined.
If lstat/stat fails with EACCES, st is left uninitialized, but its
st_dev/st_ino fields are then used in several places:
* for FTW_MOUNT check (in practice typically results in a false
positive and an early return)
* for copying to the new struct history (though the struct is not used
afterwards since we don't recurse in this case)
* for cycle detection check (could theoretically result in a false
positive and an early return)
To avoid adding FTW_NS checks to all these places, fix this by
zero-initializing st_dev/st_ino (which can never match an existing
dentry due to zero inode being reserved in Linux), and check for FTW_NS
only when handling FTW_MOUNT since we need two valid dentries there.
originally the namespace-infringing "large file support" interfaces
were included as part of glibc-ABI-compat, with the intent that they
not be used for linking, since our off_t is and always has been
unconditionally 64-bit and since we usually do not aim to support
nonstandard interfaces when there is an equivalent standard interface.
unfortunately, having the symbols present and available for linking
caused configure scripts to detect them and attempt to use them
without declarations, producing all the expected ill effects that
entails.
as a result, commit 2dd8d5e1b8 was made
to prevent this, using macros to redirect the LFS64 names to the
standard names, conditional on _GNU_SOURCE or _LARGEFILE64_SOURCE.
however, this has turned out to be a source of further problems,
especially since g++ defines _GNU_SOURCE by default. in particular,
the presence of these names as macros breaks a lot of valid code.
this commit removes all the LFS64 symbols and replaces them with a
mechanism in the dynamic linker symbol lookup failure path to retry
with the spurious "64" removed from the symbol name. in the future,
if/when the rest of glibc-ABI-compat is moved out of libc, this can be
removed.
commit 05973dc3bb made it so that lines
longer than INT_MAX can in theory be read, but did not use a suitable
type for the positions determined by sscanf. we could change to using
size_t, but since the signature for getmntent_r does not admit lines
longer than INT_MAX, it does not make sense to support them in the
legacy thread-unsafe form either -- the principle here is that there
should not be an incentive to use the unsafe function to get added
functionality.
According to fstab(5), the last two fields are optional, but this
wasn't accepted. After this change, only the first field is required,
which matches glibc's behaviour.
Using sscanf as before, it would have been impossible to differentiate
between 0 fields and 4 fields, because sscanf would have returned 0 in
both cases due to the use of assignment suppression and %n for the
string fields (which is important to avoid copying any strings). So
instead, before calling sscanf, initialize every string to the empty
string, and then we can check which strings are empty afterwards to
know how many fields were matched.
the snd_pcm_mmap_control struct used with SNDRV_PCM_IOCTL_SYNC_PTR was
mistakenly defined in the kernel uapi with "before u32" padding both
before and after the first u32 member. our conversion between the
modern struct and the legacy time32 struct was written without
awareness of that mistake, and assumed the time64 version of the
struct was the intended form with padding to match the layout on
64-bit archs. as a result, the struct was not converted correctly when
running on old kernels, with audio glitches as the likely result.
this was discovered thanks to a related bug in the kernel, whereby
32-bit userspace running on a 64-bit kernel also suffered from the
types mismatching. the mistaken layout is now the ABI and can't be
changed -- or at least making a new ioctl to change it would just
result in a worse situation.
our conversion here is changed to treat the snd_pcm_mmap_control
substruct as two separate substructs at locations dependent on
endianness (since the displacement depends on endianness), using the
existing conversion framework.
commit 2412638bb3 got the size of struct
v4l2_event wrong and failed to account for the fact that the old
struct might be either 120 bytes with time misaligned mod 8, or 128
bytes with time aligned mod 8, due to the contained union having
64-bit members whose alignment is arch-dependent.
rather than adding new logic to handle the differences, use an actual
stripped-down version of the structure in question to derive the ioctl
number, size, and offsets.
commit 2412638bb3 got the size of struct
v4l2_buffer wrong and omitted the tv_usec member slot from the offset
list, so the ioctl numbers never matched and fallback code path was
never taken. this caused the affected ioctls to fail with ENOTTY on
kernels not new enough to have the native time64 ioctls.
inability to use realpath in chroot/container without procfs access
and at early boot prior to mount of /proc has been an ongoing issue,
and it turns out realpath was one of the last remaining interfaces
that needed procfs for its core functionality. during investigation
while reimplementing, it was determined that there were also serious
problems with the procfs-based implementation. most seriously it was
unsafe on pre-O_PATH kernels, and unlike other places where O_PATH was
used, the unsafety was hard or impossible to fix because O_NOFOLLOW
can't be used (since the whole purpose was to follow symlinks).
the new implementation is a direct one, performing readlink on each
path component to resolve it. an explicit stack, as opposed to
recursion, is used to represent the remaining components to be
processed. the stack starts out holding just the input string, and
reading a link pushes the link contents onto the stack.
unlike many other implementations, this one does not call getcwd
initially for relative pathnames. instead it accumulates initial ..
components to be applied to the working directory if the result is
still a relative path. this avoids calling getcwd (which may fail) at
all when symlink traversal will eventually yield an absolute path. it
also doesn't use any form of stat operation; instead it arranges for
readlink to tell it when a non-directory is used in a context where a
directory is needed. this minimizes the number of syscalls needed,
avoids accessing inodes when the directory table suffices, and reduces
the amount of code pulled in for static linking.
as the outcome of Austin Group tracker issue #62, future editions of
POSIX have dropped the requirement that fork be AS-safe. this allows
but does not require implementations to synchronize fork with internal
locks and give forked children of multithreaded parents a partly or
fully unrestricted execution environment where they can continue to
use the standard library (per POSIX, they can only portably use
AS-safe functions).
up until recently, taking this allowance did not seem desirable.
however, commit 8ed2bd8bfc exposed the
extent to which applications and libraries are depending on the
ability to use malloc and other non-AS-safe interfaces in MT-forked
children, by converting latent very-low-probability catastrophic state
corruption into predictable deadlock. dealing with the fallout has
been a huge burden for users/distros.
while it looks like most of the non-portable usage in applications
could be fixed given sufficient effort, at least some of it seems to
occur in language runtimes which are exposing the ability to run
unrestricted code in the child as part of the contract with the
programmer. any attempt at fixing such contracts is not just a
technical problem but a social one, and is probably not tractable.
this patch extends the fork function to take locks for all libc
singletons in the parent, and release or reset those locks in the
child, so that when the underlying fork operation takes place, the
state protected by these locks is consistent and ready for the child
to use. locking is skipped in the case where the parent is
single-threaded so as not to interfere with legacy AS-safety property
of fork in single-threaded programs. lock order is mostly arbitrary,
but the malloc locks (including bump allocator in case it's used) must
be taken after the locks on any subsystems that might use malloc, and
non-AS-safe locks cannot be taken while the thread list lock is held,
imposing a requirement that it be taken last.
resource limits have been process-wide since linux 2.6.10, and the
prlimit syscall was added in 2.6.36, so prlimit can be assumed to set
the resource limits correctly for the whole process.
access always computes result with real ids not effective ones, so it
is not a valid means of determining whether the directory is readable.
instead, attempt to open it before reporting whether it's readable,
and then use fdopendir rather than opendir to open and read the
entries.
effort is made here to keep fd_limit behavior the same as before even
if it was not correct.
adding this condition makes the entire convert_ioctl_struct function
and compat_map table statically unreachable, and thereby optimized out
by dead code elimination, on archs where they are not needed.
VIDIOC_OMAP3ISP_STAT_REQ is a device-specific command for the omap3isp
video device. the command number is in a device-private range and
therefore could theoretically be used by other devices too in the
future, but problematic clashes should not be able to arise without
intentional misuse.
this commit covers all remaining ioctls I'm aware of that use
time_t-derived types in their interfaces. it may still be incomplete,
and has undergone only minimal testing for a few commands used in
audio playback.
the SNDRV_PCM_IOCTL_SYNC_PTR command is special-cased because, rather
than the whole structure expanding, it has two substructures each
padded to 64 bytes that expand within their own 64-byte reserved zone.
as long as it's the only one of its type, it doesn't really make sense
to make a general framework for it, but the existing table framework
is still used for the substructures in the special-case. one of the
substructures, snd_pcm_mmap_status, has a snd_pcm_uframes_t member
which is not a timestamp but is expanded just like one, to match the
64-bit-arch version of the structure. this is handled just like a
timestamp at offset 8, and is the motivation for the conversions table
holding offsets of individual values to be expanded rather than
timespec/timeval type pairs.
for some of the types, the size to which they expand is dependent on
whether the arch's ABI aligns 8-byte types on 8-byte boundaries.
new_req entries in the table need to reflect this size to get the
right ioctl request number that will match what callers pass, but we
don't have access to the actual structure type definitions here and
duplicating them would be cumbersome. instead, the new_misaligned
macro introduced here constructs an artificial object whose size is
the result of expanding a misaligned timespec/timeval to 64-bit and
imposing the arch's alignment on the result, which can be passed to
the _IO{R,W,WR} macros.
record offsets of individual slots that expand from 32- to 64-bit,
rather than timespec/timeval pairs. this flexibility will be needed
for some ioctls. reduce size of types in table. adjust representation
of offsets to include a count rather than needing -1 padding so that
the table is less ugly and doesn't need large diffs if we increase max
number of slots.
with the current set of supported ioctls, this conversion is hardly an
improvement, but it sets the stage for being able to do alsa, v4l2,
ppp, and other ioctls with timespec/timeval-derived types. without
this capability, a lot of functionality users depend on would stop
working with the time64 switchover.
presently the kernel does not actually define time64 versions of these
syscalls, and they're not really needed except to represent extreme
cpu time usage. however, x32's versions of the syscalls already behave
as time64 ones, meaning the functions were broken on x32 if the caller
used any part of the rusage result other than ru_utime and ru_stime.
commit 7e81711431 made it possible to
fix this by treating x32's syscalls as time64 versions.
in the non-time64-syscall case, make the syscall with the rusage
destination pointer adjusted so that all members but the timevals line
up between the libc and kernel structures. on 64-bit archs, or present
32-bit archs with 32-bit time_t, the timevals will line up too and no
further work is needed. for future 32-bit archs with 64-bit time_t,
the timevals are copied into place, contingent on time_t being larger
than long.
commit 030e526392 added optreset, a BSD
extension to getopt duplicating the functionality (also an extension)
of setting optind to 0, but failed to provide a public declaration for
it. according to the BSD documentation and headers, the application is
not supposed to need to provide its own declaration.
without this, the SIOCGSTAMP and SIOCGSTAMPNS ioctl commands, for
obtaining timestamps, would stop working on pre-5.1 kernels after
time_t is switched to 64-bit and their values are changed to the new
time64 versions.
new code is written such that it's statically unreachable on 64-bit
archs, and on existing 32-bit archs until the macro values are changed
to activate 64-bit time_t.
this further reduces the number of source files which need to include
libc.h and thereby be potentially exposed to libc global state and
internals.
this will also facilitate further improvements like adding an inline
fast-path, if we want to do so later.
the LFS64 macro was not self-documenting and barely saved any
characters. simply use weak_alias directly so that it's clear what's
being done, and doesn't depend on a header to provide a strange macro.
libc.h was intended to be a header for access to global libc state and
related interfaces, but ended up included all over the place because
it was the way to get the weak_alias macro. most of the inclusions
removed here are places where weak_alias was needed. a few were
recently introduced for hidden. some go all the way back to when
libc.h defined CANCELPT_BEGIN and _END, and all (wrongly implemented)
cancellation points had to include it.
remaining spurious users are mostly callers of the LOCK/UNLOCK macros
and files that use the LFS64 macro to define the awful *64 aliases.
in a few places, new inclusion of libc.h is added because several
internal headers no longer implicitly include libc.h.
declarations for __lockfile and __unlockfile are moved from libc.h to
stdio_impl.h so that the latter does not need libc.h. putting them in
libc.h made no sense at all, since the macros in stdio_impl.h are
needed to use them correctly anyway.
commits leading up to this one have moved the vast majority of
libc-internal interface declarations to appropriate internal headers,
allowing them to be type-checked and setting the stage to limit their
visibility. the ones that have not yet been moved are mostly
namespace-protected aliases for standard/public interfaces, which
exist to facilitate implementing plain C functions in terms of POSIX
functionality, or C or POSIX functionality in terms of extensions that
are not standardized. some don't quite fit this description, but are
"internally public" interfacs between subsystems of libc.
rather than create a number of newly-named headers to declare these
functions, and having to add explicit include directives for them to
every source file where they're needed, I have introduced a method of
wrapping the corresponding public headers.
parallel to the public headers in $(srcdir)/include, we now have
wrappers in $(srcdir)/src/include that come earlier in the include
path order. they include the public header they're wrapping, then add
declarations for namespace-protected versions of the same interfaces
and any "internally public" interfaces for the subsystem they
correspond to.
along these lines, the wrapper for features.h is now responsible for
the definition of the hidden, weak, and weak_alias macros. this means
source files will no longer need to include any special headers to
access these features.
over time, it is my expectation that the scope of what is "internally
public" will expand, reducing the number of source files which need to
include *_impl.h and related headers down to those which are actually
implementing the corresponding subsystems, not just using them.
it's not ideal, but the function is essentially an extended stdio
function specialized to getopt's needs. the only reason it exists is
avoiding pulling printf code into every program using getopt.
the public flockfile interface is significantly heavier because it has
to handle the possibility of caller returning or thread exiting while
holding the lock.
syscall.h was chosen as the header to declare it, since its intended
usage is alongside syscalls as a fallback for operations the direct
syscall does not support.
policy is that all public functions which have a public declaration
should be defined in a context where that public declaration is
visible, to avoid preventable type mismatches.
an audit performed using GCC's -Wmissing-declarations turned up the
violations corrected here. in some cases the public header had not
been included; in others, a feature test macro needed to make the
declaration visible had been omitted.
in the case of gethostent and getnetent, the omission seems to have
been intentional, as a hack to admit a single stub definition for both
functions. this kind of hack is no longer acceptable; it's UB and
would not fly with LTO or advanced toolchains. the hack is undone to
make exposure of the declarations possible.
the text of the specification for getopt's handling of options that
require an argument, which requires updating optarg and optind, does
not exclude the error case where the end of the argument list has been
reached. in that case, it is expected that optarg be assigned
argv[argc] (normally null) and optind be incremented by 2, resulting
in a value of argc+1.
for getopt_long, partial (prefix) matches of long options always begin
with "--" and thus can never be ambiguous with a short option. for
getopt_long_only, though, a single-character option can match both a
short option and as a prefix for a long option. in this case, we
wrongly interpreted it as a prefix for the long option.
introduce a new pass, only in long-only mode, to check the prefix
match against short options before accepting it. the only reason
there's a slightly nontrivial loop being introduced rather than strchr
is that our getopt already supports multibyte short options, and
getopt_long_long should handle them consistently. a temp buffer and
strstr could have been used, but the code to set it up would be just
as large as what's introduced here and it would unnecessarily pull in
relatively large code for strstr.
general policy is that all source files defining a public API or an
ABI mechanism referenced by a public header should include the public
header that declares the interface, so that the compiler or analysis
tools can check the consistency of the declarations. Alexander Monakov
pointed out a number of violations of this principle a few years back.
fix them now.
the ':' in optstring has special meaning as a flag applying to the
previous option character, or to getopt's error handling behavior when
it appears at the beginning. don't also accept a "-:" option based on
its presence.
based loosely on patch by Hauke Mehrtens; converted to wrap the public
API of the underlying getrandom function rather than direct syscalls,
so that if/when a fallback implementation of getrandom is added it
will automatically get picked up by getentropy too.
Do not retry waitpid if the child was terminated by a signal. Do not
examine status: since we are not passing any flags, we will not receive
stop or continue notifications.
Consider the first equals sign found in the option to be the delimiter
between it and its argument, even if it matches an equals sign in the
option name. This avoids consuming the equals sign, which would prevent
finding the argument. Instead, it forces a partial match of the part of
the option name before the equals sign.
Maintainer's note: GNU getopt_long does not explicitly document this
behavior, but it can be seen as a consequence of how partial matches
are specified, and at least GNU (bfd) ld is known to make use of it.
Rich Felker
Alyssa Ross
q66
Alexey Izbyshev
Stefan O'Rear
Arnd Bergmann
Bartosz Brachaczek
Alexander Monakov
Samuel Holland