riscv/qemu - qemu - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Stefan Hajnoczi	1eebdab3c3	aio-posix: add aio_add_sqe() API for user-defined io_uring requests Introduce the aio_add_sqe() API for submitting io_uring requests in the current AioContext. This allows other components in QEMU, like the block layer, to take advantage of io_uring features without creating their own io_uring context. This API supports nested event loops just like file descriptor monitoring and BHs do. This comes at a complexity cost: CQE callbacks must be placed on a list so that nested event loops can invoke pending CQE callbacks from parent event loops. If you're wondering why CqeHandler exists instead of just a callback function pointer, this is why. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-ID: <20251104022933.618123-14-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>	5 months ago
Stefan Hajnoczi	a63e41f2a4	aio-posix: unindent fdmon_io_uring_destroy() Reduce the level of indentation to make further code changes easier to read. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20251104022933.618123-12-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>	5 months ago
Stefan Hajnoczi	59202c98c0	aio-posix: gracefully handle io_uring_queue_init() failure io_uring may not be available at runtime due to system policies (e.g. the io_uring_disabled sysctl) or creation could fail due to file descriptor resource limits. Handle failure scenarios as follows: If another AioContext already has io_uring, then fail AioContext creation so that the aio_add_sqe() API is available uniformly from all QEMU threads. Otherwise fall back to epoll(7) if io_uring is unavailable. Notes: - Update the comment about selecting the fastest fdmon implementation. At this point it's not about speed anymore, it's about aio_add_sqe() API availability. - Uppercase the error message when converting from error_report() to error_setg_errno() for consistency (but there are instances of lowercase in the codebase). - It's easier to move the #ifdefs from aio-posix.h to aio-posix.c. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-ID: <20251104022933.618123-11-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>	5 months ago
Stefan Hajnoczi	ded29e64c6	aio-posix: integrate fdmon into glib event loop AioContext's glib integration only supports ppoll(2) file descriptor monitoring. epoll(7) and io_uring(7) disable themselves and switch back to ppoll(2) when the glib event loop is used. The main loop thread cannot use epoll(7) or io_uring(7) because it always uses the glib event loop. Future QEMU features may require io_uring(7). One example is uring_cmd support in FUSE exports. Each feature could create its own io_uring(7) context and integrate it into the event loop, but this is inefficient due to extra syscalls. It would be more efficient to reuse the AioContext's existing fdmon-io_uring.c io_uring(7) context because fdmon-io_uring.c will already be active on systems where Linux io_uring is available. In order to keep fdmon-io_uring.c's AioContext operational even when the glib event loop is used, extend FDMonOps with an API similar to GSourceFuncs so that file descriptor monitoring can integrate into the glib event loop. A quick summary of the GSourceFuncs API: - prepare() is called each event loop iteration before waiting for file descriptors and timers. - check() is called to determine whether events are ready to be dispatched after waiting. - dispatch() is called to process events. More details here: https://docs.gtk.org/glib/struct.SourceFuncs.html Move the ppoll(2)-specific code from aio-posix.c into fdmon-poll.c and also implement epoll(7)- and io_uring(7)-specific file descriptor monitoring code for glib event loops. Note that it's still faster to use aio_poll() rather than the glib event loop since glib waits for file descriptor activity with ppoll(2) and does not support adaptive polling. But at least epoll(7) and io_uring(7) now work in glib event loops. Splitting this into multiple commits without temporarily breaking AioContext proved difficult so this commit makes all the changes. The next commit will remove the aio_context_use_g_source() API because it is no longer needed. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-ID: <20251104022933.618123-7-stefanha@redhat.com> [kwolf: Build fixes; fix AioContext.list_lock use after destroy] Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>	5 months ago
Stefan Hajnoczi	5f8741fca5	aio-posix: fix spurious return from ->wait() due to signals io_uring_enter(2) only returns -EINTR in some cases when interrupted by a signal. Therefore the while loop in fdmon_io_uring_wait() is incomplete and can lead to a spurious early return. Handle the case when a signal interrupts io_uring_enter(2) but the syscall returns the number of SQEs submitted (that takes priority over -EINTR). This patch probably makes little difference for QEMU, but the test suite relies on the exact pattern of aio_poll() return values, so it's best to hide this io_uring syscall interface quirk. Here is the strace of test-aio receiving 3 SIGCONT signals after this fix has been applied. Notice how the io_uring_enter(2) return value is 1 the first time because an SQE was submitted, but -EINTR the other times: eventfd2(0, EFD_CLOEXEC\|EFD_NONBLOCK) = 9 io_uring_enter(7, 1, 0, 0, NULL, 8) = 1 clock_nanosleep(CLOCK_REALTIME, 0, {tv_sec=1, tv_nsec=0}, 0x7ffe38a46240) = 0 io_uring_enter(7, 1, 1, IORING_ENTER_GETEVENTS, NULL, 8) = 1 --- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=596096, si_uid=1000} --- io_uring_enter(7, 0, 1, IORING_ENTER_GETEVENTS, NULL, 8) = -1 EINTR (Interrupted system call) --- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=596096, si_uid=1000} --- io_uring_enter(7, 0, 1, IORING_ENTER_GETEVENTS, NULL, 8 <unfinished ...> <... io_uring_enter resumed>) = -1 EINTR (Interrupted system call) --- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=596096, si_uid=1000} --- io_uring_enter(7, 0, 1, IORING_ENTER_GETEVENTS, NULL, 8 <unfinished ...> <... io_uring_enter resumed>) = 0 Reported-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20251104022933.618123-4-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>	5 months ago
Stefan Hajnoczi	c31a445749	aio-posix: fix fdmon-io_uring.c timeout stack variable lifetime io_uring_prep_timeout() stashes a pointer to the timespec struct rather than copying its fields. That means the struct must live until after the SQE has been submitted by io_uring_enter(2). add_timeout_sqe() violates this constraint because the SQE is not submitted within the function. Inline add_timeout_sqe() into fdmon_io_uring_wait() so that the struct lives at least as long as io_uring_enter(2). This fixes random hangs (bogus timeout values) when the kernel loads undefined timespec struct values from userspace after the original struct on the stack has been destroyed. Reported-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20251104022933.618123-3-stefanha@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>	5 months ago
Stefan Hajnoczi	dbf70f0a03	aio-posix: fix race between io_uring CQE and AioHandler deletion When an AioHandler is enqueued on ctx->submit_list for removal, the fill_sq_ring() function will submit an io_uring POLL_REMOVE operation to cancel the in-flight POLL_ADD operation. There is a race when another thread enqueues an AioHandler for deletion on ctx->submit_list when the POLL_ADD CQE has already appeared. In that case POLL_REMOVE is unnecessary. The code already handled this, but forgot that the AioHandler itself is still on ctx->submit_list when the POLL_ADD CQE is being processed. It's unsafe to delete the AioHandler at that point in time (use-after-free). Solve this problem by keeping the AioHandler alive but setting a flag so that it will be deleted by fill_sq_ring() when it runs. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> Message-ID: <20251104022933.618123-2-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>	5 months ago
Paolo Bonzini	592d0bc030	remove unnecessary casts from uintptr_t uintptr_t, or unsigned long which is equivalent on Linux I32LP64 systems, is an unsigned type and there is no need to further cast to __u64 which is another unsigned integer type; widening casts from unsigned integers zero-extend the value. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>	2 years ago
Stefan Hajnoczi	87ec6f55af	aio-posix: zero out io_uring sqe user_data liburing does not clear sqe->user_data. We must do it ourselves to avoid undefined behavior in process_cqe() when user_data is used. Note that fdmon-io_uring is currently disabled, so this is a latent bug that does not affect users. Let's merge this fix now to make it easier to enable fdmon-io_uring in the future (and I'm working on that). Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20230426212639.82310-1-stefanha@redhat.com>	3 years ago
Stefan Hajnoczi	60f782b6b7	aio: remove aio_disable_external() API All callers now pass is_external=false to aio_set_fd_handler() and aio_set_event_notifier(). The aio_disable_external() API that temporarily disables fd handlers that were registered is_external=true is therefore dead code. Remove aio_disable_external(), aio_enable_external(), and the is_external arguments to aio_set_fd_handler() and aio_set_event_notifier(). The entire test-fdmon-epoll test is removed because its sole purpose was testing aio_disable_external(). Parts of this patch were generated using the following coccinelle (https://coccinelle.lip6.fr/) semantic patch: @@ expression ctx, fd, is_external, io_read, io_write, io_poll, io_poll_ready, opaque; @@ - aio_set_fd_handler(ctx, fd, is_external, io_read, io_write, io_poll, io_poll_ready, opaque) + aio_set_fd_handler(ctx, fd, io_read, io_write, io_poll, io_poll_ready, opaque) @@ expression ctx, notifier, is_external, io_read, io_poll, io_poll_ready; @@ - aio_set_event_notifier(ctx, notifier, is_external, io_read, io_poll, io_poll_ready) + aio_set_event_notifier(ctx, notifier, io_read, io_poll, io_poll_ready) Reviewed-by: Juan Quintela <quintela@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-Id: <20230516190238.8401-21-stefanha@redhat.com> Signed-off-by: Kevin Wolf <kwolf@redhat.com>	3 years ago
Haiyue Wang	8a947c7a58	aio-posix: fix build failure io_uring 2.2 The io_uring fixed "Don't truncate addr fields to 32-bit on 32-bit": https://git.kernel.dk/cgit/liburing/commit/?id=d84c29b19ed0b130000619cff40141bb1fc3615b This leads to build failure: ../util/fdmon-io_uring.c: In function ‘add_poll_remove_sqe’: ../util/fdmon-io_uring.c:182:36: error: passing argument 2 of ‘io_uring_prep_poll_remove’ makes integer from pointer without a cast [-Werror=int-conversion] 182 \| io_uring_prep_poll_remove(sqe, node); \| ^~~~ \| \| \| AioHandler * In file included from /root/io/qemu/include/block/aio.h:18, from ../util/aio-posix.h:20, from ../util/fdmon-io_uring.c:49: /usr/include/liburing.h:415:17: note: expected ‘__u64’ {aka ‘long long unsigned int’} but argument is of type ‘AioHandler *’ 415 \| __u64 user_data) \| ~~~~~~^~~~~~~~~ cc1: all warnings being treated as errors Use LIBURING_HAVE_DATA64 to check whether the io_uring supports 64-bit variants of the get/set userdata, to convert the paramter to the right data type. Signed-off-by: Haiyue Wang <haiyue.wang@intel.com> Message-Id: <20220221162401.45415-1-haiyue.wang@intel.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>	4 years ago
Stefan Hajnoczi	d73415a315	qemu/atomic.h: rename atomic_ to qatomic_ clang's C11 atomic_fetch_() functions only take a C11 atomic type pointer argument. QEMU uses direct types (int, etc) and this causes a compiler error when a QEMU code calls these functions in a source file that also included <stdatomic.h> via a system header file: $ CC=clang CXX=clang++ ./configure ... && make ../util/async.c:79:17: error: address argument to atomic operation must be a pointer to _Atomic type ('unsigned int ' invalid) Avoid using atomic_*() names in QEMU's atomic.h since that namespace is used by <stdatomic.h>. Prefix QEMU's APIs with 'q' so that atomic.h and <stdatomic.h> can co-exist. I checked /usr/include on my machine and searched GitHub for existing "qatomic_" users but there seem to be none. This patch was generated using: $ git grep -h -o '\<atomic$64$\?_[a-z0-9_]\+' include/qemu/atomic.h \| \ sort -u >/tmp/changed_identifiers $ for identifier in $(</tmp/changed_identifiers); do sed -i "s%\<$identifier\>%q$identifier%g" \ $(git grep -I -l "\<$identifier\>") done I manually fixed line-wrap issues and misaligned rST tables. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20200923105646.47864-1-stefanha@redhat.com>	6 years ago
Stefan Hajnoczi	de137e44f7	aio-posix: don't duplicate fd handler deletion in fdmon_io_uring_destroy() The io_uring file descriptor monitoring implementation has an internal list of fd handlers that are pending submission to io_uring. fdmon_io_uring_destroy() deletes all fd handlers on the list. Don't delete fd handlers directly in fdmon_io_uring_destroy() for two reasons: 1. This duplicates the aio-posix.c AioHandler deletion code and could become outdated if the struct changes. 2. Only handlers with the FDMON_IO_URING_REMOVE flag set are safe to remove. If the flag is not set then something still has a pointer to the fd handler. Let aio-posix.c and its user worry about that. In practice this isn't an issue because fdmon_io_uring_destroy() is only called when shutting down so all users have removed their fd handlers, but the next patch will need this! Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Tested-by: Oleksandr Natalenko <oleksandr@redhat.com> Message-id: 20200511183630.279750-2-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>	6 years ago
Stefan Hajnoczi	636b836d5f	aio-posix: signal-proof fdmon-io_uring The io_uring_enter(2) syscall returns with errno=EINTR when interrupted by a signal. Retry the syscall in this case. It's essential to do this in the io_uring_submit_and_wait() case. My interpretation of the Linux v5.5 io_uring_enter(2) code is that it shouldn't affect the io_uring_submit() case, but there is no guarantee this will always be the case. Let's check for -EINTR around both APIs. Note that the liburing APIs have -errno return values. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-id: 20200408091139.273851-1-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>	6 years ago
Stefan Hajnoczi	ae60ab7eb2	aio-posix: fix test-aio /aio/event/wait with fdmon-io_uring When a file descriptor becomes ready we must re-arm POLL_ADD. This is done by adding an sqe to the io_uring sq ring. The ->need_wait() function wasn't taking pending sqes into account and therefore io_uring_submit_and_wait() was not being called. Polling for cqes failed to detect fd readiness since we hadn't submitted the sqe to io_uring. This patch fixes the following tests/test-aio -p /aio/event/wait failure: ok 11 /aio/event/wait ** ERROR:tests/test-aio.c:374:test_flush_event_notifier: assertion failed: (aio_poll(ctx, false)) Reported-by: Cole Robinson <crobinso@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Tested-by: Cole Robinson <crobinso@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20200402145434.99349-1-stefanha@redhat.com Fixes: `73fd282e7b` ("aio-posix: add io_uring fd monitoring implementation") Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>	6 years ago
Stefan Hajnoczi	ff807d5592	aio-posix: fix io_uring with external events When external event sources are disabled fdmon-io_uring falls back to fdmon-poll. The ->need_wait() callback needs to watch for this so it can return true when external event sources are disabled. It is also necessary to call ->wait() when AioHandlers have changed because io_uring is asynchronous and we must submit new sqes. Both of these changes to ->need_wait() together fix tests/test-aio -p /aio/external-client, which failed with: test-aio: tests/test-aio.c:404: test_aio_external_client: Assertion `aio_poll(ctx, false)' failed. Reported-by: Julia Suvorova <jusual@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Stefano Garzarella <sgarzare@redhat.com> Message-id: 20200319163559.117903-1-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>	6 years ago
Stefan Hajnoczi	aa38e19f05	aio-posix: support userspace polling of fd monitoring Unlike ppoll(2) and epoll(7), Linux io_uring completions can be polled from userspace. Previously userspace polling was only allowed when all AioHandler's had an ->io_poll() callback. This prevented starvation of fds by userspace pollable handlers. Add the FDMonOps->need_wait() callback that enables userspace polling even when some AioHandlers lack ->io_poll(). For example, it's now possible to do userspace polling when a TCP/IP socket is monitored thanks to Linux io_uring. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200305170806.1313245-7-stefanha@redhat.com Message-Id: <20200305170806.1313245-7-stefanha@redhat.com>	6 years ago
Stefan Hajnoczi	73fd282e7b	aio-posix: add io_uring fd monitoring implementation The recent Linux io_uring API has several advantages over ppoll(2) and epoll(2). Details are given in the source code. Add an io_uring implementation and make it the default on Linux. Performance is the same as with epoll(7) but later patches add optimizations that take advantage of io_uring. It is necessary to change how aio_set_fd_handler() deals with deleting AioHandlers since removing monitored file descriptors is asynchronous in io_uring. fdmon_io_uring_remove() marks the AioHandler deleted and aio_set_fd_handler() will let it handle deletion in that case. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Link: https://lore.kernel.org/r/20200305170806.1313245-6-stefanha@redhat.com Message-Id: <20200305170806.1313245-6-stefanha@redhat.com>	6 years ago

18 Commits (688f20dbde87b971d81b74a56e7c307e846e0442)