riscv/qemu - qemu - Gitea: Git with a cup of tea

Author	SHA1	Message	Date
Daniel Borkmann	e53d9ec7cc	net/af-xdp: Support pinned map path for AF_XDP sockets Extend 'inhibit=on' setting with the option to specify a pinned XSK map path along with a starting index (default 0) to push the created XSK sockets into. Example usage: # ./build/qemu-system-x86_64 [...] \ -netdev af-xdp,ifname=enp2s0f0np0,id=net0,mode=native,queues=2,start-queue=14,inhibit=on,map-path=/sys/fs/bpf/xsks_map,map-start-index=14 \ -device virtio-net-pci,netdev=net0 [...] This is useful for the case where an existing XDP program with XSK map is present on the AF_XDP supported phys device and the XSK map is not yet populated. For example, the former could have been pre-loaded onto the netdevice by a control plane, which later launches QEMU to populate it with XSK sockets. Normally, the main idea behind 'inhibit=on' is that the QEMU instance doesn't need to have a lot of privileges to use the pre-loaded program and the pre-created sockets, but this mentioned use-case here is different where QEMU still needs privileges to create the sockets. The 'map-start-index' parameter is optional and defaults to 0. It allows flexible placement of the XSK sockets, and is up to the user to specify when the XDP program with XSK map was already preloaded. In the simplest case the queue-to-map-slot mapping is just 1:1 based on ctx->rx_queue_index but the user might as well have a different scheme (or smaller map size, e.g. ctx->rx_queue_index % max_size) to push the inbound traffic to one of the XSK sockets. Note that the bpf_xdp_query_id() is now only tested for 'inhibit=off' since only in the latter case the libxdp takes care of installing the XDP program which was installed based on the s->xdp_flags pointing to either driver or skb mode. For 'inhibit=on' we don't make any assumptions and neither go down the path of probing all possible options in which way the user installed the XDP program. Reviewed-by: Ilya Maximets <i.maximets@ovn.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Cc: Ilya Maximets <i.maximets@ovn.org> Cc: Jason Wang <jasowang@redhat.com> Cc: Anton Protopopov <aspsk@isovalent.com> Signed-off-by: Jason Wang <jasowang@redhat.com>	1 year ago
Daniel Borkmann	bdebcb49f4	net/af-xdp: Fix up cleanup path upon failure in queue creation While testing, it turned out that upon error in the queue creation loop, we never trigger the af_xdp_cleanup() handler. This is because we pass errp instead of a local err pointer into the various AF_XDP setup functions instead of a scheme like: bool fn(..., Error *errp) { Error err = NULL; foo(arg, &err); if (err) { handle the error... error_propagate(errp, err); return false; } ... } The same is true for the attachment probing with bpf_xdp_query_id(). With a conversion into the above format, the af_xdp_cleanup() handler is called as expected. Note the error_propagate() handles a NULL err internally. Fixes: `cb039ef3d9` ("net: add initial support for AF_XDP network backend") Reviewed-by: Ilya Maximets <i.maximets@ovn.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Cc: Ilya Maximets <i.maximets@ovn.org> Cc: Jason Wang <jasowang@redhat.com> Cc: Anton Protopopov <aspsk@isovalent.com> Signed-off-by: Jason Wang <jasowang@redhat.com>	1 year ago
Daniel Borkmann	d38c5e0d0c	net/af-xdp: Remove XDP program cleanup logic There are two issues with the XDP program removal in af_xdp_cleanup(): 1) Starting from libxdp 1.3.0 [0] the XDP program gets automatically detached when we call xsk_socket__delete() for the last successfully configured queue. libxdp internally keeps track of that. For QEMU we require libxdp >= 1.4.0. Given QEMU is not loading the program, lets also not attempt to remove it and delegate this instead. 2) The removal logic is incorrect anyway because we are setting n_queues into the last queue that never has xdp_flags on failure, so the logic is always skipped since the non-zero test for s->xdp_flags in af_xdp_cleanup() fails. Fixes: `cb039ef3d9` ("net: add initial support for AF_XDP network backend") Suggested-by: Ilya Maximets <i.maximets@ovn.org> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Reviewed-by: Ilya Maximets <i.maximets@ovn.org> Cc: Ilya Maximets <i.maximets@ovn.org> Cc: Jason Wang <jasowang@redhat.com> Cc: Anton Protopopov <aspsk@isovalent.com> Link: `38c2914988` [0] Signed-off-by: Jason Wang <jasowang@redhat.com>	1 year ago
Anastasia Belova	110d0fa2d4	net: fix buffer overflow in af_xdp_umem_create() s->pool has n_descs elements so maximum i should be n_descs - 1. Fix the upper bound. Found by Linux Verification Center (linuxtesting.org) with SVACE. Fixes: `cb039ef3d9` ("net: add initial support for AF_XDP network backend") Cc: qemu-stable@nongnu.org Reviewed-by: Ilya Maximets <i.maximets@ovn.org> Signed-off-by: Anastasia Belova <nabelova31@gmail.com> Signed-off-by: Jason Wang <jasowang@redhat.com>	1 year ago
Peter Maydell	bed150be5b	net/af-xdp.c: Don't leak sock_fds array in net_init_af_xdp() In net_init_af_xdp() we parse the arguments and allocate a buffer of ints into sock_fds. However, although we free this in the error exit path, we don't ever free it in the successful return path. Coverity spots this leak. Switch to g_autofree so we don't need to manually free the array. Resolves: Coverity CID 1534906 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Thomas Huth <thuth@redhat.com> Message-id: 20240312183810.557768-4-peter.maydell@linaro.org	2 years ago
Peter Maydell	493bc2dbc1	misc: Clean up includes This commit was created with scripts/clean-includes: ./scripts/clean-includes --git misc net/af-xdp.c plugins/.c audio/pwaudio.c util/userfaultfd.c All .c should include qemu/osdep.h first. The script performs three related cleanups: Ensure .c files include qemu/osdep.h first. * Including it in a .h is redundant, since the .c already includes it. Drop such inclusions. * Likewise, including headers qemu/osdep.h includes is redundant. Drop these, too. Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>	3 years ago
Ilya Maximets	cb039ef3d9	net: add initial support for AF_XDP network backend AF_XDP is a network socket family that allows communication directly with the network device driver in the kernel, bypassing most or all of the kernel networking stack. In the essence, the technology is pretty similar to netmap. But, unlike netmap, AF_XDP is Linux-native and works with any network interfaces without driver modifications. Unlike vhost-based backends (kernel, user, vdpa), AF_XDP doesn't require access to character devices or unix sockets. Only access to the network interface itself is necessary. This patch implements a network backend that communicates with the kernel by creating an AF_XDP socket. A chunk of userspace memory is shared between QEMU and the host kernel. 4 ring buffers (Tx, Rx, Fill and Completion) are placed in that memory along with a pool of memory buffers for the packet data. Data transmission is done by allocating one of the buffers, copying packet data into it and placing the pointer into Tx ring. After transmission, device will return the buffer via Completion ring. On Rx, device will take a buffer form a pre-populated Fill ring, write the packet data into it and place the buffer into Rx ring. AF_XDP network backend takes on the communication with the host kernel and the network interface and forwards packets to/from the peer device in QEMU. Usage example: -device virtio-net-pci,netdev=guest1,mac=00:16:35:AF:AA:5C -netdev af-xdp,ifname=ens6f1np1,id=guest1,mode=native,queues=1 XDP program bridges the socket with a network interface. It can be attached to the interface in 2 different modes: 1. skb - this mode should work for any interface and doesn't require driver support. With a caveat of lower performance. 2. native - this does require support from the driver and allows to bypass skb allocation in the kernel and potentially use zero-copy while getting packets in/out userspace. By default, QEMU will try to use native mode and fall back to skb. Mode can be forced via 'mode' option. To force 'copy' even in native mode, use 'force-copy=on' option. This might be useful if there is some issue with the driver. Option 'queues=N' allows to specify how many device queues should be open. Note that all the queues that are not open are still functional and can receive traffic, but it will not be delivered to QEMU. So, the number of device queues should generally match the QEMU configuration, unless the device is shared with something else and the traffic re-direction to appropriate queues is correctly configured on a device level (e.g. with ethtool -N). 'start-queue=M' option can be used to specify from which queue id QEMU should start configuring 'N' queues. It might also be necessary to use this option with certain NICs, e.g. MLX5 NICs. See the docs for examples. In a general case QEMU will need CAP_NET_ADMIN and CAP_SYS_ADMIN or CAP_BPF capabilities in order to load default XSK/XDP programs to the network interface and configure BPF maps. It is possible, however, to run with no capabilities. For that to work, an external process with enough capabilities will need to pre-load default XSK program, create AF_XDP sockets and pass their file descriptors to QEMU process on startup via 'sock-fds' option. Network backend will need to be configured with 'inhibit=on' to avoid loading of the program. QEMU will need 32 MB of locked memory (RLIMIT_MEMLOCK) per queue or CAP_IPC_LOCK. There are few performance challenges with the current network backends. First is that they do not support IO threads. This means that data path is handled by the main thread in QEMU and may slow down other work or may be slowed down by some other work. This also means that taking advantage of multi-queue is generally not possible today. Another thing is that data path is going through the device emulation code, which is not really optimized for performance. The fastest "frontend" device is virtio-net. But it's not optimized for heavy traffic either, because it expects such use-cases to be handled via some implementation of vhost (user, kernel, vdpa). In practice, we have virtio notifications and rcu lock/unlock on a per-packet basis and not very efficient accesses to the guest memory. Communication channels between backend and frontend devices do not allow passing more than one packet at a time as well. Some of these challenges can be avoided in the future by adding better batching into device emulation or by implementing vhost-af-xdp variant. There are also a few kernel limitations. AF_XDP sockets do not support any kinds of checksum or segmentation offloading. Buffers are limited to a page size (4K), i.e. MTU is limited. Multi-buffer support implementation for AF_XDP is in progress, but not ready yet. Also, transmission in all non-zero-copy modes is synchronous, i.e. done in a syscall. That doesn't allow high packet rates on virtual interfaces. However, keeping in mind all of these challenges, current implementation of the AF_XDP backend shows a decent performance while running on top of a physical NIC with zero-copy support. Test setup: 2 VMs running on 2 physical hosts connected via ConnectX6-Dx card. Network backend is configured to open the NIC directly in native mode. The driver supports zero-copy. NIC is configured to use 1 queue. Inside a VM - iperf3 for basic TCP performance testing and dpdk-testpmd for PPS testing. iperf3 result: TCP stream : 19.1 Gbps dpdk-testpmd (single queue, single CPU core, 64 B packets) results: Tx only : 3.4 Mpps Rx only : 2.0 Mpps L2 FWD Loopback : 1.5 Mpps In skb mode the same setup shows much lower performance, similar to the setup where pair of physical NICs is replaced with veth pair: iperf3 result: TCP stream : 9 Gbps dpdk-testpmd (single queue, single CPU core, 64 B packets) results: Tx only : 1.2 Mpps Rx only : 1.0 Mpps L2 FWD Loopback : 0.7 Mpps Results in skb mode or over the veth are close to results of a tap backend with vhost=on and disabled segmentation offloading bridged with a NIC. Signed-off-by: Ilya Maximets <i.maximets@ovn.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (docker/lcitool) Signed-off-by: Jason Wang <jasowang@redhat.com>	3 years ago