riscv/qemu - qemu - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
Juraj Marcin	7b842fe354	migration: Introduce POSTCOPY_DEVICE state Currently, when postcopy starts, the source VM starts switchover and sends a package containing the state of all non-postcopiable devices. When the destination loads this package, the switchover is complete and the destination VM starts. However, if the device state load fails or the destination side crashes, the source side is already in POSTCOPY_ACTIVE state and cannot be recovered, even when it has the most up-to-date machine state as the destination has not yet started. This patch introduces a new POSTCOPY_DEVICE state which is active while the destination machine is loading the device state, is not yet running, and the source side can be resumed in case of a migration failure. Return-path is required for this state to function, otherwise it will be skipped in favor of POSTCOPY_ACTIVE. To transition from POSTCOPY_DEVICE to POSTCOPY_ACTIVE, the source side uses a PONG message that is a response to a PING message processed just before the POSTCOPY_RUN command that starts the destination VM. Thus, this feature is effective even if the destination side does not yet support this new state. Signed-off-by: Juraj Marcin <jmarcin@redhat.com> Link: https://lore.kernel.org/r/20251103183301.3840862-9-jmarcin@redhat.com Signed-off-by: Peter Xu <peterx@redhat.com>	5 months ago
Eric Blake	181a63667a	mirror: Skip pre-zeroing destination if it is already zero When doing a sync=full mirroring, we can skip pre-zeroing the destination if it already reads as zeroes and we are not also trying to punch holes due to detect-zeroes. With this patch, there are fewer scenarios that have to pass in an explicit target-is-zero, while still resulting in a sparse destination remaining sparse. A later patch will then further improve things to skip writing to the destination for parts of the image where the source is zero; but even with just this patch, it is possible to see a difference for any source that does not report itself as fully allocated, coupled with a destination BDS that can quickly report that it already reads as zero. (For a source that reports as fully allocated, such as a file, the rest of mirror_dirty_init() still sets the entire dirty bitmap to true, so even though we avoided the pre-zeroing, we are not yet avoiding all redundant I/O). Iotest 194 detects the difference made by this patch: for a file source (where block status reports the entire image as allocated, and therefore we end up writing zeroes everywhere in the destination anyways), the job length remains the same. But for a qcow2 source and a destination that reads as all zeroes, the dirty bitmap changes to just tracking the allocated portions of the source, which results in faster completion and smaller job statistics. For the test to pass with both ./check -file and -qcow2, a new python filter is needed to mask out the now-varying job amounts (this matches the shell filters _filter_block_job_{offset,len} in common.filter). A later test will also be added which further validates expected sparseness, so it does not matter that 194 is no longer explicitly looking at how many bytes were copied. Signed-off-by: Eric Blake <eblake@redhat.com> Message-ID: <20250509204341.3553601-25-eblake@redhat.com> Reviewed-by: Sunny Zhu <sunnyzhyy@qq.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>	11 months ago
Eric Blake	eb89627899	iotests: Improve iotest 194 to mirror data Mirroring a completely sparse image to a sparse destination should be practically instantaneous. It isn't yet, but the test will be more realistic if it has some non-zero to mirror as well as the holes. Signed-off-by: Eric Blake <eblake@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Message-ID: <20250509204341.3553601-20-eblake@redhat.com>	11 months ago
Vladimir Sementsov-Ogievskiy	dcc28ab603	iotests: fix 194: filter out racy postcopy-active event The event is racy: it will not appear in the output if bitmap is migrated during downtime period of migration and postcopy phase is not started. Fixes: `ae00aa2398` "iotests: 194: test also migration of dirty bitmap" Reported-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-Id: <20230607143606.1557395-1-vsementsov@yandex-team.ru> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>	3 years ago
John Snow	3d53818ff7	iotests: replace unchecked calls to qemu_img_pipe() qemu_img_pipe() discards the return code from qemu-img in favor of returning just its output. Some tests using this function don't save, log, or check the output either, though, which is unsafe. Replace all of these calls with a checked version. Tests affected are 194, 202, 203, 234, 262, and 303. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Hanna Reitz <hreitz@redhat.com> Message-Id: <20220321201618.903471-13-jsnow@redhat.com> Signed-off-by: Hanna Reitz <hreitz@redhat.com>	4 years ago
Daniel P. Berrangé	e67d8e2928	block: remove 'dirty-bitmaps' field from 'BlockInfo' struct The same data is available in the 'BlockDeviceInfo' struct. Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>	5 years ago
Vladimir Sementsov-Ogievskiy	9dd003a998	iotests: define group in each iotest We are going to drop group file. Define group in tests as a preparatory step. The patch is generated by cd tests/qemu-iotests grep '^[0-9]\{3\} ' group \| while read line; do file=$(awk '{print $1}' <<< "$line"); groups=$(sed -e 's/^... //' <<< "$line"); awk "NR==2{print \"# group: $groups\"}1" $file > tmp; cat tmp > $file; done Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20210116134424.82867-7-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>	5 years ago
Nir Soffer	3192fad7f7	qemu-iotests: Merge FilePaths and FilePath FilePath creates now one temporary file: with FilePath("a") as a: Or more: with FilePath("a", "b", "c") as (a, b, c): This is also the behavior of the file_path() helper, used by some of the tests. Now we have only 2 helpers for creating temporary files instead of 3. Signed-off-by: Nir Soffer <nsoffer@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200828232152.205833-5-nsoffer@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>	6 years ago
Nir Soffer	a242b19e80	qemu-iotests: Support varargs syntax in FilePaths Accept variable number of names instead of a sequence: with FilePaths("a", "b", "c") as (a, b, c): The disadvantage is that base_dir must be used as kwarg: with FilePaths("a", "b", base_dir=soc_dir) as (sock1, sock2): But this is more clear and calling optional argument as positional arguments is bad idea anyway. Signed-off-by: Nir Soffer <nsoffer@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200828232152.205833-4-nsoffer@redhat.com> Signed-off-by: Max Reitz <mreitz@redhat.com>	6 years ago
Vladimir Sementsov-Ogievskiy	93d487807b	iotests: 194: wait for migration completion on target too It is possible, that shutdown on target occurs earlier than migration finish. In this case we crash in bdrv_release_dirty_bitmap_locked() on assertion "assert(!bdrv_dirty_bitmap_busy(bitmap));" as we do have busy bitmap, as bitmap migration is ongoing. We'll fix bitmap migration to gracefully cancel on early shutdown soon. Now let's fix iotest 194 to wait migration completion before shutdown. Note that in this test dest_vm.shutdown() is called implicitly, as vms used as context-providers, see __exit__() method of QEMUMachine class. Actually, not waiting migration finish is a wrong thing, but the test started to crash after commit `ae00aa2398` "iotests: 194: test also migration of dirty bitmap", which added dirty bitmaps here. So, Fixes: tag won't hurt. Fixes: `ae00aa2398` Reported-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Tested-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> [eblake: grammar tweak] Message-Id: <20200604083341.26978-1-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>	6 years ago
Vladimir Sementsov-Ogievskiy	ae00aa2398	iotests: 194: test also migration of dirty bitmap Test that dirty bitmap migration works when we deal with mirror. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-Id: <20200521220648.3255-5-vsementsov@virtuozzo.com> Signed-off-by: Eric Blake <eblake@redhat.com>	6 years ago
John Snow	7d8140595f	iotests: add script_initialize Like script_main, but doesn't require a single point of entry. Replace all existing initialization sections with this drop-in replacement. This brings debug support to all existing script-style iotests. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Max Reitz <mreitz@redhat.com> Message-Id: <20200331000014.11581-12-jsnow@redhat.com> Reviewed-by: Kevin Wolf <kwolf@redhat.com> [mreitz: Give 274 the same treatment] Signed-off-by: Max Reitz <mreitz@redhat.com>	6 years ago
Philippe Mathieu-Daudé	7c47752608	tests/qemu-iotests: Explicit usage of Python3 (scripts without __main__) Use the program search path to find the Python 3 interpreter. Patch created mechanically by running: $ sed -i "s,^#\!/usr/bin/$env\ $\?python$,#\!/usr/bin/env python3," \ $(git grep -lF '#!/usr/bin/env python' \ \| xargs grep -L 'if __name__.*__main__') Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> Suggested-by: Daniel P. Berrangé <berrange@redhat.com> Suggested-by: Stefan Hajnoczi <stefanha@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Message-Id: <20200130163232.10446-11-philmd@redhat.com> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>	6 years ago
Max Reitz	4b4d34f4f7	iotests/194: Create sockets in $SOCK_DIR Signed-off-by: Max Reitz <mreitz@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Message-id: 20191017133155.5327-15-mreitz@redhat.com Signed-off-by: Max Reitz <mreitz@redhat.com>	7 years ago
Kevin Wolf	ad53ea42fe	qemu-iotests: Remove _supported_fmt dmg qemu-iotests doesn't support dmg, and the dmg block driver doesn't support image creation. Two test cases declare dmg as supported, but that's obviously wrong for both reasons. Remove the declaration. Signed-off-by: Kevin Wolf <kwolf@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com>	8 years ago
Jeff Cody	8b2d7c364d	qemu-iotests: update unsupported image formats in 194 Test 194 checks for 'luks' to exclude as an unsupported format, However, most formats are unsupported, due to migration blockers. Rather than specifying a blacklist of unsupported formats, whitelist supported formats (specifically, qcow2, qed, raw, dmg). Tested-by: Alexey Kardashevskiy <aik@ozlabs.ru> Signed-off-by: Jeff Cody <jcody@redhat.com> Message-id: 23ca18c7f843c86a28b1529ca9ac6db4b35ca0e4.1510059970.git.jcody@redhat.com Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Reviewed-by: Denis V. Lunev <den@openvz.org> Signed-off-by: Max Reitz <mreitz@redhat.com>	9 years ago
Daniel P. Berrange	3f5c4076f1	iotests: blacklist 194 with the luks driver The 194 test has a lot of code that assumes a simple image file. Rewriting this to work with luks is possible, but non-trivial, so blacklist the luks format for now. Signed-off-by: Daniel P. Berrange <berrange@redhat.com> Message-Id: <20170901105434.3288-3-berrange@redhat.com> Reviewed-by: Eric Blake <eblake@redhat.com> Tested-by: Fam Zheng <famz@redhat.com> [eblake: commit message typo fixed] Reviewed-by: Kashyap Chamarthy <kchamart@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>	9 years ago
Stefan Hajnoczi	921a32179b	qemu-iotests: use context managers for resource cleanup in 194 Switch from atexit.register() to a more elegant idiom of declaring resources in a with statement: with FilePath('monitor.sock') as monitor_path, VM() as vm: ... The files and VMs will be automatically cleaned up whether the test passes or fails. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Message-id: 20170824072202.26818-4-stefanha@redhat.com Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>	9 years ago
Kashyap Chamarthy	2c94e2714c	qemu-iotests: Extend non-shared storage migration test (194) This is the follow-up patch that was discussed[] as part of feedback to qemu-iotest 194. Changes in this patch: - Supply 'job-id' parameter to `drive-mirror` invocation. - Once migration completes, issue QMP `block-job-cancel` command on the source QEMU to gracefully complete `drive-mirror` operation. - Once the BLOCK_JOB_COMPLETED event is emitted, stop the NBD server on the destination QEMU. - Check for both the events: MIGRATION and BLOCK_JOB_COMPLETED. With the above, the test will also be (almost) in sync with the procedure outlined in the document 'live-block-operations.rst'[+] (section: "QMP invocation for live storage migration with ``drive-mirror`` + NBD"). [] https://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg04820.html -- qemu-iotests: add 194 non-shared storage migration test [+] https://git.qemu.org/gitweb.cgi?p=qemu.git;a=blob;f=docs/interop/live-block-operations.rst Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com> Message-Id: <20170829165058.8229-1-kchamart@redhat.com> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>	9 years ago
Stefan Hajnoczi	12314f2d14	qemu-iotests: add 194 non-shared storage migration test Non-shared storage migration with NBD and drive-mirror is currently not tested by qemu-iotests. This test case covers the basic migration scenario. Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com> Based-on: <20170823134242.12080-1-famz@redhat.com> Message-Id: <20170823140506.28723-1-stefanha@redhat.com> Tested-by: Eric Blake <eblake@redhat.com> Signed-off-by: Eric Blake <eblake@redhat.com>	9 years ago

20 Commits (7ddd3aca5f2efcf6aae10f836011e445ee77a6e1)