The build fails on Windows. Replace calls to Unix programs like ´cat´,
´sed´ and ´true´ with calls to ´python´ and wrap calls to
´os.path.relpath´ in try-except because it can fail when the two paths
are on different drives. Make sure to convert the Windows paths to Unix
paths to prevent warnings in generated files.
Signed-off-by: oltolm <oleg.tolmatcev@gmail.com>
Message-id: 20250612221521.1109-2-oleg.tolmatcev@gmail.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Add new tests to check the correctness of the `-overcommit memlock`
option (possible values: off, on, on-fault) by using
`/proc/{qemu_pid}/status` file to check in VmSize, VmRSS and VmLck
values:
* if `memlock=off`, then VmLck = 0;
* if `memlock=on`, then VmLck > 0 and almost all memory is resident;
* if `memlock=on-fault`, then VmLck > 0 and only few memory is resident.
Signed-off-by: Alexandr Moshkov <dtalexundeer@yandex-team.ru>
Message-ID: <20250605065908.299979-3-dtalexundeer@yandex-team.ru>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Check whether we can run a kernel that prints something to the
serial console.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Mark Cave-Ayland <mark.caveayland@nutanix.com>
Message-ID: <20250603184007.24521-1-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
The 2023 edition of the QEMU advent calendar featured an image
that we can use to test whether the lm3s6965evb machine is basically
still working.
And for the lm3s811evb there is a small test kernel on github
which can be used to check its UART.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20250519170242.520805-1-thuth@redhat.com
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Introduce a functional test which boots Debian 12 on the imx8mp-evk board. Since
the root filesystem resides on an SD card, the test also verifies the basic
operation of the USDHC.
Signed-off-by: Bernhard Beschow <shentey@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20250409202630.19667-1-shentey@gmail.com
[PMM: added extra blank line as suggested by thuth;
set timeout to 240s]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Introduce a new test suite for ast2700fc machine.
Rename the original test_aarch64_aspeed.py to
test_aarch64_aspeed_ast2700.py.
Signed-off-by: Steven Lee <steven_lee@aspeedtech.com>
Change-Id: I3855f55c9f6e5cca1270c179445f549f8d81f36c
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250505030618.3612042-1-steven_lee@aspeedtech.com
[ clg: Added new tests in meson.build ]
Signed-off-by: Cédric Le Goater <clg@redhat.com>
This test was using cloudinit and a "dnf install" command in the guest
to exercise the NIC with SMMU enabled. Since we don't have the cloudinit
stuff in the functional framework and we should not rely on having access
to external networks (once our ASSETs have been cached), we rather boot
into the initrd first, manually mount the root disk and then use the
check_http_download() function from the functional framework here instead
for testing whether the network works as expected.
Unfortunately, there seems to be a small race when using the files
from Fedora 33: To enter the initrd shell, we have to send a "return"
once. But it does not seem to work if we send it too early. Using a
sleep(0.2) makes it work reliably for me, but to make it even more
unlikely to trigger this situation, let's better limit the Fedora 33
tests to only run with KVM.
Finally, while we're at it, we also add some lines for testing writes
to the hard disk, as we already do it in the test_intel_iommu test.
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Tested-by: Eric Auger <eric.auger@redhat.com>
Message-ID: <20250414113031.151105-14-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reuse the test function from the 32-bit big endian test to easily
convert the 64-bit big endian Wheezy mips test.
Since this was the last test in tests/avocado/linux_ssh_mips_malta.py,
we can remove this avocado file now, too.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-10-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reuse the test function from the 32-bit big endian test to easily
convert the 64-bit little endian Wheezy mips test.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-9-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reuse the test function from the big endian test to easily
convert the 32-bit little endian Wheezy mips test.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-8-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The test checks some entries in /proc and the output of some commands ...
we put these checks into exportable functions now so that they can
be reused more easily.
Additionally the linux_ssh_mips_malta.py uses SSH to test the networking
of the guest. Since we don't have a SSH module in the functional
framework yet, let's use the check_http_download() function here instead.
And while we're at it, also switch the NIC to e1000 now to get some more
test coverage, since the "pcnet" device is already tested in the test
test_mips_malta_cpio.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-7-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Since this was the last test in tests/avocado/replay_kernel.py,
we can remove that Avocado file now.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
These tests are using the gdb-related library functions from the
Avocado framework which we don't have in the functional framework
yet. So for the time being, keep those imports and skip the test
if the Avocado framework is not installed on the host.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The arm_aspeed_bletchley timeout takes more than its
current 120s timeout on a debug Rust config:
$ time (cd build/rust; PYTHONPATH=../../python:../../tests/functional QEMU_TEST_QEMU_BINARY=./qemu-system-arm ./pyvenv/bin/python3 ../../tests/functional/test_arm_aspeed_bletchley.py)
TAP version 13
ok 1 test_arm_aspeed_bletchley.BletchleyMachine.test_arm_ast2600_bletchley_openbmc
1..1
real 2m15.536s
user 3m21.444s
sys 0m11.558s
Bump it up to 480s, same as arm_aspeed_rainier.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250321155806.1888867-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
On my machine the arm_replay test takes over 2 minutes to run
in a config with Rust enabled and debug enabled:
$ time (cd build/rust ; PYTHONPATH=../../python:../../tests/functional
QEMU_TEST_QEMU_BINARY=./qemu-system-arm ./pyvenv/bin/python3
../../tests/functional/test_arm_replay.py)
TAP version 13
ok 1 test_arm_replay.ArmReplay.test_cubieboard
ok 2 test_arm_replay.ArmReplay.test_vexpressa9
ok 3 test_arm_replay.ArmReplay.test_virt
1..3
real 2m16.564s
user 2m13.461s
sys 0m3.523s
Bump up the timeout to 4 minutes.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250310102830.3752440-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Since we are about to remove all support for PPC 405, start by
removing the tests referring to the ref405ep machine.
Link: https://lore.kernel.org/qemu-devel/20250110141800.1587589-2-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20250204080649.836155-2-clg@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
./tests/functional/test_aarch64_tcg_plugins.py needs to have plugin
libinsn built. However, it's not listed as a dependency, so meson can't
know it needs to be built.
Thus, we keep track of all plugins, and add them as an explicit
dependency.
Fixes: 4c134d07b9 ("tests: add a new set of tests to exercise plugins")
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-9-alex.bennee@linaro.org>
I want to expand the number of tests to cover a wide range of
configurations. That starts with splitting off from the normal virt
test from which it doesn't really share much code. We can also reduce
the timeout of the original virt test now it is now longer burdened
with testing the GPU.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-2-alex.bennee@linaro.org>
Use do_test_arm_aspeed_openbmc() to run the latest OpenBMC firmware
build of the bletchley BMC.
Reviewed-by: Patrick Williams <patrick@stwcx.xyz>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Link: https://lore.kernel.org/qemu-devel/20250129071820.1258133-6-clg@redhat.com
Signed-off-by: Cédric Le Goater <clg@redhat.com>
We run the gitlab-CI with the untrusted tests enabled, and
the test_replay_mips64el_malta_5KEc_cpio subtest is rather slow,
so this already hit the standard 90 seconds timeout in the CI.
Increase the timeout for more headroom.
Reported-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-ID: <20250305074353.52552-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
On my local machine, for a debug build, sbsaref_alpine takes
nearly 900s:
$ (cd build/x86 && ./pyvenv/bin/meson test --setup thorough --suite func-thorough func-aarch64-aarch64_sbsaref_alpine
)
1/1 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_sbsaref_alpine
OK 896.90s
arm_aspeed_rainier can also run close to its current timeout:
6/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_rainier
OK 215.75s
and arm_aspeed_ast2500 and arm_aspeed_ast2600 can go over:
13/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_ast2600
OK 792.94s
27/44 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_aspeed_ast2500
TIMEOUT 480.01s
The sx1 test fails not on the overall meson timeout but on the
60 second timeout in some of the subtests.
Bump all these timeouts up a bit.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250221140640.786341-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Put the tests into a separate file now (in the functional framework,
each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com
Message-ID: <20250218152744.228335-14-thuth@redhat.com>
Put the tests into a separate file now (in the functional framework,
each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-13-thuth@redhat.com>
Put the tests into a separate file now (in the functional framework,
each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-12-thuth@redhat.com>
Put the tests into a separate file now (since in the functional
framework, each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-11-thuth@redhat.com>
Put the tests into a separate file now (since in the functional
framework, each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-10-thuth@redhat.com>
Put the tests into a separate file now (since in the functional
framework, each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-9-thuth@redhat.com>
Put the tests into a separate file now (since in the functional
framework, each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-8-thuth@redhat.com>
Put the tests into a separate file now (since in the functional
framework, each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-7-thuth@redhat.com>
Put the tests into a separate file now (since in the functional
framework, each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-6-thuth@redhat.com>
Put the tests into a separate file now (since in the functional
framework, each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-5-thuth@redhat.com>
While we're at it, change the machine from SS-20 to SS-10 to
increase the test coverage a little bit (SS-20 is already
tested in the test_sparc_sun4m.py file).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-4-thuth@redhat.com>
Put the tests into a separate file now (since in the functional
framework, each file is run with one specific qemu-system-* binary).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20250218152744.228335-3-thuth@redhat.com>
When a machine is first booted, all virtio balloon stats are initialized
to their default value -1 (18446744073709551615 when represented as
unsigned).
They remain that way while the firmware is loading, and early phase of
guest OS boot, until the virtio-balloon driver is activated. Thereafter
the reported stats reflect the guest OS activity.
When a machine reset is performed, however, the virtio-balloon stats are
left unchanged by QEMU, despite the guest OS no longer updating them,
nor indeed even still existing.
IOW, the mgmt app keeps getting stale stats until the guest OS starts
once more and loads the virtio-balloon driver (if ever). At that point
the app will see a discontinuity in the reported values as they sudden
jump from the stale value to the new value. This jump is indigituishable
from a valid data update.
While there is an "last-updated" field to report on the freshness of
the stats, that does not unambiguously tell the mgmt app whether the
stats are still conceptually relevant to the current running workload.
It is more conceptually useful to reset the stats to their default
values on machine reset, given that the previous guest workload the
stats reflect no longer exists. The mgmt app can now clearly identify
that there are is no stats information available from the current
executing workload.
The 'last-updated' time is also reset back to 0.
IOW, on every machine reset, the virtio stats are in the same clean
state they were when the macine first powered on.
A functional test is added to validate this behaviour with a real
world guest OS.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20250204094202.2183262-1-berrange@redhat.com>
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
The test sequence boots from kernel a sam460ex machine with a
virtio-net device to check PCI.
The buildroot is built with config :
BR2_powerpc=y
BR2_powerpc_440fp=y
and the kernel with the '44x/canyonlands' deconfig and virtio support.
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Message-ID: <20250203092606.491933-1-clg@redhat.com>
[thuth: sort meson.build alphabetically]
Signed-off-by: Thomas Huth <thuth@redhat.com>
By using the serial console instead of ssh for executing commands
in the guest, we can convert this test to the functional framework.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250130192712.19542-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This test just needs the adaption for the asset handling, then
we can move it to the functional framework.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250207144409.220006-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Move the mips big endian replay tests from tests/avocado/replay_kernel.py
to the functional framework. Since the functional tests should be run per
target, we cannot stick all replay tests in one file. Thus let's add
these tests to a separate file now.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250128152839.184599-6-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Move the mips64el replay tests from tests/avocado/replay_kernel.py to
the functional framework. Since the functional tests should be run per
target, we cannot stick all replay tests in one file. Thus let's add
these tests to a separate file there now.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250128152839.184599-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Move the mipsel replay tests from tests/avocado/replay_kernel.py to
the functional framework. Since the functional tests should be run per
target, we cannot stick all replay tests in one file. Thus let's add
these tests to a new, separate file there instead.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250128152839.184599-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The test sequence boots from disk a mac99 machine in 64-bit mode, in
which case the CPU is a PPC 970.
The buildroot rootfs is built with config :
BR2_powerpc64=y
BR2_powerpc_970=y
and the kernel with the g5 deconfig.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Message-ID: <20250128212145.1186617-1-clg@redhat.com>
[thuth: Adjusted the comment about '-nographic]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Unfortunately, this test had not been added to meson.build, so we did
not notice a regression: Looking for 'Kernel panic - not syncing: VFS:'
as the indication for the final boot state of the kernel was a bad
idea since 'Kernel panic - not syncing' is the default failure
message of the LinuxKernelTest class, and since we're now reading
the console input byte by byte instead of linewise (see commit
cdad03b74f), the failure now triggers before we fully read the
success string. Let's fix this by simply looking for the previous
line in the console output instead.
Also, replace the call to cancel() - this was only available in the
Avocado framework. In the functional framework, we must use skipTest()
instead. While we're at it, also fix the TODO here by looking for the
exact error and only skip the test if the plugins are not available.
Fixes: 3abc545e66 ("tests/functional: Convert the tcg_plugins test")
Fixes: cdad03b74f ("tests/functional: rewrite console handling to be bytewise")
Message-ID: <20250123083625.1498495-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Now that we've got a find_free_port() function in the functional
test framework, we can convert the migration test, too.
While the original avocado test was only meant to run on aarch64,
ppc64 and x86, we can turn this into a more generic test by now
and run it on all architectures that have a machine which ships
with a working firmware. To avoid overlapping with the migration
qtest, we now also test migration on machines that are not covered
by the migration qtest yet.
Acked-by: Fabiano Rosas <farosas@suse.de>
Message-ID: <20250103074308.463860-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We don't have any functional tests for this machine yet, thus let's
add a test with a MicroPython binary that is available online
(thanks to Joel Stanley for providing it, see:
https://www.mail-archive.com/qemu-devel@nongnu.org/msg606064.html ).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20250124101709.1591761-1-thuth@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Use the serial console to execute the commands in the guest instead
of using ssh since we don't have ssh support in the functional
framework yet.
Acked-by: David Woodhouse <dwmw@amazon.co.uk>
Message-ID: <20250113082516.57894-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This boot an OP-TEE environment, and launch a nested guest VM inside it
using the Realms feature. We do it for virt and sbsa-ref platforms.
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Message-Id: <20241220165212.3653495-1-pierrick.bouvier@linaro.org>
[AJB: tweak ordering of setup, strip changelog from commit]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20250108121054.1126164-16-alex.bennee@linaro.org>
We didn't have this before and as it exercises the m68k virt platform
it seems worth adding. We don't wait for the shutdown because QEMU
will auto-exit on the shutdown.
Cc: Laurent Vivier <laurent@vivier.eu>
Cc: Anders Roxell <anders.roxell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-4-alex.bennee@linaro.org>
Nothing thrilling in here, it's just a straight forward conversion.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20241218131439.255841-2-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
oltolm
Alexandr Moshkov
Thomas Huth
Bernhard Beschow
Steven Lee
Peter Maydell
Cédric Le Goater
Pierrick Bouvier
Alex Bennée
Daniel P. Berrangé
Philippe Mathieu-Daudé