This will hopefully catch containers that break because of upstream
changes as well as keep the container cache fresh.
As we have all the container jobs as dependants we tweaks the
container template to allow scheduled runs. Because we added a new
rules stanza we also need to make sure we catch the normal runs as
well.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20251117115523.3993105-19-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
By default no jobs should run under the schedule and then we can be
more explicit for the ones that we need to. Otherwise I trigger all my
custom runners every time I do a scheduled run.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20251117115523.3993105-18-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
In addition to not being triggered by schedule we should follow the
same rules about QEMU_CI. One day we may figure out how to fold the
custom runner rules into the .base_job_template but today is not that
day.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20251117115523.3993105-17-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Otherwise the mere presence of the RUNNER env vars is enough to
trigger the jobs.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20251117115523.3993105-16-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
We don't need to duplicate the if rules to get the allow_failure and
manual behaviour we want. Clean that up to keep all the rules in the
same place.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20251117115523.3993105-15-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Most of the test is pure boilerplate so to save ourselves from
repetition move all the main bits into a minimal copy of
native_build_job_template but without the caching.
We keep all the current allow_fail, manual and configure setups but do
take the opportunity to replace the inline nproc calls to using a
common JOBS variable. We also fix the namespace check to use the
QEMU_CI_UPSTREAM variable.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20251117115523.3993105-14-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
This is a plain configure build but I only run a subset of the tests
until the kinks have been worked out.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20251117115523.3993105-13-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
We need coreutils to run the IO tests so we need to include it in the
package list. Now we have the latest libvirt we can do that.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20251117115523.3993105-5-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Ubuntu is now including updated versions of Rust (up to 1.85) for
its LTS releases. Adjust the CI containers and re-add --enable-rust
to the Ubuntu jobs.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The whole point of SBSA_REF is for testing firmware which by
definition requires TCG. This means the configuration of:
--disable-tcg --with-devices-aarch64=minimal
makes no sense (and indeed is broken for the
ubuntu-24.04-aarch64-notcg) test. Drop it from minimal and remove the
allow_failure from the test case.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20251027110344.2289945-34-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Uploading the cache from the runner takes a long time in the MSYS2
job, mostly due to the size of the compiler cache.
However, looking at runs with a non-poluted cache, it seems like
you can get a build with a 99% hit rate already with ~ 160 MiB cache
size, so the compiler cache with 500 MiB certainly contains a lot of
stale files. Thus decrease the size of the ccache to a more reasonable
value to speed up the MSYS2 job in our CI.
While at it, also add a "du -sh" for the build folder to get a better
feeling for the amount of object code that is required for the build,
and publish the list of files in /var/cache to be able to better
analyze what is really clogging our cache here.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20251020161759.50241-1-thuth@redhat.com>
While working out what hoops to jump through to get a full set of
aarch32 packages installed on the aarch64 runner it was pointed out 32
bit host support is deprecated. As the extra packages where needed for
system emulation (marked deprecated since 8.0!) there didn't seem much
point keeping this in.
While the full expunging of 32 bit host support will probably be done
for 11.0 we can at least reduce the CI burden a bit now.
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Message-ID: <20251016150357.876415-3-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Most of the test is pure boilerplate so to save ourselves from
repetition move all the main bits into a minimal copy of
native_build_job_template but without the caching.
We keep all the current allow_fail and configure setups but do take
the opportunity to replace the -j`nproc --ignore=40` hack with
something that almost, but not quite, saturates the machine its being
built on.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Message-ID: <20251016150357.876415-2-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
The coverity-scan upload job has started failing as of 30th
September:
[ERROR] Cannot find '/usr/bin/file' command, and no 'file' command is found in the PATH
Coverity Capture uses this tool to identify the file type of executables.
Please ensure '/usr/bin/file' is available, or add the 'file' utility to your PATH.
This seems to have broken when we moved our containers from Fedora 40
to 41 -- probably F40 indirectly pulled in 'file' via some other
dependency, but in F41 it does not.
Explicitly install 'file' for the coverity job, in the same way we
already do for curl and wget.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20251017133156.926094-1-peter.maydell@linaro.org>
32-bit host support is deprecated since commit 6d701c9bac
("meson: Deprecate 32-bit host support"). Next commits will
remove support for 32-bit MIPS hosts. Stop cross-building
QEMU on our CI.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20251009195210.33161-3-philmd@linaro.org>
For the Windows msys2 CI job we install many packages using pacman
and use the GitLab cache to preserve the pacman cache across CI
runs. While metadata still needs downloading, this avoids pacman
re-downloading packages from msys2 if they have not changed.
The problem is that pacman never automatically purges anything
from its package cache. Thus the GitLab cache is growing without
bound and packing/unpacking the cache is consuming an increasing
amount of time in the CI job.
If we run 'pacman -Sc' /after/ installing our desired package set,
it will purge any cached downloaded packages that are not matching
any installed package.
This will (currently) cap the pacman download cache at approx
256 MB.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-ID: <20251010160545.144760-1-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20251001170947.2769296-7-alex.bennee@linaro.org>
It fails to link on fedora >= 41:
/usr/bin/ld: /usr/bin/../lib/clang/20/lib/x86_64-redhat-linux-gnu/libclang_rt.safestack.a(safestack.cpp.o): in function `__sanitizer_internal_memcpy':
(.text.__sanitizer_internal_memcpy+0x0): multiple definition of `__sanitizer_internal_memcpy'; /usr/bin/../lib/clang/20/lib/x86_64-redhat-linux-gnu/libclang_rt.ubsan_standalone.a(sanitizer_libc.cpp.o):(.text.__sanitizer_internal_memcpy+0x0): first defined here
/usr/bin/ld: /usr/bin/../lib/clang/20/lib/x86_64-redhat-linux-gnu/libclang_rt.safestack.a(safestack.cpp.o): in function `__sanitizer_internal_memmove':
(.text.__sanitizer_internal_memmove+0x0): multiple definition of `__sanitizer_internal_memmove'; /usr/bin/../lib/clang/20/lib/x86_64-redhat-linux-gnu/libclang_rt.ubsan_standalone.a(sanitizer_libc.cpp.o):(.text.__sanitizer_internal_memmove+0x0): first defined here
/usr/bin/ld: /usr/bin/../lib/clang/20/lib/x86_64-redhat-linux-gnu/libclang_rt.safestack.a(safestack.cpp.o): in function `__sanitizer_internal_memset':
(.text.__sanitizer_internal_memset+0x0): multiple definition of `__sanitizer_internal_memset'; /usr/bin/../lib/clang/20/lib/x86_64-redhat-linux-gnu/libclang_rt.ubsan_standalone.a(sanitizer_libc.cpp.o):(.text.__sanitizer_internal_memset+0x0): first defined here
cfi_debug seems to pull ubsan which has conflicting symbols with safe_stack.
See also: https://bugzilla.redhat.com/show_bug.cgi?id=2397265
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250924120426.2158655-12-marcandre.lureau@redhat.com>
The gitlab "Pipeline editor" has some warnings, and gitlab-ci-local
fails.
Read also from the docs
https://docs.gitlab.com/ci/yaml/#needs
"Supported values:
An array of jobs (maximum of 50 jobs).
An empty array ([]), to set the job to start as soon as the pipeline
is created."
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250924120426.2158655-3-marcandre.lureau@redhat.com>
In commit 52a21689cd we added a workaround for a bug in older
versions of htags where they fail with a weird error message if the
environment is too large. However, we missed one variable which
gitlab CI can set to the body of the commit message:
CI_COMMIT_DESCRIPTION.
Add this to the variables we unset when running htags, so that
the 'pages' job doesn't fail if the most recent commit happens
to have a very large commit message.
Cc: qemu-stable@nongnu.org
Fixes: 52a21689cd (".gitlab-ci.d/buildtest.yml: Work around htags bug when environment is large")
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250916163030.1467893-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The build jobs will populate build/meson-logs/ with various files
that are added as artifacts.
The test jobs preserve the state of the build jobs, so we must
delete any pre-existing logs to prevent confusion from duplicate
artifacts.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908190901.3571859-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The junit XML file produced by meson does not always have the
name 'testlog.junit.xml' - in the case of 'make check-functional'
there is a 'testlog-thorough.junit.xml' file too.
Improve CI debugging robustness by capturing all junit files that
meson produces.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908190901.3571859-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
There are files besides testlog.txt that may be useful as published
CI artifacts.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908190901.3571859-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The 'results.xml' file and 'test-results' directory were both outputs
of the avovcado test runner. Since we're now using meson with the new
functional test framework, we must reference meson results files as the
CI artifacts.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250908190901.3571859-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
mingw-w64-x86_64-curl-winssl is required for https connections.
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Message-ID: <20250806200909.507803-1-sw@weilnetz.de>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Mostly a developer aid for those who want to look at the full backlog
of multiple build units.
Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250710104531.3099313-3-alex.bennee@linaro.org>
Modernise the argument parsing so we can easily add to the script.
Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250710104531.3099313-2-alex.bennee@linaro.org>
The system tests (usually qos-test or migration-test) prove to be very
susceptible on the s390x runners. Although we have boosted memory and
virtual CPUs on the runners problems persist. For now mark test as
allow_failure so the its clear on the CI UI when checking test
results.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250627112512.1880708-2-alex.bennee@linaro.org>
Our default build enables debug info which adds hugely to the size of
the builds as well as the size of cached objects. Disable debug info
across the board to save space and reduce pressure on the CI system.
We still have a number of builds which explicitly enable debug and
related extra asserts like --enable-debug-tcg.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20250603110204.838117-3-alex.bennee@linaro.org>
Doctests are weird. They are essentially integration tests, but they're
"ran" by executing rustdoc --test, which takes a compiler-ish
command line. This is supported by Meson 1.8.0.
Because they run the linker and need all the .o files, run them in the
build jobs rather than the test jobs.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The build-previous-qemu job is intented to produce a build of the
previous QEMU release for consumption by the migration-compat-*
jobs. Keep only the pieces of the build that are necessary.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20250513133353.23022-4-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
Stefan reports that during QEMU release, pushing a series with the
VERSION bump commit, but not pushing the new git tag in the same
command will cause a failure of the build-previous-qemu job at the git
fetch step.
Since the job is intended to produce a build of the previous QEMU
version for consumption by the migration-compat-* jobs, there's no
reason to produce a build of the release commit because the migration
job would end up testing the release against itself.
Skip the job when VERSION contains the newly release version number.
I'm opting for 'exit 0' for both the build and the test jobs because
allow_failure would mask any real error in the jobs. It also avoids
having an orange ! on every release pipeline.
Reported-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20250513133353.23022-3-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
The migration compatibility tests have been running with the PYTHON
variable unset to avoid running a broken test. The faulty test has
since been removed, so we can enable the python tests once again.
Aside from the broken test, only one other test uses python and I have
been running it locally ever since, so this commit should not expose
any new bug.
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20250513133353.23022-2-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
The riscv64-debian-cross container is based on Trixie rather than sid
these days, so it is pretty much as stable as the others. Enable it
by default.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Since we don't run the Avocado jobs in the CI anymore, rename
these variables to QEMU_JOB_FUNCTIONAL and QEMU_CI_FUNCTIONAL.
Also, there was a mismatch between the documentation and the
implementation of QEMU_CI_AVOCADO_TESTING: While the documentation
said that you had to "Set this variable to have the tests using the
Avocado framework run automatically", you indeed needed to set it
to make the pipelines appear in your dashboard - but they were never
run automatically in forks and had to be triggered manually. Let's
improve this now: No need to hide these pipelines from the users
by default anymore (the functional tests should be stable enough
nowadays), and rather allow the users to run the pipelines auto-
matically with this switch now instead, as was documented.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-15-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We are going to move the remaining Avocado tests step by step
into the functional test framework. Unfortunately, Avocado fails
with an error if it cannot determine a test to run, so disable
the tests here now to avoid failures in the Gitlab-CI during the
next steps.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20250414113031.151105-2-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We want to capture potential Rust backtraces on panics in our test
logs, which isn't Rust's default behaviour. Set RUST_BACKTRACE=1 in
the add_test_setup environments, so that all our tests get run with
this environment variable set.
This makes the setting of that variable in the gitlab CI template
redundant, so we can remove it.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20250310102950.3752908-1-peter.maydell@linaro.org
We want to reduce the total number of build units in the system to get
on our way to a single binary. It will help to have some numbers so
lets add a job to gitlab to track our progress.
Cc: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Cc: Philippe Mathieu-Daudé <philmd@linaro.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-19-alex.bennee@linaro.org>
The alpine baseline has also been updated in the meantime so we need
to address that while we are at it.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-12-alex.bennee@linaro.org>
When gitlab initializes the repo checkout for a CI job, it will have
done a shallow clone with only partial history. Periodically the objects
that are omitted cause trouble with the check-patch/check-dco jobs. This
is exhibited as reporting strange errors being unable to fetch certain
objects that are known to exist.
Passing the --refetch flag to 'git fetch' causes it to not assume the
local checkout has all common objects and thus re-fetch everything that
is needed. This appears to solve the check-patch/check-dco job failures.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Michael S. Tsirkin <mst@redhat.com>
Message-ID: <20250225110525.2209854-1-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The FreeBSD job started to fail since the 14-1 image disappeared
from the cloud. Update the job to v14.2 to fix it.
Message-ID: <20250211120817.35050-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Rather than a giant sed command with a hardcoded list of env var name,
we can now use the new(ish) cirrus-vars command that libvirt has added
to the 'cirrus-run' container.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20241204194807.1472261-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
In the last week of the month we have often run out of credits on
Cirrus CI, which causes the jobs to fail, in turn causing the
overall pipeline to fail.
The cirrus-run tool can now detect the "out of credits" scenario
and exits with a code of '3'. We can tell gitlab to treat this
exit code as special and mark the job as "warning" instead of
"failed". This allows the pipeline status overall to remain
green, when we have non-technical issues with Cirrus CI.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241204194807.1472261-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Configuration of 64-bit host on 32-bit guest will shortly
be denied. Use a 32-bit guest instead.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
We will shortly need this to build our riscv64 cross container.
However to keep the delta down just do the bump first. As ccache4 is
now preferred for FreeBSD to get the latest version there is a little
update in the FreeBSD metadata.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250108121054.1126164-25-alex.bennee@linaro.org>
Alex Bennée
Paolo Bonzini
Thomas Huth
Peter Maydell
Philippe Mathieu-Daudé
Daniel P. Berrangé
Marc-André Lureau
Stefan Weil
Fabiano Rosas
Kohei Tokunaga
Richard Henderson