qemu-binfmt-conf.sh should use "-F" as short option for "--qemu-suffix".
Fix the getopt call to make this work.
Fixes: 7155be7cda ("qemu-binfmt-conf.sh: allow to provide a suffix to the interpreter name")
Signed-off-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20211129135100.3934-1-mwilck@suse.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
"meson test" starting with version 0.57 is just as capable and easy to
use as QEMU's own TAP driver. All existing options for "make check"
work. The only required code change involves how to mark "slow" tests;
they need to belong to an additional "slow" suite.
The rules for .tap output are replaced by JUnit XML; GitLab is able
to parse that output and present it in the CI pipeline report.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The "dbus" display backend exports the QEMU consoles and other
UI-related interfaces over D-Bus.
By default, the connection is established on the session bus, but you
can specify a different bus with the "addr" option.
The backend takes the "org.qemu" service name, while still allowing
further instances to queue on the same name (so you can lookup all the
available instances too). It accepts any number of clients at this
point, although this is expected to evolve with options to restrict
clients, or only accept p2p via fd passing.
The interface is intentionally very close to the internal QEMU API,
and can be introspected or interacted with busctl/dfeet etc:
$ ./qemu-system-x86_64 -name MyVM -display dbus
$ busctl --user introspect org.qemu /org/qemu/Display1/Console_0
org.qemu.Display1.Console interface - - -
.RegisterListener method h - -
.SetUIInfo method qqiiuu - -
.DeviceAddress property s "pci/0000/01.0" emits-change
.Head property u 0 emits-change
.Height property u 480 emits-change
.Label property s "VGA" emits-change
.Type property s "Graphic" emits-change
.Width property u 640 emits-change
[...]
See the interfaces XML source file and Sphinx docs for the generated API
documentations.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
g_memdup() is insecure and as been deprecated in GLib 2.68.
QEMU provides the safely equivalent g_memdup2() wrapper.
Do not allow more g_memdup() calls in the repository, provide
a hint to use g_memdup2().
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210903174510.751630-29-philmd@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
It's easier to do this in meson.build now.
Message-Id: <20211209144801.148388-1-thuth@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Older versions of Mac OS X do not support cp -a. The cp man page indicates
that -a is equivalent to -pPR.
Signed-off-by: Evan Miller <emmiller@gmail.com>
Message-Id: <40635C6E-059A-4146-B1E2-F6376700EE85@gmail.com>
[Leave out -R, these are files and not directories. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Despite all the previous fixes, it's still possible for
device-crash-test to wedge itself in the case that QEMU terminates *so
quickly* that it doesn't even begin a connection attempt to our QMP
client. Python will just joyfully wait ad infinitum for a connection
that will now never arrive.
The real fix is to use asyncio to simultaneously poll both the health of
the launched process AND the connection attempt. That's quite a bit more
invasive than just setting a connection timeout, though.
Do the very simplest thing for now.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20211118204620.1897674-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
This reverts commit ca89d15f8e.
There is already an entry for this kind of messages earlier in the
ERROR_RULE_LIST - when I added this patch, I just got fooled by
the other errors that occur due to a race between QMP connection
and QEMU terminating early (which still spit out the 'No bus found'
messages in their backtrace), but these other problems have now
fortunately been tackled by John Snow, so we certainly don't need
this duplicated entry here anymore.
Message-Id: <20211112072220.108580-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Generally, the traceback for a connection failure is uninteresting and
all we need to know is that the connection attempt failed.
Reduce the verbosity in these cases, except when debugging.
Signed-off-by: John Snow <jsnow@redhat.com>
Reported-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-id: 20211111143719.2162525-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
These errors are expected, so they shouldn't clog up terminal output. In
the event that they're *not* expected, we'll be seeing an awful lot more
output concerning the nature of the failure.
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-id: 20211111143719.2162525-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
We don't need to handle KeyboardInterruptError specifically; we can
instead tighten the scope of the broad Exception handlers to only catch
"Exception", which has the effect of allowing all BaseException classes
that do not inherit from Exception to be raised through.
KeyboardInterruptError and a few other important ones are
BaseExceptions, so this does the same thing with less code.
Signed-off-by: John Snow <jsnow@redhat.com>
Reported-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-id: 20211111143719.2162525-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
This introduces three different parts of a job designed to run
on a custom runner managed by Red Hat. The goals include:
a) propose a model for other organizations that want to onboard
their own runners, with their specific platforms, build
configuration and tests.
b) bring awareness to the differences between upstream QEMU and the
version available under CentOS Stream, which is "A preview of
upcoming Red Hat Enterprise Linux minor and major releases".
c) because of b), it should be easier to identify and reduce the gap
between Red Hat's downstream and upstream QEMU.
The components of this custom job are:
I) OS build environment setup code:
- additions to the existing "build-environment.yml" playbook
that can be used to set up CentOS/EL 8 systems.
- a CentOS Stream 8 specific "build-environment.yml" playbook
that adds to the generic one.
II) QEMU build configuration: a script that will produce binaries with
features as similar as possible to the ones built and packaged on
CentOS stream 8.
III) Scripts that define the minimum amount of testing that the
binaries built with the given configuration (point II) under the
given OS build environment (point I) should be subjected to.
IV) Job definition: GitLab CI jobs that will dispatch the build/test
jobs (see points #II and #III) to the machine specifically
configured according to #I.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211111160501.862396-2-crosa@redhat.com>
Message-Id: <20211115142915.3797652-6-alex.bennee@linaro.org>
Under SELinux, Unix domain sockets have two labels. One is on the
disk and can be set with commands such as chcon(1). There is a
different label stored in memory (called the process label). This can
only be set by the process creating the socket. When using SELinux +
SVirt and wanting qemu to be able to connect to a qemu-nbd instance,
you must set both labels correctly first.
For qemu-nbd the options to set the second label are awkward. You can
create the socket in a wrapper program and then exec into qemu-nbd.
Or you could try something with LD_PRELOAD.
This commit adds the ability to set the label straightforwardly on the
command line, via the new --selinux-label flag. (The name of the flag
is the same as the equivalent nbdkit option.)
A worked example showing how to use the new option can be found in
this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1984938
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1984938
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
[eblake: rebase to configure changes, reject --selinux-label if it is
not compiled in or not used on a Unix socket]
Note that we may relax some of these restrictions at a later date,
such as making it possible to label a TCP socket, although it may be
smarter to do so as a generic QMP action rather than more one-off
command lines in qemu-nbd.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20211115202944.615966-1-eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
[eblake: adjust meson output as suggested by thuth]
Signed-off-by: Eric Blake <eblake@redhat.com>
The following commits (released in v6.0.0) made raised the
quality of the TCI backend to the other TCG architectures,
thus is is not considerated experimental anymore:
- c6fbea47664..2f74f45e32b
- dc09f047edd..9e9acb7b348
- b6139eb0578..2fc6f16ca5e
- dbcbda2cd84..5e8892db93f
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20211106111457.517546-1-f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
And while we're at it, also provide a proper entry for this feature
in meson_options.txt, so that people who don't need it have a knob
to disable this feature.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20211028185910.1729744-3-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
This gains some bugfixes, especially:
- it fixes the introspection of array options. While technically we
still support Meson 0.58.2, this issue only appears when adding a new
option and not if the user is just building QEMU. In the relatively
rare case of a contributor using --meson to point to a 0.58 version,
review can catch spurious changes to scripts/meson-buildoptions.sh
easily.
- it fixes "meson test" when it is not the process group leader. Make is
the process group leader when "make check" invokes "meson test", so this
is a requirement for using it as a test harness.
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
(But continue to support the old ones for now, too.)
There are very few cases of any user of QEMUMachine or a subclass
thereof relying on a QMP Exception type. If you'd like to check for
yourself, you want to grep for all of the derivatives of QMPError,
excluding 'AQMPError' and its derivatives. That'd be these:
- QMPError
- QMPConnectError
- QMPCapabilitiesError
- QMPTimeoutError
- QMPProtocolError
- QMPResponseError
- QMPBadPortError
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20211026175612.4127598-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
New option parameters unstable-input and unstable-output set policy
for unstable interfaces just like deprecated-input and
deprecated-output set policy for deprecated interfaces (see commit
6dd75472d5 "qemu-options: New -compat to set policy for deprecated
interfaces"). This is intended for testing users of the management
interfaces. It is experimental.
For now, this covers only syntactic aspects of QMP, i.e. stuff tagged
with feature 'unstable'. We may want to extend it to cover semantic
aspects, or the command line.
Note that there is no good way for management application to detect
presence of these new option parameters: they are not visible output
of query-qmp-schema or query-command-line-options. Tolerable, because
it's meant for testing. If running with -compat fails, skip the test.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-10-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Doc comments fixed up]
The code to check enumeration value policy can see special feature
flag 'deprecated' in QEnumLookup member flags[value]. I want to make
feature flag 'unstable' visible there as well, so I can add policy for
it.
Instead of extending flags[], replace it by @special_features (a
bitset of QapiSpecialFeature), because that's how special features get
passed around elsewhere.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-8-armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
The code to check command policy can see special feature flag
'deprecated' as command flag QCO_DEPRECATED. I want to make feature
flag 'unstable' visible there as well, so I can add policy for it.
To let me make it visible, add member @special_features (a bitset of
QapiSpecialFeature) to QmpCommand, and adjust the generator to pass it
through qmp_register_command(). Then replace "QCO_DEPRECATED in
@flags" by QAPI_DEPRECATED in @special_features", and drop
QCO_DEPRECATED.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-7-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The generated visitor functions call visit_deprecated_accept() and
visit_deprecated() when visiting a struct member with special feature
flag 'deprecated'. This makes the feature flag visible to the actual
visitors. I want to make feature flag 'unstable' visible there as
well, so I can add policy for it.
To let me make it visible, replace these functions by
visit_policy_reject() and visit_policy_skip(), which take the member's
special features as an argument. Note that the new functions have the
opposite sense, i.e. the return value flips.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20211028102520.747396-6-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Unbreak forward visitor]
New enum QapiSpecialFeature enumerates the special feature flags.
New helper gen_special_features() returns code to represent a
collection of special feature flags as a bitset.
The next few commits will put them to use.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-5-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-4-armbru@redhat.com>
This copies the code implementing the policy from qapi/qmp-dispatch.c
to qapi/qobject-input-visitor.c. Tolerable, but if we acquire more
copies, we should look into factoring them out.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Peter Krempa <pkrempa@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
Message-Id: <20211025042405.3762351-5-armbru@redhat.com>
This is quite similar to commit 84ab008687 "qapi: Add feature flags to
struct members", only for enums instead of structs.
Special feature flag 'deprecated' is silently ignored there. This is
okay only because it will be implemented shortly.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20211025042405.3762351-3-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
The next commit will add feature flags to enum members. There's a
problem, though: query-qmp-schema shows an enum type's members as an
array of member names (SchemaInfoEnum member @values). If it showed
an array of objects with a name member, we could simply add more
members to these objects. Since it's just strings, we can't.
I can see three ways to correct this design mistake:
1. Do it the way we should have done it, plus compatibility goo.
We want a ['SchemaInfoEnumMember'] member in SchemaInfoEnum. Since
changing @values would be a compatibility break, add a new member
@members instead.
@values is now redundant. In my testing, output of
qemu-system-x86_64's query-qmp-schema grows by 11% (18.5KiB).
We can deprecate @values now and drop it later. This will break
outmoded clients. Well-behaved clients such as libvirt are
expected to break cleanly.
2. Like 1, but omit "boring" elements of @member, and empty @member.
@values does not become redundant. @members augments it. Somewhat
cumbersome, but output of query-qmp-schema grows only as we make
enum members non-boring.
There is nothing to deprecate here.
3. Versioned query-qmp-schema.
query-qmp-schema provides either @values or @members. The QMP
client can select which version it wants. There is no redundant
output.
We can deprecate old versions and eventually drop them. This will
break outmoded clients. Breaking cleanly is easier than for 1.
While 1 and 2 operate within the common rules for compatible
evolution apply (section "Compatibility considerations" in
docs/devel/qapi-code-gen.rst), 3 bypasses them. Attractive when
operating within the rules is just too awkward. Not the case here.
This commit implements 1. Libvirt developers prefer it.
Deprecate @values in favour of @members. Since query-qmp-schema
compatibility is pretty fundamental for management applications, an
extended grace period is advised.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Peter Krempa <pkrempa@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
Message-Id: <20211025042405.3762351-2-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
When we try to use 'analyze-migration.py -x' with python3,
we have the following errors:
Traceback (most recent call last):
File "scripts/analyze-migration.py", line 593, in <module>
f.write(jsonenc.encode(dump.vmsd_desc))
TypeError: a bytes-like object is required, not 'str'
Traceback (most recent call last):
File "scripts/analyze-migration.py", line 601, in <module>
f.write(jsonenc.encode(dict))
TypeError: a bytes-like object is required, not 'str'
This happens because the file 'f' is open in binary mode while
jsonenc.encode() returns a string.
The results are human-readable files, 'desc.json' and 'state.json',
so there is no reason to use the binary mode.
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211015131645.501281-3-lvivier@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
The parameters of '-d' can be either 'state' or 'desc', not 'dump'
as it is reported in the error message.
Fixes: b17425701d ("Add migration stream analyzation script")
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211015131645.501281-2-lvivier@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
The EDK2 firmware images built to test QEMU do not require
the following submodules:
- MdeModulePkg/Universal/RegularExpressionDxe/oniguruma
- UnitTestFrameworkPkg/Library/CmockaLib/cmocka
The only submodules required are:
- ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
- BaseTools/Source/C/BrotliCompress/brotli
- CryptoPkg/Library/OpensslLib/openssl
- MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
Adapt the buildsys machinery to only initialize the required
submodules.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20211018105816.2663195-3-philmd@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20211006131718.214235-7-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Right now meson_options.txt lists about 90 options. Each option
needs code in configure to parse it and pass the option down to Meson as
a -D command-line argument; in addition the default must be duplicated
between configure and meson_options.txt. This series tries to remove
the code duplication by generating the case statement for those --enable
and --disable options, as well as the corresponding help text.
About 80% of the options can be handled completely by the new mechanism.
Eight meson options are not of the --enable/--disable kind. Six more need
to be parsed in configure for various reasons documented in the patch,
but they still have their help automatically generated.
The advantages are:
- less code in configure
- parsing and help is more consistent (for example --enable-blobs was
not supported)
- options are described entirely in one place, meson_options.txt.
This make it more attractive to use Meson options instead of
hand-crafted configure options and config-host.mak
A few options change name: --enable-tcmalloc and --enable-jemalloc
become --enable-malloc={tcmalloc,jemalloc}; --disable-blobs becomes
--disable-install-blobs; --enable-trace-backend becomes
--enable-trace-backends. However, the old names are allowed
for backwards compatibility.
Message-Id: <20211007130829.632254-19-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Manually patch the introspection data to include the tracing backends.
This works around a deficiency in Meson that will be fixed by
https://github.com/mesonbuild/meson/pull/9395.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Prepare the configure script and Makefile for automatically generated
help and parsing.
Because we need to run the script to generate the full help, we
cannot rely on the user supplying the path to a Python interpreter
with --python; therefore, the introspection output is parsed into
shell functions and stored in scripts/. The converter is written
in Python as standard for QEMU, and this commit contains a stub.
Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20211007130829.632254-18-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Meson 0.58.2 does not need b_staticpic=$pie anymore, and has
stabilized the keyval module. Remove the workaround and use a few
replacements for features deprecated in the 0.57.0 release cycle.
One feature that we would like to use is passing dependencies to
summary. However, that was broken in 0.59.0 and 0.59.1. Therefore,
use the embedded Meson if the host has anything older than 0.59.2,
but allow --meson= to use 0.58.2.
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Eh. Not worth the fuss today. There are bigger fish to fry.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
The fix for this comment is forthcoming in a future commit, but this
will keep me honest. The linting configuration in ./python/setup.cfg
prohibits 'FIXME' comments. A goal of this long-running series is to
move ./scripts/qapi to ./python/qemu/qapi so that the QAPI generator is
regularly type-checked by GitLab CI.
This comment is a time-bomb to force me to address this issue prior to
that step.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Annotations do not change runtime behavior.
This commit consists of only annotations.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Adding static types causes a cycle in the QAPI generator:
[schema -> expr -> parser -> schema]. It exists because the QAPIDoc
class needs the names of types defined by the schema module, but the
schema module needs to import both expr.py/parser.py to do its actual
parsing.
Ultimately, the layering violation is that parser.py should not have any
knowledge of specifics of the Schema. QAPIDoc performs double-duty here
both as a parser *and* as a finalized object that is part of the schema.
In this patch, add the offending type hints alongside the workaround to
avoid the cycle becoming a problem at runtime. See
https://mypy.readthedocs.io/en/latest/runtime_troubles.html#import-cycles
for more information on this workaround technique.
I see three ultimate resolutions here:
(1) Just keep this patch and use the TYPE_CHECKING trick to eliminate
the cycle which is only present during static analysis.
(2) Don't bother to annotate connect_member() et al, give them 'object'
or 'Any'. I don't particularly like this, because it diminishes the
usefulness of type hints for documentation purposes. Still, it's an
extremely quick fix.
(3) Reimplement doc <--> definition correlation directly in schema.py,
integrating doc fields directly into QAPISchemaMember and relieving
the QAPIDoc class of the responsibility. Users of the information
would instead visit the members first and retrieve their
documentation instead of the inverse operation -- visiting the
documentation and retrieving their members.
My preference is (3), but in the short-term (1) is the easiest way to
have my cake (strong type hints) and eat it too (Not have import
cycles). Do (1) for now, but plan for (3).
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Here's the weird bit. QAPIDoc generally expects -- virtually everywhere
-- that it will always have a current section. The sole exception to
this is in the case that end_comment() is called, which leaves us with
*no* section. However, in this case, we also don't expect to actually
ever mutate the comment contents ever again.
NullSection is just a Null-object that allows us to maintain the
invariant that we *always* have a current section, enforced by static
typing -- allowing us to type that field as QAPIDoc.Section instead of
the more ambiguous Optional[QAPIDoc.Section].
end_section is renamed to switch_section and now accepts as an argument
the new section to activate, clarifying that no callers ever just
unilaterally end a section; they only do so when starting a new section.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
The "if self._section" clause in end_section is mysterious: In which
circumstances might we end a section when we don't have one?
QAPIDoc always expects there to be a "current section", only except
after a call to end_comment(). This actually *shouldn't* ever be 'None',
so let's remove that logic so I don't wonder why it's like this again in
three months.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
True, we do not check the validity of this symbol -- but we don't check
the validity of definition names during parse, either -- that happens
later, during the expr check. I don't want to introduce a dependency on
expr.py:check_name_str here and introduce a cycle.
Instead, rest assured that a documentation block is required for each
definition. This requirement uses the names of each section to ensure
that we fulfilled this requirement.
e.g., let's say that block-core.json has a comment block for
"Snapshot!Info" by accident. We'll see this error message:
In file included from ../../qapi/block.json:8:
../../qapi/block-core.json: In struct 'SnapshotInfo':
../../qapi/block-core.json:38: documentation comment is for 'Snapshot!Info'
That's a pretty decent error message.
Now, let's say that we actually mangle it twice, identically:
../../qapi/block-core.json: In struct 'Snapshot!Info':
../../qapi/block-core.json:38: struct has an invalid name
That's also pretty decent. If we forget to fix it in both places, we'll
just be back to the first error.
Therefore, let's just drop this FIXME and adjust the error message to
not imply a more thorough check than is actually performed.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Pylint informs us we're not using these arguments. Oops, it's
right. Correct the error message and remove the remaining unused
parameter.
Fix test output now that the error message is improved.
Fixes: e151941d1b
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-4-jsnow@redhat.com>
[Commit message formatting tweaked]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
New pylint warning. I could silence it, but this is the only occurrence
in the entire tree, including everything in iotests/ and python/. Easier
to just change this one instance.
(The warning is emitted in cases where you are fetching the values
anyway, so you may as well just take advantage of the iterator to avoid
redundant lookups.)
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Pylint 2.11.x adds this warning. We're not yet ready to pursue that
conversion, so silence it for now.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Simple unions predate flat unions. Having both complicates the QAPI
schema language and the QAPI generator. We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.
The previous commits eliminated simple union from the tree. Now drop
them from the QAPI schema language entirely, and update mentions of
"flat union" to just "union".
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-22-armbru@redhat.com>
I'm about to convert simple unions to flat unions, then drop simple
union support. The conversion involves making the implict enum types
explicit. To reduce churn, I'd like to name them exactly like the
implicit types they replace. However, these names are reserved for
the generator's use. They won't be once simple unions are gone. Stop
enforcing this naming rule now rather than then.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-3-armbru@redhat.com>
Martin Wilck
Paolo Bonzini
Marc-André Lureau
Philippe Mathieu-Daudé
Thomas Huth
Evan Miller
John Snow
Cleber Rosa
Richard W.M. Jones
Philippe Mathieu-Daudé
Markus Armbruster
Laurent Vivier
Vladimir Sementsov-Ogievskiy