riscv/qemu - qemu - Gitea: Git with a cup of tea

Commit Graph

Author	SHA1	Message	Date
John Snow	12b4267d60	python/qapi: delint import statements Missed a spot with isort, which now causes the python-minreqs test on GitLab to fail. Fix it. (Hint: the commands in python/tests/qapi-isort.sh can be run without the "-c" parameter to automatically adjust import statements according to our style rules. Maybe I should make a pre-submit hook that makes this adjustment automatically. What do you think?) Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3200 Fixes: `5bd89761a4` ("qapi/command: Avoid generating unused qmp_marshal_output_T") Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20251118200657.1043688-4-jsnow@redhat.com>	9 months ago
John Snow	cc74a4d86e	python/qapi: add an ignore for Pylint 4.x Pylint 4.x wants to use a different regex for _Stub for some reason; just silence this. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Thomas Huth <thuth@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20251118200657.1043688-3-jsnow@redhat.com>	9 months ago
Markus Armbruster	1ca08107c5	qapi/parser: Mollify mypy re.match(r'^ *', ...) can't fail, but mypy doesn't know that and complains: scripts/qapi/parser.py:444: error: Item "None" of "Match[str] \| None" has no attribute "end" [union-attr] Work around by using must_match() instead. Fixes: `8107ba47fd` (qapi: Add documentation format validation) Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20251105152219.311154-1-armbru@redhat.com>	9 months ago
Vladimir Sementsov-Ogievskiy	8107ba47fd	qapi: Add documentation format validation Add explicit validation for QAPI documentation formatting rules: 1. Lines must not exceed 70 columns in width (including '# ' prefix) 2. Sentences must be separated by two spaces Example sections and literal :: blocks (seldom case) are excluded, we don't require them to be <= 70, that would be too restrictive. Anyway, they share common 80-columns recommendations (not requirements). Add two simple tests, illustrating the change. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru> Message-ID: <20251031183129.246814-1-vsementsov@yandex-team.ru> The detection of example and literal blocks isn't quite correct, but it works well enough, and we can improve on top. Reviewed-by: Markus Armbruster <armbru@redhat.com> [Comments, error messages, and test file names tweaked] Signed-off-by: Markus Armbruster <armbru@redhat.com>	9 months ago
Markus Armbruster	5bd89761a4	qapi/command: Avoid generating unused qmp_marshal_output_T() qmp_marshal_output_T() is only ever called by qmp_marshal_C() for a command C that returns type T. We've always generated it as a static function on demand, i.e. when we generate a call. Since we split up monolithic generated code into modules (commit `252dc3105f` "qapi: Generate separate .h, .c for each module"), we do this per module. As noted in the commit message, this can result in identical (static) qmp_marshal_output_T() in several modules. Was deemed not worth avoiding. A bit later, we added 'if' conditionals to the schema language (merge commit `5dafaf4fbc`). When a conditional definition uses a type, then its condition must imply the type's condition. We made this the user's responsibility. Hasn't been an issue in practice. However, the sharing of qmp_marshal_output_T() among commands complicates matters. To avoid both undefined function errors and unused function warnings, qmp_marshal_output_T() must be defined exactly when it's used. It is used when any of the qmp_marshal_C() calling it is defined, i.e. when any C's condition holds. The generator uses T's condition instead. To avoid both error and warning, T's condition must be the conjunction of all C's conditions. Unfortunately, this can be impossible: * Conditional command returning a builtin type A builtin type cannot be conditional. This is noted in a FIXME comment. * Commands in multiple modules where the conjunction differs between modules An instance of this came up recently. we have unconditional commands returning HumanReadableText. If we add a conditional one to a module that does not have unconditional ones, compilation fails with "defined but not used". If we make HumanReadableText conditional to fix this module, we break the others. Instead of complicating the code to compute the conjunction, simplify it: generate the output marshaling code right into qmp_marshal_C(). This duplicates it when multiple commands return the same type. The impact on code size is negligible: qemu-system-x86_64's text segment grows by 1448 bytes. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20250804130602.903904-1-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> [Commit message typos fixed]	1 year ago
Paolo Bonzini	20b92da6db	treewide: remove unnessary "coding" header The "-- coding: utf-8 --" header was needed in Python 2, but in Python 3 UTF-8 is already the default encoding of source files. It is even less necessary in .css files that do not have UTF-8 sequences at all. Suggested-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org> Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Message-ID: <20250929154938.594389-4-pbonzini@redhat.com> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>	10 months ago
John Snow	636c96cd77	qapi: Fix undocumented return values by generating something Generated command documentation lacks information on return value in several cases, e.g. query-tpm. The obvious fix would be to require a Returns: section when a command returns something. However, note that many existing Returns: sections are pretty useless: the description is basically the return type, which then gets rendered like "Return: <Type> – <basically the return type>". This suggests that a description is often not really necessary, and requiring one isn't useful. Instead, generate the obvious minimal thing when Returns: is absent: "Return: <Type>". This auto-generated Return documentation is placed is as follows: 1. If we have arguments, return goes right after them. 2. Else if we have errors, return goes right before them. 3. Else if we have features, return goes right before them. 4. Else return goes right after the intro To facilitate this algorithm, a "TODO:" hack line is used to separate the intro from the remainder of the documentation block in cases where there are no other sections to separate the intro from e.g. examples and additional detail meant to appear below the key sections of interest. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20250711051045.51110-3-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [_insert_near_kind() code replaced by something simpler, commit message amended to explain why we're doing this] Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
John Snow	62c4dc4b69	qapi: lift restriction on using '=' in doc blocks We reject lines starting with '=' in definition documentation. This made sense when such lines were headings in free-form documentation, but not in definition documentation. Before the previous commit, lines starting with '=' were headings in free-form documentation, and rejected in definition documentation, where such headings could not work. The previous commit dropped the headings feature from free-form documentation, because we can simply use plain rST headings. Rejecting them in definition documentation no longer makes sense, so drop that, too. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20250618165353.1980365-6-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Amend commit message to explain why] Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
John Snow	6c10778826	docs/sphinx: remove special parsing for freeform sections Remove the QAPI doc section heading syntax, use plain rST section headings instead. Tests and documentation are updated to match. Interestingly, Plain rST headings work fine before this patch, except for over- and underlining with '=', which the doc parser rejected as invalid QAPI doc section heading in free-form comments. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20250618165353.1980365-5-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Add more detail to commit message] Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
John Snow	678868eee3	qapi: delete un-needed python static analysis configs Since the previous commit, python/setup.cfg applies to scripts/qapi/ as well. Configuration files in scripts/qapi/ override python/setup.cfg. scripts/qapi/.flake8 and scripts/qapi/.isort.cfg actually match python/setup.cfg exactly, and can go. The differences between scripts/qapi/mypy.ini and python/setup.cfg are harmless: namespace_packages being set to True is a requirement for the PEP420 nested package structure of QEMU but not for scripts/qapi, but has no effect on type checking the QAPI code. warn_unused_ignores is used in python/ to be able to target a wide variety of mypy versions; some of which that have added new ignore categories that are not present in older versions. Ultimately, scripts/qapi/mypy.ini can be removed without any real change in behavior to how mypy enforces type safety there. The pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat on how to merge them O:-) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-id: 20250604200354.459501-7-jsnow@redhat.com	1 year ago
John Snow	65aa0a1780	python: add qapi static analysis tests Update the python tests to also check QAPI and the QAPI Sphinx extensions. The docs/sphinx/qapidoc_legacy.py file is not included in these checks, as it is destined for removal soon. mypy is also not called on the QAPI Sphinx extensions, owing to difficulties supporting Sphinx 3.x - 8.x while maintaining static type checking support. mypy is called on all of the QAPI tools themselves, though. flake8, isort and mypy use the tool configuration from the existing python directory (in setup.cfg). pylint continues to use the special configuration located in scripts/qapi/ - that configuration is more permissive. If we wish to unify the two configurations, that's a separate series and a discussion for a later date. The list of pylint ignores is also updated, owing again to the wide window of pylint version support: newer versions require pragmas to occasionally silence the "too many positional arguments" warning, but older versions do not have such a warning category and will instead yelp about an unrecognized option. Silence that warning, too. As a result of this patch, one would be able to run any of the following tests locally from the qemu.git/python directory and have it cover the QAPI tooling as well. All of the following options run the python tests, static analysis tests, and linter checks; but with different combinations of dependencies and interpreters. - "make check-minreqs" Run tests specifically under our oldest supported Python and our oldest supported dependencies. This is the test that runs on GitLab as "check-python-minreqs". This helps ensure we do not regress support on older platforms accidentally. - "make check-tox" Runs the tests under the newest supported dependencies, but under each supported version of Python in turn. At time of writing, this is Python 3.8 to 3.13 inclusive. This test helps catch bleeding-edge problems before they become problems for developer workstations. This is the GitLab test "check-python-tox" and is an optionally run, may-fail test due to the unpredictable nature of new dependencies being released into the ecosystem that may cause regressions. - "make check-dev" Runs the tests under the newest supported dependencies using whatever version of Python the user happens to have installed. This is a quick convenience check that does not map to any particular GitLab test. (Note! check-dev may be busted on Fedora 41 and bleeding edge versions of setuptools. That's unrelated to this patch and I'll address it separately and soon. Thank you for your patience, --mgmt) Finally, finally, finally: this means that QAPI tooling will be linted and type-checked from the GitLab pipelines. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-id: 20250604200354.459501-5-jsnow@redhat.com [Edited license choice per review --js] Signed-off-by: John Snow <jsnow@redhat.com>	1 year ago
John Snow	dc955052b4	qapi: Add some pylint ignores This restores the linting baseline in QAPI. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-id: 20250604200354.459501-2-jsnow@redhat.com	1 year ago
Thomas Huth	d64db833d6	Drop support for Python 3.8 Python 3.8 went "end of life" in October 2024 and Fedora 42 dropped this version already, so the "python" CI job is currently failing. Thus it's time to drop support for this Python version in QEMU, too. While we're at it, also look for "python3.13" in the configure script. Message-ID: <20250425120710.879518-1-thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Signed-off-by: Thomas Huth <thuth@redhat.com>	1 year ago
Markus Armbruster	93db9c84fc	scripts/qapi/backend: Clean up create_backend()'s failure mode create_backend()'s caller catches QAPIError, and returns non-zero exit code on catch. The caller's caller passes the exit code to sys.exit(). create_backend() doesn't care: it reports errors to stderr and sys.exit()s. Change it to raise QAPIError instead. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20250311065352.992307-1-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>	1 year ago
John Snow	4d7d30b405	qapi/parser: add undocumented stub members to all_sections Parser and doc generator cooperate on generating stub documentation for undocumented members. The parser makes up an ArgSection with an empty description, and the doc generator makes up a description. Right now, the made-up ArgSections go into doc.args. However, the new doc generator uses .all_sections, not .args. So put them into .all_sections, too. Insert them right after existing 'member' sections. If there are none, insert directly after the leading section. Doesn't affect the old generator, because that one doesn't use .all_sections. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20250311034303.75779-60-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Commit message rewritten] Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
John Snow	36e4182f40	qapi/source: allow multi-line QAPISourceInfo advancing This is for the sake of the new rST generator (the "transmogrifier") so we can advance multiple lines on occasion while keeping the generated<-->source mappings accurate. next_line now simply takes an optional n parameter which chooses the number of lines to advance. The next patch will use this when converting section syntax in free-form documentation to more traditional rST section header syntax, which does not always line up 1:1 for line counts. For example: ``` ## # = Section <-- Info is pointing here, "L1" # # Lorem Ipsum ## ``` would be transformed to rST as: ``` ======= <-- L1 Section <-- L1 ======= <-- L1 <-- L2 Lorem Ipsum <-- L3 ``` After consuming the single "Section" line from the source, we want to advance the source pointer to the next non-empty line which requires jumping by more than one line. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20250311034303.75779-42-jsnow@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
John Snow	d7ca9a3a4c	qapi/schema: add __repr__ to QAPIDoc.Section Makes debugging far more pleasant when you can just print(section) and get something reasonable to display. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20250311034303.75779-35-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
John Snow	323c668934	qapi: clean up encoding of section kinds We have several kinds of sections, and to tell them apart, we use Section attribute @tag and also the section object's Python type: type @tag untagged Section None @foo: ArgSection 'foo' Returns: Section 'Returns' Errors: Section 'Errors' Since: Section 'Since' TODO: Section 'TODO' Note: * @foo can be a member or a feature description, depending on context. * tag == 'Since' can be a Since: section or a member or feature description. If it's a Section, it's the former, and if it's an ArgSection, it's the latter. Clean this up as follows. Move the member or feature name to new ArgSection attribute @name, and replace @tag by enum @kind like this: type kind name untagged Section PLAIN @foo: ArgSection MEMBER 'foo' if member or argument ArgSection FEATURE 'foo' if feature Returns: Section RETURNS Errors: Section ERRORS Since: Section SINCE TODO: Section TODO The qapi-schema tests are updated to account for the new section names; "TODO" becomes "Todo" and `None` becomes "Plain" there. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20250311034303.75779-34-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
John Snow	faeacf858b	qapi/parser: adjust info location for doc body section Instead of using the info object for the doc block as a whole (which always points to the very first line of the block), update the info pointer for each call to ensure_untagged_section when the existing section is otherwise empty. This way, Sphinx error information will match precisely to where the text actually starts. For example, this patch will move the info pointer for the "Hello!" untagged section ... > ## <-- from here ... > # Hello! <-- ... to here. > ## This doesn't seem to improve error reporting now. It will with the forthcoming QAPI doc transmogrifier. If I stick bad rST into qapi/block-core.json like this: > ## > # @SnapshotInfo: > # > +# rST syntax error: *ahh! > +# > # @id: unique shapshot id > # > # @name: user chosen name The existing code's error message will point to the beginning of the doc comment, which is less than helpful. The transmogrifier's message will point to the erroneous line, but to accomplish this, it needs this patch. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20250311034303.75779-33-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
Markus Armbruster	e6985cc440	qapi/introspect: Use @dataclass to simplify A TODO comment in class Annotated reminds us to simplify it once we can use @dataclass, new in Python 3.7. We have that now, so do it. There's a similar comment in scripts/qapi/source.py, but I can't figure out how to use @dataclass there. Left for another day. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20250227080757.3978333-4-armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>	1 year ago
Markus Armbruster	5fbc8126ac	qapi: Eliminate OrderedDict We use OrderedDict to ensure dictionary order is insertion order. Plain dict does that since Python 3.6, but it wasn't guaranteed until 3.7. Since we have 3.7 now, replace OrderedDict by dict. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20250227080757.3978333-3-armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>	1 year ago
Daniel P. Berrangé	dde279925c	qapi: pluggable backend code generators The 'qapi.backend.QAPIBackend' class defines an API contract for code generators. The current generator is put into a new class 'qapi.backend.QAPICBackend' and made to be the default impl. A custom generator can be requested using the '-k' arg which takes a fully qualified python class name qapi-gen.py -B the.python.module.QAPIMyBackend This allows out of tree code to use the QAPI generator infrastructure to create new language bindings for QAPI schemas. This has the caveat that the QAPI generator APIs are not guaranteed stable, so consumers of this feature may have to update their code to be compatible with future QEMU releases. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250224182030.2089959-1-berrange@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Error checking and messages tweaked] Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
John Snow	6a2c7fc29a	qapi: update pylintrc config If you've got a newer pylint, it'll whine about positional arguments separately from the regular ones. Update the configuration to ignore both categories of warning. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20250224033741.222749-2-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	1 year ago
Daniel P. Berrangé	2ebb09f34f	qapi: expose all schema features to code This replaces use of the constants from the QapiSpecialFeatures enum, with constants from the auto-generate QapiFeatures enum in qapi-features.h The 'deprecated' and 'unstable' features still have a little bit of special handling, being force defined to be the 1st + 2nd features in the enum, regardless of whether they're used in the schema. This retains compatibility with common code that references the features via the QapiSpecialFeatures constants. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250205123550.2754387-5-berrange@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Imports tidied up with isort] Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Daniel P. Berrangé	ba27dccc04	qapi: rename 'special_features' to 'features' This updates the QAPI code generation to refer to 'features' instead of 'special_features', in preparation for generalizing their exposure. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250205123550.2754387-4-berrange@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Imports tidied up with isort] Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Daniel P. Berrangé	696ae1ac91	qapi: change 'unsigned special_features' to 'uint64_t features' The "special_features" field / parameter holds the subset of schema features that are for internal code use. Specifically 'DEPRECATED' and 'UNSTABLE'. This special casing of internal features is going to be removed, so prepare for that by renaming to 'features'. Using a fixed size type is also best practice for bit fields. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250205123550.2754387-3-berrange@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Daniel P. Berrangé	d8a22e69fe	qapi: cope with feature names containing a '-' When we shortly expose all feature names to code, it will be valid to include a '-', which must be translated to a '_' for the enum constants. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20250205123550.2754387-2-berrange@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Daniel P. Berrangé	153b0989d8	qapi: Move and rename qapi/qmp/dispatch.h to qapi/qmp-registry.h The general expectation is that header files should follow the same file/path naming scheme as the corresponding source file. There are various historical exceptions to this practice in QEMU, with one of the most notable being the include/qapi/qmp/ directory. include/qapi/qmp/dispatch.h corresponds mostly to qapi/qmp-registry.c. Move and rename it to include/qapi/qmp-registry.h. Now just qerror.h is left in include/qapi/qmp/. Since it's deprecated & (slowly) getting eliminated anyway, it isn't worth moving. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20241118151235.2665921-3-armbru@redhat.com>	2 years ago
Daniel P. Berrangé	407bc4bf90	qapi: Move include/qapi/qmp/ to include/qobject/ The general expectation is that header files should follow the same file/path naming scheme as the corresponding source file. There are various historical exceptions to this practice in QEMU, with one of the most notable being the include/qapi/qmp/ directory. Most of the headers there correspond to source files in qobject/. This patch corrects most of that inconsistency by creating include/qobject/ and moving the headers for qobject/ there. This also fixes MAINTAINERS for include/qapi/qmp/dispatch.h: scripts/get_maintainer.pl now reports "QAPI" instead of "No maintainers found". Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Acked-by: Halil Pasic <pasic@linux.ibm.com> #s390x Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20241118151235.2665921-2-armbru@redhat.com> [Rebased]	2 years ago
Markus Armbruster	7b29353fdd	qapi: Smarter camel_to_upper() to reduce need for 'prefix' camel_to_upper() converts its argument from camel case to upper case with '_' between words. Used for generated enumeration constant prefixes. When some of the words are spelled all caps, where exactly to insert '_' is guesswork. camel_to_upper()'s guesses are bad enough in places to make people override them with a 'prefix' in the schema. Rewrite it to guess better: 1. Insert '_' after a non-upper case character followed by an upper case character: OneTwo -> ONE_TWO One2Three -> ONE2_THREE 2. Insert '_' before the last upper case character followed by a non-upper case character: ACRONYMWord -> ACRONYM_Word Except at the beginning (as in OneTwo above), or when there is already one: AbCd -> AB_CD This changes the default enumeration constant prefix for a number of enums. Generated enumeration constants change only where the default is not overridden with 'prefix'. The following enumerations without a 'prefix' change: enum old camel_to_upper() new camel_to_upper() ------------------------------------------------------------------ DisplayGLMode DISPLAYGL_MODE DISPLAY_GL_MODE EbpfProgramID EBPF_PROGRAMID EBPF_PROGRAM_ID HmatLBDataType HMATLB_DATA_TYPE HMAT_LB_DATA_TYPE HmatLBMemoryHierarchy HMATLB_MEMORY_HIERARCHY HMAT_LB_MEMORY_HIERARCHY MultiFDCompression MULTIFD_COMPRESSION MULTI_FD_COMPRESSION OffAutoPCIBAR OFF_AUTOPCIBAR OFF_AUTO_PCIBAR QCryptoBlockFormat Q_CRYPTO_BLOCK_FORMAT QCRYPTO_BLOCK_FORMAT QCryptoBlockLUKSKeyslotState Q_CRYPTO_BLOCKLUKS_KEYSLOT_STATE QCRYPTO_BLOCK_LUKS_KEYSLOT_STATE QKeyCode Q_KEY_CODE QKEY_CODE XDbgBlockGraphNodeType X_DBG_BLOCK_GRAPH_NODE_TYPE XDBG_BLOCK_GRAPH_NODE_TYPE TestUnionEnumA TEST_UNION_ENUMA TEST_UNION_ENUM_A Add a 'prefix' so generated code doesn't change now. Subsequent commits will remove most of them again. Two will remain: MULTIFD_COMPRESSION, because migration code generally spells "multifd" that way, and Q_KEY_CODE, because that one is baked into subprojects/keycodemapdb/tools/keymap-gen. The following enumerations with a 'prefix' change so that the prefix is now superfluous: enum old camel_to_upper() new camel_to_upper() [equal to prefix] ------------------------------------------------------------------ BlkdebugIOType BLKDEBUGIO_TYPE BLKDEBUG_IO_TYPE QCryptoTLSCredsEndpoint Q_CRYPTOTLS_CREDS_ENDPOINT QCRYPTO_TLS_CREDS_ENDPOINT QCryptoSecretFormat Q_CRYPTO_SECRET_FORMAT QCRYPTO_SECRET_FORMAT QCryptoCipherMode Q_CRYPTO_CIPHER_MODE QCRYPTO_CIPHER_MODE QCryptodevBackendType Q_CRYPTODEV_BACKEND_TYPE QCRYPTODEV_BACKEND_TYPE QType [builtin] Q_TYPE QTYPE Drop these prefixes. The following enumerations with a 'prefix' change without making the 'prefix' superfluous: enum old camel_to_upper() new camel_to_upper() [equal to prefix] prefix ------------------------------------------------------------------ CpuS390Entitlement CPUS390_ENTITLEMENT CPU_S390_ENTITLEMENT S390_CPU_ENTITLEMENT CpuS390Polarization CPUS390_POLARIZATION CPU_S390_POLARIZATION S390_CPU_POLARIZATION CpuS390State CPUS390_STATE CPU_S390_STATE S390_CPU_STATE QAuthZListFormat Q_AUTHZ_LIST_FORMAT QAUTH_Z_LIST_FORMAT QAUTHZ_LIST_FORMAT QAuthZListPolicy Q_AUTHZ_LIST_POLICY QAUTH_Z_LIST_POLICY QAUTHZ_LIST_POLICY QCryptoAkCipherAlgorithm Q_CRYPTO_AK_CIPHER_ALGORITHM QCRYPTO_AK_CIPHER_ALGORITHM QCRYPTO_AKCIPHER_ALG QCryptoAkCipherKeyType Q_CRYPTO_AK_CIPHER_KEY_TYPE QCRYPTO_AK_CIPHER_KEY_TYPE QCRYPTO_AKCIPHER_KEY_TYPE QCryptoCipherAlgorithm Q_CRYPTO_CIPHER_ALGORITHM QCRYPTO_CIPHER_ALGORITHM QCRYPTO_CIPHER_ALG QCryptoHashAlgorithm Q_CRYPTO_HASH_ALGORITHM QCRYPTO_HASH_ALGORITHM QCRYPTO_HASH_ALG QCryptoIVGenAlgorithm Q_CRYPTOIV_GEN_ALGORITHM QCRYPTO_IV_GEN_ALGORITHM QCRYPTO_IVGEN_ALG QCryptoRSAPaddingAlgorithm Q_CRYPTORSA_PADDING_ALGORITHM QCRYPTO_RSA_PADDING_ALGORITHM QCRYPTO_RSA_PADDING_ALG QCryptodevBackendAlgType Q_CRYPTODEV_BACKEND_ALG_TYPE QCRYPTODEV_BACKEND_ALG_TYPE QCRYPTODEV_BACKEND_ALG QCryptodevBackendServiceType Q_CRYPTODEV_BACKEND_SERVICE_TYPE QCRYPTODEV_BACKEND_SERVICE_TYPE QCRYPTODEV_BACKEND_SERVICE Subsequent commits will tweak things to remove most of these prefixes. Only QAUTHZ_LIST_FORMAT and QAUTHZ_LIST_POLICY will remain. Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Message-ID: <20240904111836.3273842-2-armbru@redhat.com>	2 years ago
John Snow	3c5f6114d9	qapi: remove "Example" doc section Fully eliminate the "Example" sections in QAPI doc blocks now that they have all been converted to arbitrary rST syntax using the ".. qmp-example::" directive. Update tests to match. Migrating to the new syntax --------------------------- The old "Example:" or "Examples:" section syntax is now caught as an error, but "Example::" is stil permitted as explicit rST syntax for an un-lexed, generic preformatted text block. ('Example' is not special in this case, any sentence that ends with "::" will start an indented code block in rST.) Arbitrary rST for Examples is now possible, but it's strongly recommended that documentation authors use the ".. qmp-example::" directive for consistent visual formatting in rendered HTML docs. The ":title:" directive option may be used to add extra information into the title bar for the example. The ":annotated:" option can be used to write arbitrary rST instead, with nested "::" blocks applying QMP formatting where desired. Other choices available are ".. code-block:: QMP" which will not create an "Example:" box, or the short-form "::" code-block syntax which will not apply QMP highlighting when used outside of the qmp-example directive. Why? ---- This patch has several benefits: 1. Example sections can now be written more arbitrarily, mixing explanatory paragraphs and code blocks however desired. 2. Example sections can now use fully arbitrary rST. 3. All code blocks are now lexed and validated as QMP; increasing usability of the docs and ensuring validity of example snippets. (To some extent - This patch only gaurantees it lexes correctly, not that it's valid under the JSON or QMP grammars. It will catch most small mistakes, however.) 4. Each qmp-example can be titled or annotated independently without bypassing the QMP lexer/validator. (i.e. code blocks are now for code only, so we don't have to sacrifice exposition for having lexically valid examples.) NOTE: As with the "Notes" conversion (`d461c27973`), this patch (and the three preceding) may change the rendering order for Examples in the current generator. The forthcoming qapidoc rewrite will fix this by always generating documentation in source order. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20240717021312.606116-10-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
John Snow	65fa48c79f	qapi/parser: don't parse rST markup as section headers The double-colon synax is rST formatting that precedes a literal code block. We do not want to capture these as QAPI-specific sections. Coerce blocks that start with e.g. "Example::" to be parsed as untagged paragraphs instead of special tagged sections. Signed-off-by: John Snow <jsnow@redhat.com> Message-ID: <20240626222128.406106-14-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Indentation tweaked for consistency] Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
John Snow	d461c27973	qapi: convert "Note" sections to plain rST We do not need a dedicated section for notes. By eliminating a specially parsed section, these notes can be treated as normal rST paragraphs in the new QMP reference manual, and can be placed and styled much more flexibly. Convert all existing "Note" and "Notes" sections to pure rST. As part of the conversion, capitalize the first letter of each sentence and add trailing punctuation where appropriate to ensure notes look sensible and consistent in rendered HTML documentation. Markup is also re-aligned to the de-facto standard of 3 spaces for directives. Update docs/devel/qapi-code-gen.rst to reflect the new paradigm, and update the QAPI parser to prohibit "Note" sections while suggesting a new syntax. The exact formatting to use is a matter of taste, but a good candidate is simply: .. note:: lorem ipsum ... ... dolor sit amet ... ... consectetur adipiscing elit ... ... but there are other choices, too. The Sphinx readthedocs theme offers theming for the following forms (capitalization unimportant); all are adorned with a (!) symbol () in the title bar for rendered HTML docs. See https://sphinx-rtd-theme.readthedocs.io/en/stable/demo/demo.html#admonitions for examples of each directive/admonition in use. These are rendered in orange: .. Attention:: ... .. Caution:: ... .. WARNING:: ... These are rendered in red: .. DANGER:: ... .. Error:: ... These are rendered in green: .. Hint:: ... .. Important:: ... .. Tip:: ... These are rendered in blue: .. Note:: ... .. admonition:: custom title admonition body text This patch uses ".. note::" almost everywhere, with just two "caution" directives. Several instances of "Notes:" have been converted to merely ".. note::", or multiple ".. note::" where appropriate. ".. admonition:: notes" is used in a few places where we had an ordered list of multiple notes that would not make sense as standalone/separate admonitions. Two "Note:" following "Example:" have been turned into ordinary paragraphs within the example. NOTE: Because qapidoc.py does not attempt to preserve source ordering of sections, the conversion of Notes from a "tagged section" to an "untagged section" means that rendering order for some notes may change as a result of this patch. The forthcoming qapidoc.py rewrite strictly preserves source ordering in the rendered documentation, so this issue will be rectified in the new generator. Signed-off-by: John Snow <jsnow@redhat.com> Acked-by: Stefan Hajnoczi <stefanha@redhat.com> [for block*.json] Message-ID: <20240626222128.406106-11-jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> [Commit message clarified slightly, period added to one more note] Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
John Snow	83deda8769	qapi/parser: fix comment parsing immediately following a doc block If a comment immediately follows a doc block, the parser doesn't ignore that token appropriately. Fix that. e.g. > ## > # = Hello World! > ## > > # I'm a comment! will break the parser, because it does not properly ignore the comment token if it immediately follows a doc block. Fixes: `3d035cd2cc` (qapi: Rewrite doc comment parser) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240626222128.406106-7-jsnow@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
John Snow	939c639e1d	qapi/parser: preserve indentation in QAPIDoc sections Change get_doc_indented() to preserve indentation on all subsequent text lines, and create a compatibility dedent() function for qapidoc.py that removes indentation the same way get_doc_indented() did. This is being done for the benefit of a new qapidoc generator which requires that indentation in argument and features sections are preserved. Prior to this patch, a section like this: ``` @name: lorem ipsum dolor sit amet consectetur adipiscing elit ``` would have its body text be parsed into: ``` lorem ipsum dolor sit amet consectetur adipiscing elit ``` We want to preserve the indentation for even the first body line so that the entire block can be parsed directly as rST. This patch would now parse that segment into: ``` lorem ipsum dolor sit amet consectetur adipiscing elit ``` This is helpful for formatting arguments and features as field lists in rST, where the new generator will format this information as: ``` :arg type name: lorem ipsum dolor sit amet consectetur apidiscing elit ``` ...and can be formed by the simple concatenation of the field list construct and the body text. The indents help preserve the continuation of a block-level element, and further allow the use of additional rST block-level constructs such as code blocks, lists, and other such markup. This understandably breaks the existing qapidoc.py; so a new function is added there to dedent the text for compatibility. Once the new generator is merged, this function will not be needed any longer and can be dropped. I verified this patch changes absolutely nothing by comparing the md5sums of the QMP ref html pages both before and after the change, so it's certified inert. QAPI test output has been updated to reflect the new strategy of preserving indents for rST. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240626222128.406106-6-jsnow@redhat.com> [Lost commit message paragraph restored] Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
John Snow	f64e753149	qapi: linter fixups Fix minor irritants to pylint/flake8 et al. (Yes, these need to be guarded by the Python tests. That's a work in progress, a series that's quite likely to follow once I finish this Sphinx project. Please pardon the temporary irritation.) Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240626222128.406106-3-jsnow@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Markus Armbruster	285a8f209a	qapi: Simplify QAPISchemaVariants @tag_member For union types, the tag member is known only after .check(). We used to code this in a simple way: QAPISchemaVariants attribute .tag_member was None for union types until .check(). Since this complicated typing, recent commit "qapi/schema: fix typing for QAPISchemaVariants.tag_member" hid it behind a property. The previous commit lets us treat .tag_member just like the other attributes that become known only in .check(): declare, but don't initialize it in .__init__(). Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Markus Armbruster	8152bc7de6	qapi: Move conditional code from QAPISchemaVariants to its subtypes QAPISchemaVariants.check()'s code is almost entirely conditional on union vs. alternate type. Move the conditional code to QAPISchemaBranches.check() and QAPISchemaAlternatives.check(), where the conditions are always satisfied. Attribute QAPISchemaVariants.tag_name is now only used by QAPISchemaBranches. Move it there. Refactor the three types' .__init__() to make them a bit simpler. Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Markus Armbruster	e0a28f39b4	qapi: Rename QAPISchemaAlternateType.variants to .alternatives A previous commit narrowed the type of QAPISchemaAlternateType.variants from QAPISchemaVariants to QAPISchemaAlternatives. Rename it to .alternatives. Same for .__init__() parameter @variants. Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Markus Armbruster	3ff2a5a35c	qapi: Rename QAPISchemaObjectType.variants to .branches A previous commit narrowed the type of QAPISchemaObjectType.variants from QAPISchemaVariants to QAPISchemaBranches. Rename it to .branches. Same for .__init__() parameter @variants. Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Markus Armbruster	41d0ad1d04	qapi: Rename visitor parameter @variants to @alternatives A previous commit narrowed the type of .visit_alternate_type() parameter @variants from QAPISchemaVariants to QAPISchemaAlternatives. Rename it to @alternatives. One of them passes @alternatives to helper function gen_visit_alternate(). Rename its @variants parameter to @alternatives as well. Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Markus Armbruster	d1da8af897	qapi: Rename visitor parameter @variants to @branches The previous commit narrowed the type of .visit_object_type() parameter @variants from QAPISchemaVariants to QAPISchemaBranches. Rename it to @branches. Same for .visit_object_type_flat(). A few of these pass @branches to helper functions: QAPISchemaGenRSTVisitor.visit_object_type() to ._nodes_for_members() and ._nodes_for_variant_when(), and QAPISchemaGenVisitVisitor.visit_object_type() to gen_visit_object_members(). Rename the helpers' @variants parameters to @branches as well. Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Markus Armbruster	1d067e3953	qapi: New QAPISchemaBranches, QAPISchemaAlternatives QAPISchemaVariants represents either a union type's branches, or an alternate type's alternatives. Much of its code is conditional on which one it actually is. Create QAPISchemaBranches for branches, and QAPISchemaAlternatives for alternatives, both subtypes of QAPISchemaVariants. Replace QAPISchemaVariants by one of them where possible. Keep it only where we actually deal with either of them. QAPISchemaVariants.__init__() takes @tag_name and @tag_member, where exactly one must be None: @tag_name for alternatives, @tag_member for branches. Let QAPISchemaBranches.__init__() take just @tag_name, and QAPISchemaAlternatives.__init__() take just @tag_member. A later patch will move the conditional code to the subtypes. Signed-off-by: Markus Armbruster <armbru@redhat.com>	2 years ago
Markus Armbruster	060b5a9323	qapi: Dumb down QAPISchema.lookup_entity() QAPISchema.lookup_entity() takes an optional type argument, a subtype of QAPISchemaDefinition, and returns that type or None. Callers can use this to save themselves an isinstance() test. The only remaining user of this convenience feature is .lookup_type(). But we don't actually save anything anymore there: we still need the isinstance() to help mypy over the hump. Drop the .lookup_entity() argument, and adjust .lookup_type(). Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-26-armbru@redhat.com> Reviewed-by: John Snow <jsnow@redhat.com> [Commit message typo fixed]	2 years ago
Markus Armbruster	99e75d8c2a	qapi: Tighten check whether implicit object type already exists Entities with names starting with q_obj_ are implicit object types. Therefore, QAPISchema._make_implicit_object_type()'s .lookup_entity() can only return a QAPISchemaObjectType. Assert that. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-25-armbru@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: John Snow <jsnow@redhat.com>	2 years ago
John Snow	8d413dbd5d	qapi/schema: remove unnecessary asserts With strict typing enabled, these runtime statements aren't necessary anymore; we can prove them statically. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-24-armbru@redhat.com>	2 years ago
John Snow	aa1fed9f54	qapi/schema: turn on mypy strictness This patch can be rolled in with the previous one once the series is ready for merge, but for work-in-progress' sake, it's separate here. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-23-armbru@redhat.com>	2 years ago
John Snow	4ed3fe0822	qapi/schema: add type hints This patch only adds type hints, which aren't utilized at runtime and don't change the behavior of this module in any way. In a scant few locations, type hints are removed where no longer necessary due to inference power from typing all of the rest of creation; and any type hints that no longer need string quotes are changed. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-22-armbru@redhat.com>	2 years ago
John Snow	d5e2f3d03c	qapi/parser.py: assert member.info is present in connect_member Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-21-armbru@redhat.com>	2 years ago
John Snow	7c6e446476	qapi/parser: demote QAPIExpression to Dict[str, Any] Dict[str, object] is a stricter type, but with the way that code is currently arranged, it is infeasible to enforce this strictness. In particular, although expr.py's entire raison d'être is normalization and type-checking of QAPI Expressions, that type information is not "remembered" in any meaningful way by mypy because each individual expression is not downcast to a specific expression type that holds all the details of each expression's unique form. As a result, all of the code in schema.py that deals with actually creating type-safe specialized structures has no guarantee (myopically) that the data it is being passed is correct. There are two ways to solve this: (1) Re-assert that the incoming data is in the shape we expect it to be, or (2) Disable type checking for this data. (1) is appealing to my sense of strictness, but I gotta concede that it is asinine to re-check the shape of a QAPIExpression in schema.py when expr.py has just completed that work at length. The duplication of code and the nightmare thought of needing to update both locations if and when we change the shape of these structures makes me extremely reluctant to go down this route. (2) allows us the chance to miss updating types in the case that types are updated in expr.py, but it is an awful lot simpler and, importantly, gets us closer to type checking schema.py at all. Something is better than nothing, I'd argue. So, do the simpler dumber thing and worry about future strictness improvements later. Signed-off-by: John Snow <jsnow@redhat.com> Reviewed-by: Markus Armbruster <armbru@redhat.com> Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-ID: <20240315152301.3621858-20-armbru@redhat.com>	2 years ago

1 2 3 4 5 ...

516 Commits (master)