The include directive permits modular QAPI schemata, but the generated
code is monolithic all the same.  To permit generating modular code,
the front end needs to pass more information on inclusions to the back
ends.  The commit before last added the necessary information to the
parse tree.  This commit adds it to the intermediate representation
and its QAPISchemaVisitor.  A later commit will use this to to
generate modular code.

New entity QAPISchemaInclude represents inclusions.  Call new visitor
method visit_include() for it, so visitors can see the sub-modules a
module includes.

Note that unlike other entities, QAPISchemaInclude has no name, and is
therefore not added to entity_dict.

New QAPISchemaEntity attribute @module names the entity's source file.
Call new visitor method visit_module() when it changes during a visit,
so visitors can keep track of the module being visited.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-18-armbru@redhat.com>
[eblake: avoid accidental deletion of self._predefining]
Signed-off-by: NEric Blake <eblake@redhat.com>

The generators' conversion to visitors (merge commit 9e72681d)
changed the processing order of entities from source order to
alphabetical order.  The next commit needs source order, so change it
back.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-17-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

The parse tree is a list of expressions.  Except include expressions
currently get replaced by the included file's parse tree.

Instead of throwing away the include expression, keep it with the file
name expanded so you don't have to track the including file's
directory to make sense of it.

A future commit will put this include expression to use.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-16-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
[eblake: fix check of expr after assignment]
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-15-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-14-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-13-armbru@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Error messages print absolute file names of included files even if the
user gave a relative one on the command line:

    $ PYTHONPATH=scripts python -B tests/qapi-schema/test-qapi.py tests/qapi-schema/include-cycle.json
    In file included from tests/qapi-schema/include-cycle.json:1:
    In file included from /work/armbru/qemu/tests/qapi-schema/include-cycle-b.json:1:
    /work/armbru/qemu/tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json

Improve this to

    In file included from tests/qapi-schema/include-cycle.json:1:
    In file included from tests/qapi-schema/include-cycle-b.json:1:
    tests/qapi-schema/include-cycle-c.json:1: Inclusion loop for include-cycle.json

The error message when an include file can't be opened prints the
include directive's file name, which is relative to the including
file.  Change this to print the file name relative to the working
directory.  Visible in tests/qapi-schema/include-no-file.err.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-12-armbru@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

A massive number of objects depends on QAPI-generated headers.  In my
"build everything" tree, it's roughly 4800 out of 5100.  This is
particularly annoying when only some of the generated files change,
say for a doc fix.

Improve qapi-gen.py to touch its output files only if they actually
change.  Rebuild time for a QAPI doc fix drops from many minutes to a
few seconds.  Rebuilds get faster for certain code changes, too.  For
instance, adding a simple QMP event now recompiles less than 200
instead of 4800 objects.  But adding a QAPI type is as bad as ever;
we've clearly got more work to do.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20180211093607.27351-11-armbru@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
[eblake: fix octal constant for python3]
Signed-off-by: NEric Blake <eblake@redhat.com>

argparse is nicer to use than getopt, and gives us --help almost for
free.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-10-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
[eblake: Fix --output-dir editing accident]
Signed-off-by: NEric Blake <eblake@redhat.com>

Whenever qapi-schema.json changes, we run six programs eleven times to
update eleven files.  Similar for qga/qapi-schema.json.  This is
silly.  Replace the six programs by a single program that spits out
all eleven files.

The programs become modules in new Python package qapi, along with the
helper library.  This requires moving them to scripts/qapi/.  While
moving them, consistently drop executable mode bits.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-9-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
[eblake: move change to one-line 'blurb' earlier in series, mention mode
bit change as intentional, update qapi-code-gen.txt to match actual
generated events.c file]
Signed-off-by: NEric Blake <eblake@redhat.com>

The next commit will introduce a common driver program for all
generators.  The generators need to be modules for that.  qapi2texi.py
already is.  Make the other generators follow suit.

The changes are actually trivial.  Obvious in the diffs once you view
them with whitespace changes ignored.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-8-armbru@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
[eblake: minor tweak to keep 'blurb' one line]
Signed-off-by: NEric Blake <eblake@redhat.com>

In preparation of the next commit, which will turn the generators into
modules.  These global variables will become local to main() then.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-7-armbru@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

These classes encapsulate accumulating and writing output.

Convert C code generation to QAPIGenC and QAPIGenH.  The conversion is
rather shallow: most of the output accumulation is not converted.
Left for later.

The indentation machinery uses a single global variable indent_level,
even though we generally interleave creation of a .c and its .h.  It
should become instance variable of QAPIGenC.  Also left for later.

Documentation generation isn't converted, and QAPIGenDoc isn't used.
This will change shortly.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-6-armbru@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
[eblake: fix nits spotted by Michael]
Signed-off-by: NEric Blake <eblake@redhat.com>

Rename the variable holding the QAPISchemaGenFOOVisitor from gen to
vis, to avoid confusion in the next commit.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-5-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Each generator carries a copyright notice for the generator itself,
and another one for the files it generates.  Only the former have been
updated along the way, the latter have not, and are all out of date.

Fix by copying the generator's copyright notice to the generated files
instead.  Note that the fix doesn't copy the "Authors:" part; the
generated files' outdated Authors list goes away without replacement.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-4-armbru@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
[eblake: Flatten each 'blurb' to one line]
Signed-off-by: NEric Blake <eblake@redhat.com>

Every generator has separate boilerplate for .h and .c, and their
differences are boring.  All of them repeat the license note.

Reduce the repetition as follows.  Move common text like the license
note to common open_output(), next to the existing common text there.
For each generator, replace the two separate descriptions by a single
one.

While there, emit an "automatically generated" note into generated
documentation, too.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-3-armbru@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20180211093607.27351-2-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

tricore patches

# gpg: Signature made Fri 02 Mar 2018 10:59:26 GMT
# gpg:                using RSA key 0AD2C6396B69CA14
# gpg: Good signature from "Bastian Koppelmann <kbastian@mail.uni-paderborn.de>"
# Primary key fingerprint: 6E63 6A7E 83F2 DD0C FA6E  6E37 0AD2 C639 6B69 CA14

* remotes/bkoppelmann/tags/pull-tricore-2018-03-02:
  tricore: renamed masking of PIE
  tricore: renamed masking of IE
  tricore: added CORE_ID
  tricore: added some missing cpu instructions
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

target-arm queue:
 * implement FCMA and RDM v8.1 and v8.3 instructions
 * enable Cortex-M33 v8M core, and provide new mps2-an505 board model
   that uses it
 * decodetree: Propagate return value from translate subroutines
 * xlnx-zynqmp: Implement the RTC device

# gpg: Signature made Fri 02 Mar 2018 11:05:40 GMT
# gpg:                using RSA key 3C2525ED14360CDE
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>"
# gpg:                 aka "Peter Maydell <pmaydell@gmail.com>"
# gpg:                 aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>"
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83  15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-target-arm-20180302: (39 commits)
  target/arm: Enable ARM_FEATURE_V8_FCMA
  target/arm: Decode t32 simd 3reg and 2reg_scalar extension
  target/arm: Decode aa32 armv8.3 2-reg-index
  target/arm: Decode aa32 armv8.3 3-same
  target/arm: Decode aa64 armv8.3 fcmla
  target/arm: Decode aa64 armv8.3 fcadd
  target/arm: Add ARM_FEATURE_V8_FCMA
  target/arm: Enable ARM_FEATURE_V8_RDM
  target/arm: Decode aa32 armv8.1 two reg and a scalar
  target/arm: Decode aa32 armv8.1 three same
  target/arm: Decode aa64 armv8.1 scalar/vector x indexed element
  target/arm: Decode aa64 armv8.1 three same extra
  target/arm: Decode aa64 armv8.1 scalar three same extra
  target/arm: Refactor disas_simd_indexed size checks
  target/arm: Refactor disas_simd_indexed decode
  target/arm: Add ARM_FEATURE_V8_RDM
  mps2-an505: New board model: MPS2 with AN505 Cortex-M33 FPGA image
  hw/arm/iotkit: Model Arm IOT Kit
  hw/misc/iotkit-secctl: Add remaining simple registers
  hw/misc/iotkit-secctl: Add handling for PPCs
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

nbd patches for 2018-03-01

- Eric Blake: nbd: Honor server's advertised minimum block size
- Vladimir Sementsov-Ogievskiy: partial: nbd block status base:allocation

# gpg: Signature made Thu 01 Mar 2018 21:01:22 GMT
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2018-03-01:
  nbd/client: fix error messages in nbd_handle_reply_err
  nbd: BLOCK_STATUS constants
  nbd: change indenting in nbd.h
  nbd: Honor server's advertised minimum block size
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Enable it for the "any" CPU used by *-linux-user.
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 20180228193125.20577-17-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Happily, the bits are in the same places compared to a32.
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-16-richard.henderson@linaro.org
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-15-richard.henderson@linaro.org
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 20180228193125.20577-14-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-13-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
[PMM: renamed e1/e2/e3/e4 to use the same naming as the version
 of the pseudocode in the Arm ARM]
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>

Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-12-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Not enabled anywhere yet.
Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-11-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Enable it for the "any" CPU used by *-linux-user.
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 20180228193125.20577-10-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-9-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-8-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-7-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-6-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-5-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

The integer size check was already outside of the opcode switch;
move the floating-point size check outside as well.  Unify the
size vs index adjustment between fp and integer paths.
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 20180228193125.20577-4-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Include the U bit in the switches rather than testing separately.
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 20180228193125.20577-3-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Not enabled anywhere yet.
Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180228193125.20577-2-richard.henderson@linaro.org
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Define a new board model for the MPS2 with an AN505 FPGA image
containing a Cortex-M33. Since the FPGA images for TrustZone
cores (AN505, and the similar AN519 for Cortex-M23) have a
significantly different layout of devices to the non-TrustZone
images, we use a new source file rather than shoehorning them
into the existing mps2.c.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180220180325.29818-20-peter.maydell@linaro.org

Model the Arm IoT Kit documented in
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ecm0601256/index.html

The Arm IoT Kit is a subsystem which includes a CPU and some devices,
and is intended be extended by adding extra devices to form a
complete system.  It is used in the MPS2 board's AN505 image for the
Cortex-M33.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180220180325.29818-19-peter.maydell@linaro.org

Add remaining easy registers to iotkit-secctl:
 * NSCCFG just routes its two bits out to external GPIO lines
 * BRGINSTAT/BRGINTCLR/BRGINTEN can be dummies, because QEMU's
   bus fabric can never report errors
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Message-id: 20180220180325.29818-18-peter.maydell@linaro.org

The IoTKit Security Controller includes various registers
that expose to software the controls for the Peripheral
Protection Controllers in the system. Implement these.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Message-id: 20180220180325.29818-17-peter.maydell@linaro.org