Previous commits demonstrated that the generator had several
flaws with less-than-perfect unions:
- a simple union that listed the same branch twice (or two variant
names that map to the same C enumerator, including the implicit
MAX sentinel) ended up generating invalid C code
- an anonymous union that listed two branches with the same qtype
ended up generating invalid C code
- the generator crashed on anonymous union attempts to use an
array type
- the generator was silently ignoring a base type for anonymous
unions
- the generator allowed unknown types or nested anonymous unions
as a branch in an anonymous union
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

None of the existing QMP or QGA interfaces uses a union with a
base type but no discriminator; it is easier to avoid this in the
generator to save room for other future extensions more likely to
be useful.  An earlier commit added a union-base-no-discriminator
test to ensure that we eventually give a decent error message;
likewise, removing UserDefUnion outright is okay, because we moved
all the tests we wish to keep into the tests of the simple union
UserDefNativeListUnion in the previous commit.  Now is the time to
actually forbid simple union with base, and remove the last
vestiges from the testsuite.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

The previous commit demonstrated that the generator had several
flaws with less-than-perfect enums:
- an enum that listed the same string twice (or two variant
strings that map to the same C enumerator) ended up generating
an invalid C enum
- because the generator adds a _MAX terminator to each enum,
the use of an enum member 'max' can also cause this clash
- if an enum omits 'data', the generator left a python stack
trace rather than a graceful message
- an enum that used a non-array 'data' was silently accepted by
the parser
- an enum that used non-string members in the 'data' member
was silently accepted by the parser

Add check_enum to cover these situations, and update testcases
to match.  While valid .json files won't trigger any of these
cases, we might as well be nicer to developers that make a typo
while trying to add new QAPI code.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Python 2 and Python 3 have a wild history of whether strings
default to ascii or unicode, where Python 3 requires checking
isinstance(foo, basestr) to cover all strings, but where that
code is not portable to Python 2.  It's simpler to just state
that we don't care about Unicode strings, and to just always
use the simpler isinstance(foo, str) everywhere.

I'm no python expert, so I'm basing it on this conversation:
https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg05278.htmlSigned-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

We were missing the 'size' builtin type (which means that QAPI using
[ 'size' ] would fail to compile).
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

There was some redundancy between builtin_types[] and
builtin_type_qtypes{}.  Merge them into one.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

There is one instance of any() in qapi.py that breaks builds on older
distros that ship Python 2.4 (like RHEL5):

  GEN   qmp-commands.h
Traceback (most recent call last):
  File "build/scripts/qapi-commands.py", line 445, in ?
    exprs = parse_schema(input_file)
  File "build/scripts/qapi.py", line 329, in parse_schema
    schema = QAPISchema(open(input_file, "r"))
  File "build/scripts/qapi.py", line 110, in __init__
    if any(include_path == elem[1]
NameError: global name 'any' is not defined
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NBenoît Canet <benoit.canet@nodalink.com>

This patch improve docs and uses c_type(argentry, is_param=True)
in script.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

qapi-event.py will parse the schema and generate qapi-event.c, then
the API in qapi-event.c can be used to handle events in qemu code.
All API have prefix "qapi_event".

The script mainly includes two parts: generate API for each event
define, generate an enum type for all defined events.

Since in some cases the real emit behavior may change, for example,
qemu-img would not send a event, a callback layer is used to
control the behavior. As a result, the stubs at compile time
can be saved, the binding of block layer code and monitor code
will become looser.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

We always generate a space between type and identifier in parameter
and variable declarations, even when idiomatic C style doesn't have
a space there.  Suppress it.
Signed-off-by: NAmos Kong <akong@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

It's ugly to add const prefix for parameter type by an if statement
outside c_type(). This patch adds a parameter to do it.
Signed-off-by: NAmos Kong <akong@redhat.com>
Suggested-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

The Python "except Foo as x" syntax was only introduced in
Python 2.6, but we aim to support Python 2.4 and later.
Use the old-style "except Foo, x" syntax instead, thus
fixing configure/compile on systems with older Python.
Reported-by: NPeter Maydell <peter.maydell@linaro.org>
Tested-by: NAndreas Färber <andreas.faerber@web.de>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

The purpose of this change is to help create a json file containing
common definitions; each bit of generated C code must be emitted
only one time.

A second history global to all QAPISchema instances has been added
to detect when a file is included more than one time and skip these
includes.
It does not act as a stack and the changes made to it by the
__init__ function are propagated back to the caller so it's really
a global state.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

The primitive uses JSON syntax, and include paths are relative to the file using the directive:

  { 'include': 'path/to/file.json' }
Signed-off-by: NLluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Use an explicit input file on the command-line instead of reading from standard
input.

It also outputs the proper file name when there's an error.
Signed-off-by: NLluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Now "enum AIOContext" will generate AIO_CONTEXT instead of A_I_O_CONTEXT,
"X86CPU" will generate X86_CPU instead of X86_C_P_U.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Since enum based discriminators provide better type-safety and
ensure that future qapi additions do not forget to adjust dependent
unions, forbid using string as discriminator from now on.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

By default, any union will automatically generate a enum type as
"[UnionName]Kind" in C code, and it is duplicated when the discriminator
is specified as a pre-defined enum type in schema. After this patch,
the pre-defined enum type will be really used as the switch case
condition in generated C code, if discriminator is an enum field.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Prior to this patch, qapi-visit.py used custom code to generate enum
names used for handling a qapi union. Fix it to instead reuse common
code, with identical generated results, and allowing future updates to
generation to only need to touch one place.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Later both qapi-types.py and qapi-visit.py need a common function
for enum name generation.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Since line info is remembered as QAPISchema.line now, this patch
uses it as additional info for every expr in QAPISchema inside qapi.py,
then improves error message with it in checking of exprs.

For common union the patch will check whether base is a valid complex
type if specified. For flat union it will check whether base presents,
whether discriminator is found in base, whether the key of every branch
is correct when discriminator is an enum type.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Before this patch, 'QAPISchemaError' scans whole input until 'pos'
to get error line number. After this patch, the scan is avoided since
line number is remembered in schema parsing. This patch also benefits
other error report functions, which would be introduced later.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

It is bad that same key was specified twice, especially when a union has
two branches with same condition. This patch can prevent it.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Later other scripts will need to check the enum values.
Signed-off-by: NWenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

The scripts carry this copyright notice:

    # This work is licensed under the terms of the GNU GPLv2.
    # See the COPYING.LIB file in the top-level directory.

The sentences contradict each other, as COPYING.LIB contains the LGPL
2.1.  Michael Roth says this was a simple pasto, and he meant to refer
COPYING.  Let's fix that.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Using "errno" directly as an identifier results in various syntax
errors; therefore it should be added to the list of polluted words.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Add c++ keywords to avoid errors in compiling with c++ compiler.
This also renames class member of PciDeviceInfo to q_class.
Signed-off-by: NTomoki Sekiyama <tomoki.sekiyama@hds.com>
Reviewed-by: NLaszlo Ersek <lersek@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NMichael Roth <mdroth@linux.vnet.ibm.com>

The Python "except Foo as x" syntax was only introduced in
Python 2.6, but we aim to support Python 2.4 and later.
Use the old-style "except Foo, x" syntax instead, thus
fixing configure/compile on systems with older Python.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-10-git-send-email-armbru@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-9-git-send-email-armbru@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Report syntax error instead of crashing.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-8-git-send-email-armbru@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Fixes at least the following parser bugs:

* accepts any token in place of a colon

* treats comma as optional

* crashes when closing braces or brackets are missing
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-7-git-send-email-armbru@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-6-git-send-email-armbru@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-5-git-send-email-armbru@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The parser has a rather unorthodox structure:

    Until EOF:

        Read a section:

            Generator function get_expr() yields one section after the
            other, as a string.  An unindented, non-empty line that
            isn't a comment starts a new section.

        Lexing:

            Split section into a list of tokens (strings), with help
            of generator function tokenize().

        Parsing:

            Parse the first expression from the list of tokens, with
            parse(), throw away any remaining tokens.

            In parse_schema(): record value of an enum, union or
            struct key (if any) in the appropriate global table,
            append expression to the list of expressions.

    Return list of expressions.

Known issues:

(1) Indentation is significant, unlike in real JSON.

(2) Neither lexer nor parser have any idea of source positions.  Error
    reporting is hard, let's go shopping.

(3) The one error we bother to detect, we "report" via raise.

(4) The lexer silently ignores invalid characters.

(5) If everything in a section gets ignored, the parser crashes.

(6) The lexer treats a string containing a structural character exactly
    like the structural character.

(7) Tokens trailing the first expression in a section are silently
    ignored.

(8) The parser accepts any token in place of a colon.

(9) The parser treats comma as optional.

(10) parse() crashes on unexpected EOF.

(11) parse_schema() crashes when a section's expression isn't a JSON
    object.

Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others.  Generated source files remain unchanged.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

The discriminator for anonymous unions is the data type. This allows to
have a union type that allows both of these:

    { 'file': 'my_existing_block_device_id' }
    { 'file': { 'filename': '/tmp/mydisk.qcow2', 'read-only': true } }

Unions like this are specified in the schema with an empty dict as
discriminator. For this example you could take:

    { 'union': 'BlockRef',
      'discriminator': {},
      'data': { 'definition': 'BlockOptions',
                'reference': 'str' } }
    { 'type': 'ExampleObject',
      'data: { 'file': 'BlockRef' } }
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

If 'data' for a command definition isn't a dict, but a string, it is
taken as a (struct) type name and the fields of this struct are directly
used as parameters.

This is useful for transactionable commands that can use the same type
definition for both the transaction action and the arguments of the
standalone command.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

The code that interprets the read JSON expression and appends types to
the respective global variables was duplicated. We can avoid that by
splitting off the part that reads from the file.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>

Teach type generators about native types so they can generate the
appropriate linked list types.
Signed-off-by: NMichael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: NLaszlo Ersek <lersek@redhat.com>
Reviewed-by: NAmos Kong <akong@redhat.com>
Signed-off-by: NLuiz Capitulino <lcapitulino@redhat.com>