What's more meta than using qapi to define qapi? :)

Convert QType into a full-fledged[*] builtin qapi enum type, so
that a subsequent patch can then use it as the discriminator
type of qapi alternate types.  Fortunately, the judicious use of
'prefix' in the qapi definition avoids churn to the spelling of
the enum constants.

To avoid circular definitions, we have to flip the order of
inclusion between "qobject.h" vs. "qapi-types.h".  Back in commit
28770e05, we had the latter include the former, so that we could
use 'QObject *' for our implementation of 'any'.  But that usage
also works with only a forward declaration, whereas the
definition of QObject requires QType to be a complete type.

[*] The type has to be builtin, rather than declared in
qapi/common.json, because we want to use it for alternates even
when common.json is not included. But since it is the first
builtin enum type, we have to add special cases to qapi-types
and qapi-visit to only emit definitions once, even when two
qapi files are being compiled into the same binary (the way we
already handled builtin list types like 'intList').  We may
need to revisit how multiple qapi files share common types,
but that's a project for another day.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-4-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

The name QType matches our CODING_STYLE conventions for type names
in CamelCase.  It also matches the fact that we are already naming
all the enum members with a prefix of QTYPE, not QTYPE_CODE.  And
doing the rename will also make it easier for the next patch to use
QAPI for providing the enum, which also wants CamelCase type names.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1449033659-25497-3-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

When munging enum values, the fact that we were passing the entire
prefix + value through camel_to_upper() meant that enum values
spelled with CamelCase could be turned into CAMEL_CASE.  However,
this provides a potential collision (both OneTwo and One-Two would
munge into ONE_TWO) for enum types, when the same two names are
valid side-by-side as QAPI member names.  By changing the generation
of enum constants to always be prefix + '_' + c_name(value,
False).upper(), and ensuring that there are no case collisions (in
the next patches), we no longer have to worry about names that
would be distinct as QAPI members but collide as variant tag names,
without having to think about what munging the heuristics in
camel_to_upper() will actually perform on an enum value.

Making the change will affect enums that did not follow coding
conventions, using 'CamelCase' rather than desired 'lower-case'.

Thankfully, there are only two culprits: InputButton and ErrorClass.
We already tweaked ErrorClass to make it an alias of QapiErrorClass,
where only the alias needs changing rather than the whole tree.  So
the bulk of this change is modifying INPUT_BUTTON_WHEEL_UP to the
new INPUT_BUTTON_WHEELUP (and likewise for WHEELDOWN).  That part
of this commit may later need reverting if we rename the enum
constants from 'WheelUp' to 'wheel-up' as part of moving
x-input-send-event to a stable interface; but at least we have
documentation bread crumbs in place to remind us (commit 513e7cdb),
and it matches the fact that SDL constants are also spelled
SDL_BUTTON_WHEELUP.

Suggested by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-27-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Now that we no longer collide with an implicit _MAX enum member,
we no longer need to reject it in the ad hoc parser, and can
remove several tests that are no longer needed.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-24-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Now that we guarantee the user doesn't have any enum values
beginning with a single underscore, we can use that for our
own purposes.  Renaming ENUM_MAX to ENUM__MAX makes it obvious
that the sentinel is generated.

This patch was mostly generated by applying a temporary patch:

|diff --git a/scripts/qapi.py b/scripts/qapi.py
|index e6d014b..b862ec9 100644
|--- a/scripts/qapi.py
|+++ b/scripts/qapi.py
|@@ -1570,6 +1570,7 @@ const char *const %(c_name)s_lookup[] = {
|     max_index = c_enum_const(name, 'MAX', prefix)
|     ret += mcgen('''
|     [%(max_index)s] = NULL,
|+// %(max_index)s
| };
| ''',
|                max_index=max_index)

then running:

$ cat qapi-{types,event}.c tests/test-qapi-types.c |
    sed -n 's,^// \(.*\)MAX,s|\1MAX|\1_MAX|g,p' > list
$ git grep -l _MAX | xargs sed -i -f list

The only things not generated are the changes in scripts/qapi.py.

Rejecting enum members named 'MAX' is now useless, and will be dropped
in the next patch.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-23-git-send-email-eblake@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
[Rebased to current master, commit message tweaked]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

We already documented that qapi names should match specific
patterns (such as starting with a letter unless it was an enum
value or a downstream extension).  Tighten that from a suggestion
into a hard requirement, which frees up names beginning with a
single underscore for qapi internal usage.

The tighter regex doesn't forbid everything insane that a user
could provide (for example, a user could name a type 'Foo-lookup'
to collide with the generated 'Foo_lookup[]' for an enum 'Foo'),
but does a good job at protecting the most obvious uses, and
also happens to reserve single leading underscore for later use.

The handling of enum values starting with a digit is tricky:
commit 9fb081e0 introduced a subtle bug by using c_name() on
a munged value, which would allow an enum to include the
member 'q-int' in spite of our reservation.  Furthermore,
munging with a leading '_' would fail our tighter regex.  So
fix it by only munging for leading digits (which are never
ticklish in c_name()) and by using a different prefix (I
picked 'D', although any letter should do).

Add new tests, reserved-member-underscore and reserved-enum-q,
to demonstrate the tighter checking.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-22-git-send-email-eblake@redhat.com>
Message-Id: <1447883135-18020-1-git-send-email-eblake@redhat.com>
[Eric's fixup squashed in]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

The method c_name() is supposed to do two different actions: munge
'-' into '_', and add a 'q_' prefix to ticklish names.  But it did
these steps out of order, making it possible to submit input that
is not ticklish until after munging, where the output then lacked
the desired prefix.

The failure is exposed easily if you have a compiler that recognizes
C11 keywords, and try to name a member '_Thread-local', as it would
result in trying to compile the declaration 'uint64_t _Thread_local;'
which is not valid.  However, this name violates our conventions
(ultimately, want to enforce that no qapi names start with single
underscore), so the test is slightly weaker by instead testing
'wchar-t'; the declaration 'uint64_t wchar_t;' is valid in C (where
wchar_t is only a typedef) but would fail with a C++ compiler (where
it is a keyword).

Fix things by reversing the order of actions within c_name().
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-18-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Detect attempts to declare two object members that would result
in the same C member name, by keying the 'seen' dictionary off
of the C name rather than the qapi name.  It also requires passing
info through the check_clash() methods.

This addresses a TODO and fixes the previously-broken
args-name-clash test.  The resulting error message demonstrates
the utility of the .describe() method added previously.  No change
to generated code.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-17-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Future commits will migrate semantic checking away from parsing
and over to the various QAPISchema*.check() methods.  But to
report an error message about an incorrect semantic use of a
member of an object type, it helps to know which type, command,
or event owns the member.  In particular, when a member is
inherited from a base type, it is desirable to associate the
member name with the base type (and not the type calling
member.check()).

Rather than packing additional information into the seen array
passed to each member.check() (as in seen[m.name] = {'member':m,
'owner':type}), it is easier to have each member track the name
of the owner type in the first place (keeping things simpler
with the existing seen[m.name] = m).  The new member.owner field
is set via a new set_owner() method, called when registering
the members and variants arrays with an object or variant type.
Track only a name, and not the actual type object, to avoid
creating a circular python reference chain.

Note that Variants.set_owner() method does not set the owner
for the tag_member field; this field is set earlier either as
part of an object's non-variant members, or explicitly by
alternates.

The source information is intended for human consumption in
error messages, and a new describe() method is added to access
the resulting information.  For example, given the qapi:
  { 'command': 'foo', 'data': { 'string': 'str' } }
an implementation of visit_command() that calls
  arg_type.members[0].describe()
will see "'string' (parameter of foo)".

To make the human-readable name of implicit types work without
duplicating efforts, the describe() method has to reverse the
name of implicit types, via the helper _pretty_owner().

No change to generated code.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-16-git-send-email-eblake@redhat.com>
[Incorrect & unused -wrapper case in _pretty_owner() dropped]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Now that branches are in a separate C namespace, we can remove
the restrictions in the parser that claim a branch name would
collide with QMP, and delete the negative tests that are no
longer problematic.  A separate patch can then add positive
tests to qapi-schema-test to test that any corner cases will
compile correctly.

This reverts the scripts/qapi.py portion of commit 7b2a5c2f,
now that the assertions that it plugged are no longer possible.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-15-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Checking that a given QAPISchemaObjectTypeVariant.name is a
member of the corresponding QAPISchemaEnumType of the owning
QAPISchemaObjectTypeVariants.tag_member ensures that there are
no collisions in the generated C union for those tag values
(since the enum itself should have no collisions).

However, ever since its introduction in f51d8c3d, this was the
only additional action of of Variant.check(), beyond calling
the superclass Member.check().  This forces a difference in
.check() signatures, just to pass the enum type down.

Simplify things by instead doing the tag name check as part of
Variants.check(), at which point we can rely on inheritance
instead of overriding Variant.check().
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-14-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Consolidate two common sequences of clash detection into a
new QAPISchemaObjectType.check_clash() helper method.

No change to generated code.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-13-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Right now, our ad hoc parser ensures that we cannot have a
flat union that introduces any members that would clash with
non-variant members inherited from the union's base type (see
flat-union-clash-member.json).  We want QAPISchemaObjectType.check()
to make the same check, so we can later reduce some of the ad
hoc checks.

We already have a map 'seen' of all non-variant members. We
still need to check for collisions between each variant type's
members and the non-variant ones.

To know the variant type's members, we need to call
variant.type.check().  This also detects when a type contains
itself in a variant, exactly like the existing base.check()
detects when a type contains itself as a base.  (Except that
we currently forbid anything but a struct as the type of a
variant, so we can't actually trigger this type of loop yet.)

Slight complication: an alternate's variant can have arbitrary
type, but only an object type's check() may be called outside
QAPISchema.check(). We could either skip the call for variants
of alternates, or skip it for non-object types.  For now, do
the latter, because it's easier.

Then we call each variant member's check_clash() with the
appropriate 'seen' map.  Since members of different variants
can't clash, we have to clone a fresh seen for each variant.
Wrap this in a new helper method
QAPISchemaObjectTypeVariants.check_clash().

Note that cloning 'seen' inside .check_clash() resembles
the one we just removed from .check() in 'qapi: Drop
obsolete tag value collision assertions'; the difference here is
that we are now checking for clashes among the qapi members of
the variant type, rather than for a single clash with the variant
tag name itself.

Note that, by construction, collisions can't actually happen for
simple unions: each variant's type is a wrapper with a single
member 'data', which will never collide with the only non-variant
member 'type'.

For alternates, there's nothing for a variant object type's
members to clash with, and therefore no need to call the new
variants.check_clash().

No change to generated code.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-12-git-send-email-eblake@redhat.com>
[Commit message tweaked]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Reduce the ugly flat union / simple union conditional by doing just
the essential work here, namely setting self.tag_member.
Move the rest to callers.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-7-git-send-email-armbru@redhat.com>
[rebase to earlier changes that moved tag_member.check() of
alternate types, and tweak commit title and wording]
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-11-git-send-email-eblake@redhat.com>

While there, stick in a TODO change key of seen from QAPI name to C
name.  Can't do it right away, because it would fail the assertion for
tests/qapi-schema/args-has-clash.json.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-6-git-send-email-armbru@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-10-git-send-email-eblake@redhat.com>

We can use seen.values() instead if we make it an OrderedDict.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-5-git-send-email-armbru@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-9-git-send-email-eblake@redhat.com>

This hunk

    @@ -964,6 +965,7 @@ class QAPISchemaObjectType(QAPISchemaType):
                 members = []
             seen = {}
             for m in members:
    +            assert c_name(m.name) not in seen
                 seen[m.name] = m
             for m in self.local_members:
                 m.check(schema, members, seen)

is plainly broken.

Asserting the members inherited from base don't clash is somewhat
redundant, because self.base.check() just checked that.  But it
doesn't hurt.

The idea to use c_name(m.name) instead of m.name for collision
checking is sound, because we need to catch clashes between the m.name
and between the c_name(m.name), and when two m.name clash, then their
c_name() also clash.

However, using c_name(m.name) instead of m.name in one of several
places doesn't work.  See the very next line.

Keep the assertion, but drop the c_name() for now.  A future commit
will bring it back.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-4-git-send-email-armbru@redhat.com>
[change TABs in commit message to space]
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-8-git-send-email-eblake@redhat.com>

QAPISchemaObjectTypeVariants.check() parameter members and
QAPISchemaObjectTypeVariant.check() parameter seen are no longer used,
drop them.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-3-git-send-email-armbru@redhat.com>
[rebase to earlier changes that moved tag_member.check() of
alternate types]
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-7-git-send-email-eblake@redhat.com>

QAPISchemaObjectTypeMember.check() currently does four things:

1. Compute self.type

2. Accumulate members in all_members

   Only one caller cares: QAPISchemaObjectType.check() uses it to
   compute self.members.  The other callers pass a throw-away
   accumulator.

3. Accumulate a map from names to members in seen

   Only one caller cares: QAPISchemaObjectType.check() uses it to
   compute its local variable seen, for self.variants.check(), which
   uses it to compute self.variants.tag_member from
   self.variants.tag_name.  The other callers pass a throw-away
   accumulator.

4. Check for collisions

   This piggybacks on 3: before adding a new entry, we assert it's new.

   Only one caller cares: QAPISchemaObjectType.check() uses it to
   assert non-variant members don't clash.

Simplify QAPISchemaObjectType.check(): move 2.-4. to
QAPISchemaObjectType.check(), and drop parameters all_members and
seen.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-2-git-send-email-armbru@redhat.com>
[rebase to earlier changes that moved tag_member.check() of
alternate types, commit message typo fix]
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-6-git-send-email-eblake@redhat.com>

Union tag values can't clash with member names in generated C anymore
since commit e4ba22b3, but QAPISchemaObjectTypeVariants.check() still
asserts they don't.  Drop it.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <1446559499-26984-1-git-send-email-armbru@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-5-git-send-email-eblake@redhat.com>

We were previously creating all unions with an empty list for
local_members.  However, it will make it easier to unify struct
and union generation if we include the generated tag member in
local_members.  That way, we can have a common code pattern:
visit the base (if any), visit the local members (if any), visit
the variants (if any).  The local_members of a flat union
remains empty (because the discriminator is already visited as
part of the base).  Then, by visiting tag_member.check() during
AlternateType.check(), we no longer need to call it during
Variants.check().

The various front end entities now exist as follows:
struct: optional base, optional local_members, no variants
simple union: no base, one-element local_members, variants with tag_member
  from local_members
flat union: base, no local_members, variants with tag_member from base
alternate: no base, no local_members, variants

With the new local members, we require a bit of finesse to
avoid assertions in the clients.

No change to generated code.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1447836791-369-2-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Now that we have separated union tag values from colliding with
non-variant C names, by naming the union 'u', we should reserve
this name for our use.  Note that we want to forbid 'u' even in
a struct with no variants, because it is possible for a future
qemu release to extend QMP in a backwards-compatible manner while
converting from a struct to a flat union.  Fortunately, no
existing clients were using this member name.  If we ever find
the need for QMP to have a member 'u', we could at that time
relax things, perhaps by having c_name() munge the QMP member to
'q_u'.

Note that we cannot forbid 'u' everywhere (by adding the
rejection code to check_name()), because the existing QKeyCode
enum already uses it; therefore we only reserve it as a struct
type member name.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1445898903-12082-24-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

We have two issues with our qapi union layout:
1) Even though the QMP wire format spells the tag 'type', the
C code spells it 'kind', requiring some hacks in the generator.
2) The C struct uses an anonymous union, which places all tag
values in the same namespace as all non-variant members. This
leads to spurious collisions if a tag value matches a non-variant
member's name.

This patch is the front end for a series that converts to a
saner qapi union layout.  By the end of the series, we will no
longer have the type/kind mismatch, and all tag values will be
under a named union, which requires clients to access
'obj->u.value' instead of 'obj->value'.  But since the
conversion touches a number of files, it is easiest if we
temporarily support BOTH layouts simultaneously.

Given a simple union qapi type:

{ 'union':'Foo', 'data': { 'a':'int', 'b':'bool' } }

make the following changes in generated qapi-types.h:

| struct Foo {
|-    FooKind kind;
|-    union { /* union tag is @kind */
|+    union {
|+        FooKind kind;
|+        FooKind type;
|+    };
|+    union { /* union tag is @type */
|         void *data;
|         int64_t a;
|         bool b;
|+        union { /* union tag is @type */
|+            void *data;
|+            int64_t a;
|+            bool b;
|+        } u;
|     };
| };

Flat unions do not need the anonymous union for the tag member,
as we already fixed that to use the member name instead of 'kind'
back in commit 0f61af3e.

One additional change is needed in qapi.py: check_union() now
needs to check for collisions with 'type' in addition to those
with 'kind'.

Later, when the conversions are complete, we will remove the
duplication hacks, and also drop the check_union() restrictions.

Note, however, that we do not rename the generated enum, which
is still 'FooKind'.  A further patch could generate implicit
enums as 'FooType', but while the generator already reserved
the '*Kind' namespace (commit 4dc2e690), there are already QMP
constructs with '*Type' naming, which means changing our
reservation namespace would have lots of churn to C code to
deal with a forced name change.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1445898903-12082-13-git-send-email-eblake@redhat.com>
[Commit message tweaked slightly]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

c_name() produces names starting with 'q_' when protecting a
dictionary member name that would fail to directly compile, but
in doing so can cause clashes with any member name already
beginning with 'q-' or 'q_'.  Likewise, we create a C name 'has_'
for any optional member that can clash with any member name
beginning with 'has-' or 'has_'.

Technically, rather than blindly reserving the namespace,
we could try to complain about user names only when an actual
collision occurs, or even teach c_name() how to munge names
to avoid collisions.  But it is not trivial, especially when
collisions can occur across multiple types (such as via
inheritance or flat unions).  Besides, no existing .json
files are trying to use these names.  So it's easier to just
outright forbid the potential for collision.  We can always
relax things in the future if a real need arises for QMP to
express member names that have been forbidden here.

'has_' only has to be reserved for struct/union member names,
while 'q_' is reserved everywhere (matching the fact that
only members can be optional, while we use c_name() for munging
both members and entities).  Note that we could relax 'q_'
restrictions on entities independently from member names; for
example, c_name('qmp_' + 'unix') would result in a different
function name than our current 'qmp_' + c_name('unix').

Update and add tests to cover the new error messages.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1445898903-12082-6-git-send-email-eblake@redhat.com>
[Consistently pass protect=False to c_name(); commit message tweaked
slightly]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Type names ending in 'List' can clash with qapi list types in
generated C.  We don't currently use such names. It is easier to
outlaw them now than to worry about how to resolve such a clash
in the future. For precedence, see commit 4dc2e690, which did the
same for names ending in 'Kind' versus implicit enum types for
qapi unions.

Update the testsuite to match.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1445898903-12082-5-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Rather than slicing the end of a string, we can use python's
endswith().  And rather than creating a set of characters,
we can search for a character within a string.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1445898903-12082-3-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

A future patch will move some error checking from the parser
to the various QAPISchema*.check() methods, which run only
after parsing completes.  It will thus be possible to create
a python instance representing an implicit QAPI type that
parses fine but will fail validation during check().  Since
all errors have to have an associated 'info' location, we
need a location to be associated with those implicit types.
The intuitive info to use is the location of the enclosing
entity that caused the creation of the implicit type.

Note that we do not anticipate builtin types being used in
an error message (as they are not part of the user's QAPI
input, the user can't cause a semantic error in their
behavior), so we exempt those types from requiring info, by
setting a flag to track the completion of _def_predefineds(),
and tracking that flag in _def_entity().

No change to the generated code.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1444710158-8723-13-git-send-email-eblake@redhat.com>
[Missing QAPISchemaArrayType.is_implicit() supplied]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

For simple unions, we were creating the implicit 'type' tag
member during the QAPISchemaObjectTypeVariants constructor.
This is different from every other implicit QAPISchemaEntity
object, which get created by QAPISchema methods.  Hoist the
creation to the caller (renaming _make_tag_enum() to
_make_implicit_tag()), and pass the entity rather than the
string name, so that we have the nice property that no
entities are created as a side effect within a different
entity.  A later patch will then have an easier time of
associating location info with each entity creation.

No change to generated code.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1444710158-8723-10-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Commit ac88219a had several TODO markers about whether we needed
to automatically create the corresponding array type alongside
any other type.  It turns out that most of the time, we don't!

There are a few exceptions: 1) We have a few situations where we
use an array type in internal code but do not expose that type
through QMP; fix it by declaring a dummy type that forces the
generator to see that we want to use the array type.

2) The builtin arrays (such as intList for QAPI ['int']) must
always be generated, because of the way our QAPI_TYPES_BUILTIN
compile guard works: we have situations (at the very least
tests/test-qmp-output-visitor.c) that include both top-level
"qapi-types.h" (via "error.h") and a secondary
"test-qapi-types.h". If we were to only emit the builtin types
when used locally, then the first .h file would not include all
types, but the second .h does not declare anything at all because
the first .h set QAPI_TYPES_BUILTIN, and we would end up with
compilation error due to things like unknown type 'int8List'.

Actually, we may need to revisit how we do type guards, and
change from a single QAPI_TYPES_BUILTIN over to a different
usage pattern that does one #ifdef per qapi type - right now,
the only types that are declared multiple times between two qapi
.json files for inclusion by a single .c file happen to be the
builtin arrays.  But now that we have QAPI 'include' statements,
it is logical to assume that we will soon reach a point where
we want to reuse non-builtin types (yes, I'm thinking about what
it will take to add introspection to QGA, where we will want to
reuse the SchemaInfo type and friends).  One #ifdef per type
will help ensure that generating the same qapi type into more
than one qapi-types.h won't cause collisions when both are
included in the same .c file; but we also have to solve how to
avoid creating duplicate qapi-types.c entry points.  So that
is a problem left for another day.

Generated code for qapi-types and qapi-visit is drastically
reduced; less than a third of the arrays that were blindly
created were actually needed (a quick grep shows we dropped
from 219 to 69 *List types), and the .o files lost more than
30% of their bulk.  [For best results, diff the generated
files with 'git diff --patience --no-index pre post'.]

Interestingly, the introspection output is unchanged - this is
because we already cull all types that are not indirectly
reachable from a command or event, so introspection was already
using only a subset of array types.  The subset of types
introspected is now a much larger percentage of the overall set
of array types emitted in qapi-types.h (since the larger set
shrunk), but still not 100% (evidence that the array types
emitted for our new Dummy structs, and the new struct itself,
don't affect QMP).
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1444710158-8723-9-git-send-email-eblake@redhat.com>
[Moved array info tracking to a later patch]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

A future patch will enable error reporting from the various
QAPISchema*.check() methods.  But to report an error related
to an implicit type, we'll need to associate a location with
the type (the same location as the top-level entity that is
causing the creation of the implicit type), and once we do
that, keying off of whether foo.info exists is no longer a
viable way to determine if foo is an implicit type.

Instead, add an is_implicit() method to QAPISchemaEntity, and use it.
It can be overridden later for ObjectType and EnumType, when implicit
instances of those classes gain info.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1444710158-8723-8-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

qapi-schema-test was already testing that we could have a
command returning int, but burned a command name in the whitelist.
Merge the redundant positive test returns-int, and pick a name
that reduces the whitelist size.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1444710158-8723-6-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

The next few patches will start migrating error checking from
ad hoc parse methods into the QAPISchema*.check() methods.  But
for an error message to display, we first have to fix the
overall 'try' to catch those errors.  We also want to enable a
few more assertions, such as making sure every attempt to
raise a semantic error is passed a valid location info, or that
various preconditions hold.

The general approach for moving error checking will then be to
relax an assertion into an if that raises an exception if the
condition does not hold, and removing the counterpart ad hoc
check done during the parse phase.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1444710158-8723-3-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Previously, qapi-types and qapi-visit filtered out implicit
objects during visit_object_type() by using 'info' (works since
implicit objects do not [yet] have associated info); meanwhile
qapi-introspect filtered out all schema types on the first pass
by returning a python type from visit_begin(), which was then
used at a distance in QAPISchema.visit() to do the filtering.

Rather than keeping these ad hoc approaches, add a new visitor
callback visit_needed() which returns False to skip a given
entity, and which defaults to True unless overridden.  Use the
new mechanism to simplify all three filtering visitors.

No change to the generated code.
Suggested-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1444710158-8723-2-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Since we have consolidated all generated code to use 'err' as
the name of the local variable for error detection, we can
simplify the decision on whether to skip error detection (useful
for deallocation paths) to be a boolean.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-18-git-send-email-eblake@redhat.com>
[Change to gen_visit_fields() simplified]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Consolidate the code between visit, command marshalling, and
event generation that iterates over the members of a struct.
It reduces code duplication in the generator, so that a future
patch can reduce the size of generated code while touching only
one instead of three locations.

There are no changes to the generated marshal code.

The visitor code becomes slightly more verbose, but remains
semantically equivalent, and is actually easier to read as
it follows a more common idiom:

|     visit_optional(v, &(*obj)->has_device, "device", &err);
|-    if (!err && (*obj)->has_device) {
|-        visit_type_str(v, &(*obj)->device, "device", &err);
|-    }
|     if (err) {
|         goto out;
|     }
|+    if ((*obj)->has_device) {
|+        visit_type_str(v, &(*obj)->device, "device", &err);
|+        if (err) {
|+            goto out;
|+        }
|+    }

The event code becomes slightly more verbose, but this is
arguably a bug fix: although the visitors are not well
documented, use of an optional member should not be attempted
unless guarded by a prior call to visit_optional().  Works only
because the output qmp visitor has a no-op visit_optional():

|+    visit_optional(v, &has_offset, "offset", &err);
|+    if (err) {
|+        goto out;
|+    }
|     if (has_offset) {
|         visit_type_int(v, &offset, "offset", &err);
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-17-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

qapi-commands has a nice helper gen_err_check(), but did not
use it everywhere. In fact, using it in more places makes it
easier to reduce the lines of code used for generating error
checks.  This in turn will make it easier for later patches
to consolidate another common pattern among the generators.

The generated code has fewer blank lines in qapi-event.c functions,
but has no semantic difference.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-16-git-send-email-eblake@redhat.com>
[Drop another blank line for symmetry]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Rather than open-code the check for a valid base type, we
should reuse the common functionality. This allows for
consistent error messages, and also makes it easier for a
later patch to turn on support for inline anonymous base
structures.

Test flat-union-inline is updated to test only one feature
(anonymous branch dictionaries), which can be implemented
independently (test flat-union-bad-base already covers the
idea of an anonymous base dictionary).
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-10-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

The previous commit added two tests that triggered an assertion
failure. It's fairly straightforward to avoid the failure by
just outright forbidding the collision between a union's tag
values and its discriminator name (including the implicit name
'kind' supplied for simple unions [*]).  Ultimately, we'd like
to move the collision detection into QAPISchema*.check(), but
for now it is easier just to enhance the existing checks.

[*] Of course, down the road, we have plans to rename the simple
union tag name to 'type' to match the QMP wire name, but the
idea of the collision will still be present even then.

Technically, we could avoid the collision by naming the C union
members representing each enum value as '_case_value' rather
than 'value'; but until we have an actual qapi client (and not
just our testsuite) that has a legitimate reason to match a
case label to the name of a QMP key and needs the name munging
to satisfy the compiler, it's easier to just reject the qapi
as invalid.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-7-git-send-email-eblake@redhat.com>
[Polished a few comments]
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Silence pep8, and make pylint a bit happier.  Just style cleanups,
plus killing a useless comment in camel_to_upper(); no semantic
changes.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-5-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

pylint recommends that every exception class should explicitly
invoke the superclass __init__, even though things seem to work
fine without it.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <1443565276-4535-4-git-send-email-eblake@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>