Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Cc: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

When we find a IOC bit set on a setup trb and therefore queue an event,
that should not stop events being generated for following data trbs.
So clear the 'reported' flag.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

This makes xhci generate multiple short packet events in case of
multi-trb transfers.  Which is wrong.  We need to fix this in a
different way.

This reverts commit aa685789.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

The dev->config pointer isn't set until guest
system initializes usb devices (via
usb_desc_set_config). However qemu networking can
go through some motions prior to that, e.g.:

 #0  is_rndis (s=0x555557261970) at hw/usb/dev-network.c:653
 #1  0x000055555585f723 in usbnet_can_receive (nc=0x55555641e820) at hw/usb/dev-network.c:1315
 #2  0x000055555587635e in qemu_can_send_packet (sender=0x5555572660a0) at net/net.c:470
 #3  0x0000555555878e34 in net_hub_port_can_receive (nc=0x5555562d7800) at net/hub.c:101
 #4  0x000055555587635e in qemu_can_send_packet (sender=0x5555562d7980) at net/net.c:470
 #5  0x000055555587dbca in tap_can_send (opaque=0x5555562d7980) at net/tap.c:172

The command to reproduce most reliably was:

 qemu-system-i386 -usb -device usb-net,vlan=0 -net tap,vlan=0

This wasn't strictly a problem with tap. Other
networking endpoints (vde, user) could trigger
this problem as well.

Fixes: https://bugs.launchpad.net/qemu/+bug/1050823
Cc: qemu-stable@nongnu.org
Signed-off-by: NMichal Kazior <michal.kazior@tieto.com>
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

migration/next for 20150507

# gpg: Signature made Thu May  7 17:42:19 2015 BST using RSA key ID 5872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>"
# gpg:                 aka "Juan Quintela <quintela@trasno.org>"

* remotes/juanquintela/tags/migration/20150507-1:
  migration: Fix migration state update issue
  migration: avoid divide by zero in xbzrle cache miss rate
  migration: Add hmp interface to set and query parameters
  migration: Add qmp commands to set and query parameters
  migration: Use an array instead of 3 parameters
  migration: Add interface to control compression
  migration: Add the core code for decompression
  migration: Make compression co-work with xbzrle
  migration: Add the core code of multi-thread compression
  migration: Split save_zero_page from ram_save_page
  arch_init: Add and free data struct for decompression
  arch_init: Alloc and free data struct for compression
  qemu-file: Add compression functions to QEMUFile
  migration: Add the framework of multi-thread decompression
  migration: Add the framework of multi-thread compression
  docs: Add a doc about multiple thread compression
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

If live migration is very fast and can be completed in 1 second,
the dirty_sync_count of MigrationState will not be updated.
Then you will see "dirty sync count: 0" in qemu monitor even if
the actual dirty sync count is not 0.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NDr.David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

This bug manifested itself as a VM that could not be resumed by libvirt
following a migration:

  # virsh resume example
  error: Failed to resume domain example
  error: internal error: cannot parse json {"return":
    {"xbzrle-cache":
      {..., "cache-miss-rate": -nan, ...},
      ...
    }
  }: lexical error: malformed number, a digit is required after the minus sign.

This patch also ensures xbzrle_cache_miss_prev and iterations_prev are
cleared at the start of the migration.
Signed-off-by: NMichael Chapman <mike@very.puzzling.org>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add the hmp interface to tune and query the parameters used in
live migration.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add the qmp commands to tune and query the parameters used in live
migration.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Put the three parameters related to multiple thread (de)compression
into an int array, and use an enum type to index the parameter.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

The multiple compression threads can be turned on/off through
qmp and hmp interface before doing live migration.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NDr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Implement the core logic of multiple thread decompression,
the decompression can work now.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Now, multiple thread compression can co-work with xbzrle. when
xbzrle is on, multiple thread compression will only work at the
first round of RAM data sync.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NDr.David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Implement the core logic of the multiple thread compression. At this
point, multiple thread compression can't co-work with xbzrle yet.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Split the function save_zero_page from ram_save_page so that we can
reuse it later.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Define the data structure and variables used to do multiple thread
decompression, and add the code to initialize and free them.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NDr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Define the data structure and variables used to do multiple thread
compression, and add the code to initialize and free them.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NDr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

qemu_put_compression_data() compress the data and put it to QEMUFile.
qemu_put_qemu_file() put the data in the buffer of source QEMUFile to
destination QEMUFile.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add the code to create and destroy the multiple threads those will be
used to do data decompression. Left some functions empty just to keep
clearness, and the code will be added later.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NDr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Add the code to create and destroy the multiple threads those will
be used to do data compression. Left some functions empty to keep
clearness, and the code will be added later.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NDr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Give some details about the multiple thread (de)compression and
how to use it in live migration.
Signed-off-by: NLiang Li <liang.z.li@intel.com>
Signed-off-by: NYang Zhang <yang.z.zhang@intel.com>
Reviewed-by: NDr.David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

size reduction merge

# gpg: Signature made Wed May  6 00:21:43 2015 BST using RSA key ID 4DD0279B
# gpg: Good signature from "Richard Henderson <rth7680@gmail.com>"
# gpg:                 aka "Richard Henderson <rth@redhat.com>"
# gpg:                 aka "Richard Henderson <rth@twiddle.net>"

* remotes/rth/tags/tcg-next-20150505:
  tcg: optimise memory layout of TCGTemp
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

drop qapi nested structs

# gpg: Signature made Tue May  5 17:40:40 2015 BST using RSA key ID EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"

* remotes/armbru/tags/pull-qmp-2015-05-05: (40 commits)
  qapi: Check for member name conflicts with a base class
  qapi: Support (subset of) \u escapes in strings
  qapi: Tweak doc references to QMP when QGA is also meant
  qapi: Drop dead visitor code related to nested structs
  qapi: Drop support for inline nested types
  qapi: Drop inline nested structs in query-pci
  qapi: Drop inline nested struct in query-version
  qapi: Drop tests for inline nested structs
  qapi: Merge UserDefTwo and UserDefNested in tests
  qapi: Forbid 'type' in schema
  qapi: Use 'struct' instead of 'type' in schema
  qapi: Document 'struct' metatype
  qapi: Prefer 'struct' over 'type' in generator
  qapi: More rigorous checking for type safety bypass
  qapi: Whitelist commands that don't return dictionary
  qapi: Require valid names
  qapi: More rigourous checking of types
  qapi: Add some type check tests
  qapi: Unify type bypass and add tests
  qapi: Allow true, false and null in schema json
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Our type inheritance for both 'struct' and for flat 'union' merges
key/value pairs from the base class with those from the type in
question.  Although the C code currently boxes things so that there
is a distinction between which member is referred to, the QMP wire
format does not allow passing a key more than once in a single
object.  Besides, if we ever change the generated C code to not be
quite so boxy, we'd want to avoid duplicate member names there,
too.

Fix a testsuite entry added in an earlier patch, as well as adding
a couple more tests to ensure we have appropriate coverage.  Ensure
that collisions are detected, regardless of whether there is a
difference in opinion on whether the member name is optional.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings).  For other things, like '\n', it resulted in a
literal 'n' instead of a newline.

Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that.  But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX.  This gets us
closer, by supporting Unicode escapes in the ASCII range.

Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

We have more than one qapi schema in use by more than one protocol.
Add a new term 'Client JSON Protocol' for use throughout the
document, to avoid confusion on whether something refers only to
QMP and not QGA.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Now that we no longer have nested structs to visit, the use of
prefix strings is no longer required.  Remove the code that is
no longer reachable.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument
(see previous commit messages for more details why); but existing
use of inline nested structs conflicts with that goal. Now that
all commands have been changed to avoid inline nested structs,
nuke support for them, and turn it into a hard error. Update the
testsuite to reflect tighter parsing rules.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument
(see previous commit message for more details why); but existing
use of inline nested structs conflicts with that goal. This patch
fixes one of only two commands relying on nested types, by
breaking the nesting into an explicit type; it means that the
type is now boxed instead of unboxed in C code, but the QMP wire
format is unaffected by this change.

Prefer the safer g_new0() while making the conversion, and reduce
some long lines.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument
(see previous commit message for more details why); but existing
use of inline nested structs conflicts with that goal. This patch
fixes one of only two commands relying on nested types, by
breaking the nesting into an explicit type; it means that the
type is now boxed instead of unboxed in C code, but the QMP wire
format is unaffected by this change.

Prefer the safer g_new0() while making the conversion.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument;
but existing use of inline nested structs conflicts with that goal.

More precisely, a definition in the QAPI schema associates a name
with a set of properties:

Example 1: { 'struct': 'Foo', 'data': { MEMBERS... } }
associates the global name 'Foo' with properties (meta-type struct)
and MEMBERS...

Example 2: 'mumble': TYPE
within MEMBERS... above associates 'mumble' with properties (type
TYPE) and (optional false) within type Foo

The syntax of example 1 is extensible; if we need another property,
we add another name/value pair to the dictionary (such as
'base':TYPE).  The syntax of example 2 is not extensible, because
the right hand side can only be a type.

We have used name encoding to add a property: "'*mumble': 'int'"
associates 'mumble' with (type int) and (optional true).  Nice,
but doesn't scale.  So the solution is to change our existing uses
to be syntactic sugar to an extensible form:

   NAME: TYPE   --> NAME:  { 'type': TYPE, 'optional': false }
   *ONAME: TYPE --> ONAME: { 'type': TYPE, 'optional': true }

This patch fixes the testsuite to avoid inline nested types, by
breaking the nesting into explicit types; it means that the type
is now boxed instead of unboxed in C code, but makes no difference
on the wire (and if desired, a later patch could change the
generator to not do so much boxing in C).  When touching code to
add new allocations, also convert existing allocations to
consistently prefer typesafe g_new0 over g_malloc0 when a type
name is involved.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

In the testsuite, UserDefTwo and UserDefNested were identical
structs other than the member names.  Reduce code duplication by
having just one type, and choose names that also favor reuse.
This will also make it easier for a later patch to get rid of
inline nested types in QAPI.  When touching code related to
allocations, convert g_malloc0(sizeof(Type)) to the more typesafe
g_new0(Type, 1).

Ensure that 'make check-qapi-schema check-unit' still passes.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing.  Finish up the
conversion to using "struct" in qapi schema by removing the hack
in the generator that allowed 'type'.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing.  Do the bulk of
the conversion to "struct" in qapi schema, with a fairly
mechanical:

for f in `find -name '*.json'; do sed -i "s/'type'/'struct'/"; done

followed by manually filtering out the places where we have a
'type' embedded in 'data'.  Then tweak a couple of tests whose
output changes slightly due to longer lines.

I also verified that the generated files for QMP and QGA (such
as qmp-commands.h) are the same before and after, as assurance
that I didn't leave in any accidental member name changes.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing.  Now that the
generator accepts 'struct' as a synonym for 'type', update all
documentation to use saner wording.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing.  The confusion
is only made worse by the fact that the generator mostly already
refers to struct even when dealing with expr['type'].  This
commit changes the generator to consistently refer to it as
struct everywhere, plus a single back-compat tweak that allows
accepting the existing .json files as-is, so that the meat of
this change is separate from the mindless churn of that change.

Fix the testsuite fallout for error messages that change, and
in some cases, become more legible.  Improve comments to better
match our intentions where a struct (rather than any complex
type) is required.  Note that in some cases, an error message
now refers to 'struct' while the schema still refers to 'type';
that will be cleaned up in the later commit to the schema.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Now that we have a way to validate every type, we can also be
stricter about enforcing that callers that want to bypass
type safety in generated code.  Prior to this patch, it didn't
matter what value was associated with the key 'gen', but it
looked odd that 'gen':'yes' could result in bypassing the
generated code.  These changes also enforce the changes made
earlier in the series for documentation and consolidation of
using '**' as the wildcard type, as well as 'gen':false as the
canonical spelling for requesting type bypass.

Note that 'gen':false is a one-way switch away from the default;
we do not support 'gen':true (similar for 'success-response').
In practice, this doesn't matter.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

...or an array of dictionaries.  Although we have to cater to
existing commands, returning a non-dictionary means the command
is not extensible (no new name/value pairs can be added if more
information must be returned in parallel).  By making the
whitelist explicit, any new command that falls foul of this
practice will have to be self-documenting, which will encourage
developers to either justify the action or rework the design to
use a dictionary after all.

It's a little bit sloppy that we share a single whitelist among
three clients (it's too permissive for each).  If this is a
problem, a future patch could tighten things by having the
generator take the whitelist as an argument (as in
scripts/qapi-commands.py --legacy-returns=...), or by having
the generator output C code that requires explicit use of the
whitelist (as in:
 #ifndef FROBNICATE_LEGACY_RETURN_OK
 # error Command 'frobnicate' should return a dictionary
 #endif
then having the callers define appropriate macros).  But until
we need such fine-grained separation (if ever), this patch does
the job just fine.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>