This patch enables migrating vcpu's TSC rate. If KVM on the
destination machine supports TSC scaling, guest programs will
observe a consistent TSC rate across the migration.

If TSC scaling is not supported on the destination machine, the
migration will not be aborted and QEMU on the destination will
not set vcpu's TSC rate to the migrated value.

If vcpu's TSC rate specified by CPU option 'tsc-freq' on the
destination machine is inconsistent with the migrated TSC rate,
the migration will be aborted.

For backwards compatibility, the migration of vcpu's TSC rate is
disabled on pc-*-2.5 and older machine types.
Signed-off-by: NHaozhong Zhang <haozhong.zhang@intel.com>
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
[ehabkost: Rewrote comment at kvm_arch_put_registers()]
[ehabkost: Moved compat code to pc-2.5]
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Following changes are made to the TSC rate setting code in
kvm_arch_init_vcpu():
 * The code is moved to a new function kvm_arch_set_tsc_khz().
 * If kvm_arch_set_tsc_khz() fails, i.e. following two conditions are
   both satisfied:
   * KVM does not support the TSC scaling or it fails to set vcpu's
     TSC rate by KVM_SET_TSC_KHZ,
   * the TSC rate to be set is different than the value currently used
     by KVM, then kvm_arch_init_vcpu() will fail. Prevously,
   * the lack of TSC scaling never failed kvm_arch_init_vcpu(),
   * the failure of KVM_SET_TSC_KHZ failed kvm_arch_init_vcpu()
     unconditionally, even though the TSC rate to be set is identical
     to the value currently used by KVM.
Signed-off-by: NHaozhong Zhang <haozhong.zhang@intel.com>
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

If no user-specified TSC rate is present, we will try to set
env->tsc_khz to the value returned by KVM_GET_TSC_KHZ. This patch
does not change the current functionality of QEMU and just
prepares for later patches to enable migrating vcpu's TSC rate.
Signed-off-by: NHaozhong Zhang <haozhong.zhang@intel.com>
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

This will ensure we never use the MMX_* and ZMM_* macros with the
wrong struct type.
Reviewed-by: NRichard Henderson <rth@twiddle.net>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

This will simplify the definitions of ZMMReg and MMXReg.
Reviewed-by: NRichard Henderson <rth@twiddle.net>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Add a new field and reorder MMXReg fields, to make MMXReg and
ZMMReg field lists look the same (except for the array sizes).
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

They are helpers for the ZMMReg fields, so name them accordingly.

This is just a global search+replace, no other changes are being
introduced.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

The struct represents a 512-bit register, so name it accordingly.

This is just a global search+replace, no other changes are being
introduced.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Make MMXReg use the same field names used on XMMReg, so we can
try to reuse macros and other code later.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

We have a MMX_Q macro in addition to MMX_{B,W,L}. Use it.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Rename the function so that the reason for its existence is
clearer: it does x86-specific initialization of TCG structures.
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Block layer patches

# gpg: Signature made Wed 20 Jan 2016 15:37:57 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream:
  iotests: Test that throttle values ranges
  blockdev: Error out on negative throttling option values
  vmdk: Create streamOptimized as version 3
  qcow2: Make image inaccessible after failed qcow2_invalidate_cache()
  qcow2: Fix BDRV_O_INACTIVE handling in qcow2_invalidate_cache()
  qcow2: Implement .bdrv_inactivate
  block: Inactivate BDS when migration completes
  block: Rename BDRV_O_INCOMING to BDRV_O_INACTIVE
  block: Fix error path in bdrv_invalidate_cache()
  block: Assert no write requests under BDRV_O_INCOMING
  qcow2: Write full header on image creation
  qcow2: Write feature table only for v3 images
  block: Clean up includes
  qemu-iotests: Reduce racy output in 028
  qemu-img: Speed up comparing empty/zero images
  block/raw-posix: avoid bogus fixup for cylinders on DASD disks
  block: Fix .bdrv_open flags
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

I/O channels fixes 2016/01/20 v1

# gpg: Signature made Wed 20 Jan 2016 11:31:47 GMT using RSA key ID 15104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>"
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>"

* remotes/berrange/tags/pull-io-next-2016-01-20-1:
  io: use memset instead of { 0 } for initializing array
  io: fix description of @errp parameter initialization
  io: some fixes to handling of /dev/null when running commands
  io: increment counter when killing off subcommand
  io: fix sign of errno value passed to error report
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Convert qemu-socket to use QAPI exclusively, update MAINTAINERS.

# gpg: Signature made Wed 20 Jan 2016 06:49:07 GMT using RSA key ID D3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"

* remotes/kraxel/tags/pull-socket-20160120-1:
  vnc: distiguish between ipv4/ipv6 omitted vs set to off
  sockets: remove use of QemuOpts from socket_dgram
  sockets: remove use of QemuOpts from socket_connect
  sockets: remove use of QemuOpts from socket_listen
  sockets: remove use of QemuOpts from header file
  add MAINTAINERS entry for qemu socket code
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

VFIO updates 2016-01-19

 - Performance fix for devices with poorly placed MSI-X PBA regions
 - Quirk fix for hosts with broken MMCONFIG access

# gpg: Signature made Tue 19 Jan 2016 19:00:21 GMT using RSA key ID 3BB08B22
# gpg: Good signature from "Alex Williamson <alex.williamson@redhat.com>"
# gpg:                 aka "Alex Williamson <alex@shazbot.org>"
# gpg:                 aka "Alex Williamson <alwillia@redhat.com>"
# gpg:                 aka "Alex Williamson <alex.l.williamson@gmail.com>"

* remotes/awilliam/tags/vfio-update-20160119.0:
  vfio/pci: Lazy PBA emulation
  vfio/pci-quirks: Only quirk to size of PCI config space
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

extract_common_blockdev_options() uses qemu_opt_get_number() to parse
the bps/iops numbers to uint64_t, then converts to double and stores in
ThrottleConfig.  The actual parsing is done by strtoull() in
parse_option_number().  Negative numbers are wrapped to large positive
ones, and stored.

We used to reject negative numbers since 7d81c141, but this regressed
when the option parsing code was changed later. Now fix this again.

This time, define an arbitrary large upper limit (1e15),  and check the
values so both negative and impractically big numbers are caught and
reported.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

VMware products accept only version 3 for streamOptimized, let's bump
the version.
Reported-by: NRadoslav Gerganov <rgerganov@vmware.com>
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

If qcow2_invalidate_cache() fails, we are in a state where qcow2_close()
has already been completed, but the image hasn't been reopened yet.
Calling into any qcow2 function for an image in this state will cause
crashes.

The real solution would be to get rid of the close/open pair and instead
do an atomic reset of the involved data structures, but this isn't
trivial, so let's just make the image inaccessible for now.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

What qcow2_invalidate_cache() should do is close the image with
BDRV_O_INACTIVE set and reopen it with the flag cleared. In fact, it
used to do exactly the opposite: qcow2_close() relied on bs->open_flags,
which is already updated to have cleared BDRV_O_INACTIVE at this point,
whereas qcow2_open() was called with s->flags, which has the flag still
set. Fix this.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The callback has to ensure that closing or flushing the image afterwards
wouldn't cause a write access to the image files. This means that just
the caches have to be written out, which is part of the existing
.bdrv_close implementation.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

So far, live migration with shared storage meant that the image is in a
not-really-ready don't-touch-me state on the destination while the
source is still actively using it, but after completing the migration,
the image was fully opened on both sides. This is bad.

This patch adds a block driver callback to inactivate images on the
source before completing the migration. Inactivation means that it goes
to a state as if it was just live migrated to the qemu instance on the
source (i.e. BDRV_O_INACTIVE is set). You're then supposed to continue
either on the source or on the destination, which takes ownership of the
image.

A typical migration looks like this now with respect to disk images:

1. Destination qemu is started, the image is opened with
   BDRV_O_INACTIVE. The image is fully opened on the source.

2. Migration is about to complete. The source flushes the image and
   inactivates it. Now both sides have the image opened with
   BDRV_O_INACTIVE and are expecting the other side to still modify it.

3. One side (the destination on success) continues and calls
   bdrv_invalidate_all() in order to take ownership of the image again.
   This removes BDRV_O_INACTIVE on the resuming side; the flag remains
   set on the other side.

This ensures that the same image isn't written to by both instances
(unless both are resumed, but then you get what you deserve). This is
important because .bdrv_close for non-BDRV_O_INACTIVE images could write
to the image file, which is definitely forbidden while another host is
using the image.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>

Instead of covering only the state of images on the migration
destination before the migration is completed, the flag will also cover
the state of images on the migration source after completion. This
common state implies that the image is technically still open, but no
writes will happen and any cached contents will be reloaded from disk if
and when the image leaves this state.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

We can only clear BDRV_O_INCOMING if the caches were actually
invalidated.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

As long as BDRV_O_INCOMING is set, the image file is only opened so we
have a file descriptor for it. We're definitely not supposed to modify
the image, it's still owned by the migration source.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

When creating a qcow2 image, we didn't necessarily call
qcow2_update_header(), but could end up with the basic header that
qcow2_create2() created manually. One thing that this basic header
lacks is the feature table. Let's make sure that it's always present.

This requires a few updates to test cases as well.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Version 2 images don't have feature bits, so writing a feature table to
those images is kind of pointless.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

On my machine, './check -qcow2 028' was failing about 80% of the
time, due to a race in how many times the repeated attempts
to run 'info block-jobs' could occur before the job was done,
showing up as a failure of fewer '(qemu) ' prompts than in the
expected output.  Silence the output during the repetitions, then
add a final clean command to keep the expected output useful;
once patched, I was finally able to run the test 20 times in a
row with no failures.
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Two empty raw files are always compared by actually reading data even if
there is no data, because BDRV_BLOCK_ZERO is considered "allocated" in
bdrv_is_allocated_above().  That is inefficient.

Use bdrv_get_block_status_above() for more information, and skip the
consecutive zero sectors.

This brings a huge speed up in comparing sparse/empty raw images:

    $ qemu-img create a 1G

    $ time ~/build/master/bin/qemu-img compare a a
    Images are identical.

    real    0m6.583s
    user    0m0.191s
    sys     0m6.367s

    $ time qemu-img compare a a
    Images are identical.

    real    0m0.033s
    user    0m0.003s
    sys     0m0.031s
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Some versions of GCC on OS-X complain about CMSG_SPACE
not being constant size, which prevents use of { 0 }

io/channel-socket.c: In function 'qio_channel_socket_writev':
io/channel-socket.c:497:18: error: variable-sized object may not be initialized
     char control[CMSG_SPACE(sizeof(int) * SOCKET_MAX_FDS)] = { 0 };

The compiler is at fault here, but it is nicer to avoid
tickling this compiler bug by using memset instead.
Reviewed-By: NJohn Arbuckle <programmingkidx@gmail.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The "Error **errp" parameters must be NULL initialized
not uninitialized.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The /dev/null file handle was leaked in a couple of places.
There is also the possibility that both readfd and writefd
point to the same /dev/null file handle, so care must be
taken not to close the same file handle twice.
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The PCI spec recommends devices use additional alignment for MSI-X
data structures to allow software to map them to separate processor
pages.  One advantage of doing this is that we can emulate those data
structures without a significant performance impact to the operation
of the device.  Some devices fail to implement that suggestion and
assigned device performance suffers.

One such case of this is a Mellanox MT27500 series, ConnectX-3 VF,
where the MSI-X vector table and PBA are aligned on separate 4K
pages.  If PBA emulation is enabled, performance suffers.  It's not
clear how much value we get from PBA emulation, but the solution here
is to only lazily enable the emulated PBA when a masked MSI-X vector
fires.  We then attempt to more aggresively disable the PBA memory
region any time a vector is unmasked.  The expectation is then that
a typical VM will run entirely with PBA emulation disabled, and only
when used is that emulation re-enabled.
Reported-by: NShyam Kaushik <shyam.kaushik@gmail.com>
Tested-by: NShyam Kaushik <shyam.kaushik@gmail.com>
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>

For quirks that support the full PCIe extended config space, limit the
quirk to only the size of config space available through vfio.  This
allows host systems with broken MMCONFIG regions to still make use of
these quirks without generating bad address faults trying to access
beyond the end of config space exposed through vfio.  This may expose
direct access to the mirror of extended config space, only trapping
the sub-range of standard config space, but allowing this makes the
quirk, and thus the device, functional.  We expect that only device
specific accesses make use of the mirror, not general extended PCI
capability accesses, so any virtualization in this space is likely
unnecessary anyway, and the device is still IOMMU isolated, so it
should only be able to hurt itself through any bogus configurations
enabled by this space.

Link: https://www.redhat.com/archives/vfio-users/2015-November/msg00192.htmlReported-by: NRonnie Swanink <ronnie@ronnieswanink.nl>
Reviewed-by: NLaszlo Ersek <lersek@redhat.com>
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>

large volume DASD that have > 64k cylinders do claim to have
0xFFFE cylinders as special value in the old 16 bit field. We
want to pass this "token" along to the guest, instead of
calculating the real number. Otherwise qemu might fail with
"cyls must be between 1 and 65535"

Cc: qemu-stable@nongnu.org
Acked-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: NMarkus Armbruster <armbru@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

bdrv_common_open() modified bs->open_flags after inferring the set of
options to pass to the driver's .bdrv_open callback. This means that the
cache options were correctly set in bs->open_flags (and therefore
correctly displayed in 'info block'), but the image would actually be
opened with the default cache mode instead.

This patch removes the flags parameter to bdrv_common_open() (except for
BDRV_O_NO_BACKING it's the same as bs->open_flags anyway, and having two
names for the same thing is confusing), and moves the assignment of
open_flags down to immediately before calling into the block drivers. In
all other places, bs->open_flags is now used consistently.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Tested-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

The VNC code for interpreting QemuOpts does not currently
distinguish between ipv4/ipv6 being omitted, and being
set to 'off', because historically the 'ipv4' and 'ipv6'
options were just flags which did not accept a value.

The upshot is that if someone runs

  $QEMU -vnc localhost:1,ipv6=off

QEMU still uses PF_UNSPEC and thus may still bind to IPv6,
when it should use PF_INET.

This is another instance of the problem previously fixed
for chardevs in

  commit b77e7c8e
  Author: Paolo Bonzini <pbonzini@redhat.com>
  Date:   Mon Oct 12 15:35:16 2015 +0200

    qemu-sockets: fix conversion of ipv4/ipv6 JSON to QemuOpts
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Message-id: 1452518225-11751-6-git-send-email-berrange@redhat.com
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

The socket_dgram method accepts a QAPI SocketAddress object
which it then turns into QemuOpts before calling the
inet_dgram_opts helper method. By converting the latter to
use QAPI SocketAddress directly, the QemuOpts conversion
step can be eliminated.

This removes the very last use of QemuOpts from the
sockets code, so the socket_optslist[] array is also
removed.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1452518225-11751-5-git-send-email-berrange@redhat.com
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

The socket_connect method accepts a QAPI SocketAddress object
which it then turns into QemuOpts before calling the
inet_connect_opts/unix_connect_opts helper methods. By
converting the latter to use QAPI SocketAddress directly,
the QemuOpts conversion step can be eliminated
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-id: 1452518225-11751-4-git-send-email-berrange@redhat.com
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>