Block patches

# gpg: Signature made Fri 21 Feb 2014 21:42:24 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (54 commits)
  iotests: Mixed quorum child device specifications
  quorum: Simplify quorum_open()
  quorum: Add unit test.
  quorum: Add quorum_open() and quorum_close().
  quorum: Implement recursive .bdrv_recurse_is_first_non_filter in quorum.
  quorum: Add quorum_co_flush().
  quorum: Add quorum_invalidate_cache().
  quorum: Add quorum_getlength().
  quorum: Add quorum mechanism.
  quorum: Add quorum_aio_readv.
  blkverify: Extract qemu_iovec_clone() and qemu_iovec_compare() from blkverify.
  quorum: Add quorum_aio_writev and its dependencies.
  quorum: Create BDRVQuorumState and BlkDriver and do init.
  quorum: Create quorum.c, add QuorumChildRequest and QuorumAIOCB.
  check-qdict: Test termination of qdict_array_split()
  check-qdict: Adjust test for qdict_array_split()
  qdict: Extract non-QDicts in qdict_array_split()
  qemu-config: Sections must consist of keys
  qemu-iotests: Check qemu-img command line parsing
  qemu-img: Allow -o help with incomplete argument list
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

* remotes/sstabellini/xen-140220:
  xen_disk: fix io accounting
  Call pci_piix3_xen_ide_unplug from unplug_disks
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

* remotes/bonzini/configure:
  build: softmmu targets do not have a "main.o" file
  configure: Disable libtool if -fPIE does not work with it (bug #1257099)
  block: convert block drivers linked with libs to modules
  Makefile: introduce common-obj-m and block-obj-m for DSO
  Makefile: install modules with "make install"
  module: implement module loading
  rules.mak: introduce DSO rules
  darwin: do not use -mdynamic-no-pic
  block: use per-object cflags and libs
  rules.mak: allow per object cflags and libs
  rules.mak: fix $(obj) to a real relative path
  util: Split out exec_dir from os_find_datadir
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Check that the C++ compiler works with the C compiler; if it
does not, then don't pass CXX to the build process. This
fixes a regression where QEMU was no longer building if the
build environment didn't have a C++ compiler (introduced
in commit 3144f78b, which incorrectly assumed that rules.mak
would only see a non-empty $(CXX) if configure had actually
found a working C++ compiler).
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reported-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Reported-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Tested-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Tested-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Message-id: 1392909016-14028-1-git-send-email-peter.maydell@linaro.org

Add a test case to test 081 for mixing full option dicts and reference
strings of specifying the quorum child block devices through QMP.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NBenoit Canet <benoit@irqsave.net>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Although it may not look like it, this patch simplifies quorum_open().
qdict_array_split() is now able to return QLists with different objects
than only QDicts, therefore it will now do all the work and
quorum_open() does not have to handle reference strings by itself.

This allows mixing full option dicts and reference strings for
specifying the child block devices of quorum; furthermore, it improves
handling of malformed specifications.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NBenoit Canet <benoit@irqsave.net>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Example of command line:

-drive if=virtio,driver=quorum,\
children.0.file.filename=1.raw,\
children.0.node-name=1.raw,\
children.0.driver=raw,\
children.1.file.filename=2.raw,\
children.1.node-name=2.raw,\
children.1.driver=raw,\
children.2.file.filename=3.raw,\
children.2.node-name=3.raw,\
children.2.driver=raw,\
vote-threshold=2

blkverify=on with vote-threshold=2 and two files can be passed to
emulate blkverify.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This is used to activate quorum snapshot.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Makes a vote to select error if any.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

We really want that live migration works with quorum so implement
quorum_invalidate_cache().
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Check that every bs file returns the same length.
Otherwise, return -EIO to disable the quorum and
avoid length discrepancy.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patchset enables the core of the quorum mechanism.
The num_children reads are compared to get the majority version and if this
version exists more than threshold times the guest won't see the error at all.

If a block is corrupted or if an error occurs during an IO or if the quorum
cannot be established QMP events are used to report to the management.

Use gnutls's SHA-256 to compare versions.

--enable-quorum must be used to enable the feature.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Add code to do num_children reads in parallel and cleanup the structures
afterwards.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

qemu_iovec_compare() will be used to compare IOs vectors in quorum blkverify
mode. The patch extracts these functions in order to factorize the code.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Writes are mirrored num_children times on num_children devices.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Create the structure holding the quorum settings and write the minimal block
driver instanciation boilerplate.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Quorum is a block filter mirroring writes to num_children children.
For reads quorum reads each children and does a vote.
If more than vote_threshold versions are identical the quorum is reached and
this winning version is returned to the guest. So quorum prevents bit corruption.
For high availability purpose minority errors are reported via QMP but the guest
does not see them.

This patch creates the driver C source file and introduces the structures that
will be used in asynchronous reads and writes.
Signed-off-by: NBenoit Canet <benoit@irqsave.net>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

qdict_array_split() should terminate if it encounters both an entry with
a key of "%u" and entries with keys prefixed "%u." for the same index.

This patch adds a test for this case.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Test the new functionality of qdict_array_split(), that is, splitting
off single objects.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Currently, qdict_array_split() only splits off entries with a key prefix
of "%u.", packing them into a new QDict. This patch makes it support
entries with the plain key "%u" as well, directly putting them into the
new QList without creating a QDict.

If there is both an entry with a key of "%u" and other entries with keys
prefixed "%u." (for the same index), the function simply terminates.

To do this, this patch also adds a static function which tests whether a
given QDict contains any keys with the given prefix. This is used to test
whether entries with a key prefixed "%u." do exist in the source QDict
without modifying it.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

In config_parse_qdict_section(), the QList returned by
qdict_array_split() is assumed to only contain QDicts. Currently, this
is true but it may (and will) change in the future. Therefore, check
whether the assumption actually holds.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

This patch allows using 'qemu-img $subcmd -o help' for the create,
convert and amend subcommands, without specifying the previously
required filename arguments.

Note that it's still allowed and meaningful to specify a filename: An
invocation like 'qemu-img create -o help sheepdog:foo' will also display
options that are provided by the Sheepdog driver.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Instead of ignoring all option values but the last one, multiple -o
options now have the same meaning as having a single option with all
settings in the order of their respective -o options.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Instead of ignoring all option values but the last one, multiple -o
options now have the same meaning as having a single option with all
settings in the order of their respective -o options.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

If you specified multiple -o options for qemu-img create, it would
silently ignore all but the last one. This patch fixes the problem.

Now multiple -o options has the same meaning as having a single option
with all settings in the order of their respective -o options.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

has_help_option() checks if any help option ('help' or '?') occurs
anywhere in an option string, so that things like 'cluster_size=4k,help'
are recognised.

is_valid_option_list() ensures that the option list doesn't have options
with leading commas or trailing unescaped commas.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Instead of just putting it in debugging output, we can now put the
value in an Error.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Returning "Wrong medium type" for an image that does not have a valid
header is a bit weird.  Improve the error by mentioning what format
was trying to open it.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Now that we can return the "right" errors, use the Error** parameter
to pass them back instead of just printing them.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This prepares for propagating errors from vmdk_open_sparse and
vmdk_open_desc_file up to the caller of vmdk_open.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Currently, we just try reading a VMDK file as both image and descriptor.
This makes it hard to choose which of the two attempts gave the best error.
We'll decide in advance if the file looks like an image or a descriptor,
and this patch is the first step to that end.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Before:
    $ ./qemu-io-old
    qemu-io-old> open -r -o driver=vvfat,fat-type=24,dir=i386-softmmu
    Valid FAT types are only 12, 16 and 32
    qemu-io-old: can't open device (null): Could not open image: Invalid argument

After:
    $ ./qemu-io
    qemu-io> open -r -o driver=vvfat,fat-type=24,dir=i386-softmmu
    qemu-io: can't open device (null): Valid FAT types are only 12, 16 and 32
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>