In some cases it is possible that query-io-status is called just
before the job is completed, causing

    -{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "src", "len": 31457280, "offset": OFFSET, "speed": 0, "type": "mirror", "error": "Operation not permitted"}}
    -{"return": []}
    +{"return": [{"io-status": "ok", "device": "src", "busy": true, "len": 31457280, "offset": OFFSET, "paused": false, "speed": 0, "ready": false, "type": "mirror"}]}

Assert that the completeion event eventually happens.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 20161109162008.27287-1-pbonzini@redhat.com
Reviewed-by: NJeff Cody <jcody@redhat.com>
Signed-off-by: NJeff Cody <jcody@redhat.com>

Add a regression test for the case found by Vladimir.
Reported-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1478587839-9834-7-git-send-email-jsnow@redhat.com
Signed-off-by: NJeff Cody <jcody@redhat.com>

Refactor backup_start as backup_job_create, which only creates the job,
but does not automatically start it. The old interface, 'backup_start',
is not kept in favor of limiting the number of nearly-identical interfaces
that would have to be edited to keep up with QAPI changes in the future.

Callers that wish to synchronously start the backup_block_job can
instead just call block_job_start immediately after calling
backup_job_create.

Transactions are updated to use the new interface, calling block_job_start
only during the .commit phase, which helps prevent race conditions where
jobs may finish before we even finish building the transaction. This may
happen, for instance, during empty block backup jobs.
Reported-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-id: 1478587839-9834-6-git-send-email-jsnow@redhat.com
Signed-off-by: NJeff Cody <jcody@redhat.com>

Instead of automatically starting jobs at creation time via backup_start
et al, we'd like to return a job object pointer that can be started
manually at later point in time.

For now, add the block_job_start mechanism and start the jobs
automatically as we have been doing, with conversions job-by-job coming
in later patches.

Of note: cancellation of unstarted jobs will perform all the normal
cleanup as if the job had started, particularly abort and clean. The
only difference is that we will not emit any events, because the job
never actually started.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-id: 1478587839-9834-5-git-send-email-jsnow@redhat.com
Signed-off-by: NJeff Cody <jcody@redhat.com>

Add an explicit start field to specify the entrypoint. We already have
ownership of the coroutine itself AND managing the lifetime of the
coroutine, let's take control of creation of the coroutine, too.

This will allow us to delay creation of the actual coroutine until we
know we'll actually start a BlockJob in block_job_start. This avoids
the sticky question of how to "un-create" a Coroutine that hasn't been
started yet.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1478587839-9834-4-git-send-email-jsnow@redhat.com
Signed-off-by: NJeff Cody <jcody@redhat.com>

Cleaning up after we have deferred to the main thread but before the
transaction has converged can be dangerous and result in deadlocks
if the job cleanup invokes any BH polling loops.

A job may attempt to begin cleaning up, but may induce another job to
enter its cleanup routine. The second job, part of our same transaction,
will block waiting for the first job to finish, so neither job may now
make progress.

To rectify this, allow jobs to register a cleanup operation that will
always run regardless of if the job was in a transaction or not, and
if the transaction job group completed successfully or not.

Move sensitive cleanup to this callback instead which is guaranteed to
be run only after the transaction has converged, which removes sensitive
timing constraints from said cleanup.

Furthermore, in future patches these cleanup operations will be performed
regardless of whether or not we actually started the job. Therefore,
cleanup callbacks should essentially confine themselves to undoing create
operations, e.g. setup actions taken in what is now backup_start.
Reported-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1478587839-9834-3-git-send-email-jsnow@redhat.com
Signed-off-by: NJeff Cody <jcody@redhat.com>

Though it is not intended to be reached through normal circumstances,
if we do not gracefully deconstruct the transaction QLIST, we may wind
up with stale pointers in the list.

The rest of this series attempts to address the underlying issues,
but this should fix list inconsistencies.
Signed-off-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-id: 1478587839-9834-2-git-send-email-jsnow@redhat.com
[Rewrote commit message. --js]
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NJeff Cody <jcody@redhat.com>

# gpg: Signature made Mon 14 Nov 2016 04:16:48 PM GMT
# gpg:                using RSA key 0x7DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>"
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* jsnow/tags/ide-pull-request:
  ahci-test: add QMP tray test for ATAPI
  libqos/ahci: Add get_sense and test_ready
  libqos/ahci: Add ATAPI tray macros
  libqos/ahci: Support expected errors
  libqtest: add qmp_eventwait_ref
  block-backend: Always notify on blk_eject
  ahci-test: test atapi read_cd with bcl, nb_sectors = 0
  ahci-test: Create smaller test ISO images
  atapi: classify read_cd as conditionally returning data

Message-id: 1479140746-22142-1-git-send-email-jsnow@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Test QMP events for a CDROM device with or without a media inserted,
including both guest-initiated and hw-initiated eject/load requests.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1478553214-497-7-git-send-email-jsnow@redhat.com
Signed-off-by: NJohn Snow <jsnow@redhat.com>

Required for tray tests once a medium may have changed.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1478553214-497-6-git-send-email-jsnow@redhat.com
[Line length edit --js]
Signed-off-by: NJohn Snow <jsnow@redhat.com>

(1) Add START_STOP_UNIT command to ahci-test suite
(2) Add eject/start macro commands; this is not a data transfer
    command so it is not well-served by the existing generic pipeline.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1478553214-497-5-git-send-email-jsnow@redhat.com
Signed-off-by: NJohn Snow <jsnow@redhat.com>

Sometimes we know we'll get back an error, so let's have the
test framework understand that.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1478553214-497-4-git-send-email-jsnow@redhat.com
Signed-off-by: NJohn Snow <jsnow@redhat.com>

Wait for an event, but return a copy so we can investigate parameters.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1478553214-497-3-git-send-email-jsnow@redhat.com
Signed-off-by: NJohn Snow <jsnow@redhat.com>

blk_eject is only used by scsi-disk and atapi, and in both cases we
only attempt to invoke blk_eject if we have a bona-fide change in
tray state.

The "issue" here is that the tray state does not generate a QMP event
unless there is a medium/BDS attached to the device, so if libvirt et al
are waiting for a tray event to occur from an empty-but-closed drive,
software opening that drive will not emit an event and libvirt will
wait forever.

Change this by modifying blk_eject to always emit an event, instead of
conditionally on a "real" backend eject.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1373264Reported-by: NPeter Krempa <pkrempa@redhat.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1478553214-497-2-git-send-email-jsnow@redhat.com
Signed-off-by: NJohn Snow <jsnow@redhat.com>

Commit 9ef2e93f introduced the concept of tagging ATAPI commands as
NONDATA, but this introduced a regression for certain commands better
described as CONDDATA. read_cd is such a command that both requires
a non-zero BCL if a transfer size is set, but is perfectly content to
accept a zero BCL if the transfer size is 0.

This test adds a regression test for the case where BCL and nb_sectors
are both 0.

Flesh out the CDROM tests by:

(1) Allowing the test to specify a BCL
(2) Allowing the buffer comparison test to compare a 0-size buffer
(3) Fix the BCL specification in libqos (It is LE, not BE)
(4) Add a nice human-readable message for future SCSI command additions
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1477970211-25754-4-git-send-email-jsnow@redhat.com
[Line length edit --js]
Signed-off-by: NJohn Snow <jsnow@redhat.com>

These can simply be the size of the number of sectors we're reading,
plus one for a buffer. We don't need them to be any larger.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1477970211-25754-3-git-send-email-jsnow@redhat.com
Signed-off-by: NJohn Snow <jsnow@redhat.com>

For the purposes of byte_count_limit verification, add a new flag that
identifies read_cd as sometimes returning data, then check the BCL in
its command handler after we know that it will indeed return data.
Reported-by: NHervé Poussineau <hpoussin@reactos.org>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1477970211-25754-2-git-send-email-jsnow@redhat.com
Signed-off-by: NJohn Snow <jsnow@redhat.com>

Block layer patches for 2.8.0-rc0

# gpg: Signature made Fri 11 Nov 2016 03:46:12 PM GMT
# gpg:                using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* kwolf/tags/for-upstream:
  raw-posix: Rename 'raw_s' to 'rs'
  iotests: Always use -machine accel=qtest
  iotests: Skip test 162 if there is no SSH support
  block: Emit modules in bdrv_iterate_format()
  block: Fix bdrv_iterate_format() sorting
  nfs: Fix memory leak in nfs_file_create()
  qcow2: Remove stale FIXME comment
  raw_bsd: don't check size alignment when only offset is set
  raw_bsd: move check to prevent overflow
  hmp: Make block_stream set an explicit job ID
  block/ssh: Code cleanup for unused parameter
  block/nbd: Fix the leaked visitor

Message-id: 1478883311-24052-1-git-send-email-kwolf@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

* remotes/fam/tags/for-upstream:
  test-uuid: fix leak

Message-id: 20161111131818.GC12800@lemon
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Block patches for qemu 2.8

# gpg: Signature made Fri Nov 11 15:56:59 2016 CET
# gpg:                using RSA key 0xF407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* mreitz/tags/pull-block-2016-11-11:
  raw-posix: Rename 'raw_s' to 'rs'
  iotests: Always use -machine accel=qtest
  iotests: Skip test 162 if there is no SSH support
  block: Emit modules in bdrv_iterate_format()
  block: Fix bdrv_iterate_format() sorting
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

It is too confusing because it sounds like a BDRVRawState variable.
Suggested-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NFam Zheng <famz@redhat.com>
Message-id: 1477565117-17230-1-git-send-email-famz@redhat.com
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Currently, we only use -machine accel=qtest when qemu is invoked through
the common.qemu functions. However, we always want to use it, so move it
from common.qemu directly into QEMU_OPTIONS.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20161017183917.8837-1-mreitz@redhat.com
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Signed-off-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20161012204907.25941-4-mreitz@redhat.com
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Some block drivers may not be loaded yet, but qemu supports them
nonetheless. bdrv_iterate_format() should report them, too.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20161012204907.25941-3-mreitz@redhat.com
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

bdrv_iterate_format() did not actually sort the formats by name but by
"pointer interpreted as string". That is probably not what we intended
to do, so fix it (by changing qsort_strcmp() so it matches the example
from qsort()'s manual page).
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Message-id: 20161012204907.25941-2-mreitz@redhat.com
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

The leak was introduced in commit 94d6a7a7.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

It was from the time when none of the global functions had a qcow2_
prefix.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

We make sure that the size is aligned to sector length to prevent any
round ups. Otherwise we could end up reading/writing data outside the
area specified by user. This is only needed when user supplies the size
option to avoid any surprises. It is not necessary when only offset is
set.

More over, the check made it difficult to use the offset option without
size option. The check puts unneeded restriction on the offset which had
to be aligned too. Because bdrv_getlength() returns aligned value having
unaligned offset would make the check fail.
Signed-off-by: NTomáš Golembiovský <tgolembi@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

When only offset is specified but no size and the offset is greater than
the real size of the containing device an overflow occurs when parsing
the options. This overflow is harmless because we do check for this
exact situation little bit later, but it leads to an error message with
weird values. It is better to do the check is sooner and prevent the
overflow.
Signed-off-by: NTomáš Golembiovský <tgolembi@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

A job ID is always required in order to create a block job on a
non-root node. The default ID (obtained with bdrv_get_device_name())
is otherwise empty in this scenario and the job cannot be created.

The HMP block_stream command doesn't set a job ID and therefore it
doesn't allow streaming to intermediate nodes. One solution is to add
an extra parameter to set a job ID. The other solution is to simply
use the node name passed to block_stream as job ID. This won't work
if it's automatically generated (because it contains a '#') but is
otherwise simple enough for all other cases.

This way 'block_stream node3' will create a job with the ID 'node3'
and the good old 'block_stream virtio0' will keep the previous
behaviour and use 'virtio0' for the job ID.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch drops the unused parameter "BDRVSSHState" being passed into
the ssh_config() function and does code cleanup. The unused parameter
was introduced by the commit c322712.
Signed-off-by: NAshijeet Acharya <ashijeetacharya@gmail.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch frees the leaked visitor in nbd_refresh_filename() and uses
visit_free() to fix it. The leak was introduced by the commit 491d6c7c.
Signed-off-by: NAshijeet Acharya <ashijeetacharya@gmail.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

ASAN spotted:
SUMMARY: AddressSanitizer: 74 byte(s) leaked in 2 allocation(s).
Signed-off-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20161109110210.25925-1-marcandre.lureau@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NFam Zheng <famz@redhat.com>

Small fixes for hard freeze.

# gpg: Signature made Thu 10 Nov 2016 03:34:24 PM GMT
# gpg:                using RSA key 0xBFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* bonzini/tags/for-upstream:
  nbd: Don't inf-loop on early EOF
  target-i386: document how x86 gdb_num_core_regs is computed.
  qdev: fix use-after-free regression from becdfa00
  target-i386/machine: fix migrate faile because of Hyper-V HV_X64_MSR_VP_RUNTIME
  vl.c: move pidfile creation up the line
  target-i386: fix typo

Message-id: 1478800362-18138-1-git-send-email-pbonzini@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

There are only very old and orphaned stable branches listed
in the MAINTAINERS file - so this section is pretty useless
nowadays. Let's remove it.
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NThomas Huth <thuth@redhat.com>

These files are currently unmaintained.

I'm proposing that Fam and I co-maintain them; under the model that
whomever between us isn't authoring a given series will be responsible
for reviewing it.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Acked-by: NFam Zheng <famz@redhat.com>
Acked-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NThomas Huth <thuth@redhat.com>

I recently added new files to the source tree that are not
covered by any maintainer yet -- and since every new source
file should have a maintainer nowadays, I volunteer to look
after these files now, too.
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NThomas Huth <thuth@redhat.com>

disas/m68k.c obviously belong to the m68k CPU section in
the MAINTAINERS file, but remove the hw/m68k/ directory
here since it only contains machine (not CPU) related
files, as requested by Laurent. Add the machine related
files to the right machine sections instead.
Reviewed-by: NLaurent Vivier <laurent@vivier.eu>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NThomas Huth <thuth@redhat.com>

Both files seem to belong to the Sun4m machine.
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NThomas Huth <thuth@redhat.com>

The files w/cpu/a*mpcore.c are already assigned to the ARM CPU
section, but the corresponding headers include/hw/cpu/a*mpcore.h
are still missing.

The file hw/*/imx* are already assigned to the i.MX31 machine, but
the corresponding header files include/hw/*/imx* are still missing.

The file hw/misc/arm_integrator_debug.c seems to belong to Integrator
CP, hw/cpu/realview_mpcore.c seems to belong to Real View, and
hw/misc/mst_fpga.c seems to belong to PXA2XX.

And the files hw/misc/zynq* and include/hw/misc/zynq* seem to belong
to the Xilinx Zynq machine.
Reviewed-by: NAlistair Francis <alistair.francis@xilinx.com>
Acked-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NThomas Huth <thuth@redhat.com>