This patch is necessary to suppress the "probed raw" warning when
running raw over nbd tests.
Signed-off-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

There is a bug in the recently added sys.platform test, and we no longer
run python tests, because "linux2" is the value to compare here. So do a
prefix match. According to python doc [1], the way to use sys.platform
is "unless you want to test for a specific system version, it is
therefore recommended to use the following idiom":

if sys.platform.startswith('freebsd'):
    # FreeBSD-specific code here...
elif sys.platform.startswith('linux'):
    # Linux-specific code here...

[1]: https://docs.python.org/2.7/library/sys.html#sys.platformSigned-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

this adds a knob to disable request merging for debugging or benchmarks if dedired.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

this patch finally introduces multiread support to virtio-blk. While
multiwrite support was there for a long time, read support was missing.

The complete merge logic is moved into virtio-blk.c which has
been the only user of request merging ever since. This is required
to be able to merge chunks of requests and immediately invoke callbacks
for those requests. Secondly, this is required to switch to
direct invocation of coroutines which is planned at a later stage.

The following benchmarks show the performance of running fio with
4 worker threads on a local ram disk. The numbers show the average
of 10 test runs after 1 run as warmup phase.

              |        4k        |       64k        |        4k
MB/s          | rd seq | rd rand | rd seq | rd rand | wr seq | wr rand
--------------+--------+---------+--------+---------+--------+--------
master        | 1221   | 1187    | 4178   | 4114    | 1745   | 1213
multiread     | 1829   | 1189    | 4639   | 4110    | 1894   | 1216
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPeter Lieven <pl@kamp.de>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

As it was not obvious (at least for me) where the 32 comes from;
add a constant for it.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NPeter Lieven <pl@kamp.de>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The commit 533ffb17 that removed qed_aiocb_info.cancel said to remove
this but didn't do it.
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This is the same way vl.c handles this.
Signed-off-by: NDon Slutz <dslutz@verizon.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

do not trim requests if the driver does not supply a limit
through BlockLimits. For write zeroes we still keep a limit
for the unsupported path to avoid allocating a big bounce buffer.
Suggested-by: NKevin Wolf <kwolf@redhat.com>
Suggested-by: NDenis V. Lunev <den@openvz.org>
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This sequence works efficiently if FALLOC_FL_ZERO_RANGE is not supported.
Unfortunately, FALLOC_FL_ZERO_RANGE is supported on really modern systems
and only for a couple of filesystems. FALLOC_FL_PUNCH_HOLE is much more
mature.

The sequence of 2 operations FALLOC_FL_PUNCH_HOLE and 0 is necessary due
to the following reasons:
- FALLOC_FL_PUNCH_HOLE creates a hole in the file, the file becomes
  sparse. In order to retain original functionality we must allocate
  disk space afterwards. This is done using fallocate(0) call
- fallocate(0) without preceeding FALLOC_FL_PUNCH_HOLE will do nothing
  if called above already allocated areas of the file, i.e. the content
  will not be zeroed

This should increase the performance a bit for not-so-modern kernels.

CC: Max Reitz <mreitz@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Peter Lieven <pl@kamp.de>
CC: Fam Zheng <famz@redhat.com>
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

There is a possibility that we are extending our image and thus writing
zeroes beyond the end of the file. In this case we do not need to care
about the hole to make sure that there is no data in the file under
this offset (pre-condition to fallocate(0) to work). We could simply call
fallocate(0).

This improves the performance of writing zeroes even on really old
platforms which do not have even FALLOC_FL_PUNCH_HOLE.

Before the patch do_fallocate was used when either
CONFIG_FALLOCATE_PUNCH_HOLE or CONFIG_FALLOCATE_ZERO_RANGE are defined.
Now the story is different. CONFIG_FALLOCATE is defined when Linux
fallocate is defined, posix_fallocate is completely different story
(CONFIG_POSIX_FALLOCATE). CONFIG_FALLOCATE is mandatory prerequite
for both CONFIG_FALLOCATE_PUNCH_HOLE and CONFIG_FALLOCATE_ZERO_RANGE
thus we are on the safe side.

CC: Max Reitz <mreitz@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Peter Lieven <pl@kamp.de>
CC: Fam Zheng <famz@redhat.com>
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This efficiently writes zeroes on Linux if the kernel is capable enough.
FALLOC_FL_ZERO_RANGE correctly handles all cases, including and not
including file expansion.

CC: Kevin Wolf <kwolf@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Peter Lieven <pl@kamp.de>
CC: Fam Zheng <famz@redhat.com>
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

move code dealing with a block device to a separate function. This will
allow to implement additional processing for ordinary files.

Please note, that xfs_code has been moved before checking for
s->has_write_zeroes as xfs_write_zeroes does not touch this flag inside.
This makes code a bit more consistent.

CC: Kevin Wolf <kwolf@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Peter Lieven <pl@kamp.de>
CC: Fam Zheng <famz@redhat.com>
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The pattern
    do {
        if (fallocate(s->fd, mode, offset, len) == 0) {
            return 0;
        }
    } while (errno == EINTR);
    ret = translate_err(-errno);
will be commonly useful in next patches. Create helper for it.

CC: Kevin Wolf <kwolf@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Peter Lieven <pl@kamp.de>
CC: Fam Zheng <famz@redhat.com>
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

actually the code
    if (ret == -ENODEV || ret == -ENOSYS || ret == -EOPNOTSUPP ||
        ret == -ENOTTY) {
        ret = -ENOTSUP;
    }
is present twice and will be added a couple more times. Create helper
for this.

CC: Kevin Wolf <kwolf@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Peter Lieven <pl@kamp.de>
CC: Fam Zheng <famz@redhat.com>
Signed-off-by: NDenis V. Lunev <den@openvz.org>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Reviewed-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

(With the previous atapi_dma flag recovery)
If migration happens between the ATAPI command being written and the
bmdma being started, the DMA is dropped.  Eventually the guest times
out and recovers, but that can take many seconds.
(This is rare, on a pingpong reading the CD continuously I hit
this about ~1/30-1/50 migrates)

I don't think we've got enough state to be able to recover safely
at this point, so I throw a 'medium error, no seek complete'
that I'm assuming guests will try and recover from an apparently
dirty CD.

OK, it's a hack, the real solution is probably to push a lot of
ATAPI state into the migration stream, but this is a fix that
works with no stream changes. Tested only on Linux (both RHEL5
(pre-libata) and RHEL7).
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

If a migration happens just after the guest has kicked
off an ATAPI command and kicked off DMA, we lose the atapi_dma
flag, and the destination tries to complete the command as PIO
rather than DMA.  This upsets Linux; modern libata based kernels
stumble and recover OK, older kernels end up passing bad data
to userspace.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

# gpg: Signature made Fri 06 Feb 2015 14:10:40 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/net-pull-request:
  monitor: more accurate completion for host_net_remove()
  net: del hub port when peer is deleted
  net: remove the wrong comment in net_init_hubport()
  monitor: print hub port name during info network
  rtl8139: simplify timer logic
  MAINTAINERS: add Jason Wang as net subsystem maintainer
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Current completion for host_net_remove will show hub ports and clients
that were not peered with hub ports. Fix this.

Cc: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: NJason Wang <jasowang@redhat.com>
Message-id: 1422860798-17495-4-git-send-email-jasowang@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

We should del hub port when peer is deleted since it will not be reused
and will only be freed during exit.
Signed-off-by: NJason Wang <jasowang@redhat.com>
Message-id: 1422860798-17495-3-git-send-email-jasowang@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Not only nic could be the one to peer.
Signed-off-by: NJason Wang <jasowang@redhat.com>
Message-id: 1422860798-17495-2-git-send-email-jasowang@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NJason Wang <jasowang@redhat.com>
Message-id: 1422860798-17495-1-git-send-email-jasowang@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Pavel Dovgalyuk reports that TimerExpire and the timer are not restored
correctly on the receiving end of migration.

It is not clear to me whether this is really the case, but we can take
the occasion to get rid of the complicated code that computes PCSTimeout
on the fly upon changes to IntrStatus/IntrMask.  Just always keep a
timer running, it will fire every ~130 seconds at most if the interrupt
is masked with TimerInt != 0.

This makes rtl8139_set_next_tctr_time idempotent (when the virtual clock
is stopped between two calls, as is the case during migration).

Tested with Frediano's qtest.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 1421765099-26190-1-git-send-email-pbonzini@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

# gpg: Signature made Fri 06 Feb 2015 13:45:06 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/tracing-pull-request:
  trace: Print PID and time in stderr traces
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

When debugging migration it's useful to know the PID of
each trace message so you can figure out if it came from the source
or the destination.

Printing the time makes it easy to do latency measurements or timings
between trace points.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Message-id: 1421746875-9962-1-git-send-email-dgilbert@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

migration/next for 20150205

# gpg: Signature made Thu 05 Feb 2015 16:17:08 GMT using RSA key ID 5872D723
# gpg: Can't check signature: public key not found

* remotes/juanquintela/tags/migration/20150205:
  fix mc146818rtc wrong subsection name to avoid vmstate_subsection_load() fail
  Tracify migration/rdma.c
  Add migration stream analyzation script
  migration: Append JSON description of migration stream
  qemu-file: Add fast ftell code path
  QJSON: Add JSON writer
  Print errors in some of the early migration failure cases.
  Migration: Add lots of trace events
  savevm: Convert fprintf to error_report
  vmstate-static-checker: update whitelist
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

coverity: Improve and extend model

# gpg: Signature made Thu 05 Feb 2015 16:20:49 GMT using RSA key ID EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"

* remotes/armbru/tags/pull-cov-model-2015-02-05:
  MAINTAINERS: Add myself as Coverity model maintainer
  coverity: Model g_free() isn't necessarily free()
  coverity: Model GLib string allocation partially
  coverity: Improve model for GLib memory allocation
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

fix mc146818rtc wrong subsection name to avoid vmstate_subsection_load() fail
during incoming migration or loadvm.
Signed-off-by: NZhang Haoyu <zhanghy@sangfor.com.cn>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Suggested-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>

Turn all the D/DD/DDDPRINTFs into trace events
Turn most of the fprintf(stderr, into error_report
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

This patch adds a python tool to the scripts directory that can read
a dumped migration stream if it contains the JSON description of the
device states. I constructs a human readable JSON stream out of it.

It's very simple to use:

  $ qemu-system-x86_64
    (qemu) migrate "exec:cat > mig"
  $ ./scripts/analyze_migration.py -f mig
Signed-off-by: NAlexander Graf <agraf@suse.de>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

One of the annoyances of the current migration format is the fact that
it's not self-describing. In fact, it's not properly describing at all.
Some code randomly scattered throughout QEMU elaborates roughly how to
read and write a stream of bytes.

We discussed an idea during KVM Forum 2013 to add a JSON description of
the migration protocol itself to the migration stream. This patch
adds a section after the VM_END migration end marker that contains
description data on what the device sections of the stream are composed of.

This approach is backwards compatible with any QEMU version reading the
stream, because QEMU just stops reading after the VM_END marker and ignores
any data following it.

With an additional external program this allows us to decipher the
contents of any migration stream and hopefully make migration bugs easier
to track down.
Signed-off-by: NAlexander Graf <agraf@suse.de>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

For ftell we flush the output buffer to ensure that we don't have anything
lingering in our internal buffers. This is a very safe thing to do.

However, with the dynamic size measurement that the dynamic vmstate
description will bring this would turn out quite slow.

Instead, we can fast path this specific measurement and just take the
internal buffers into account when telling the kernel our position.

I'm sure I overlooked some corner cases where this doesn't work, so
instead of tuning the safe, existing version, this patch adds a fast
variant of ftell that gets used by the dynamic vmstate description code
which isn't critical when it fails.
Signed-off-by: NAlexander Graf <agraf@suse.de>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

To support programmatic JSON assembly while keeping the code that generates it
readable, this patch introduces a simple JSON writer. It emits JSON serially
into a buffer in memory.

The nice thing about this writer is its simplicity and low memory overhead.
Unlike the QMP JSON writer, this one does not need to spawn QObjects for every
element it wants to represent.

This is a prerequisite for the migration stream format description generator.
Signed-off-by: NAlexander Graf <agraf@suse.de>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Mostly on the load side, so that when we get a complaint about
a migration failure we can figure out what it didn't like.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Convert a bunch of fprintfs to error_reports
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Commit 22382bb9 renamed the
'hw_cursor_x' and 'hw_cursor_y' fields in cirrus_vga.  Update the static
checker's whitelist to allow matching against the old and new names.
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Reviewed-by: NGerd Hoffmann <kraxel@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NJuan Quintela <quintela@redhat.com>

Memory allocated with GLib needs to be freed with GLib.  Freeing it
with free() instead of g_free() is a common error.  Harmless when
g_free() is a trivial wrapper around free(), which is commonly the
case.  But model the difference anyway.

In a local scan, this flags four ALLOC_FREE_MISMATCH.  Requires
--enable ALLOC_FREE_MISMATCH, because the checker is still preview.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Acked-by: NPaolo Bonzini <pbonzini@redhat.com>