The bug is that the EventNotifiers do have a NULL io_flush callback.
Because _none_ of the callbacks on the dataplane AioContext have such a
callback, aio_poll will simply do nothing.  Fixed by adding the callbacks:
the ioeventfd will always be polled (this can change in the future to
pause/resume the processing during live snapshots or similar operations);
the ioqueue will be polled if there are outstanding requests.

I must admit I have screwed up my testing somehow, because commit
2c20e711 does not work even if cherry-picked on top of 1.4.0, and this
patch fixes it there as well.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

CoQueue uses a BH to awake coroutines that were made ready to run again
using qemu_co_queue_next() or qemu_co_queue_restart_all().  The BH
currently runs in the iothread AioContext and would break coroutines
that run in a different AioContext.

This is a slightly tricky problem because the lifetime of the BH exceeds
that of the CoQueue.  This means coroutines can be awoken after CoQueue
itself has been freed.  Also, there is no qemu_co_queue_destroy()
function which we could use to handle freeing resources.

Introducing qemu_co_queue_destroy() has a ripple effect of requiring us
to also add qemu_co_mutex_destroy() and qemu_co_rwlock_destroy(), as
well as updating all callers.  Avoid doing that.

We also cannot switch from BH to GIdle function because aio_poll() does
not dispatch GIdle functions.  (GIdle functions make memory management
slightly easier because they free themselves.)

Finally, I don't want to move unlock_queue and unlock_bh into
AioContext.  That would break encapsulation - AioContext isn't supposed
to know about CoQueue.

This patch implements a different solution: each qemu_co_queue_next() or
qemu_co_queue_restart_all() call creates a new BH and list of coroutines
to wake up.  Callers tend to invoke qemu_co_queue_next() and
qemu_co_queue_restart_all() occasionally after blocking I/O, so creating
a new BH for each call shouldn't be massively inefficient.

Note that this patch does not add an interface for specifying the
AioContext.  That is left to future patches which will convert CoQueue,
CoMutex, and CoRwlock to expose AioContext.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>

Now that each AioContext has a ThreadPool and the main loop AioContext
can be fetched with bdrv_get_aio_context(), we can eliminate the concept
of a global thread pool from thread-pool.c.

The submit functions must take a ThreadPool* argument.

block/raw-posix.c and block/raw-win32.c use
aio_get_thread_pool(bdrv_get_aio_context(bs)) to fetch the main loop's
ThreadPool.

tests/test-thread-pool.c must be updated to reflect the new
thread_pool_submit() function prototypes.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>

For now bdrv_get_aio_context() is just a stub that calls
qemu_aio_get_context() since the block layer is currently tied to the
main loop AioContext.

Add the stub now so that the block layer can begin accessing its
AioContext.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>

This patch adds a ThreadPool to AioContext.  It's possible that some
AioContext instances will never use the ThreadPool, so defer creation
until aio_get_thread_pool().

The reason why AioContext should have the ThreadPool is because the
ThreadPool is bound to a AioContext instance where the work item's
callback function is invoked.  It doesn't make sense to keep the
ThreadPool pointer anywhere other than AioContext.  For example,
block/raw-posix.c can get its AioContext's ThreadPool and submit work.

Special note about headers: I used struct ThreadPool in aio.h because
there is a circular dependency if aio.h includes thread-pool.h.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>

ThreadPool is tied to an AioContext through its event notifier, which
dictates in which AioContext the work item's callback function will be
invoked.

In order to support multiple AioContexts we need to support multiple
ThreadPool instances.

This patch adds the new/free functions.  The free function deserves
special attention because it quiesces remaining worker threads.  This
requires a new condition variable and a "stopping" flag to let workers
know they should terminate once idle.

We never needed to do this before since the global threadpool was not
explicitly destroyed until process termination.

Also stash the AioContext pointer in ThreadPool so that we can call
aio_set_event_notifier() in thread_pool_free().  We didn't need to hold
onto AioContext previously since there was no free function.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>

Move global variables into a struct so multiple thread pools can be
supported in the future.

This patch does not change thread-pool.h interfaces.  There is still a
global thread pool and it is not yet possible to create/destroy
individual thread pools.  Moving the variables into a struct first makes
later patches easier to review.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>

It is very useful to get the main loop AioContext, which is a static
variable in main-loop.c.

I'm not sure whether qemu_get_aio_context() will be necessary in the
future once devices focus on using their own AioContext instead of the
main loop AioContext, but for now it allows us to refactor code to
support multiple AioContext while actually passing the main loop
AioContext.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>

If an io_flush handler is not set, qemu_aio_wait doesn't invoke
callbacks.
Signed-off-by: NMORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Using a blocking socket in the coroutine context reduces the chance of
switching to other work.  This patch makes the sheepdog driver use a
non-blocking fd always.
Signed-off-by: NMORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Otherwise, live migration of the top layer will miss zero clusters and
let the backing file show through.  This also matches what is done in qed.

QCOW2_CLUSTER_ZERO clusters are invalid in v2 image files.  Check this
directly in qcow2_get_cluster_offset instead of replicating the test
everywhere.

Cc: qemu-stable@nongnu.org
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

We already flush when the function completes.  There is no need to flush
after every compressed cluster.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The update_cluster_refcount() function increments/decrements a cluster's
refcount and then returns the new refcount value.

There is no need to flush since both update_cluster_refcount() callers
already take care of this:

1. qcow2_alloc_bytes() calls update_cluster_refcount() when compressed
   sectors will be appended to an existing cluster with enough free
   space.  qcow2_alloc_bytes() already flushes so there is no need to do
   so in update_cluster_refcount().

2. qcow2_update_snapshot_refcount() sets a cache dependency on refcounts
   if it needs to update L2 entries.  It also flushes before completing.

Removing this flush significantly speeds up qcow2 snapshot creation:

  $ qemu-img create -f qcow2 test.qcow2 -o size=50G,preallocation=metadata
  $ time qemu-img snapshot -c new test.qcow2

Time drops from more than 3 minutes to under 1 second.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Users of qcow2_update_snapshot_refcount() do not flush consistently.
qcow2_snapshot_create() flushes but qcow2_snapshot_goto() and
qcow2_snapshot_delete() do not.

Solve this by moving the bdrv_flush() into
qcow2_update_snapshot_refcount().
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Compressed writes use qcow2_alloc_bytes() to allocate space with byte
granularity.  The affected clusters' refcounts will be incremented but
we do not need to flush yet.

Set a L2 cache dependency on the refcount block cache, so that the
refcounts get written out before the L2 updates.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Since qcow2 metadata is cached we need to flush the caches, not just the
underlying file.  Use bdrv_flush(bs) instead of bdrv_flush(bs->file).

Also add the error return path when bdrv_flush() fails and move the
flush after checking for qcow2_alloc_clusters() failure so that the
qcow2_alloc_clusters() error return value takes precedence.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

update_refcount() affects the refcount cache, it does not write to disk.
Therefore bdrv_flush(bs->file) does nothing.  We need to flush the
refcount cache in order to write out the refcount updates!

While we're here also add error returns when qcow2_cache_flush() fails.
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

qcow2 images now accept a boolean lazy_refcounts options. Use it like
this:

  -drive file=test.qcow2,lazy_refcounts=on

If the option is specified on the command line, it overrides the default
specified by the qcow2 header flags that were set when creating the
image.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Any non-default -drive options are now passed down to the block drivers.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Pointing to a QemuOpts element is surprising and can lead to subtle
use-after-free errors when the QemuOpts is freed after all options are
parsed.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This adds a function that adds all entries of a QDict to a QemuOpts if
the keys are known, and leaves only the rest in the QDict.

This way a single QDict of -drive options can be processed in multiple
places (generic block layer, block driver, backing file block driver,
etc.), where each part picks the options it knows. If at the end of the
process the QDict isn't empty, the user specified an invalid option.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

The options are passed down to the block drivers, which are supposed to
remove all options they have processed. Anything that is left over in
the end is an unknown option and results in an error.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

It doesn't do anything yet except storing the options QDict in the
BlockDriverState.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

These functions don't exist until glib version 2.26. QEMU is currently only
mandating glib 2.12.

This patch replaces the functions with g_key_file_get/set_integer.

Unbreaks the build on Ubuntu 10.04 and RHEL 5.6.

Regression was introduced by 39097dafSigned-off-by: NPeter Crosthwaite <peter.crosthwaite@xilinx.com>
Message-id: 1363323879-682-1-git-send-email-peter.crosthwaite@xilinx.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

* kraxel/chardev.5:
  spice-qemu-char: Remove dead debugging code
  spice-qemu-char: Fix name parameter issues after qapi-ifying
  qemu-char.c: fix waiting for telnet connection message
  Revert "hmp: Disable chardev-add and chardev-remove"
  chardev: add udp support to qapi
  chardev: add memory (ringbuf) support to qapi
  chardev: add vc support to qapi
  chardev: add spice support to qapi
  chardev: add pipe support to qapi
  chardev: add console support to qapi
  chardev: switch pty init to qapi
  chardev: switch parallel init to qapi
  chardev: switch serial/tty init to qapi
  chardev: add stdio support to qapi
  chardev: switch file init to qapi
  chardev: add braille support to qapi
  chardev: add msmouse support to qapi
  chardev: switch null init to qapi
  chardev: add mux chardev support to qapi
  chardev: add support for qapi-based chardev initialization

Conflicts:
	ui/console.c
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

# By Andreas Färber (16) and Igor Mammedov (1)
# Via Andreas Färber
* afaerber/qom-cpu:
  target-lm32: Update VMStateDescription to LM32CPU
  target-arm: Override do_interrupt for ARMv7-M profile
  cpu: Replace do_interrupt() by CPUClass::do_interrupt method
  cpu: Pass CPUState to cpu_interrupt()
  exec: Pass CPUState to cpu_reset_interrupt()
  cpu: Move halted and interrupt_request fields to CPUState
  target-cris/helper.c: Update Coding Style
  target-i386: Update VMStateDescription to X86CPU
  cpu: Introduce cpu_class_set_vmsd()
  cpu: Register VMStateDescription through CPUState
  stubs: Add a vmstate_dummy struct for CONFIG_USER_ONLY
  vmstate: Make vmstate_register() static inline
  target-sh4: Move PVR/PRR/CVR into SuperHCPUClass
  target-sh4: Introduce SuperHCPU subclasses
  cpus: Replace open-coded CPU loop in qmp_memsave() with qemu_get_cpu()
  monitor: Use qemu_get_cpu() in monitor_set_cpu()
  cpu: Fix qemu_get_cpu() to return NULL if CPU not found

# By Laszlo Ersek (3) and others
# Via Michael Roth
* mdroth/qga-pull-3-11-2013:
  qga: implement qmp_guest_set_vcpus() for Linux with sysfs
  qga: implement qmp_guest_get_vcpus() for Linux with sysfs
  qga: introduce guest-get-vcpus / guest-set-vcpus with stubs
  qga: add guest-set-time command
  qga: add guest-get-time command
  qemu-ga: use key-value store to avoid recycling fd handles after restart
  qemu-ga: make guest-sync-delimited available during fsfreeze
  qemu-ga: fix confusing GAChannelMethod comparison
  qga: cast to int for DWORD type

# By Peter Maydell (5) and others
# Via Riku Voipio
* riku/linux-user-for-upstream:
  linux-user/syscall.c: Don't warn about unimplemented get_robust_list
  linux-user: Implement accept4
  linux-user: Implement sendfile and sendfile64
  linux-user: make bogus negative iovec lengths fail EINVAL
  linux-user: Fix layout of usage table to account for option text
  linux-user: Add more sparc syscall numbers
  linux-user: Support setgroups syscall with no groups
  linux-user: fix futex strace of FUTEX_CLOCK_REALTIME
  linux-user/syscall.c: handle FUTEX_WAIT_BITSET in do_futex
  linux-user: improve print_fcntl()
  linux-user: Add Alpha socket constants

Since commit d62e5f7036a018b2ad09f17ebd481bd28953d783
"chardev: add spice support to qapi"

It is impossible to set the debug parameter, so all the dprintf calls
are essentially nops. Since we've not needed the debug parameter in ages this
is not a problem, if it later turns out we do need some more debugging options
we can add more trace-points.
Signed-off-by: NHans de Goede <hdegoede@redhat.com>
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

The strings passed in through the qapi calls are dynamic memory, since
we want to have them stick around longer then just the call to
qemu_chr_open_spice_* we need to strdup them.
Signed-off-by: NHans de Goede <hdegoede@redhat.com>
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

    commit 01f45d98
    Author: Anthony Liguori <aliguori@us.ibm.com>
    Date:   Tue Mar 5 23:21:32 2013 +0530

       qemu-char: move text console init to console.c

Broke vc initialization for GTK.  It's a simple typo.
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Current colon position in "waiting for telnet connection" message template
produces messages like:
QEMU waiting for connection on: telnet::127.0.0.16666,server

After moving a colon to the right, we will get a correct messages like:
QEMU waiting for connection on: telnet:127.0.0.1:6666,server
Signed-off-by: NIgor Mitsyanko <i.mitsyanko@gmail.com>
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

This reverts commit 8a14952c.

This patch adds 'udp' support to qapi.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

This patch adds 'memory' support to qapi and also switches over
the memory chardev initialization to the new qapi code path.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

This patch adds 'vc' support to qapi and also switches over the
vc chardev initialization to the new qapi code path.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

This patch adds 'spicevmc' and 'spiceport' support to qapi and also
switches over the spice chardev initialization to the new qapi code
path.

This patch adds 'pipe' support to qapi and also switches over the
pipe chardev initialization to the new qapi code path.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>