NUMA queue, 2015-07-03

# gpg: Signature made Fri Jul  3 21:49:58 2015 BST using RSA key ID 984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/numa-pull-request:
  numa: API to lookup NUMA node by address
  numa: Store boot memory address range in node_info
  numa,pc-dimm: Store pc-dimm memory information in numa_info
  pc: Abort if HotplugHandlerClass::plug() fails
  pc,pc-dimm: Factor out reusable parts in pc_dimm_plug to a separate routine
  pc,pc-dimm: Extract hotplug related fields in PCMachineState to a structure
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Introduce an API numa_get_node(ram_addr_t addr, Error **errp) that
returns the NUMA node to which the given address belongs to. This
API works uniformly for both boot time as well as hotplugged memory.

This API is needed by sPAPR PowerPC to support
ibm,dynamic-reconfiguration-memory device tree node which is needed for
memory hotplug.
Signed-off-by: NBharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Tested-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Store memory address range information of boot memory  in address
range list of numa_info.

This helps to have a common NUMA node lookup by address function that
works for both boot-time memory and hotplugged memory.
Signed-off-by: NBharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Tested-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Start storing the (start_addr, end_addr) of the pc-dimm memory
in corresponding numa_info[node] so that this information can be used
to lookup node by address.
Signed-off-by: NBharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Tested-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

HotplugHandlerClass::plug() shouldn't fail and hence use error_abort
to abort if it fails.
Signed-off-by: NBharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Tested-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

pc_dimm_plug() has code that will be needed for memory plug handlers
in other archs too. Extract code from pc_dimm_plug() into a generic
routine pc_dimm_memory_plug() that resides in pc-dimm.c. Also
correspondingly refactor re-usable unplug code into pc_dimm_memory_unplug().
Signed-off-by: NBharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Tested-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Move hotplug_memory_base and hotplug_memory fields of PCMachineState
into a separate structure so that the same can be made use of from
other architectures supporing memory hotplug.
Signed-off-by: NBharata B Rao <bharata@linux.vnet.ibm.com>
Reviewed-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Tested-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

virtio-input: add input routing support, update multiseat docs.

# gpg: Signature made Fri Jul  3 11:22:33 2015 BST using RSA key ID D3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"

* remotes/kraxel/tags/pull-input-20150703-1:
  update pci-bridge-seat section in docs/multiseat.txt
  virtio-input: add input routing support
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Add display and head properties for input routing to
virtio-input devices, update multiseat documentation.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

Several s390x patches including:
- missing virtio-1 related code for virtio-ccw
- bugfixes in ipl device, gdb, virtio-ccw
- bugfix in s390-ccw bios + rebuild
- introduce versioned machines for s390-ccw-virtio

# gpg: Signature made Thu Jul  2 15:05:34 2015 BST using RSA key ID C6F02FAF
# gpg: Good signature from "Cornelia Huck <huckc@linux.vnet.ibm.com>"
# gpg:                 aka "Cornelia Huck <cornelia.huck@de.ibm.com>"

* remotes/cohuck/tags/s390x-20150702-v3:
  s390x/migration: Introduce 2.4 machine
  s390x/gdb: synchronize cpu state after modifying acrs
  s390x/ipl: Fix boot if no bootindex was specified
  virtio-ccw: migrate ->revision
  s390x/virtio-ccw: support virtio-1 set_vq format
  s390x/virtio-ccw: add virtio set-revision call
  s390x/css: Add a callback for when subchannel gets disabled
  s390-ccw.img: update
  s390-ccw.img: Consume service interrupts
  css: mss/mcss-e vs. migration
  virtio-ccw: complete handling of guest-initiated resets
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

The section footer changes commit f68945d4 ("Add a protective
section footer") and commit 37fb569c ("Disable section footers
on older machine types") broke migration for any non-versioned
machines.

This pinpoints a problem of s390-ccw machines: it needs to
be versioned to be compatible with future changes in common
code data structures such as section footers.

Let's introduce a version scheme for s390-ccw-virtio machines.
We will use the old s390-ccw-virtio name as alias to the latest
version as all existing libvirt XML for the ccw type were expanded
by libvirt to that name.

The only downside of this patch is, that the old alias s390-ccw
will no longer be available as machines can have only one alias,
but it should not really matter.

Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Cc: Boris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Cc: Jason J. Herne <jjherne@linux.vnet.ibm.com>
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Message-Id: <1435742217-62246-1-git-send-email-borntraeger@de.ibm.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Whenever we touch the access control registers, we have to make sure that
the values will make it into kvm. Otherwise the change will simply be lost.

When synchronizing qemu and kvm, a normal KVM_PUT_RUNTIME_STATE does not take
care of these registers. Let's simply trigger a KVM_PUT_FULL_STATE sync,
so the values will directly be written to kvm. The performance overhead can
be ignored and this is much cleaner than manually writing these registers to kvm
via our two supported ways.
Reviewed-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

commit fa92e218 ("s390x/ipl: avoid sign extension") introduced
a regression:

qemu-system-s390x -drive file=image.qcow,format=qcow2
does not boot, the bios states
"No virtio-blk device found!"

adding bootindex=1 does boot.

The reason is that the uint32_t as return value will not do the right
thing for the return -1 (default without bootindex).
The bios itself, will interpret a 64bit -1 as autodetect (but it will
interpret 32bit -1 as ccw device address ff.ff.ffff)
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Cornelia Huck <cornelia.huck@de.ibm.com>
Cc: qemu-stable@nongnu.org # v2.3.0
Tested-by: NAurelien Jarno <aurelien@aurel32.net>
Reviewed-by: NAurelien Jarno <aurelien@aurel32.net>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

We need to migrate the revision field as well. No compatibility
concerns as we already introduced migration of ->config_vector in
this release.
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Support the new CCW_CMD_SET_VQ format for virtio-1 devices.

While we're at it, refactor the code a bit and enforce big endian
fields (which had always been required, even for legacy).
Reviewed-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>

Handle the virtio-ccw revision according to what the guest sets.
When revision 1 is selected, we have a virtio-1 standard device
with byteswapping for the virtio rings.

When a channel gets disabled, we have to revert to the legacy behavior
in case the next user of the device does not negotiate the revision 1
anymore (e.g. the boot firmware uses revision 1, but the operating
system only uses the legacy mode).

Note that revisions > 0 are still disabled.

[CH: assure memory accesses are always BE]
Signed-off-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Acked-by: NMichael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Thu Jul  2 10:10:39 2015 BST using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request:
  block: remove redundant check before g_slist_find()
  block/nfs: limit maximum readahead size to 1MB
  block/iscsi: restore compatiblity with libiscsi 1.9.0
  iotests: Use event_wait in wait_ready
  qemu-iotests: Add test case for mirror with unmap
  qemu-iotests: Make block job methods common
  block: Remove bdrv_reset_dirty
  block: Fix dirty bitmap in bdrv_co_discard
  mirror: Do zero write on target if sectors not allocated
  qmp: Add optional bool "unmap" to drive-mirror
  block: Add bdrv_get_block_status_above
  timer: Use a single definition of NSEC_PER_SEC for the whole codebase
  timer: Move NANOSECONDS_PER_SECONDS to timer.h
  blockdev: no need to drain+flush in hmp_drive_del
  qapi: Rename 'dirty-bitmap' mode to 'incremental'
  qcow2: Handle EAGAIN returned from update_refcount
  block/iscsi: add support for request timeouts
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

An empty GSList is represented by a NULL pointer, therefore it's a
perfectly valid argument for g_slist_find() and there's no need to
make any additional check.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Message-id: 1435583533-5758-1-git-send-email-berto@igalia.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

a malicious caller could otherwise specify a very
large value via the URI and force libnfs to allocate
a large amount of memory for the readahead buffer.

Cc: qemu-stable@nongnu.org
Signed-off-by: NPeter Lieven <pl@kamp.de>
Message-id: 1435317241-25585-1-git-send-email-pl@kamp.de
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

RHEL7 and others are stuck with libiscsi 1.9.0 since there
unfortunately was an ABI breakage after that release.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-id: 1435313881-19366-1-git-send-email-pl@kamp.de
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Only poll the specific type of event we are interested in, to avoid
stealing events that should be consumed by someone else.
Suggested-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

This checks that the discard on mirror source that effectively zeroes
data is also reflected by the data of target.
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Using this function would always be wrong because a dirty bitmap must
have a specific owner that consumes the dirty bits and calls
bdrv_reset_dirty_bitmap().

Remove the unused function to avoid future misuse.
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Unsetting dirty globally with discard is not very correct. The discard may zero
out sectors (depending on can_write_zeroes_with_unmap), we should replicate
this change to destination side to make sure that the guest sees the same data.

Calling bdrv_reset_dirty also troubles mirror job because the hbitmap iterator
doesn't expect unsetting of bits after current position.

So let's do it the opposite way which fixes both problems: set the dirty bits
if we are to discard it.

Reported-by: wangxiaolong@ucloud.cn
Signed-off-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

If guest discards a source cluster, mirroring with bdrv_aio_readv is overkill.
Some protocols do zero upon discard, where it's best to use
bdrv_aio_write_zeroes, otherwise, bdrv_aio_discard will be enough.
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

If specified as "true", it allows discarding on target sectors where source is
not allocated.
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Like bdrv_is_allocated_above, this function follows the backing chain until seeing
BDRV_BLOCK_ALLOCATED.  Base is not included.

Reimplement bdrv_is_allocated on top.

[Initialized bdrv_co_get_block_status_above() ret to 0 to silence
mingw64 compiler warning about the unitialized variable.  assert(bs !=
base) prevents that case but I suppose the program could be compiled
with -DNDEBUG.
--Stefan]
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Signed-off-by: NAlberto Garcia <berto@igalia.com>
Message-id: c6e55468856ba0b8f95913c4da111cc0ef266541.1434113783.git.berto@igalia.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

We want to be able to reuse this define by making it common to
multiple QEMU modules.

This also makes it an integer since there's no need for it to be a
float.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Message-id: 6375912849da2ab561046dd013684535ccecca44.1434113783.git.berto@igalia.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

bdrv_close already does that, and in fact hmp_drive_del would need
another drain after the flush (which bdrv_close does).  So remove
the duplication.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Message-id: 1432822629-25401-1-git-send-email-pbonzini@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

If we wish to make differential backups a feature that's easy to access,
it might be pertinent to rename the "dirty-bitmap" mode to "incremental"
to make it clear what /type/ of backup the dirty-bitmap is helping us
perform.

This is an API breaking change, but 2.4 has not yet gone live,
so we have this flexibility.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-id: 1433463642-21840-2-git-send-email-jsnow@redhat.com
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Fixes a crash during image compression
Signed-off-by: NJindřich Makovička <makovick@gmail.com>
Tested-by: NRichard W.M. Jones <rjones@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

libiscsi starting with 1.15 will properly support timeout of iscsi
commands. The default will remain no timeout, but this can
be changed via cmdline parameters, e.g.:

qemu -iscsi timeout=30 -drive file=iscsi://...

If a timeout occurs a reconnect is scheduled and the timed out command
will be requeued for processing after a successful reconnect.

The required API call iscsi_set_timeout is present since libiscsi
1.10 which was released in October 2013. However, due to some bugs
in the libiscsi code the use is not recommended before version 1.15.

Please note that this patch bumps the libiscsi requirement to 1.10
to have all function and macros defined. The patch fixes also a
off-by-one error in the NOP timeout calculation which was fixed
while touching these code parts.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Message-id: 1434455107-19328-1-git-send-email-pl@kamp.de
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

We need a possibility to run code when a subchannel gets disabled.
This patch adds the necessary infrastructure.
Signed-off-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>

Update for "s390-ccw.img: Consume service interrupts".
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

We have to consume the outstanding service interrupt after each
service call, otherwise a correct implementation will return
CC=2 on subsequent service calls.
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Our main channel_subsys structure is not a device (yet), but we need
to setup mss/mcss-e again if the guest had enabled it before. Use
a hack that should catch most configurations (assuming that the guest
will have enabled at least one device in higher subchannel sets or
channel subsystems if it enabled the functionality.)
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

For a guest-initiated reset, we need to not only reset the virtio device,
but also reset the VirtioCcwDevice into a clean state. This includes
resetting the indicators, or else a guest will not be able to e.g.
switch from classic interrupts to adapter interrupts.

Split off this routine into a new function virtio_ccw_reset_virtio()
to make the distinction between resetting the virtio-related devices
and the base subchannel device clear.

CC: qemu-stable@nongnu.org
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Reviewed-by: NChristian Borntraeger <borntraeger@de.ibm.com>