The post-copy code does some I/O to/from an intermediate
in-memory buffer rather than direct to the underlying
I/O channel. Switch this code to use QIOChannelBuffer
instead of QEMUSizedBuffer.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Message-Id: <1461751518-12128-12-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Currently if an application initiates an outgoing migration,
it may or may not, get an error reported back on failure. If
the error occurs synchronously to the 'migrate' command
execution, the client app will see the error message. This
is the case for DNS lookup failures. If the error occurs
asynchronously to the monitor command though, the error
will be thrown away and the client left guessing about
what went wrong. This is the case for failure to connect
to the TCP server (eg due to wrong port, or firewall
rules, or other similar errors).

In the future we'll be adding more scope for errors to
happen asynchronously with the TLS protocol handshake.
TLS errors are hard to diagnose even when they are well
reported, so discarding errors entirely will make it
impossible to debug TLS connection problems.

Management apps which do migration are already using
'query-migrate' / 'info migrate' to check up on progress
of background migration operations and to see their end
status. This is a fine place to also include the error
message when things go wrong.

This patch thus adds an 'error-desc' field to the
MigrationInfo struct, which will be populated when
the 'status' is set to 'failed':

(qemu) migrate -d tcp:localhost:9001
(qemu) info migrate
capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: off compress: off events: off x-postcopy-ram: off
Migration status: failed (Error connecting to socket: Connection refused)
total time: 0 milliseconds

In the HMP, when doing non-detached migration, it is
also possible to display this error message directly
to the app.

(qemu) migrate tcp:localhost:9001
Error connecting to socket: Connection refused

Or with QMP

  {
    "execute": "query-migrate",
    "arguments": {}
  }
  {
    "return": {
      "status": "failed",
      "error-desc": "address resolution failed for myhost:9000: No address associated with hostname"
    }
  }
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-11-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Currently creating a QEMUFile instance from a QIOChannel is
quite simple only requiring a single call to
qemu_fopen_channel_input or  qemu_fopen_channel_output
depending on the end of migration connection.

When QEMU gains TLS support, however, there will need to be
a TLS negotiation done inbetween creation of the QIOChannel
and creation of the final QEMUFile. Introduce some helper
methods that will encapsulate this logic, isolating the
migration protocol drivers from knowledge about TLS.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Acked-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-10-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Introduce a new QEMUFile implementation that is based on
the QIOChannel objects. This impl is different from existing
impls in that there is no file descriptor that can be made
available, as some channels may be based on higher level
protocols such as TLS.

Although the QIOChannel based implementation can trivially
provide a bi-directional stream, initially we have separate
functions for opening input & output directions to fit with
the expectation of the current QEMUFile interface.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Message-Id: <1461751518-12128-9-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Instead of relying on the default QEMUFile I/O blocking flag
state, explicitly turn on blocking I/O for outgoing migration
since it takes place in a background thread.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-8-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Remove the assumption that every QEMUFile implementation has
a file descriptor available by introducing a new function
in QEMUFileOps to change the blocking state of a QEMUFile.

If not set, it will fallback to the original code using
the get_fd method.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-7-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

The QEMUFileOps struct contains the I/O subsystem callbacks
and the migration stage hooks. Split the hooks out into a
separate QEMUFileHooks struct to make it easier to refactor
the I/O side of QEMUFile without affecting the hooks.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-6-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

The QEMUFile writev_buffer / put_buffer functions are expected
to write out the full set of requested data, blocking until
complete. The qemu_fflush() caller does not expect to deal with
partial writes. Clarify the function comments and add a sanity
check to the code to catch mistaken implementations.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-5-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Some of the test-vmstate.c test cases use a temporary file
while others use a memory buffer. To facilitate the future
removal of the qemu_bufopen() function, convert all the tests
to use a temporary file.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-4-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

The QIOChannelBuffer's close implementation will free
the internal data buffer. It failed to reset the pointer
to NULL though, so when the object is later finalized
it will free it a second time with predictable crash.
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-3-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

The s390 skeys monitor command needs to write out a plain text
file. Currently it is using the QEMUFile class for this, but
work is ongoing to refactor QEMUFile and eliminate much code
related to it. The only feature qemu_fopen() gives over fopen()
is support for QEMU FD passing, but this can be achieved with
qemu_open() + fdopen() too. Switching to regular stdio FILE
APIs avoids the need to sprintf via an intermedia buffer which
slightly simplifies the code.
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-2-git-send-email-berrange@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

X86 queue, 2016-05-23

# gpg: Signature made Mon 23 May 2016 23:48:27 BST using RSA key ID 984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"

* remotes/ehabkost/tags/x86-pull-request:
  target-i386: kvm: Eliminate kvm_msr_entry_set()
  target-i386: kvm: Simplify MSR setting functions
  target-i386: kvm: Simplify MSR array construction
  target-i386: kvm: Increase MSR_BUF_SIZE
  target-i386: kvm: Allocate kvm_msrs struct once per VCPU
  target-i386: Call cpu_exec_init() on realize
  target-i386: Move TCG initialization to realize time
  target-i386: Move TCG initialization check to tcg_x86_init()
  cpu: Eliminate cpudef_init(), cpudef_setup()
  target-i386: Set constant model_id for qemu64/qemu32/athlon
  pc: Set CPU model-id on compat_props for pc <= 2.4
  osdep: Move default qemu_hw_version() value to a macro
  target-i386: kvm: Use X86XSaveArea struct for xsave save/load
  target-i386: Use xsave structs for ext_save_area
  target-i386: Define structs for layout of xsave area
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

migration fixes:

- ensure src block devices continue fine after a failed migration
- fail on migration blockers; helps 9p savevm/loadvm
- move autoconverge commands out of experimental state
- move the migration-specific qjson in migration/

# gpg: Signature made Mon 23 May 2016 18:15:09 BST using RSA key ID 657EF670
# gpg: Good signature from "Amit Shah <amit@amitshah.net>"
# gpg:                 aka "Amit Shah <amit@kernel.org>"
# gpg:                 aka "Amit Shah <amitshah@gmx.net>"

* remotes/amit-migration/tags/migration-2.7-1:
  migration: regain control of images when migration fails to complete
  savevm: fail if migration blockers are present
  migration: Promote improved autoconverge commands out of experimental state
  migration/qjson: Drop gratuitous use of QOM
  migration: Move qjson.[ch] to migration/
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

rng: rename RndRandom to RndRandom

# gpg: Signature made Mon 23 May 2016 16:44:58 BST using RSA key ID 657EF670
# gpg: Good signature from "Amit Shah <amit@amitshah.net>"
# gpg:                 aka "Amit Shah <amit@kernel.org>"
# gpg:                 aka "Amit Shah <amitshah@gmx.net>"

* remotes/amit-virtio-rng/tags/rng-2.7-1:
  rng-random: rename RndRandom to RngRandom
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

opencores_eth cleanups:
- use mii.h
- reduce stack usage in open_eth_start_xmit.

# gpg: Signature made Mon 23 May 2016 20:14:20 BST using RSA key ID F83FA044
# gpg: Good signature from "Max Filippov <max.filippov@cogentembedded.com>"
# gpg:                 aka "Max Filippov <jcmvbkbc@gmail.com>"

* remotes/xtensa/tags/20160523-opencores_eth:
  hw/net/opencores_eth: Allocating Large sized arrays to heap
  hw/net/opencores_eth: use mii.h
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Inline the function inside kvm_msr_entry_add().
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Simplify kvm_put_tscdeadline_msr() and
kvm_put_msr_feature_control() using kvm_msr_buf and the
kvm_msr_entry_add() helper.
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Add a helper function that appends new entries to the MSR buffer
and checks for the buffer size limit.
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

We are dangerously close to the array limits in kvm_put_msrs()
and kvm_get_msrs(): with the default mcg_cap configuration, we
can set up to 148 MSRs in kvm_put_msrs(), and if we allow mcg_cap
to be changed, we can write up to 236 MSRs.

Use 4096 bytes for the buffer, that can hold 255 kvm_msr_entry
structs.
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Instead of using 2400 bytes in the stack for 150 MSR entries in
kvm_get_msrs() and kvm_put_msrs(), allocate a buffer once for
each VCPU.
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

QOM instance_init functions are not supposed to have any side-effects,
as new objects may be created at any moment for querying property
information (see qmp_device_list_properties()).

Calling cpu_exec_init() also affects QEMU's ability to handle errors
during CPU creation, as some actions done by cpu_exec_init() can't be
reverted.

Move cpu_exec_init() call to realize so a simple object_new() won't
trigger it, and so that it is called after some basic validation of CPU
parameters.
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

QOM instance_init functions are not supposed to have any side-effects,
as new objects may be created at any moment for querying property
information (see qmp_device_list_properties()).

Move TCG initialization to realize time so it won't be called when just
doing object_new() on a X86CPU subclass.
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Instead of requiring cpu.c to check if TCG was already initialized,
simply let the function be called multiple times.
Suggested-by: NIgor Mammedov <imammedo@redhat.com>
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

x86_cpudef_init() doesn't do anything anymore, cpudef_init(),
cpudef_setup(), and x86_cpudef_init() can be finally removed.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Newer PC machines don't set hw_version, and older machines set
model-id on compat_props explicitly, so we don't need the
x86_cpudef_setup() code that sets model_id using
qemu_hw_version() anymore.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

open_eth_start_xmit has a huge stack usage of 65536 bytes approx.
Moving large arrays to heap to reduce stack usage.

Reduce size of a buffer allocated on stack to 0x600 bytes, which is the
maximal frame length when HUGEN bit is not set in MODER, only allocate
buffer on heap when that is too small. Thus heap is not used in typical
use case.
Signed-off-by: NZhou Jie <zhoujie2011@cn.fujitsu.com>
Signed-off-by: NMax Filippov <jcmvbkbc@gmail.com>

Drop local definitions of MII registers and use constants from mii.h for
registers and register bits. No functional changes.
Signed-off-by: NMax Filippov <jcmvbkbc@gmail.com>

We currently have an error path during migration that can cause
the source QEMU to abort:

migration_thread()
  migration_completion()
    runstate_is_running() ----------------> true if guest is running
    bdrv_inactivate_all() ----------------> inactivate images
    qemu_savevm_state_complete_precopy()
     ... qemu_fflush()
           socket_writev_buffer() --------> error because destination fails
         qemu_fflush() -------------------> set error on migration stream
  migration_completion() -----------------> set migrate state to FAILED
migration_thread() -----------------------> break migration loop
  vm_start() -----------------------------> restart guest with inactive
                                            images

and you get:

qemu-system-ppc64: socket_writev_buffer: Got err=104 for (32768/18446744073709551615)
qemu-system-ppc64: /home/greg/Work/qemu/qemu-master/block/io.c:1342:bdrv_co_do_pwritev: Assertion `!(bs->open_flags & 0x0800)' failed.
Aborted (core dumped)

If we try postcopy with a similar scenario, we also get the writev error
message but QEMU leaves the guest paused because entered_postcopy is true.

We could possibly do the same with precopy and leave the guest paused.
But since the historical default for migration errors is to restart the
source, this patch adds a call to bdrv_invalidate_cache_all() instead.
Signed-off-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Message-Id: <146357896785.6003.11983081732454362715.stgit@bahia.huguette.org>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Instead of relying on x86_cpudef_setup() calling
qemu_hw_version(), just make old machines set model-id explicitly
on compat_props for qemu64, qemu32, and athlon. This will allow
us to eliminate x86_cpudef_setup() later.
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

The macro will be used by code that will stop calling
qemu_hw_version() at runtime and just need a constant value.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Instead of using offset macros and bit operations in a uint32_t
array, use the X86XSaveArea struct to perform the loading/saving
operations in kvm_put_xsave() and kvm_get_xsave().
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

This doesn't introduce any change in the code, as the offsets and
struct sizes match what was present in the table. This can be
validated by the QEMU_BUILD_BUG_ON lines on target-i386/cpu.h,
which ensures the struct sizes and offsets match the existing
values in ext_save_area.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

Add structs that define the layout of the xsave areas used by
Intel processors. Add some QEMU_BUILD_BUG_ON lines to ensure the
structs match the XSAVE_* macros in target-i386/kvm.c and the
offsets and sizes at target-i386/cpu.c:ext_save_areas.
Signed-off-by: NEduardo Habkost <ehabkost@redhat.com>

QEMU has currently two ways to prevent migration to occur:
- migration blocker when it depends on runtime state
- VMStateDescription.unmigratable when migration is not supported at all

This patch gathers all the logic into a single function to be called from
both the savevm and the migrate paths.

This fixes a bug with 9p, at least, where savevm would succeed and the
following would happen in the guest after loadvm:

$ ls /host
ls: cannot access /host: Protocol error

With this patch:

(qemu) savevm foo
Migration is disabled when VirtFS export path '/' is mounted in the guest
using mount_tag 'host'
Signed-off-by: NGreg Kurz <gkurz@linux.vnet.ibm.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Message-Id: <146239057139.11271.9011797645454781543.stgit@bahia.huguette.org>

[Update subject according to Paolo's suggestion - Amit]
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

* NMI cleanups (Bandan)
* RAMBlock/Memory cleanups and fixes (Dominik, Gonglei, Fam, me)
* first part of linuxboot support for fw_cfg DMA (Richard)
* IOAPIC fix (Peter Xu)
* iSCSI SG_IO fix (Vadim)
* Various infrastructure bug fixes (Zhijian, Peter M., Stefan)
* CVE fixes (Prasad)

# gpg: Signature made Mon 23 May 2016 16:06:18 BST using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"

* remotes/bonzini/tags/for-upstream: (24 commits)
  cpus: call the core nmi injection function
  nmi: remove x86 specific nmi handling
  target-i386: add a generic x86 nmi handler
  coccinelle: add g_assert_cmp* to macro file
  iscsi: pass SCSI status back for SG_IO
  esp: check dma length before reading scsi command(CVE-2016-4441)
  esp: check command buffer length before write(CVE-2016-4439)
  scripts/signrom.py: Check for magic in option ROMs.
  scripts/signrom.py: Allow option ROM checksum script to write the size header.
  Remove config-devices.mak on 'make clean'
  cpus.c: Use pthread_sigmask() rather than sigprocmask()
  memory: remove unnecessary masking of MemoryRegion ram_addr
  memory: Drop FlatRange.romd_mode
  memory: Remove code for mr->may_overlap
  exec: adjust rcu_read_lock requirement
  memory: drop find_ram_block()
  vl: change runstate only if new state is different from current state
  ioapic: clear remote irr bit for edge-triggered interrupts
  ioapic: keep RO bits for IOAPIC entry
  target-i386: key sfence availability on CPUID_SSE, not CPUID_SSE2
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

We can call the common function here directly since
x86 specific actions will be taken care of by the arch
specific nmi handler
Signed-off-by: NBandan Das <bsd@redhat.com>
Message-Id: <1463761717-26558-4-git-send-email-bsd@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

nmi_monitor_handle is wired to call the x86 nmi
handler. So, we can directly use it at call sites.
Signed-off-by: NBandan Das <bsd@redhat.com>
Message-Id: <1463761717-26558-3-git-send-email-bsd@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Instead of having x86 ifdefs in core nmi code, this
change adds a arch specific handler that the nmi common
code can call.
Signed-off-by: NBandan Das <bsd@redhat.com>
Message-Id: <1463761717-26558-2-git-send-email-bsd@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This helps applying semantic patches to unit tests.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Signed-off-by: NVadim Rozenfeld <vrozenfe@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>