A set of patches collected over the holidays.  Mix of optimizations and
fixes.

# gpg: Signature made Fri 16 Jan 2015 07:42:00 GMT using RSA key ID 854083B6
# gpg: Good signature from "Amit Shah <amit@amitshah.net>"
# gpg:                 aka "Amit Shah <amit@kernel.org>"
# gpg:                 aka "Amit Shah <amitshah@gmx.net>"

* remotes/amit-migration/tags/mig-2.3-1:
  vmstate: type-check sub-arrays
  migration_cancel: shutdown migration socket
  Handle bi-directional communication for fd migration
  socket shutdown
  Tests: QEMUSizedBuffer/QEMUBuffer
  QEMUSizedBuffer: only free qsb that qemu_bufopen allocated
  xbzrle: rebuild the cache_is_cached function
  xbzrle: optimize XBZRLE to decrease the cache misses
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

While we cannot check against the type of the full array, we can check
against the type of the fields.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Force shutdown on migration socket on cancel to cause the cancel
to complete even if the socket is blocked on a dead network.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

libvirt prefers opening the TCP connection itself, for two reasons.
First, connection failed errors can be detected easier, without having
to parse qemu's error output.
Second, libvirt might be asked to secure the transfer by tunnelling the
communication through an TLS layer.
Therefore, libvirt opens the TCP connection itself and passes an FD to qemu
using QMP and a POSIX-specific mechanism.

Hence, in order to make the reverse-path work in such cases, qemu needs to
distinguish if the transmitted FD is a socket (reverse-path available)
or not (reverse-path might not be available) and use the corresponding
abstraction.
Signed-off-by: NCristian Klein <cristian.klein@cs.umu.se>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Add QEMUFile interface to allow a socket to be 'shut down' - i.e. any
reads/writes will fail (and any blocking read/write will be woken).
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NAmit Shah <amit.shah@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Modify some of tests/test-vmstate.c due to qemu_bufopen() change.
If you create a QEMUSizedBuffer yourself, you have to explicitly
free it.
Signed-off-by: NYang Hongyang <yanghy@cn.fujitsu.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Cc: Amit Shah <amit.shah@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Only free qsb that qemu_bufopen allocated, and also allow
qemu_bufopen accept qsb as input for write operation. It
will make the API more logical:
1.If you create the QEMUSizedBuffer yourself, you need to
  free it by using qsb_free() but not depends on other API
  like qemu_fclose.
2.allow qemu_bufopen() accept QEMUSizedBuffer as input for
  write operation, otherwise, it will be a little strange
  for this API won't accept the second parameter.

This brings API change, since there are only 3
users of this API currently, this change only impact the
first one which will be fixed in patch 2 of this patchset,
so I think it is safe to do this change.

1     70  tests/test-vmstate.c <<open_mem_file_read>>
            return qemu_bufopen("r", qsb);
2    404  tests/test-vmstate.c <<test_save_noskip>>
            QEMUFile *fsave = qemu_bufopen("w", NULL);
3    424  tests/test-vmstate.c <<test_save_skip>>
            QEMUFile *fsave = qemu_bufopen("w", NULL);
Signed-off-by: NYang Hongyang <yanghy@cn.fujitsu.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Cc: Amit Shah <amit.shah@redhat.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Rebuild the cache_is_cached function by cache_get_by_addr. And
drops the asserts because the caller is also asserting the same
thing.
Signed-off-by: NChenLiang <chenliang88@huawei.com>
Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Reviewed-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

Avoid hot pages being replaced by others to remarkably decrease cache
misses

Sample results with the test program which quote from xbzrle.txt ran in
vm:(migrate bandwidth:1GE and xbzrle cache size 8MB)

the test program:

include <stdlib.h>
include <stdio.h>
int main()
 {
        char *buf = (char *) calloc(4096, 4096);
        while (1) {
            int i;
            for (i = 0; i < 4096 * 4; i++) {
                buf[i * 4096 / 4]++;
            }
            printf(".");
        }
 }

before this patch:
virsh qemu-monitor-command test_vm '{"execute": "query-migrate"}'
{"return":{"expected-downtime":1020,"xbzrle-cache":{"bytes":1108284,
"cache-size":8388608,"cache-miss-rate":0.987013,"pages":18297,"overflow":8,
"cache-miss":1228737},"status":"active","setup-time":10,"total-time":52398,
"ram":{"total":12466991104,"remaining":1695744,"mbps":935.559472,
"transferred":5780760580,"dirty-sync-counter":271,"duplicate":2878530,
"dirty-pages-rate":29130,"skipped":0,"normal-bytes":5748592640,
"normal":1403465}},"id":"libvirt-706"}

18k pages sent compressed in 52 seconds.
cache-miss-rate is 98.7%, totally miss.

after optimizing:
virsh qemu-monitor-command test_vm '{"execute": "query-migrate"}'
{"return":{"expected-downtime":2054,"xbzrle-cache":{"bytes":5066763,
"cache-size":8388608,"cache-miss-rate":0.485924,"pages":194823,"overflow":0,
"cache-miss":210653},"status":"active","setup-time":11,"total-time":18729,
"ram":{"total":12466991104,"remaining":3895296,"mbps":937.663549,
"transferred":1615042219,"dirty-sync-counter":98,"duplicate":2869840,
"dirty-pages-rate":58781,"skipped":0,"normal-bytes":1588404224,
"normal":387794}},"id":"libvirt-266"}

194k pages sent compressed in 18 seconds.
The value of cache-miss-rate decrease to 48.59%.
Signed-off-by: NChenLiang <chenliang88@huawei.com>
Signed-off-by: NGonglei <arei.gonglei@huawei.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NAmit Shah <amit.shah@redhat.com>

trivial patches for 2015-01-15

# gpg: Signature made Thu 15 Jan 2015 08:26:26 GMT using RSA key ID A4C3D7DB
# gpg: Good signature from "Michael Tokarev <mjt@tls.msk.ru>"
# gpg:                 aka "Michael Tokarev <mjt@corpit.ru>"
# gpg:                 aka "Michael Tokarev <mjt@debian.org>"

* remotes/mjt/tags/pull-trivial-patches-2015-01-15:
  vl.c: fix some alignment issues
  blizzard: do not depend on VGA internals
  Makefile: Remove config.status and common.env during 'make distclean'
  target-openrisc: bugfix for dec_sys to decode instructions correctly
  Do not hang on full PTY
  misc: Fix new typos in comments
  target-arm: Fix typo in comment (seperately -> separately)
  target-tricore: Fix new typos
  migration/qemu-file.c: Don't shift left into sign bit
  translate-all: Mark map_exec() with the 'unused' attribute
  tests/hd-geo-test.c: Remove unused test_image variable
  vt82c686: avoid out-of-bounds read
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

The misalignment was caused by tabs which were used instead of spaces.
Signed-off-by: NMarcel Apfelbaum <marcel@redhat.com>
Reviewed-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

There is nothing that is used by this ARM-specific device.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

config.status and tests/qemu-iotests/common.env are generated files
that should be deleted during 'make distclean'.
Signed-off-by: NThomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Fixed the decoding of "system" instructions (starting with 0x2)
in dec_sys() in translate.c.  In particular, the l.trap instruction
is now correctly decoded, which enables for singlestepping and
breakpoints to be set in GDB.
Signed-off-by: NDavid R. Morrison <dmorrison@invlim.com>
Acked-by: NJia Liu <proljc@gmail.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Signed-off-by: NDon Slutz <dslutz@verizon.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

recieve -> receive
suprise -> surprise

Cc: Igor Mammedov <imammedo@redhat.com>
Cc: John Snow <jsnow@redhat.com>
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Cc: Peter Maydell <peter.maydell@linaro.org>
Cc: Greg Bellows <greg.bellows@linaro.org>
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

adress -> address
managment -> management

Cc: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Add a cast in qemu_get_be32() to avoid shifting left into the sign
bit of a signed integer (which is undefined behaviour in C).
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Mark map_exec() with the 'unused' attribute to avoid '-Wunused-function'
warnings on clang 3.4 or later. This means we don't need to mark it
'inline', which is what we were previously using to suppress the warning
(a trick which only works with gcc, not clang).
Signed-off-by: NSeokYeon Hwang <syeon.hwang@samsung.com>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
[PMM: tweaked comment message a little]
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

Remove unused variable test_image; this silences a clang warning.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>

superio_ioport_readb can read the 256th element of the array.
Coverity reports an out-of-bounds write in superio_ioport_writeb,
but it does not show the corresponding out-of-bounds read
because it cannot prove that it can happen.  Fix the root
cause of the problem (zhanghailang's patch instead fixes
the logic in superio_ioport_writeb).
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Nzhanghailiang <zhang.zhanghailiang@huawei.com>
Signed-off-by: NMichael Tokarev <mjt@tls.msk.ru>
Cc: qemu-stable@nongnu.org

Mostly bugfixes and cleanups from qemu-devel.  Yet another small patch from
the record/replay series, and a few SCSI and i386 patches as well.

# gpg: Signature made Wed 14 Jan 2015 09:39:14 GMT using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
  cpus: consistently use QEMU_CLOCK_VIRTUAL_RT for icount_warp_rt timer
  qemu-timer: rename timer_init to timer_init_tl
  scsi: fix cancellation when I/O was completed but DMA was not.
  rules.mak: Fix module build
  hw/scsi/lsi53c895a: add support for additional diag / debug registers
  qemu-common.h: optimise muldiv64 if int128 is available
  target-i386: do not memcpy in and out of xmm_regs
  target-i386: fix movntsd on big-endian hosts
  vl.c: fix regression when reading memory size from config file
  vl: Don't silently change topology when all -smp options were set
  vl: fix max_cpus check
  vl: Avoid unnecessary 'if' nesting
  9pfs: changed to use event_notifier instead of qemu_pipe
  vl.c: fix regression when reading machine type from config file
  char: restore stdio echo on resume from suspend.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Fix mismatch between timer_new_ms and timer_mod.
Signed-off-by: NPavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

timer_init is not called that often.  Free the name for an equivalent
of timer_new.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Commit d5776465 (scsi: Introduce scsi_req_cancel_complete, 2014-09-25)
was supposed to have no semantic change, but it missed a case.  When
r->aiocb has already been NULLed, but DMA was not complete and the
SCSI layer was waiting for scsi_req_continue, after the patch the
SCSI layer will not call the .cancel callback of SCSIBusInfo.

Fixes: d5776465
Cc: qemu-stable@nongnu.org
Reported-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Tested-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Module build is broken since commit c261d774 ( rules.mak: Fix DSO
build by pulling in archive symbols). That commit added .mo placeholders
of DSO to -y variables, in order to pull stub symbols to executable. But
the placeholders are unintentionally expanded in -y, rather than
filtered out while linking.

Fix it by moving the -objs expanding to before inserting .mo
placeholders.  Note that passing -cflags and -libs to member objects are
also moved to keep it happening before object expanding.
Reported-by: NBharata B Rao <bharata.rao@gmail.com>
Tested-by: NBharata B Rao <bharata.rao@gmail.com>
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Some ancient Linux kernels read from registers 0x09 and 0x3c-3f during
boot. According to the spec these registers are for diag and debug
purposes only. If they are absend qemu aborts on read.
Signed-off-by: NPeter Lieven <pl@kamp.de>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Let compiler do the job to optimise the function.
Signed-off-by: NFrediano Ziglio <frediano.ziglio@huawei.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NFrediano Ziglio <freddy77@gmail.com>

After the next patch, we will move the high parts of AVX and AVX512 registers
in the same array as the SSE registers.  This will make it impossible to
memcpy an array of 128-bit values in and out of xmm_regs in one swoop.
Use a for loop instead.

Similarly, always use XMM_Q in translate.c.  This avoids introducing bugs
such as the one fixed in the previous patch.
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This was accessing an XMM register's low half without going through XMM_Q.

Cc: qemu-stable@nongnu.org
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

This is happening because an actual logic is performed on the memory
arguments inside the main's switch, disregarding the config file content.

Solved by extracting the logic on a separate function and calling it
after the switch.
Signed-off-by: NMarcel Apfelbaum <marcel@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

* remotes/sstabellini/xen-2015-01-13:
  xen-hvm: increase maxmem before calling xc_domain_populate_physmap
  xen-pt: Fix PCI devices re-attach failed
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Increase maxmem before calling xc_domain_populate_physmap_exact to
avoid the risk of running out of guest memory. This way we can also
avoid complex memory calculations in libxl at domain construction
time.

This patch fixes an abort() when assigning more than 4 NICs to a VM.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Signed-off-by: NDon Slutz <dslutz@verizon.com>

# gpg: Signature made Tue 13 Jan 2015 13:48:06 GMT using RSA key ID 81AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"

* remotes/stefanha/tags/block-pull-request: (38 commits)
  NVMe: Set correct VS Value for 1.1 Compliant Controllers
  MAINTAINERS: Add migration/block* to block subsystem
  MAINTAINERS: Update email addresses for Chrysostomos Nanakos
  nvme: Fix get/set number of queues feature
  ide: Implement VPD response for ATAPI
  block: Split BLOCK_OP_TYPE_COMMIT to BLOCK_OP_TYPE_COMMIT_{SOURCE, TARGET}
  block: limited request size in write zeroes unsupported path
  coroutine: try harder not to delete coroutines
  coroutine: drop qemu_coroutine_adjust_pool_size
  coroutine: rewrite pool to avoid mutex
  QSLIST: add lock-free operations
  test-coroutine: avoid overflow on 32-bit systems
  qemu-thread: add per-thread atexit functions
  coroutine-ucontext: use __thread
  qemu-iotests: Add supported os parameter for python tests
  qemu-iotests: Add "_supported_os Linux" to 058
  qemu-iotests: Replace "/bin/true" with "true"
  .gitignore: Ignore generated "common.env"
  libqos: Convert malloc-pc allocator to a generic allocator
  migration/block: fix pending() return value
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

According to NVMe specifications Bits 15:08 represent Minor Version number.
Signed-off-by: NAnubhav Rakshit <anubhav.rakshit@gmail.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

We are moving block-migration.c to the separated migration directory,
keep this file watched by block maintainers is a good idea.
Signed-off-by: NFam Zheng <famz@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Remove first email address and let the one from which I am contributing.
Signed-off-by: NChrysostomos Nanakos <chris@include.gr>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

According to the specification, the low 16 bits should contain the number of
I/O submission queues, and the high 16 bits should contain the number of
I/O completion queues.
Signed-off-by: NAlex Friedman <alex@e8storage.com>
Acked-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

SCSI devices have multiple kinds of queries they need to respond
to, as defined in the "cmd inquiry" section in MMC-6 and SPC-3.

Relevent sections:
MMC-6 revision 2g:
      Non-VPD response data and pointer to SPC-3;
      Section 6.8 "Inquiry Command"
SPC-3 revision 23:
      Inquiry command and error handling:
      Section 6.4 "INQUIRY command"
      VPD data pages format:
      Section 7.6 "Vital product data parameters"

We implement these Vital Product Data queries for SCSI, but not for
ATAPI through IDE. The result is that if you are looking for the WWN
identifier via tools such as sg3_utils, you will be unable to query
our CD/DVD rom device to obtain it.

This patch adds the minimum number of mandatory responses as defined
by SPC-3, which include the "supported pages" response (page 0x00)
and the "Device Identification" response (page 0x83). It also correctly
responds when it receives a request for an illegal page to improve
error output from related tools.

The Device ID page contains an arbitrary list of identification
strings of various formats; the ID strings included in this patch
were chosen to mimic those provided by the libata driver when
emulating this SCSI query (model, serial, and wwn when present.)

Example:

# libata emulated response
[root@localhost ~]# sg_inq --id /dev/sda
VPD INQUIRY: Device Identification page
  Designation descriptor number 1, descriptor length: 24
    designator_type: vendor specific [0x0],  code_set: ASCII
    associated with the addressed logical unit
      vendor specific: QM00001
  Designation descriptor number 2, descriptor length: 72
    designator_type: T10 vendor identification,  code_set: ASCII
    associated with the addressed logical unit
      vendor id: ATA
      vendor specific: QEMU HARDDISK                           QM00001

# QEMU generated ATAPI response, with WWN
[root@localhost ~]# sg_inq --id /dev/sr0
VPD INQUIRY: Device Identification page
  Designation descriptor number 1, descriptor length: 24
    designator_type: vendor specific [0x0],  code_set: ASCII
    associated with the addressed logical unit
      vendor specific: QM00005
  Designation descriptor number 2, descriptor length: 72
    designator_type: T10 vendor identification,  code_set: ASCII
    associated with the addressed logical unit
      vendor id: ATA
      vendor specific: QEMU DVD-ROM                            QM00005
  Designation descriptor number 3, descriptor length: 12
    designator_type: NAA,  code_set: Binary
    associated with the addressed logical unit
      NAA 5, IEEE Company_id: 0xc50
      Vendor Specific Identifier: 0x15ea71bb
      [0x5000c50015ea71bb]

See also: hw/scsi/scsi-disk.c, scsi_disk_emulate_inquiry()
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>