Lazy refcounts is a performance optimization for qcow2 that postpones
refcount metadata updates and instead marks the image dirty.  In the
case of crash or power failure the image will be left in a dirty state
and repaired next time it is opened.

Reducing metadata I/O is important for cache=writethrough and
cache=directsync because these modes guarantee that data is on disk
after each write (hence we cannot take advantage of caching updates in
RAM).  Refcount metadata is not needed for guest->file block address
translation and therefore does not need to be on-disk at the time of
write completion - this is the motivation behind the lazy refcount
optimization.

The lazy refcount optimization must be enabled at image creation time:

  qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on a.qcow2 10G
  qemu-system-x86_64 -drive if=virtio,file=a.qcow2,cache=writethrough

Update qemu-iotests 031 and 036 since the extension header size changes
when we add feature bit table entries.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Hide the default lazy_refcounts=off output from qemu-img like we do with
other image creation options.  This ensures that existing golden outputs
continue to pass despite the new option that has been added.

Note that this patch applies before the one that actually introduces the
lazy_refcounts=on|off option.  This ensures git-bisect(1) continues to
work.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The lazy refcounts bit indicates that this image can take advantage of
the dirty bit and that refcount updates can be postponed.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch adds an incompatible feature bit to mark images that have not
been closed cleanly.  When a dirty image file is opened a consistency
check and repair is performed.

Update qemu-iotests 031 and 036 since the extension header size changes
when we add feature bit table entries.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The dirty bit will make it possible to perform lazy refcount updates,
where the image file is not kept consistent all the time.  Upon opening
a dirty image file, it is necessary to perform a consistency check and
repair any incorrect refcounts.

Therefore the dirty bit must be an incompatible feature bit.  We don't
want old programs accessing a file with stale refcounts.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The qed.py utility can inspect and manipulate QED image files.  It can
be used for testing to see the state of image metadata and also to
inject corruptions into the image file.  It also has a scrubbing feature
to copy just the metadata out of an image file, allowing users to share
broken image files without revealing data in bug reports.

This has lived in my local repo for a long time but could be useful
to others.
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Talk about background operations in general, rather than specifically
about streaming.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Legacy -drive cyls=... are now ignored completely when the drive
doesn't back a hard disk device.  Before, they were first checked
against a hard disk's limits, then ignored.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Acked-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Commit b1f416aa breaks vhost_net
because it always registers the virtio_pci_host_notifier_read() handler
function on the ioeventfd, even when vhost_net.ko is using the ioeventfd.
The result is both QEMU and vhost_net.ko polling on the same eventfd
and the virtio_net.ko guest driver seeing inconsistent results:

  # ifconfig eth0 192.168.0.1 netmask 255.255.255.0
  virtio_net virtio0: output:id 0 is not a head!

To fix this, proceed the same as we do for irqfd: add a parameter to
virtio_queue_set_host_notifier_fd_handler and in that case only set
the notifier, not the handler.

Cc: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Tested-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Reviewed-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

* kiszka/queues/slirp:
  slirp: Handle whole 127.0.0.0/8 network as local addresses.

* 'axp-next' of git://repo.or.cz/qemu/rth:
  alpha-linux-user: Fix the getpriority syscall
  alpha-linux-user: Properly handle the non-rt sigprocmask syscall.
  alpha-linux-user: Fix a3 error return with v0 error bypass.
  linux-user: Translate pipe2 flags; add to strace
  linux-user: Allocate the right amount of space for non-fixed file maps
  linux-user: Handle O_SYNC, O_NOATIME, O_CLOEXEC, O_PATH
  linux-user: Sync fcntl.h bits with the kernel
  alpha-linux-user: Handle TARGET_SSI_IEEE_RAISE_EXCEPTION properly
  alpha-linux-user: Work around hosted mmap allocation problems
  alpha-linux-user: Fix signal handling

Alpha uses unbiased priority values in the syscall, with the a3
return value signaling error conditions.  Therefore, properly
interpret the libc getpriority as needed for the guest rather
than passing the host value through unchanged.
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Name the syscall properly for QEMU, kernel source notwithstanding.
Fix syntax errors in the code thus enabled within do_syscall.
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

We were failing to initialize a3 for syscalls that bypass the
negative return value error check.
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

If we let the kernel handle the implementation of mmap_find_vma,
via an anon mmap, we must use the size as indicated by the user
and not the size truncated to the filesize.

This happens often in ld.so, where we initially mmap the file to
the size of the text+data+bss to reserve an area, then mmap+fixed
over the top to properly handle data and bss.
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

For each target, only define the bits that appear in
arch/target/include/asm/fcntl.h.  Mirror the kernel's
asm-generic layout by handling anything undefined afterward.
Signed-off-by: NRichard Henderson <rth@twiddle.net>

We weren't aggregating the exceptions, nor raising signals properly.
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Signed-off-by: NRichard Henderson <rth@twiddle.net>

Proper signal numbers were not defined, and EXCP_INTERRUPT
was unhandled, leading to all sorts of subtle confusion.
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NRichard Henderson <rth@twiddle.net>

Qualifier 'volatile' is not useful for applications, it's too strict
for single threaded code but does not give the real atomicity guarantees
needed for multithreaded code.

Drop them and now useless casts.
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

The Xen 4.1 probe never uses the return value from xc_interface_open(),
so was provoking a compiler warning on newer gcc. Fix by not bothering
to put the return value anywhere.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

The Xen compile checks are currently run inside subshells. This
is unnecessary and has the effect that if do_cc() exits with
an error message then this only causes the subshell to exit,
not the whole of configure, which is confusing. Remove the
subshells, changing:
  if ( cat ; compile_prog ) ; then ...
to
  if cat && compile_prog ; then ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

The -numa option to qemu is used to create [fake] numa nodes
and expose them to the guest OS instance.

There are a couple of issues with the -numa option:

a) Max VCPU's that can be specified for a guest while using
   the qemu's -numa option is 64. Due to a typecasting issue
   when the number of VCPUs is > 32 the VCPUs don't show up
   under the specified [fake] numa nodes.

b) KVM currently has support for 160VCPUs per guest. The
   qemu's -numa option has only support for upto 64VCPUs
   per guest.
This patch addresses these two issues.

Below are examples of (a) and (b)

a) >32 VCPUs are specified with the -numa option:

/usr/local/bin/qemu-system-x86_64 \
-enable-kvm \
71:01:01 \
-net tap,ifname=tap0,script=no,downscript=no \
-vnc :4

...
Upstream qemu :
--------------

QEMU 1.1.50 monitor - type 'help' for more information
(qemu) info numa
6 nodes
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 32 33 34 35 36 37 38 39 40 41
node 0 size: 131072 MB
node 1 cpus: 10 11 12 13 14 15 16 17 18 19 42 43 44 45 46 47 48 49 50 51
node 1 size: 131072 MB
node 2 cpus: 20 21 22 23 24 25 26 27 28 29 52 53 54 55 56 57 58 59
node 2 size: 131072 MB
node 3 cpus: 30
node 3 size: 131072 MB
node 4 cpus:
node 4 size: 131072 MB
node 5 cpus: 31
node 5 size: 131072 MB

With the patch applied :
-----------------------

QEMU 1.1.50 monitor - type 'help' for more information
(qemu) info numa
6 nodes
node 0 cpus: 0 1 2 3 4 5 6 7 8 9
node 0 size: 131072 MB
node 1 cpus: 10 11 12 13 14 15 16 17 18 19
node 1 size: 131072 MB
node 2 cpus: 20 21 22 23 24 25 26 27 28 29
node 2 size: 131072 MB
node 3 cpus: 30 31 32 33 34 35 36 37 38 39
node 3 size: 131072 MB
node 4 cpus: 40 41 42 43 44 45 46 47 48 49
node 4 size: 131072 MB
node 5 cpus: 50 51 52 53 54 55 56 57 58 59
node 5 size: 131072 MB

b) >64 VCPUs specified with -numa option:

/usr/local/bin/qemu-system-x86_64 \
-enable-kvm \
-cpu Westmere,+rdtscp,+pdpe1gb,+dca,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pclmuldq,+pbe,+tm,+ht,+ss,+acpi,+d-vnc :4

...

Upstream qemu :
--------------

only 63 CPUs in NUMA mode supported.
only 64 CPUs in NUMA mode supported.
QEMU 1.1.50 monitor - type 'help' for more information
(qemu) info numa
8 nodes
node 0 cpus: 6 7 8 9 38 39 40 41 70 71 72 73
node 0 size: 65536 MB
node 1 cpus: 10 11 12 13 14 15 16 17 18 19 42 43 44 45 46 47 48 49 50 51 74 75 76 77 78 79
node 1 size: 65536 MB
node 2 cpus: 20 21 22 23 24 25 26 27 28 29 52 53 54 55 56 57 58 59 60 61
node 2 size: 65536 MB
node 3 cpus: 30 62
node 3 size: 65536 MB
node 4 cpus:
node 4 size: 65536 MB
node 5 cpus:
node 5 size: 65536 MB
node 6 cpus: 31 63
node 6 size: 65536 MB
node 7 cpus: 0 1 2 3 4 5 32 33 34 35 36 37 64 65 66 67 68 69
node 7 size: 65536 MB

With the patch applied :
-----------------------

QEMU 1.1.50 monitor - type 'help' for more information
(qemu) info numa
8 nodes
node 0 cpus: 0 1 2 3 4 5 6 7 8 9
node 0 size: 65536 MB
node 1 cpus: 10 11 12 13 14 15 16 17 18 19
node 1 size: 65536 MB
node 2 cpus: 20 21 22 23 24 25 26 27 28 29
node 2 size: 65536 MB
node 3 cpus: 30 31 32 33 34 35 36 37 38 39
node 3 size: 65536 MB
node 4 cpus: 40 41 42 43 44 45 46 47 48 49
node 4 size: 65536 MB
node 5 cpus: 50 51 52 53 54 55 56 57 58 59
node 5 size: 65536 MB
node 6 cpus: 60 61 62 63 64 65 66 67 68 69
node 6 size: 65536 MB
node 7 cpus: 70 71 72 73 74 75 76 77 78 79
Signed-off-by: NChegu Vinod &lt;chegu_vinod@hp.com&gt;, Jim Hull &lt;jim.hull@hp.com&gt;, Craig Hada <craig.hada@hp.com>
Tested-by: NEduardo Habkost <ehabkost@redhat.com>
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Signed-off-by: NBlue Swirl <blauwirbel@gmail.com>

More details:
http://buildbot.b1-systems.de/qemu/builders/xen_unstable/builds/83/steps/compile/logs/stdio

VLANState has been removed since commit a005d073, so "vlan id" should be
fetched using net_hub_id_for_client().
Signed-off-by: NDunrong Huang <riegamaths@gmail.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Commit 0f66998f makes -enable-fips conditional on Linux hosts but then uses it
unconditionally in vl.c.

Fix this by moving the fips handling to os-posix.c and adding a condition.

Cc: Paul Moore <pmoore@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Changes so translation of remote address to the host's ip address in
the virtual network happens for all addresses in the 127.0.0.0/8
network, not just 127.0.0.1.

This fixes so that hostfwd bound to addresses such as 127.0.0.2 works.
Signed-off-by: NAnders Waldenborg <anders@0x63.nu>
Signed-off-by: NJan Kiszka <jan.kiszka@siemens.com>

* bonzini/scsi-next:
  scsi: add support for ATA_PASSTHROUGH_xx scsi command
  esp: add missing const on TypeInfo structures
  esp: enable for all PCI machines
  Revert "megasas: disable due to build breakage"
  megasas: static SAS addresses
  scsi-disk: fix compilation with DEBUG_SCSI
  megasas: Update function megasys_scsi_uninit
  SCSI: STARTSTOPUNIT only eject/load media if powercondition is 0
  SCSI: Update the sense code for PREVENT REMOVAL errors

Correct the command names of opcode 0x85 and 0xa1, and calculate
their xfer size from CDB.
Signed-off-by: NCong Meng <mc@linux.vnet.ibm.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

FIPS 140-2 requires disabling certain ciphers, including DES, which is used
by VNC to obscure passwords when they are sent over the network.  The
solution for FIPS users is to disable the use of VNC password auth when the
host system is operating in FIPS compliance mode and the user has specified
'-enable-fips' on the QEMU command line.

This patch causes QEMU to emit a message to stderr when the host system is
running in FIPS mode and a VNC password was specified on the commend line.
If the system is not running in FIPS mode, or is running in FIPS mode but
VNC password authentication was not requested, QEMU operates normally.
Signed-off-by: NPaul Moore <pmoore@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

* qmp/queue/qmp:
  hmp: show the backing file depth
  block: Use bdrv_get_backing_file_depth()
  block: create bdrv_get_backing_file_depth()
  qapi: qapi.py: allow the "'" character to be escaped

* afaerber-or/qom-cpu-4:
  cpu: Move thread_kicked to CPUState
  cpu: Move thread field into CPUState
  cpu: Move CPU_COMMON_THREAD into CPUState
  qemu-thread: Let qemu_thread_is_self() return bool

* sstabellini/xen-fixes-20120801:
  fix Xen compilation
  configure: Fix xen probe with Xen 4.2 and later

* stefanha/net:
  net: add the support for -netdev socket, listen
  net: fix the coding style
  hub: add the support for hub own flow control
  net: determine if packets can be sent before net queue deliver packets
  net: cleanup deliver/deliver_iov func pointers
  net: Make "info network" output more readable info
  net: Rename qemu_del_vlan_client() to qemu_del_net_client()
  net: Rename vc local variables to nc
  net: Rename VLANClientState to NetClientState
  net: Rename non_vlan_clients to net_clients
  net: Remove VLANState
  net: Remove vlan code from net.c
  net: Convert qdev_prop_vlan to peer with hub
  net: Drop vlan argument to qemu_new_net_client()
  hub: Check that hubs are configured correctly
  net: Look up 'vlan' net clients using hubs
  net: Use hubs for the vlan feature
  net: Add a hub net client
  net: Add interface to bridge when SIOCBRADDIF isn't available

* stefanha/trivial-patches:
  exec.c: Remove out of date comment
  exec.c: Use subpages for large unaligned mappings
  exec.c: Fix off-by-one error in register_subpage
  socket: clean up redundant assignment
  qom: Clean libuser object and dependency files
  usb: Clean common object and dependency files

* aneesh/for-upstream:
  hw/9pfs: Fix assert when disabling migration
  configure: Fix build with capabilities

Remove an out of date comment: this comment used to be attached to
cpu_register_physical_memory_log(), before commit 0f0cb164 accidentally
inserted a couple of other functions between the comment and its function.
It is in any case obsolete since (a) the function arguments it refers
to have been replaced with a single MemoryRegionSection* argument and
(b) the inability to handle regions whose offset_within_address_space
and offset_within_region aren't equally aligned was fixed as part of
the rewrite of this code.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>

Registering a multi-page memory region that is non-page-aligned results
in a subpage from the start to the page boundary, some number of full
pages, and possibly another subpage from the last page boundary to the
end. The full pages will have a value for offset_within_region that is
not a multiple of TARGET_PAGE_SIZE. Accesses through softmmu are unable
to handle this and will segfault.

Handling full pages through subpages is not optimal, but only
non-page-aligned mappings take the penalty.
Signed-off-by: NTyler Hall <tylerwhall@gmail.com>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NAvi Kivity <avi@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>

subpage_register() expects "end" to be the last byte in the mapping.
Registering a non-page-aligned memory region that extends up to or
beyond a page boundary causes subpage_register() to silently fail
through the (end >= PAGE_SIZE) check.

This bug does not cause noticeable problems for mappings that do not
extend to a page boundary, though they do register an extra byte.
Signed-off-by: NTyler Hall <tylerwhall@gmail.com>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NAvi Kivity <avi@redhat.com>
Signed-off-by: NStefan Hajnoczi <stefanha@linux.vnet.ibm.com>