The array of uint8_t's which is introduced here, contains access metadata
about the MAC registers: if a register is accessible, but partly implemented,
or if a register requires a certain compatibility flag in order to be
accessed. Currently, 6 hypothetical flags are supported (3 exist for e1000
so far) but in the future, if more than 6 flags will be needed, the datatype
of this array can simply be swapped for a larger one.

This patch is intended to solve the following current problems:

1) In a scenario of migration between different versions of QEMU, which
differ by the MAC registers implemented in them, some registers need not to
be active if a compatibility flag is set, in order to preserve the machine's
state perfectly for the older version. Checking this for each register
individually, would create a lot of clutter in the code.

2) Some registers are (or may be) only partly implemented (e.g.
placeholders that allow reading and writing, but lack other functions).
In such cases it is better to print a debug warning on read/write attempts.
As above, dealing with this functionality on a per-register level, would
require longer and more messy code.
Signed-off-by: NLeonid Bloch <leonid.bloch@ravellosystems.com>
Signed-off-by: NDmitry Fleytman <dmitry.fleytman@ravellosystems.com>
Signed-off-by: NJason Wang <jasowang@redhat.com>

This patch makes the migration of the entire array of MAC registers
possible during live migration. The entire array is just 128 KB long, so
practically no penalty should be felt when transmitting it, additionally
to the previously transmitted individual registers. The advantage here is
eliminating the need to introduce new vmstate subsections in the future,
when additional MAC registers will be implemented.

Backward compatibility is preserved by introducing a e1000-specific
boolean parameter (in a later patch), which will be on by default.
Setting it to off would enable migration to older versions of QEMU.

Additionally, this parameter will be used to control the access to the
extra MAC registers in the future.
Signed-off-by: NLeonid Bloch <leonid.bloch@ravellosystems.com>
Signed-off-by: NDmitry Fleytman <dmitry.fleytman@ravellosystems.com>
Signed-off-by: NJason Wang <jasowang@redhat.com>

This fixes some alignment and cosmetic issues. The changes are made
in order that the following patches in this series will look like
integral parts of the code surrounding them, while conforming to the
coding style. Although some changes in unrelated areas are also made.
Signed-off-by: NLeonid Bloch <leonid.bloch@ravellosystems.com>
Signed-off-by: NDmitry Fleytman <dmitry.fleytman@ravellosystems.com>
Signed-off-by: NJason Wang <jasowang@redhat.com>

Casting pointers to long won't work on 64 bit Windows.
It is not needed with the right format strings.
Signed-off-by: NStefan Weil <sw@weilnetz.de>
Signed-off-by: NJason Wang <jasowang@redhat.com>

Hopefully last big batch of s390x patches, including:
- bugfixes for LE host and for pci translation
- MAINTAINERS update
- hugetlbfs enablement (kernel patches pending)
- boot from El Torito iso images on virtio-blk
  (boot from scsi pending)
- cleanup in the ipl device code

There's also a helper function for resetting busless devices in the
qdev core in there.

# gpg: Signature made Wed 11 Nov 2015 17:49:58 GMT using RSA key ID C6F02FAF
# gpg: Good signature from "Cornelia Huck <huckc@linux.vnet.ibm.com>"
# gpg:                 aka "Cornelia Huck <cornelia.huck@de.ibm.com>"

* remotes/cohuck/tags/s390x-20151111:
  s390: deprecate the non-ccw machine in 2.5
  s390x/ipl: switch error reporting to error_setg
  s390x/ipl: clean up qom definitions and turn into TYPE_DEVICE
  qdev: provide qdev_reset_all_fn()
  pc-bios/s390-ccw: rebuild image
  pc-bios/s390-ccw: El Torito 16-bit boot image size field workaround
  pc-bios/s390-ccw: El Torito s390x boot entry check
  pc-bios/s390-ccw: ISO-9660 El Torito boot implementation
  pc-bios/s390-ccw: Always adjust virtio sector count
  s390x/kvm: don't enable CMMA when hugetlbfs will be used
  s390x: switch to memory_region_allocate_system_memory
  MAINTAINERS: update virtio-ccw/s390 git tree
  MAINTAINERS: update s390 file patterns
  s390x/pci : fix up s390 pci iommu translation function
  s390x/css: sense data endianness
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Block layer patches

# gpg: Signature made Wed 11 Nov 2015 16:03:19 GMT using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (41 commits)
  iotests: Check for quorum support in test 139
  qcow2: Fix qcow2_get_cluster_offset() for zero clusters
  iotests: Add tests for the x-blockdev-del command
  block: Add 'x-blockdev-del' QMP command
  block: Add blk_get_refcnt()
  mirror: block all operations on the target image during the job
  qemu-iotests: fix -valgrind option for check
  qemu-iotests: fix cleanup of background processes
  qemu-io: Correct error messages
  qemu-io: Check for trailing chars
  qemu-io: fix cvtnum lval types
  block: test 'blockdev-snapshot' using a file BDS as the overlay
  block: Remove inner quotation marks in iotest 085
  block: Disallow snapshots if the overlay doesn't support backing files
  throttle: Use bs->throttle_state instead of bs->io_limits_enabled
  throttle: Check for pending requests in throttle_group_unregister_bs()
  qemu-img: add check for zero-length job len
  qcow2: avoid misaligned 64bit bswap
  qemu-iotests: Test the reopening of overlay_bs in 'block-commit'
  commit: reopen overlay_bs before base
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

The non-ccw machine for s390 (s390-virtio) is not very well maintained
and caused several issues in the past:
- aliases like virtio-blk did not work for s390
- virtio refactoring failed due to long standing bugs (e.g.see
commit cb927b8a "s390-virtio: Accommodate guests using virtqueues too early")
- some features like memory hotplug will cause trouble due to virtio storage
  being above guest memory
- the boot loader bios no longer seems to work. the source code of that
  loader is also no longer maintained

2.4 changed the default to the ccw machine, let's deprecate the old
machine for 2.5.
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Acked-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Message-Id: <1446811645-25565-1-git-send-email-borntraeger@de.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Now that we can report errors in the realize function, let's replace
the fprintf's and hw_error's with error_setg.
Signed-off-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Let's move the qom definitions of the ipl device into ipl.h, replace
"s390-ipl" by a proper type define, turn it into a TYPE_DEVICE
and remove the unneeded class definition.
Signed-off-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

For TYPE_DEVICE, the dc->reset() function is not called on system resets
yet. Until that is changed, we have to manually register a reset handler.
Let's provide qdev_reset_all_fn(), that can directly be used - just like
the reset handler that is already available for qbus.
Reviewed-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Contains:
  pc-bios/s390-ccw: Always adjust virtio sector count
  pc-bios/s390-ccw: ISO-9660 El Torito boot implementation
  pc-bios/s390-ccw: El Torito s390x boot entry check
  pc-bios/s390-ccw: El Torito 16-bit boot image size field workaround
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Because of El Torito spec flaw boot image size needs to be verified.

Boot catalog entry size field has 16-bit width, and specifies size
in 512-byte units.

Thus, boot image size cannot exceed 32M.

We actually search for the file to get the file size.

This is done by scanning the ISO directory tree for the ISO block number
and reading the file size from the directory entry.
Signed-off-by: NMaxim Samoylov <max7255@linux.vnet.ibm.com>
Reviewed-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Boot entry is considered compatible if boot image is Linux kernel
with matching S390 Linux magic string.

Empty boot images with sector_count == 0 are considered broken.
Signed-off-by: NMaxim Samoylov <max7255@linux.vnet.ibm.com>
Reviewed-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

This patch enables boot from media formatted according to
ISO-9660 and El Torito bootable CD specification.

We try to boot from device as ISO-9660 media when SCSI IPL failed.

The first boot catalog entry with bootable flag is used.

ISO-9660 media with default 2048-bytes sector size only is supported.
Signed-off-by: NMaxim Samoylov <max7255@linux.vnet.ibm.com>
Reviewed-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Let's always adjust the sector number to be read using the current
virtio block size value.

This prepares for the implementation of IPL from ISO-9660 media.
Signed-off-by: NMaxim Samoylov <max7255@linux.vnet.ibm.com>
Reviewed-by: NDavid Hildenbrand <dahi@linux.vnet.ibm.com>
Reviewed-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

On hugetlbfs CMMA will not be useful as every ESSA instruction will trap.
So don't offer CMMA to guests with a hugepages backing.
Signed-off-by: NDominik Dingel <dingel@linux.vnet.ibm.com>
Acked-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

By replacing memory_region_init_ram with memory_region_allocate_system_memory
we gain goodies like mem-path backends. This will allow us to use hugetlbfs
once the kernel supports it.
Signed-off-by: NDominik Dingel <dingel@linux.vnet.ibm.com>
Acked-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

Let's reference the git branch I actually use, and add Christian's
git tree.
Acked-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

We were missing some files, and some files should get an additional
entry to add the people actually looking after the code.
Reported-by: NThomas Huth <thuth@redhat.com>
Acked-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

On s390x, each pci device has its own iommu, which is only properly
setup in qemu once the mpcifc instruction used to register the
translation table has been intercepted. Therefore, for a pci device that
is not configured or has not been initialized, proper translation is
neither required nor possible. Moreover, we may not have a host bridge
device ready yet.

This was exposed by a recent vfio change that triggers iommu translation
during the initialization of the vfio pci device. Let's do an early exit
in that case.
Reviewed-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Reviewed-by: NJens Freimann <jfrei@linux.vnet.ibm.com>
Signed-off-by: NYi Min Zhao <zyimin@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

We keep the device's sense data in a byte array (following the
architecture), but the ecws are an array of 32 bit values. If we
just blindly copy the values, the sense data will change from
de-facto BE data to de-facto cpu-endian data, which means we end
up doing an incorrect conversion on LE hosts.

Let's just explicitly convert to cpu-endianness while assembling
the irb.
Reported-by: NAndy Lutomirski <luto@kernel.org>
Tested-by: NAndy Lutomirski <luto@kernel.org>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>
Reviewed-by: NChristian Borntraeger <borntraeger@de.ibm.com>

Block patches from 2015-10-26 until 2015-11-11.

# gpg: Signature made Wed Nov 11 17:00:50 2015 CET using RSA key ID E838ACAD
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"

* mreitz/tags/pull-block-for-kevin-2015-11-11:
  iotests: Check for quorum support in test 139
  qcow2: Fix qcow2_get_cluster_offset() for zero clusters
  iotests: Add tests for the x-blockdev-del command
  block: Add 'x-blockdev-del' QMP command
  block: Add blk_get_refcnt()
  mirror: block all operations on the target image during the job
  qemu-iotests: fix -valgrind option for check
  qemu-iotests: fix cleanup of background processes
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The quorum driver is always built in, but it is disabled during
run-time if there's no SHA256 support available (see commit e94867ed).

This patch skips the quorum test in iotest 139 in that case.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Message-id: 1447172891-20410-1-git-send-email-berto@igalia.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

When searching for contiguous zero clusters, we only need to check the
cluster type. Before this patch, an increasing offset (L2E_OFFSET_MASK)
was expected, so that the function never returned more than a single
zero cluster in practice. This patch fixes it to actually return as many
contiguous zero clusters as it can.

Cc: qemu-stable@nongnu.org
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Message-id: 1446657384-5907-1-git-send-email-kwolf@redhat.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Signed-off-by: NAlberto Garcia <berto@igalia.com>
Message-id: 57c3b0d4d0c73ddadd19e5bded9492c359cc4568.1446475331.git.berto@igalia.com
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

This command is still experimental, hence the name.

This is the companion to 'blockdev-add'. It allows deleting a
BlockBackend with its associated BlockDriverState tree, or a
BlockDriverState that is not attached to any backend.

In either case, the command fails if the reference count is greater
than 1 or the BlockDriverState has any parents.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 6cfc148c77aca1da942b094d811bfa3fcf7ac7bb.1446475331.git.berto@igalia.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

This function returns the reference count of a given BlockBackend.
For convenience, it returns 0 if the BlockBackend pointer is NULL.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: dfdd8a17dbe3288842840636d2cfe5bb895abcb0.1446475331.git.berto@igalia.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

There's nothing preventing the target image from being used by other
operations during the 'drive-mirror' job, so we should block them all
until the job is done.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Message-id: 82b88fd04cde918a08a6f9a4ab85626d7fd7b502.1446475331.git.berto@igalia.com
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Commit 934659c4 switched the iotests to run qemu-io from a bash subshell,
in order to catch segfaults.  This method is incompatible with the
current valgrind_qemu_io() bash function.

Move the valgrind usage into the exec subshell in _qemu_io_wrapper(),
while making sure the original return value is passed back to the
caller.

Update test output for tests 039, 061, and 137 as it looks for the
specific subshell command when the process is terminated.
Reported-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NJeff Cody <jcody@redhat.com>
Message-id: 0066fd85d26ca641a1c25135ff2479b7985701cf.1446232490.git.jcody@redhat.com
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Commit 934659c4 switched the iotests to run qemu and qemu-nbd from a bash
subshell, in order to catch segfaults.  Unfortunately, this means the
process PID cannot be captured via '$!'. We stopped killing qemu and
qemu-nbd processes, leaving a lot of orphaned, running qemu processes
after executing iotests.

Since the process is using exec in the subshell, the PID is the
same as the subshell PID.

Track these PIDs for cleanup using pidfiles in the $TEST_DIR. Only
track the qemu PID, however, if requested - not all usage requires
killing the process.
Reported-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NJeff Cody <jcody@redhat.com>
Message-id: 9e4f958b3895b7259b98d845bb46f000ba362869.1446232490.git.jcody@redhat.com
[mreitz@redhat.com: Replaced '! -z "..."' by '-n "..."']
Signed-off-by: NMax Reitz <mreitz@redhat.com>

Reported-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Make sure there's not trailing garbage, e.g.
"64k-whatever-i-want-here"
Reported-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

cvtnum() returns int64_t: we should not be storing this
result inside of an int.

In a few cases, we need an extra sprinkling of error handling
where we expect to pass this number on towards a function that
expects something smaller than int64_t.
Reported-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This test checks that it is not possible to create a snapshot if the
requested overlay node is a BDS which does not support backing images.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This patch removes the inner quotation marks in all cases like this:

   cmd=" ... "${variable}" ... "
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This addresses scenarios like this one:

  { 'execute': 'blockdev-add', 'arguments':
    { 'options': { 'driver': 'qcow2',
                   'node-name': 'new0',
                   'file': { 'driver': 'file',
                             'filename': 'new.qcow2',
                             'node-name': 'file0' } } } }

  { 'execute': 'blockdev-snapshot', 'arguments':
    { 'node': 'virtio0',
      'overlay': 'file0' } }
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NMax Reitz <mreitz@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

There are two ways to check for I/O limits in a BlockDriverState:

- bs->throttle_state: if this pointer is not NULL, it means that this
  BDS is member of a throttling group, its ThrottleTimers structure
  has been initialized and its I/O limits are ready to be applied.

- bs->io_limits_enabled: if true it means that the throttle_state
  pointer is valid _and_ the limits are currently enabled.

The latter is used in several places to check whether a BDS has I/O
limits configured, but what it really checks is whether requests
are being throttled or not. For example, io_limits_enabled can be
temporarily set to false in cases like bdrv_read_unthrottled() without
otherwise touching the throtting configuration of that BDS.

This patch replaces bs->io_limits_enabled with bs->throttle_state in
all cases where what we really want to check is the existence of I/O
limits, not whether they are currently enabled or not.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

throttle_group_unregister_bs() removes a BlockDriverState from its
throttling group and destroys the timers. This means that there must
be no pending throttled requests at that point (because it would be
impossible to complete them), so the caller has to drain them first.

At the moment throttle_group_unregister_bs() is only called from
bdrv_io_limits_disable(), which already takes care of draining the
requests, so there's nothing to worry about, but this patch makes
this invariant explicit in the documentation and adds the relevant
assertions.
Signed-off-by: NAlberto Garcia <berto@igalia.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The mirror job doesn't update its total length until
it has already started running, so we should translate
a zero-length job-len as meaning 0%.

Otherwise, we may get divide-by-zero faults.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NJeff Cody <jcody@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

If we create a buffer directly on the stack by using 12 bytes, there's
no guarantee the 64bit value we want to swap will be aligned, which
could cause errors with undefined behavior.

Spotted with clang -fsanitize=undefined and observed in iotests 15, 26,
44, 115 and 121.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>