nbd patches for 2018-07-07

Minor improvement for tracing of NBD block status

- Vladimir Sementsov-Ogievskiy: nbd/server: fix nbd_co_send_block_status

# gpg: Signature made Sun 08 Jul 2018 02:45:18 BST
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2018-07-07:
  nbd/server: fix nbd_co_send_block_status
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Call nbd_co_send_extents() with correct length parameter
(extent.length may be smaller than original length).

Also, switch length parameter type to uint32_t, to correspond with
request->len and similar nbd_co_send_bitmap().
Signed-off-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20180704112302.471456-2-vsementsov@virtuozzo.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Bug fixes.

# gpg: Signature made Fri 06 Jul 2018 17:40:06 BST
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
  checkpatch: handle token pasting better
  ioapic: remove useless lower bounds check
  pr-manager-helper: fix memory leak on event
  qemu-char: check errno together with ret < 0
  i386: fix '-cpu ?' output for host cpu type
  qtest: Use cpu address space instead of system memory
  pr-helper: Rework socket path handling
  pr-helper: avoid error on PR IN command with zero request size
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

The mechanism to find possible type tokens can sometimes be confused and go into an
infinite loop.  This happens for example in QEMU for a line that looks like

         uint## BITS ##_t S = _S, T = _T;                            \
         uint## BITS ##_t as, at, xs, xt, xd;                        \

Because the token pasting operator does not have a space before _t, it does not
match $notPermitted.  However, (?x) is turned on in the regular expression for
modifiers, and thus ##_t matches the empty string.  As a result, annotate_values
goes in an infinite loop.

The solution is simply to remove token pasting operators from the string before
looking for modifiers.  In the example above, the string uintBITS_t will be
evaluated as a candidate modifier.  This is not optimal, but it works as long
as people do not write things like a##s##m, and it fits nicely into sub
possible.

For a similar reason, \# should be rejected always, even if it is not
at end of line or followed by whitespace.

The same patch was sent to the Linux kernel mailing list.
Reported-by: NAleksandar Markovic <amarkovic@wavecomp.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

The vector cannot be negative.  Coverity now reports this because it sees an
array access before the check, in ioapic_stat_update_irq.
Reviewed-by: NPeter Xu <peterx@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Reported by Coverity.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

In the tcp_chr_write function, we checked errno,
but errno was not reset before a read or write operation.
Therefore, this check of errno's actions is often
incorrect after EAGAIN has occurred.
we need check errno together with ret < 0.
Signed-off-by: Nxinhua.Cao <caoxinhua@huawei.com>
Message-Id: <20180704033642.15996-1-caoxinhua@huawei.com>
Reviewed-by: NMarc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>
Fixes: 9fc53a10Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Since commit d6dcc558, '-cpu ?' shows the description of the
X86_CPU_TYPE_NAME("max") for the host CPU model:

Enables all features supported by the accelerator in the current host

instead of the expected:

KVM processor with all supported host features

or

HVF processor with all supported host features

This is caused by the early use of kvm_enabled() and hvf_enabled() in
a class_init function. Since the accelerator isn't configured yet, both
helpers return false unconditionally.

A QEMU binary will only be compiled with one of these accelerators, not
both. The appropriate description can thus be decided at build time.
Signed-off-by: NGreg Kurz <groug@kaod.org>
Message-Id: <153055056654.212317.4697363278304826913.stgit@bahia.lan>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Some devices (like nvic in armv7m) are not accessable through
address_space_memory, therefore can not be tested with qtest.
Signed-off-by: NJulia Suvorova <jusual@mail.ru>
Message-Id: <20180702065237.27899-1-jusual@mail.ru>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

When reviewing Paolo's pr-helper patches I've noticed couple of
problems:

1) socket_path needs to be calculated at two different places
(one for printing out help, the other if socket activation is NOT
used),

2) even though the default socket_path is allocated in
compute_default_paths() it is the only default path the function
handles. For instance, pidfile is allocated outside of this
function. And yet again, at different places than 1)
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
Message-Id: <c791ba035f26ea957e8f3602e3009b621769b1ba.1530611283.git.mprivozn@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

After reading a PR IN command with zero request size in prh_read_request,
the resp->result field will be uninitialized and the resp.sz field will
be also uninitialized when returning to prh_co_entry.

If resp->result == GOOD (from a previous successful reply or just luck),
then the assert in prh_write_response might not be triggered and
uninitialized response will be sent.

The fix is to remove the whole handling of sz == 0 in prh_co_entry.
Those errors apply only to PR OUT commands and it's perfectly okay to
catch them later in do_pr_out and multipath_pr_out; the check for
too-short parameters in fact doesn't apply in the easy SG_IO case, as
it can be left to the target firmware even.

The result is that prh_read_request does not fail requests anymore and
prh_co_entry becomes simpler.
Reported-by: NDima Stepanov <dimastep@yandex-team.ru>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

PPC tcg seems to be failing migration tests quite regularly;
we believe this is TCG bugs in dirty bit updating; it's
not clear why PPC fails more but lets skip for the moment.

$ ./tests/migration-test
/ppc64/migration/deprecated: OK
/ppc64/migration/bad_dest: Skipping test: kvm_hv not available OK
/ppc64/migration/postcopy/unix: Skipping test: kvm_hv not available OK
/ppc64/migration/precopy/unix: Skipping test: kvm_hv not available OK
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NLaurent Vivier <lvivier@redhat.com>
Message-id: 20180706143105.93472-1-dgilbert@redhat.com
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

RISC-V: SoCify SiFive boards and connect GEM

This series has three tasks:
 1. To convert the SiFive U and E machines into SoCs and boards
 2. To connect the Cadence GEM device to the SiFive U board
 3. Fix some device tree problems with the SiFive U board

After this series the SiFive E and U boards have their SoCs split into
seperate QEMU objects, which can be used on future boards if desired.

The RISC-V Virt and Spike boards have not been converted. They haven't
been converted as they aren't physical boards, so it doesn't make a
whole lot of sense to split them into an SoC and board. The only
disadvantage with this is that they now differ to the SiFive boards.

This series also connect the Cadence GEM device to the SiFive U board.
There are some interrupt line changes requried before this is possible.

# gpg: Signature made Fri 06 Jul 2018 02:17:21 BST
# gpg:                using RSA key 21E10D29DF977054
# gpg: Good signature from "Alistair Francis <alistair@alistair23.me>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: F6C4 AC46 D493 4868 D3B8  CE8F 21E1 0D29 DF97 7054

* remotes/alistair/tags/pull-riscv-pull-20180705:
  hw/riscv/sifive_u: Connect the Cadence GEM Ethernet device
  hw/riscv/sifive_u: Move the uart device tree node under /soc/
  hw/riscv/sifive_u: Set the interrupt controller number of interrupts
  hw/riscv/sifive_u: Set the soc device tree node as a simple-bus
  hw/riscv/sifive_plic: Use gpios instead of irqs
  hw/riscv/sifive_e: Create a SiFive E SoC object
  hw/riscv/sifive_u: Create a SiFive U SoC object
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Connect the Cadence GEM ethernet device. This also requires us to
expose the plic interrupt lines.
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>
Reviewed-by: NMichael Clark <mjc@sifive.com>

Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>
Reviewed-by: NMichael Clark <mjc@sifive.com>

Set the interrupt-controller ndev to the correct number taken from the
HiFive Unleashed board.
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>
Reviewed-by: NMichael Clark <mjc@sifive.com>

To allow Linux to ennumerate devices on the /soc/ node set it as a
"simple-bus".
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>
Reviewed-by: NMichael Clark <mjc@sifive.com>

Instead of creating the interrupt in lines with qemu_allocate_irq() use
qdev_init_gpio_in() as this gives us the ability to use the qdev*gpio*()
helpers later on.
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>
Suggested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NMichael Clark <mjc@sifive.com>

Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>
Reviewed-by: NMichael Clark <mjc@sifive.com>

Create a SiFive Unleashed U54 SoC and use that in the sifive_u machine.

We leave the SoC, RAM, device tree and reset/fdt loading as part of the
machine. All the other device creation has been moved to the SoC.
Signed-off-by: NAlistair Francis <alistair.francis@wdc.com>
Reviewed-by: NMichael Clark <mjc@sifive.com>

Merge remote-tracking branch 'remotes/stsquad/tags/pull-code-coverage-and-build-tweaks-050718-3' into staging

Code coverage and other build tweaks

  - revert 208ecb3e (and drop filter for mingw, tweak for check-tcg)
  - some travis speed-ups
  - modernise code coverage support
  - docker image cleanups
  - clean-up binfmt_misc docker infrastructure
  - add debian-powerpc-user-cross image for ppc32 build

# gpg: Signature made Thu 05 Jul 2018 17:00:02 BST
# gpg:                using RSA key FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>"
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-code-coverage-and-build-tweaks-050718-3:
  docker: add linux-user powered cross builder for QEMU
  docker: add special rule for deboostrapped images
  docker: add special handling for FROM:debian-%-user targets
  docker: debian-bootstrap.pre allow customising of variant/url
  docker: drop QEMU build-dep from bootstrap
  docker: Do not run tests in 'intermediate' images
  docker: Clean the MXE base image
  docker: ubuntu: Use SDL2
  docker: ubuntu: Update the package list before installing new ones
  linux-user: add gcov support to preexit_cleanup
  linux-user: introduce preexit_cleanup
  build-system: add coverage-report target
  build-system: add clean-coverage target
  travis: add gcovr summary for GCOV build
  docker: add gcovr to travis image
  .gitignore: add .gcov files
  build-system: remove per-test GCOV reporting
  travis: test out-of-tree builds
  travis: do not waste time cloning unused submodules
  Revert "Makefile: Rename TARGET_DIRS to TARGET_LIST"
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

We can't use cross compilers in the current Debian stable and Debian
sid is sketchy as hell. So for powerpc fall back to dog-fooding our
own linux-user to do the build.

As we can only build the base image with a suitably configured
source tree we fall back to checking for its existence when we can't
build it from scratch. However this does mean you don't have to keep
a static powerpc-linux-user in your active configuration just to
update the cross build image.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>

Coverity model patches for 2018-07-05

# gpg: Signature made Thu 05 Jul 2018 14:11:03 BST
# gpg:                using RSA key 3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-cov-model-2018-07-05:
  coverity-model: Fix replay_get_byte()
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

We might as well have a custom rule for this. For one thing the
dependencies are different. As the primary dependency for
docker-image-% could never be docker-image-debian-bootstrap we can
drop that test in the main rule as well.

Missing EXECUTABLE, DEB_ARCH and DEB_TYPE are treated as hard faults
now. We also error out if the EXECUTABLE file isn't there. We should
really do this with a dependency on any source rules but currently
subdir-FOO-linux-user isn't enough on a clean build.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>

These will have been build with debootstrap so we need to check
against the debian-bootstrap dockerfile. This does mean sticking to
debian-FOO-user as the naming conventions for boot-strapped images.
The actual cross image is built on top.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>

We default to the buildd variant as most of our images are for
building. However lets give the user the ability to specify "minbase"
if they want to create a simple base image for experimentation.

Allowing the tweaking of DEB_URL means we can also bootstrap other
Debian based OS's. For example:

  make docker-binfmt-image-debian-ubuntu-bionic-arm64 \
       DEB_ARCH=arm64 DEB_TYPE=bionic \
       DEB_VARIANT=minbase DEB_URL=http://ports.ubuntu.com/ \
       EXECUTABLE=./aarch64-linux-user/qemu-aarch64
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>

This is best done with any child images that actually need it.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>

We can still build the DOCKER_INTERMEDIATE_IMAGES images,
but they won't appear in 'make test*@$IMAGE'.
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

Using the duplicated same package is confusing.
Reported-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

Do not test the deprecated API versions (see cabd3584).
Debian MXE MinGW cross images are already using SDL2.
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

Since docker caches the different layers, updating the package
list does not invalidate the previous "apt-get update" layer,
and it is likely "apt-get install" hits an outdated repository.

See https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get

This fixes:

  $ make docker-image-ubuntu V=1
  ./tests/docker/docker.py build qemu:ubuntu tests/docker/dockerfiles/ubuntu.docker   --add-current-user
  Sending build context to Docker daemon  3.072kB
  [...]
  E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa_17.0.7-0ubuntu0.16.04.2_amd64.deb  404  Not Found
  E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa-dev_17.0.7-0ubuntu0.16.04.2_amd64.deb  404  Not Found
  E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
  The command '/bin/sh -c apt-get -y install $PACKAGES' returned a non-zero code: 100
  tests/docker/Makefile.include:40: recipe for target 'docker-image-ubuntu' failed
  make: *** [docker-image-ubuntu] Error 1
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

As we don't always take the normal exit path when running a guest we
can skip the normal exit destructors where gcov normally dumps it's
info. The GCC manual suggests long running programs use __gcov_dump()
to flush out the coverage state periodically so we use that here.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>

To avoid repeating ourselves move our preexit clean-up code into a
helper function. I figured the continuing effort to split of the
syscalls made it worthwhile creating a new file for it now.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>
Reviewed-by: NLaurent Vivier <laurent@vivier.eu>

This will build a coverage report under the current directory in
reports/coverage. At the users option a report can be generated by
directly invoking something like:

  make foo/bar/coverage-report.html
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

This can be used to remove any stale coverage data before any
particular test run. This is useful for analysing individual tests.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>---

This gives a more useful summary, sorted by descending % coverage,
after the tests have run. The final numbers will give an idea if our
coverage is getting better or worse.

To keep the width sane we need to post process the file that the old
gcovr tool generates. This is done with a mix of sed, awk and column
in the scripts/coverage-summary.sh script.

As quite a lot of lines don't get covered at all we filter out all the
0% lines. If the file doesn't appear it is not being exercised.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

Useful for debugging if nothing else as the gcovr on the Travis images
are a little old.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

These are temporary files generated on gcov runs and shouldn't be
included in the source tree.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

I'm not entirely sure who's using this information and certainly in a
CI environment it just washes over as additional noise. Later patches
will provide new reporting options so a user who wants to analyse
individual tests will be able to use that to get the information.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

Force one config to build 'out-of-tree' (object files and executables
are created in a tree outside the project source code).
Signed-off-by: NPhilippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>