Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Message-Id: <20181016112232.23241-1-dgilbert@redhat.com>
Signed-off-by: NLaurent Vivier <laurent@vivier.eu>

We have now managed to eradicate all the places in the codebase
that triggered clang's -Waddress-of-packed-member warning. Remove
the compiler flag that exempted it from our usual -Werror policy.
This will prevent any new problematic code being added in future.
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NThomas Huth <thuth@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190208132112.31493-1-peter.maydell@linaro.org>
Signed-off-by: NLaurent Vivier <laurent@vivier.eu>

The NetBSD support in Intel HAXM has beem merged upstream and is functional.
Signed-off-by: NKamil Rytarowski <n54@gmx.com>
Message-Id: <20190207233704.29978-1-n54@gmx.com>
Signed-off-by: NLaurent Vivier <laurent@vivier.eu>

Signed-off-by: NCleber Rosa <crosa@redhat.com>
Reviewed-by: NLaurent Vivier <laurent@vivier.eu>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190207193605.25676-3-crosa@redhat.com>
Signed-off-by: NLaurent Vivier <laurent@vivier.eu>

This looks like a leftover that was never implemented.
Signed-off-by: NCleber Rosa <crosa@redhat.com>
Reviewed-by: NLaurent Vivier <laurent@vivier.eu>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190207193605.25676-2-crosa@redhat.com>
Signed-off-by: NLaurent Vivier <laurent@vivier.eu>

The option is only a dummy since a long time. We've finally deprecated
it in QEMU v3.0, so it's time to remove it now.
Signed-off-by: NThomas Huth <thuth@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <1549545296-18903-3-git-send-email-thuth@redhat.com>
Signed-off-by: NLaurent Vivier <laurent@vivier.eu>

The two sections have accidentally been added again during the
merge of Paolo's and Gerd's trees.

Fixes: 3e29da9fSigned-off-by: NThomas Huth <thuth@redhat.com>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Message-Id: <1549545296-18903-2-git-send-email-thuth@redhat.com>
Signed-off-by: NLaurent Vivier <laurent@vivier.eu>

Present since commit b25b387f (2.10).
Signed-off-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NThomas Huth <thuth@redhat.com>
Message-Id: <20190206202848.16999-1-eblake@redhat.com>
Signed-off-by: NLaurent Vivier <laurent@vivier.eu>

Pull request

# gpg: Signature made Tue 12 Feb 2019 03:58:58 GMT
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  virtio-blk: cleanup using VirtIOBlock *s and VirtIODevice *vdev
  qemugdb/coroutine: fix arch_prctl has unknown return type
  iothread: fix iothread hang when stop too soon
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

nbd patches for 2019-02-11

- Add qcow2 bitmap details to 'qemu-img info'

# gpg: Signature made Mon 11 Feb 2019 20:38:32 GMT
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-02-11:
  qcow2: list of bitmaps new test 242
  qcow2: Add list of bitmaps to ImageInfoSpecificQCow2
  bdrv_query_image_info Error parameter added
  nbd/server: Kill pointless shadowed variable
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

In several part we still using req->dev or VIRTIO_DEVICE(req->dev)
when we have already defined s and vdev pointers:
    VirtIOBlock *s = req->dev;
    VirtIODevice *vdev = VIRTIO_DEVICE(s);
Signed-off-by: NStefano Garzarella <sgarzare@redhat.com>
Reviewed-by: NLiam Merwick <liam.merwick@oracle.com>
Message-id: 20190208142347.214815-1-sgarzare@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

qemu coroutine command results in following error output:

Python Exception <class 'gdb.error'> 'arch_prctl' has unknown return
type; cast the call to its declared return type: Error occurred in
Python command: 'arch_prctl' has unknown return type; cast the call to
its declared return type

Fix it by giving it what it wants: arch_prctl return type.

Information on the topic:
   https://sourceware.org/gdb/onlinedocs/gdb/Calling.htmlSigned-off-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190206151425.105871-1-vsementsov@virtuozzo.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

Lukas reported an hard to reproduce QMP iothread hang on s390 that
QEMU might hang at pthread_join() of the QMP monitor iothread before
quitting:

  Thread 1
  #0  0x000003ffad10932c in pthread_join
  #1  0x0000000109e95750 in qemu_thread_join
      at /home/thuth/devel/qemu/util/qemu-thread-posix.c:570
  #2  0x0000000109c95a1c in iothread_stop
  #3  0x0000000109bb0874 in monitor_cleanup
  #4  0x0000000109b55042 in main

While the iothread is still in the main loop:

  Thread 4
  #0  0x000003ffad0010e4 in ??
  #1  0x000003ffad553958 in g_main_context_iterate.isra.19
  #2  0x000003ffad553d90 in g_main_loop_run
  #3  0x0000000109c9585a in iothread_run
      at /home/thuth/devel/qemu/iothread.c:74
  #4  0x0000000109e94752 in qemu_thread_start
      at /home/thuth/devel/qemu/util/qemu-thread-posix.c:502
  #5  0x000003ffad10825a in start_thread
  #6  0x000003ffad00dcf2 in thread_start

IMHO it's because there's a race between the main thread and iothread
when stopping the thread in following sequence:

    main thread                       iothread
    ===========                       ==============
                                      aio_poll()
    iothread_get_g_main_context
      set iothread->worker_context
    iothread_stop
      schedule iothread_stop_bh
                                        execute iothread_stop_bh [1]
                                          set iothread->running=false
                                          (since main_loop==NULL so
                                           skip to quit main loop.
                                           Note: although main_loop is
                                           NULL but worker_context is
                                           not!)
                                      atomic_read(&iothread->worker_context) [2]
                                        create main_loop object
                                        g_main_loop_run() [3]
    pthread_join() [4]

We can see that when execute iothread_stop_bh() at [1] it's possible
that main_loop is still NULL because it's only created until the first
check of the worker_context later at [2].  Then the iothread will hang
in the main loop [3] and it'll starve the main thread too [4].

Here the simple solution should be that we check again the "running"
variable before check against worker_context.

CC: Thomas Huth <thuth@redhat.com>
CC: Dr. David Alan Gilbert <dgilbert@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Lukáš Doktor <ldoktor@redhat.com>
CC: Markus Armbruster <armbru@redhat.com>
CC: Eric Blake <eblake@redhat.com>
CC: Paolo Bonzini <pbonzini@redhat.com>
Reported-by: NLukáš Doktor <ldoktor@redhat.com>
Signed-off-by: NPeter Xu <peterx@redhat.com>
Tested-by: NThomas Huth <thuth@redhat.com>
Message-id: 20190129051432.22023-1-peterx@redhat.com
Signed-off-by: NStefan Hajnoczi <stefanha@redhat.com>

A new test file 242 added to the qemu-iotests set. It checks
the format of qcow2 specific information for the new added
section that lists details of bitmaps.
Signed-off-by: NAndrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-Id: <1549638368-530182-4-git-send-email-andrey.shinkevich@virtuozzo.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
[eblake: pep8 compliance, avoid trailing blank line]
Reviewed-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

In the 'Format specific information' section of the 'qemu-img info'
command output, the supplemental information about existing QCOW2
bitmaps will be shown, such as a bitmap name, flags and granularity:

image: /vz/vmprivate/VM1/harddisk.hdd
file format: qcow2
virtual size: 64G (68719476736 bytes)
disk size: 3.0M
cluster_size: 1048576
Format specific information:
    compat: 1.1
    lazy refcounts: true
    bitmaps:
        [0]:
            flags:
                [0]: in-use
                [1]: auto
            name: back-up1
            granularity: 65536
        [1]:
            flags:
                [0]: in-use
                [1]: auto
            name: back-up2
            granularity: 65536
    refcount bits: 16
    corrupt: false
Signed-off-by: NAndrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <1549638368-530182-3-git-send-email-andrey.shinkevich@virtuozzo.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Inform a user in case qcow2_get_specific_info fails to obtain
QCOW2 image specific information. This patch is preliminary to
the one "qcow2: Add list of bitmaps to ImageInfoSpecificQCow2".
Signed-off-by: NAndrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: NKevin Wolf <kwolf@redhat.com>
Message-Id: <1549638368-530182-2-git-send-email-andrey.shinkevich@virtuozzo.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

lgtm.com pointed out that commit 678ba275 introduced a shadowed
declaration of local variable 'bs'; thankfully, the inner 'bs'
obtained by 'blk_bs(blk)' matches the outer one given that we had
'blk_insert_bs(blk, bs, errp)' a few lines earlier, and there are
no later uses of 'bs' beyond the scope of the 'if (bitmap)' to
care if we change the value stored in 'bs' while traveling the
backing chain to find a bitmap.  So simply get rid of the extra
declaration.
Reported-by: NPeter Maydell <peter.maydell@linaro.org>
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190207191357.6665-1-eblake@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Fix dynamic tlb resize
Fix x86 host vector saturation
Diagnose missing tcg labels

# gpg: Signature made Mon 11 Feb 2019 16:57:52 GMT
# gpg:                using RSA key 64DF38E8AF7E215F
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A  05C0 64DF 38E8 AF7E 215F

* remotes/rth/tags/pull-tcg-20190211:
  cputlb: update TLB entry/index after tlb_fill
  exec-all: document that tlb_fill can trigger a TLB resize
  tcg/i386: fix unsigned vector saturating arithmetic
  tcg: Diagnose referenced labels that have not been emitted
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

We are failing to take into account that tlb_fill() can cause a
TLB resize, which renders prior TLB entry pointers/indices stale.
Fix it by re-doing the TLB entry lookups immediately after tlb_fill.

Fixes: 86e1eff8 ("tcg: introduce dynamic TLB sizing", 2019-01-28)
Reported-by: NMax Filippov <jcmvbkbc@gmail.com>
Tested-by: NMax Filippov <jcmvbkbc@gmail.com>
Signed-off-by: NEmilio G. Cota <cota@braap.org>
Message-Id: <20190209162745.12668-3-cota@braap.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

Signed-off-by: NEmilio G. Cota <cota@braap.org>
Message-Id: <20190209162745.12668-2-cota@braap.org>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

Due to a cut/paste error in the original implementation, the unsigned
vector saturating arithmetic was erroneously being calculated as signed
vector saturating arithmetic.

Fixes: 8ffafbce ("tcg/i386: Implement vector saturating arithmetic")
Signed-off-by: NMark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <20190207224258.426-1-mark.cave-ayland@ilande.co.uk>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

Currently, a jump to a label that is not defined anywhere will
be emitted not be relocated.  This results in a jump to a random
jump target.  With tcg debugging, print a diagnostic to the -d op
file and abort.

This could help debug or detect errors like
c2d9644e ("target/arm: Fix crash on conditional instruction in an IT block")
Reported-by: NRoman Kapl <code@rkapl.cz>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: NRichard Henderson <richard.henderson@linaro.org>

Testing updates:

  - .travis.yml tweaks and optimisations
  - .cirrus.yml enabled for FreeBSD CI
  - docker.py clean-ups for binfmt_misc
  - more control of vm-test builds

# gpg: Signature made Mon 11 Feb 2019 13:03:14 GMT
# gpg:                using RSA key F715F7CD46F94435F4F588658E520D61289519AE
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44
#      Subkey fingerprint: F715 F7CD 46F9 4435 F4F5  8865 8E52 0D61 2895 19AE

* remotes/stsquad/tags/pull-testing-next-110219-1:
  tests/vm: Be verbose while extracting compressed images
  docs/devel/testing: Add -a option to usermod command on docker setup
  scripts/qemu.py: allow arches use KVM for their 32bit cousins
  tests/vm: expose BUILD_TARGET, TARGET_LIST and EXTRA_CONFIGURE_OPTS
  tests/vm: add --build-target option
  tests/vm: call make check directly for netbsd/freebsd/ubuntu.i386
  tests/vm: move images to $HOME/.cache/qemu-vm/images
  tests: PEP8 cleanup of docker.py, mostly white space
  tests: docker.py be even smarter with persistent binfmt_misc
  tests: make docker.py check for persistent configs
  tests: make docker.py update use configured binfmt path
  docker: add debian-buster-arm64-cross
  archive-source.sh: Clone the submodules locally
  MAINTAINERS: Add an entry for scripts/archive-source.sh
  .travis.yml: fold --disable-tcg into alternate coroutine builds
  .travis.yml: separate tools and docs into another entry
  .travis.yml: stop requesting libffi & gettext from homebrew
  .cirrus.yml: basic compile and test for FreeBSD
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Depending of the host hardware, copying and extracting VM images can
take up to few minutes. Add verbosity to avoid the user to worry about
VMs hanging.
Signed-off-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190129175403.18017-2-philmd@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

The option -G of usermod command will remove user from other groups
not listed, i.e.: $USER will belong only to group 'docker' after
following the documentation as is.

From usermod(8) manual page:

    If the user is currently a member of a group which is not listed,
    the user will be removed from the group. This behaviour can be
    changed via the -a option, which appends the user to the current
    supplementary group list.

This patch improves the situation by adding the -a option to the
usermod command, which will just append user to the supplementary
group list.

Cc: qemu-trivial@nongnu.org
Signed-off-by: NMurilo Opsfelder Araujo <muriloo@linux.ibm.com>
Message-Id: <20190207184346.6840-1-muriloo@linux.ibm.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

A lot of architectures can run their 32 bit cousins on KVM so the
kvm_available function needs to be a little less restricting when
deciding if KVM is available.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

Now the underlying basevm support passes these along we can expose
some additional variables to our Makefile to allow more customised
tweaking of the build. For example:

  make vm-build-freebsd TARGET_LIST=aarch64-softmmu \
    EXTRA_CONFIGURE_OPTS="--disable-tools --disable-docs" \
    BUILD_TARGET=check-softfloat
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

This allows us to invoke the build with a custom target (for the VMs
that use the {target} format string specifier). Currently OpenBSD is
still hardwired due to problems running check.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

The "make check" target calls check-qtest which has the appropriate
system binaries as dependencies so we shouldn't need to do two steps
of make invocation. Doing it in two steps was a hangover from when our
make check couldn't run tests in parallel.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

It's easier to move around the images then, by replacing the
subdirectory with a symlink.  Allows to share the images between
multiple qemu checkouts for example.
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

My editor keeps putting squiggly lines under a bunch of the python
lines to remind me how non-PEP8 compliant it is. Clean that up so it's
easier to spot new errors.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

If we have a persistent mapping we don't need the QEMU binary copied
into the container as the kernel has already opened the file and will
pass the fd in. However the support libraries will still need to be
there.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

binfmt_misc configured with the "F" flag opens the interpreter at
config time. This means it can use an already open file-descriptor to
run QEMU so there is no point trying to copy the binary into a
container.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

When copying a QEMU binary into a linux-user docker image we should
check what the current configured binfmt_misc path is rather than
just assuming "/usr/bin/qemu-bin". Obviously if the user changes the
configuration afterwards they will break their images again.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

We can't build QEMU with this but we can use this image to build newer
arm64 testcases which need more up to date tools.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

We cloned the QEMU repository from the local storage. Since the
submodules are also available there, clone them too. This is
quicker and reduce network use.
Signed-off-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
[AJB: incorporated review suggestions from danpb]
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

The scripts/archive-source.sh is used by the VM tests, it makes
sense to add it in the "Build and test automation" section.
Signed-off-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

The alternate coroutine builds are really only of interest to people
running KVM (although I think you could use them for TCG if you really
tried). As they tend to run long lets kill two birds with one stone
and fold the --disable-tcg build into them.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

Re-building the tools and documents by default is a little wasteful as
they are not really affected by the main build options. Split tools
and documents into their own task with a minimal softmmu and
linux-user target list just to check they don't interact badly.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

The default package set installed on macOS builders from Travis already
includes libffi and gettext as shown by log messages:

  Skipping install of libffi formula. It is already up-to-date.
  Using libffi
  Skipping install of gettext formula. It is already up-to-date.
  Using gettext
Signed-off-by: NDaniel P. Berrangé <berrange@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>