This makes use of wide curses functions instead of 8bit functions. This
allows to type e.g. accented letters.

Unfortunately, key codes are then returned with values that could be
confused with wide characters by ncurses, so we need to add a maybe_keycode
variable to know whether the returned value is a key code or a character
(curses with wide support), or possibly both (curses without wide support).

The translation tables thus also need to be separated into key code
translation and character translation.  The curses2foo helper makes it easier
to use them.
Signed-off-by: NSamuel Thibault <samuel.thibault@ens-lyon.org>
Message-id: 20190304210532.7840-1-samuel.thibault@ens-lyon.org
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

By default, curses will only report single ESC key event after 1s delay,
since ESC is also used for keypad escape sequences. This however makes
users believe that ESC is not working. Reducing to 25ms provides good user
experience, while still allowing 25ms for keypad sequences to get in, which
should be enough.
Signed-off-by: NSamuel Thibault <samuel.thibault@ens-lyon.org>
Message-Id: <20190303172557.17139-1-samuel.thibault@ens-lyon.org>
Signed-off-by: NGerd Hoffmann <kraxel@redhat.com>

nbd patches for 2019-03-08

- support TLS client authorization in NBD servers
- iotest 223 race fix

# gpg: Signature made Fri 08 Mar 2019 17:37:59 GMT
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2019-03-08:
  iotests: Wait for qemu to end in 223
  nbd: fix outdated qapi docs syntax for tls-creds
  nbd: allow authorization with nbd-server-start QMP command
  qemu-nbd: add support for authorization of TLS clients
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Pull request

# gpg: Signature made Fri 08 Mar 2019 16:53:34 GMT
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  iothread: document about why we need explicit aio_poll()
  iothread: push gcontext earlier in the thread_fn
  iothread: create main loop unconditionally
  iothread: create the gcontext unconditionally
  iothread: replace init_done_cond with a semaphore
  hw/block/virtio-blk: Clean req->dev repetitions
  MAINTAINERS: add missing support status fields
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Block layer patches:

- qcow2: Support for external data files
- qcow2: Default to 4KB for the qcow2 cache entry size
- Apply block driver whitelist for -drive format=help
- Several qemu-iotests improvements

# gpg: Signature made Fri 08 Mar 2019 12:54:27 GMT
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (33 commits)
  qcow2 spec: Describe string header extensions
  qemu-iotests: Add dependency to qemu-nbd tool
  ahci-test: Add dependency to qemu-img tool
  qemu-iotests: amend with external data file
  qemu-iotests: General tests for qcow2 with external data file
  qemu-iotests: Preallocation with external data file
  qcow2: Implement data-file-raw create option
  qcow2: Store data file name in the image
  qcow2: Creating images with external data file
  qcow2: Add basic data-file infrastructure
  qcow2: Support external data file in qemu-img check
  qcow2: Return error for snapshot operation with data file
  qcow2: External file I/O
  qcow2: Prepare qcow2_co_block_status() for data file
  qcow2: Return 0/-errno in qcow2_alloc_compressed_cluster_offset()
  qcow2: Don't assume 0 is an invalid cluster offset
  qcow2: Prepare count_contiguous_clusters() for external data file
  qcow2: Prepare qcow2_get_cluster_type() for external data file
  qcow2: Pass bs to qcow2_get_cluster_type()
  qcow2: Basic definitions for external data files
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Fixes mixed up operands in CADDN and CADD

# gpg: Signature made Fri 08 Mar 2019 09:45:05 GMT
# gpg:                using RSA key 6E636A7E83F2DD0CFA6E6E370AD2C6396B69CA14
# gpg:                issuer "kbastian@mail.uni-paderborn.de"
# gpg: Good signature from "Bastian Koppelmann <kbastian@mail.uni-paderborn.de>" [full]
# Primary key fingerprint: 6E63 6A7E 83F2 DD0C FA6E  6E37 0AD2 C639 6B69 CA14

* remotes/bkoppelmann/tags/pull-tricore-2019-03-08:
  tricore: fixed RCR_CADDN instruction
  tricore: fixed RCR_CADD instruction
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

- qtest fixes
- Some generic clean-ups by Philippe
- macOS CI testing via cirrus-ci.com

# gpg: Signature made Fri 08 Mar 2019 08:58:47 GMT
# gpg:                using RSA key 2ED9D774FE702DB5
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2019-03-08:
  cirrus.yml: Add macOS continuous integration task
  tests/bios-tables: Improve portability by searching bash in the $PATH
  vhost-user-test: fix leaks
  tests: Do not use "\n" in g_test_message() strings
  hw/devices: Remove unused TC6393XB_RAM definition
  hw: Remove unused 'hw/devices.h' include
  tests: Move qdict-test-data.txt to tests/data/qobject/
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

# Conflicts:
#	tests/vhost-user-test.c

Fix use after free on temporary.
Optmize branch to next insn via br r0.

# gpg: Signature made Fri 08 Mar 2019 01:53:16 GMT
# gpg:                using RSA key 64DF38E8AF7E215F
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A  05C0 64DF 38E8 AF7E 215F

* remotes/rth/tags/pull-hppa-20190307:
  target/hppa: Optimize blr r0,rn
  target/hppa: Do not return freed temporary
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

qgraph project from GSoC 2018

# gpg: Signature made Thu 07 Mar 2019 16:29:17 GMT
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream-qgraph: (57 commits)
  qos-test: megasas test node
  qos-test: e1000 test node
  qos-test: eepro100 test node
  qos-test: es1370 test node
  qos-test: vmxnet3 test node
  qos-test: usb-hcd-ohci test node
  qos-test: spapr-phb test node
  qos-test: pcnet test node
  qos-test: nvme test node
  qos-test: ne2k_pci test node
  qos-test: ipoctal232 test node
  qos-test: tpci200 test node
  qos-test: ac97 test node
  tests: move virtio entirely to qos-test
  tests/libqos: remove pre-qgraph QVirtioPCIDevice API
  qos-test: virtio-scsi test node
  tests/libqos: virtio-scsi driver and interface nodes
  qos-test: vhost-user test node
  vhost-user-test: always use 256 MiB of guest memory
  tests/libqos: support multiqueue for virtio-net
  ...
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Slirp updates

Greg Kurz (1):
  slirp: Fix build with gcc 9

Marc-André Lureau (7):
  slirp: adapt a subset of QEMU vmstate code
  slirp: use libslirp migration code
  slirp: use "slirp_" prefix for inet_aton() win32 implementation
  slirp: move sources to src/ subdirectory
  slirp: add a standalone Makefile
  build-sys: link with slirp as an external project
  slirp: remove QEMU Makefile.objs

Samuel Thibault (2):
  slirp: fix big/little endian conversion in ident protocol
  slirp: Mark pieces missing IPv6 support

Vic Lee (1):
  slirp: check for ioctlsocket error and 0-length udp payload.

William Bowling (1):
  slirp: check sscanf result when emulating ident

# gpg: Signature made Thu 07 Mar 2019 11:51:20 GMT
# gpg:                using RSA key E61DBB15D4172BDEC97E92D9DB550E89F0FA54F3
# gpg: Good signature from "Samuel Thibault <samuel.thibault@aquilenet.fr>" [unknown]
# gpg:                 aka "Samuel Thibault <sthibault@debian.org>" [marginal]
# gpg:                 aka "Samuel Thibault <samuel.thibault@gnu.org>" [unknown]
# gpg:                 aka "Samuel Thibault <samuel.thibault@inria.fr>" [marginal]
# gpg:                 aka "Samuel Thibault <samuel.thibault@labri.fr>" [marginal]
# gpg:                 aka "Samuel Thibault <samuel.thibault@ens-lyon.org>" [marginal]
# gpg:                 aka "Samuel Thibault <samuel.thibault@u-bordeaux.fr>" [unknown]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 900C B024 B679 31D4 0F82  304B D017 8C76 7D06 9EE6
#      Subkey fingerprint: E61D BB15 D417 2BDE C97E  92D9 DB55 0E89 F0FA 54F3

* remotes/thibault/tags/samuel-thibault:
  slirp: remove QEMU Makefile.objs
  build-sys: link with slirp as an external project
  slirp: add a standalone Makefile
  slirp: move sources to src/ subdirectory
  slirp: use "slirp_" prefix for inet_aton() win32 implementation
  slirp: use libslirp migration code
  slirp: adapt a subset of QEMU vmstate code
  slirp: Mark pieces missing IPv6 support
  slirp: fix big/little endian conversion in ident protocol
  slirp: check sscanf result when emulating ident
  slirp: check for ioctlsocket error and 0-length udp payload.
  slirp: Fix build with gcc 9
Signed-off-by: NPeter Maydell <peter.maydell@linaro.org>

Be more specific about the string representation in header extensions.
Suggested-by: NStefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Reviewed-by: NStefan Hajnoczi <stefanha@redhat.com>

Since a9660664, some iotests use qemu-nbd.
Add a dependency to build it before using it.
This fixes:

  $ make check-block
    GEN     qemu-img-cmds.h
    CC      qemu-img.o
    LINK    qemu-img
    CC      qemu-io.o
    LINK    qemu-io
    CC      tests/qemu-iotests/socket_scm_helper.o
    LINK    tests/qemu-iotests/socket_scm_helper
  tests/qemu-iotests-quick.sh
  check: qemu-nbd not found
  make: *** [tests/Makefile.include:1059: check-tests/qemu-iotests-quick.sh] Error 1
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Since the ahci-test uses qemu-img, add a dependency to build it
before using it.
This fixes:

  $ gmake check-qtest V=1
  QTEST_QEMU_BINARY=x86_64-softmmu/qemu-system-x86_64 QTEST_QEMU_IMG=qemu-img tests/ahci-test
  Failed to execute child process "/tmp/qemu-test.19tMRF/qemu-img" (No such file or directory)
  ERROR:tests/libqos/libqos.c:192:mkimg: assertion failed: (ret && !err)
Reviewed-by: NJohn Snow <jsnow@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Test that preallocating metadata results in a somewhat larger qcow2
file, but preallocating data only affects the disk usage of the data
file and the qcow2 file stays small.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Provide an option to force QEMU to always keep the external data file
consistent as a standalone read-only raw image.

At the moment, this means making sure that write_zeroes requests are
forwarded to the data file instead of just updating the metadata, and
checking that no backing file is used.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Rather than requiring that the external data file node is passed
explicitly when creating the qcow2 node, store the filename in the
designated header extension during .bdrv_create and read it from there
as a default during .bdrv_open.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This adds a .bdrv_create option to use an external data file.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This adds a .bdrv_open option to specify the external data file node.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

For external data files, data clusters must be excluded from the
refcount calculations. Instead, an implicit refcount of 1 is assumed for
the COPIED flag.

Compressed clusters and internal snapshots are incompatible with
external data files, so print an error if they are in use for images
with an external data file.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Internal snapshots and an external data file are incompatible because
snapshots require refcounting and non-linear mapping. Return an error
for all of the snapshot operations if an external data file is in use.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This changes the qcow2 implementation to direct all guest data I/O to
s->data_file rather than bs->file, while metadata I/O still uses
bs->file. At the moment, this is still always the same, but soon we'll
add options to set s->data_file to an external data file.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Offset 0 cannot be assumed to mean an unallocated cluster any more.
Instead, the cluster type needs to be checked.

*file must refer to the data file instead of the image file if a valid
offset is returned from qcow2_co_block_status().
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

qcow2_alloc_compressed_cluster_offset() used to return the cluster
offset for success and 0 for error. This doesn't only conflict with 0 as
a valid host offset, but also loses the error code.

Similar to the change made to qcow2_alloc_cluster_offset() for
uncompressed clusters in commit 148da7ea, make the function return
0/-errno and return the allocated cluster offset in a by-reference
parameter.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

The cluster allocation code uses 0 as an invalid offset that is used in
case of errors or as "offset not yet determined". With external data
files, a host cluster offset of 0 becomes valid, though.

Define a constant INV_OFFSET (which is not cluster aligned and will
therefore never be a valid offset) that can be used for such purposes.

This removes the additional host_offset == 0 check that commit
ff52aab2 introduced; the confusion between an invalid offset and
(erroneous) allocation at offset 0 is removed with this change.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Offset 0 can be valid for normal (allocated) clusters now, so use
qcow2_get_cluster_type() instead.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This adds basic constants, struct fields and helper function for
external data file support to the implementation.

QCOW2_INCOMPAT_MASK and QCOW2_AUTOCLEAR_MASK are not updated yet so that
opening images with an external data file still fails (we don't handle
them correctly yet).
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

This adds external data file to the qcow2 spec as a new incompatible
feature.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Image creation already involves a bdrv_co_truncate() call, which allows
to specify a preallocation mode. Just pass the right mode there and
remove the code that is made redundant by this.
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Signed-off-by: NKevin Wolf <kwolf@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Various sed regexp from common.filter use sed GNU extensions.
Instead of spending time to write these regex to be POSIX compliant,
verify the GNU sed is available and use it.
Signed-off-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Bash is not always installed as /bin/bash. In particular on OpenBSD,
the package installs it in /usr/local/bin.
Use the 'env' shebang to search bash in the $PATH.

Patch created mechanically by running:

  $ git grep -lE '#! ?/bin/bash' -- tests/qemu-iotests \
    | while read f; do \
      sed -i 's|^#!.\?/bin/bash$|#!/usr/bin/env bash|' $f; \
    done
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Bash is not always installed as /bin/bash. In particular on OpenBSD,
the package installs it in /usr/local/bin.
Use the 'env' shebang to search bash in the $PATH.
Reviewed-by: NKamil Rytarowski <n54@gmx.com>
Reviewed-by: NIgor Mammedov <imammedo@redhat.com>
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: NThomas Huth <thuth@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Bash is not always installed as /bin/bash. In particular on OpenBSD,
the package installs it in /usr/local/bin.
Use the 'env' shebang to search bash in the $PATH.
Reviewed-by: NKamil Rytarowski <n54@gmx.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Signed-off-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Some test cases require specific formats. The method decorator
skip_if_unsupported() checks if requested formats are whitelisted.
The test #139 was selected for a sample output, after running
$ ./check -qcow2 131-140

137 3s ...
138 0s ...
139 2s ...
    [case not run] testBlkDebug (__main__.TestBlockdevDel): formats ['blkdebug'] are not whitelisted
    [case not run] testBlkVerify (__main__.TestBlockdevDel): formats ['blkverify'] are not whitelisted
    [case not run] testQuorum (__main__.TestBlockdevDel): formats ['quorum'] are not whitelisted
140 0s ...
Not run: 131 135 136
Some cases not run in: 139
Passed all 7 tests
Signed-off-by: NAndrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

Supported formats listed by 'qemu' may differ from those listed by
'qemu-img' due to whitelists. Some test cases require specific formats
that may be used with qemu. They can be inquired directly by running
'qemu -drive format=help'. The response takes whitelists into account.
The method supported_formats() serves for that. The method decorator
skip_if_unsupported() checks if all requested formats are whitelisted.
If not, the test case will be skipped. That has been implemented in
the 'check' file in the way similar to the 'test notrun' mechanism.
Suggested-by: NRoman Kagan <rkagan@virtuozzo.com>
Suggested-by: NVladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: NKevin Wolf <kwolf@redhat.com>
Signed-off-by: NAndrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>

bdrv_iterate_format (which is currently only used for printing out the
formats supported by the block layer) doesn't take format whitelisting
into account.

This creates a problem for tests: they enumerate supported formats to
decide which tests to enable, but then discover that QEMU doesn't let
them actually use some of those formats.

To avoid that, exclude formats that are not whitelisted from
enumeration, if whitelisting is in use.  Since we have separate
whitelists for r/w and r/o, take this a parameter to
bdrv_iterate_format, and print two lists of supported formats (r/w and
r/o) in main qemu.
Signed-off-by: NRoman Kagan <rkagan@virtuozzo.com>
Signed-off-by: NAndrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: NKevin Wolf <kwolf@redhat.com>