Despite our attempts in 4d26c7fe to keep this going it still gets in
the way of "make docker-test-build" completing because of course we
can't build a modern QEMU with the image. Let's put the thing out of
its misery and remove it.

People who really care about building on powerpc can still use the
binfmt_misc support to manually build an image (or just run the build
from pre this commit).
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>

As part of the push to drop python2 support, replace any explicit python2
dependencies with python3 versions.

For centos, python2 still exists as an implicit dependency, but by adding
python3 we will be able to build even if the configure script begins to
require python 3.5+.

Tested with centos7, fedora, ubuntu, ubuntu1804, and debian 9 (amd64).
Tested under a custom configure script that requires Python 3.5+.

the travis dockerfile is also moved to using python3, which was tested
by running `make docker-test-build@travis`, which I hope is sufficient.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-7-jsnow@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NEduardo Habkost <ehabkost@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: NCleber Rosa <crosa@redhat.com>
Tested-by: NCleber Rosa <crosa@redhat.com>

debian-sid is listed as a partial image, so we cannot run tests against it.
Since it isn't used by any other testable image, remove it for now as it
is prone to bitrot.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-6-jsnow@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Acked-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

debian-ports is listed as a partial image, so we cannot run tests against it.
Since it isn't used by any other testable image, remove it for now as it
is prone to bitrot.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-5-jsnow@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Acked-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

There isn't a debian.dockerfile anymore,
so perform some ghost-busting.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-4-jsnow@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

debian8 partial base is also not consumed by any image, so remove it.
For QEMU's development cycle, we only support debian9 (stretch) and
debian10 (buster).
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190923181140.7235-3-jsnow@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

We don't have a debian8-mxe dockerfile anymore.

Fixes: 67bd36beSigned-off-by: NJohn Snow <jsnow@redhat.com>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190923181140.7235-2-jsnow@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

This adds two new tests that re-use the memory test to check basic
record replay functionality is still working. We have to define our
own runners rather than using the default pattern as we want to change
the test name but re-use the memory binary.

We declare the test binaries as PHONY as they don't really exist.

[AJB: A better test would output some sort of timer value or other
otherwise variable value so we could compare the record and replay
outputs and ensure they match]
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Cc: Pavel Dovgalyuk <dovgaluk@ispras.ru>

This is broadly similar to the existing fcvt test for ARM but using
the generic float testing framework. We should be able to pare down
the ARM fcvt test case to purely half-precision with or without the
Alt HP provision.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

This is a generic floating point multiply and accumulate test for
single precision floating point values. I've split of the common float
functions into a helper library so additional tests can use the same
common code.

As I don't have references for all architectures I've allowed some
flexibility for tests to pass without reference files. They can be
added as we get collect them.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

Now we have fixed the signal delivary bug we can remove this horrible
hack from the system.

Cc: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

These were missed in the recent de-tangling so have been updated to be
more actuate. I've also built up ARM_TESTS in a manner similar to
AARCH64_TESTS for better consistency.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPeter Maydell <peter.maydell@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

Oops; there's no argv here.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-Id: <20190913193821.17756-1-jsnow@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NCleber Rosa <crosa@redhat.com>
Tested-by: NCleber Rosa <crosa@redhat.com>
Reviewed-by: NThomas Huth <thuth@redhat.com>
Tested-by: NThomas Huth <thuth@redhat.com>

There was in the clean-up code caused by attempting to inspect images
which finished before we got there. Clean up the clean up code by:

  - only track the one instance at a time
  - use --filter for docker ps instead of doing it by hand
  - just call docker rm -f to be done with it
  - use uuid.uuid4() for a random uid
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NRichard Henderson <richard.henderson@linaro.org>

When it was based on debian8 which uses python-minimal, it needed this.
It no longer does.

Goodbye, python2.7.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-Id: <20190918222546.11696-1-jsnow@redhat.com>
[AJB: fixed up commit message]
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>

Finger trouble in a previous clean-up inadvertently set
DEBIAN_PARTIAL_IMAGES instead of DOCKER_PARTIAL_IMAGES. Also fix the
typo to debian-9-mxe.

Fixes: 44d5a8bfSigned-off-by: NJohn Snow <jsnow@redhat.com>
[AJB: merged fix from Message-Id: <20190917185537.25417-1-jsnow@redhat.com>]
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NPhilippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: NCleber Rosa <crosa@redhat.com>

Fedora23 is but a distant twinkle. The sanitizer works again, and even
if not, we have --enable-sanitizers now.
Signed-off-by: NJohn Snow <jsnow@redhat.com>
Message-Id: <20190912014442.5757-1-jsnow@redhat.com>
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

On slow hosts with tcg we were sometimes finding that the migration
would complete during precopy and never get into the postcopy test.
Trim back the bandwidth a bit to make that much less likely.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190923131022.15498-3-dgilbert@redhat.com>
Reviewed-by: NCleber Rosa <crosa@redhat.com>
Acked-by: NAlex Bennée <alex.bennee@linaro.org>
Tested-by: NCleber Rosa <crosa@redhat.com>
Tested-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPeter Xu <peterx@redhat.com>
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

We've got various places where we wait for a migration to enter
a given state; but if we enter an unexpected state we tend to fail
in odd ways; add a mechanism for explicitly testing for any state
which we shouldn't be in.
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190923131022.15498-2-dgilbert@redhat.com>
Reviewed-by: NCleber Rosa <crosa@redhat.com>
Tested-by: NCleber Rosa <crosa@redhat.com>
Reviewed-by: NAlex Bennée <alex.bennee@linaro.org>
Reviewed-by: NJuan Quintela <quintela@redhat.com>
Reviewed-by: NPeter Xu <peterx@redhat.com>
Signed-off-by: NDr. David Alan Gilbert <dgilbert@redhat.com>

Doing so catches the bugs we just fixed with NBD not properly using
correct contexts.
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190920220729.31801-1-eblake@redhat.com>

I received an off-list report of failure to connect to an NBD server
expecting an x509 certificate, when the client was attempting something
similar to this command line:

$ ./x86_64-softmmu/qemu-system-x86_64 -name 'blah' -machine q35 -nodefaults \
  -object tls-creds-x509,id=tls0,endpoint=client,dir=$path_to_certs \
  -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0,addr=0x6 \
  -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=raw,file=nbd:localhost:9000,werror=stop,rerror=stop,tls-creds=tls0 \
  -device scsi-hd,id=image1,drive=drive_image1,bootindex=0
qemu-system-x86_64: -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=raw,file=nbd:localhost:9000,werror=stop,rerror=stop,tls-creds=tls0: TLS negotiation required before option 7 (go)
server reported: Option 0x7 not permitted before TLS

The problem?  As specified, -drive is trying to pass tls-creds to the
raw format driver instead of the nbd protocol driver, but before we
get to the point where we can detect that raw doesn't know what to do
with tls-creds, the nbd driver has already failed because the server
complained.  The fix to the broken command line?  Pass
'...,file.tls-creds=tls0' to ensure the tls-creds option is handed to
nbd, not raw.  But since the error message was rather cryptic, I'm
trying to improve the error message.

With this patch, the error message adds a line:

qemu-system-x86_64: -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=raw,file=nbd:localhost:9000,werror=stop,rerror=stop,tls-creds=tls0: TLS negotiation required before option 7 (go)
Did you forget a valid tls-creds?
server reported: Option 0x7 not permitted before TLS

And with luck, someone grepping for that error message will find this
commit message and figure out their command line mistake.  Sadly, the
only mention of file.tls-creds in our docs relates to an --image-opts
use of PSK encryption with qemu-img as the client, rather than x509
certificate encryption with qemu-kvm as the client.

CC: Tingting Mao <timao@redhat.com>
CC: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190907172055.26870-1-eblake@redhat.com>
[eblake: squash in iotest 233 fix]
Reviewed-by: NDaniel P. Berrangé <berrange@redhat.com>

Commit 87adbbff..3e270dca "qapi: Add 'if' to (implicit
struct|union|alternate) members" (v4.0.0) neglected test coverage, and
promptly failed to check the conditions.  Review fail.

Recent commit "tests/qapi-schema: Demonstrate insufficient 'if'
checking" added test coverage, demonstrating the bug.  Fix it by add
the missing check_if().
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-13-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

"'if': 'COND'" generates "#if COND".  We reject empty COND because it
won't compile.  Blank COND won't compile any better, so reject that,
too.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-12-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

check_union() checks the discriminator exists in base and makes sense.
Two error messages mention the base.  These are broken for anonymous
bases, as demonstrated by tests flat-union-invalid-discriminator and
flat-union-invalid-if-discriminator.err.  The third one doesn't
bother.

First broken when commit ac4338f8 "qapi: Allow anonymous base for
flat union" (v2.6.0) neglected to adjust the "not a member of base"
error message.  Commit ccadd6bc "qapi: Add 'if' to implicit struct
members" (v4.0.0) then cloned the flawed error message.

Dumb them down not to mention the base.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-11-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

We represent the parse tree as OrderedDict.  We fetch optional dict
members with .get().  So far, so good.

We represent null literals as None.  .get() returns None both for
"absent" and for "present, value is the null literal".  Uh-oh.

Test features-if-invalid exposes this bug: "'if': null" is
misinterpreted as absent "if".

We added null to the schema language to "allow [...] an explicit
default value" (commit e53188ad "qapi: Allow true, false and null in
schema json", v2.4.0).  Hasn't happened; null is still unused except
as generic invalid value in tests/.

To fix, we'd have to replace .get() by something more careful, or
represent null differently.  Feasible, but we got more and bigger fish
to fry right now.  Remove the null literal from the schema language.
Replace null in tests by another invalid value.

Test features-if-invalid now behaves as it should.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-10-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Show text up to next structural character, whitespace, or quote
character instead of just the first character.

Forgotten quotes now get reported like "Stray 'command'" instead of
"Stray 'c'".
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-9-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Consistently enclose error messages in double quotes.  Use single
quotes within, except for one case of "'".
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-8-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The error message for forgotten quotes around a name shows just the
name's first character, which isn't as nice as it could be.  Same for
attempting to use a number.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-7-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Cover invalid 'if' in struct members, features, union and alternate
branches.  Four out of four are broken.  Mark FIXME.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-6-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
[Comment typo fixed]

When the union definition's base is an object, some error messages
show it as an OrderedDict.  Oops.  Mark FIXME.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-5-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Test flat-union-optional-discriminator declares its union tag as
'*switch': 'Enum', and points to it with 'discriminator': '*switch'.
This gets rejected as "discriminator of flat union 'MyUnion' uses
invalid name '*switch'".  Correct; member 'discriminator' doesn't
accept a '*' prefix.

However, this merely tests name validity checking, which we already
cover elsewhere.  More interesting is testing the valid name 'switch'.
This reports "discriminator 'switch' is not a member of base struct
'Base'", which is misleading.

Copy the existing 'discriminator': '*switch' test to
flat-union-discriminator-bad-name, and rewrite its comment.  Change
flat-union-optional-discriminator to test 'discriminator': 'switch',
and mark it FIXME.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-4-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Tests duplicate-key and double-data test the same thing.  The former
predates the latter, and it has a better name.  Delete the latter, and
tweak the former's comment.

Tests include-format-err and include-extra-junk test the same thing.
The former predates the latter, but the latter has a better name and a
comment.  Delete the former.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-3-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-2-armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>

The previous commit made qapi-code-gen.txt define "(top-level)
expression" as either "directive" or "definition".  The code still
uses "expression" when it really means "definition".  Tidy up.

The previous commit made qapi-code-gen.txt use "object" rather than
"dictionary".  The code still uses "dictionary".  Tidy up.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-17-armbru@redhat.com>

For consistency with docs/devel/qapi-code-gen.txt.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-12-armbru@redhat.com>

Absent flat union branches default to the empty struct (since commit
800877bb "qapi: allow empty branches in flat unions").  But an
attempt to omit all of them is rejected with "Union 'FOO' has no
branches".  Harmless oddity, but it's easy to avoid, so do that.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-11-armbru@redhat.com>
[Commit message typo fixed]

A union or alternate without branches makes no sense and doesn't work:
it can't be instantiated.  A union or alternate with just one branch
works, but is degenerate.  We accept the former, but reject the
latter.  Weird.  docs/devel/qapi-code-gen.txt doesn't mention the
difference.  It claims an alternate definition is "is similar to a
simple union type".

Permit degenerate alternates to make them consistent with unions.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-10-armbru@redhat.com>

We reject empty types with 'boxed': true.  We don't really need that
to work, but making it work is actually simpler than rejecting it, so
do that.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-9-armbru@redhat.com>

Since the previous commit restricted strings to printable ASCII,
\uXXXX's only use is obfuscation.  Drop it.

This leaves \\, \/, \', and \".  Since QAPI schema strings are all
names, and names are restricted to ASCII letters, digits, hyphen, and
underscore, none of them is useful.

The latter three have no test coverage.  Drop them.

Keep \\ to avoid (more) gratuitous incompatibility with JSON.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-8-armbru@redhat.com>

RFC 8259 on string contents:

   All Unicode characters may be placed within the quotation marks,
   except for the characters that MUST be escaped: quotation mark,
   reverse solidus, and the control characters (U+0000 through
   U+001F).

The QAPI schema parser accepts both less and more than JSON: it
accepts only ASCII with \u (less), and accepts control characters
other than LF (new line) unescaped.  How it treats unescaped non-ASCII
input differs between Python 2 and Python 3.

Make it accept strictly less: require printable ASCII.  Drop support
for \b, \f, \n, \r, \t.
Signed-off-by: NMarkus Armbruster <armbru@redhat.com>
Reviewed-by: NEric Blake <eblake@redhat.com>
Message-Id: <20190913201349.24332-7-armbru@redhat.com>