Make it obvious to 'dumpxml' readers what unit we are using,
since our default of KiB for memory (1024) differs from qemu's
default of MiB; and differs from our use of bytes for storage.

Tests were updated via:

$ find tests/*data tests/*out -name '*.xml' | \
  xargs sed -i 's/<\(memory\|currentMemory\|hard_limit\|soft_limit\|min_guarantee\|swap_hard_limit\)>/<\1 unit='"'KiB'>/"
$ find tests/*data tests/*out -name '*.xml' | \
  xargs sed -i 's/<\(capacity\|allocation\|available\)>/<\1 unit='"'bytes'>/"

followed by a few fixes for the stragglers.

Note that with this patch, the RNG for <memory> still forbids
validation of anything except unit='KiB', since the code silently
ignores the attribute; a later patch will expand <memory> to allow
scaled input in the code and update the RNG to match.

* docs/schemas/basictypes.rng (unit): Add 'bytes'.
(scaledInteger): New define.
* docs/schemas/storagevol.rng (sizing): Use it.
* docs/schemas/storagepool.rng (sizing): Likewise.
* docs/schemas/domaincommon.rng (memoryKBElement): New define; use
for memory elements.
* src/conf/storage_conf.c (virStoragePoolDefFormat)
(virStorageVolDefFormat): Likewise.
* src/conf/domain_conf.h (_virDomainDef): Document unit used
internally.
* src/conf/storage_conf.h (_virStoragePoolDef, _virStorageVolDef):
Likewise.
* tests/*data/*.xml: Update all tests.
* tests/*out/*.xml: Likewise.
* tests/define-dev-segfault: Likewise.
* tests/openvzutilstest.c (testReadNetworkConf): Likewise.
* tests/qemuargv2xmltest.c (blankProblemElements): Likewise.

The bulk of this patch was done with:

sed -i 's/\(\bfree *(/VIR_FREE(/g' tests/*.c

followed by fixing the few compile errors that resulted.

* cfg.mk (exclude_file_name_regexp--sc_prohibit_raw_allocation):
Remove tests from exemption.
* tests/testutils.h: Add common header.
* tests/commandhelper.c: Fix offenders.
* tests/cputest.c: Likewise.
* tests/domainsnapshotxml2xmltest.c: Likewise.
* tests/interfacexml2xmltest.c: Likewise.
* tests/networkxml2argvtest.c: Likewise.
* tests/networkxml2xmltest.c: Likewise.
* tests/nodedevxml2xmltest.c: Likewise.
* tests/nodeinfotest.c: Likewise.
* tests/nwfilterxml2xmltest.c: Likewise.
* tests/qemuargv2xmltest.c: Likewise.
* tests/qemuxml2argvtest.c: Likewise.
* tests/qemuxml2xmltest.c: Likewise.
* tests/qemuxmlnstest.c: Likewise.
* tests/qparamtest.c: Likewise.
* tests/sexpr2xmltest.c: Likewise.
* tests/storagepoolxml2xmltest.c: Likewise.
* tests/storagevolxml2xmltest.c: Likewise.
* tests/testutils.c: Likewise.
* tests/virshtest.c: Likewise.
* tests/xencapstest.c: Likewise.
* tests/xmconfigtest.c: Likewise.
* tests/xml2sexprtest.c: Likewise.

QEMU supports a bunch of CPUID features that are tied to the kvm CPUID
nodes rather than the processor's.  They are "kvmclock",
"kvm_nopiodelay", "kvm_mmu", "kvm_asyncpf".  These are not known to
libvirt and their CPUID leaf might move if (for example) the Hyper-V
extensions are enabled. Hence their handling would anyway require some
special-casing.

However, among these the most useful is kvmclock; an additional
"property" of this feature is that a <timer> element is a better model
than a CPUID feature.  Although, creating part of the -cpu command-line
from something other than the <cpu> XML element introduces some
ugliness.
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

I got this failure on FreeBSD:

shunloadtest.c: In function 'main':
shunloadtest.c:150: error: 'EXIT_AM_SKIP' undeclared (first use in this function)

but inspection showed several other problems, all fixed here.

* tests/domainsnapshotxml2xmltest.c [!WITH_QEMU]: Ensure
EXIT_AM_SKIP is defined.
* tests/esxutilstest.c [!WITH_ESX]: Likewise.
* tests/openvzutilstest.c [!WITH_OPENVZ]: Likewise.
* tests/qemuargv2xmltest.c [!WITH_QEMU]: Likewise.
* tests/qemuhelptest.c [!WITH_QEMU]: Likewise.
* tests/qemuxml2argvtest.c [!WITH_QEMU]: Likewise.
* tests/qemuxml2xmltest.c [!WITH_QEMU]: Likewise.
* tests/qemuxmlnstest.c [!WITH_QEMU]: Likewise.
* tests/shunloadtest.c [!linux]: Likewise.
* tests/vmx2xmltest.c [!WITH_VMX]: Likewise.
* tests/xml2vmxtest.c [!WITH_VMX]: Likewise.

This improves the support for qemu rbd devices by adding support for a few
key features (e.g., authentication) and cleaning up the way in which
rbd configuration options are passed to qemu.

An <auth> member of the disk source xml specifies how librbd should
authenticate. The username attribute is the Ceph/RBD user to authenticate as.
The usage or uuid attributes specify which secret to use. Usage is an
arbitrary identifier local to libvirt.

The old RBD support relied on setting an environment variable to
communicate information to qemu/librbd.  Instead, pass those options
explicitly to qemu.  Update the qemu argument parsing and tests
accordingly.
Signed-off-by: NSage Weil <sage@newdream.net>
Signed-off-by: NJosh Durgin <josh.durgin@dreamhost.com>

Previously libvirt's disk device XML only had a single attribute,
error_policy, to control both read and write error policy, but qemu
has separate options for controlling read and write. In one case
(enospc) a policy is allowed for write errors but not read errors.

This patch adds a separate attribute that sets only the read error
policy. If just error_policy is set, it will apply to both read and
write error policy (previous behavior), but if the new rerror_policy
attribute is set, it will override error_policy for read errors only.
Possible values for rerror_policy are "stop", "report", and "ignore"
("report" is the qemu-controlled default for rerror_policy when
error_policy isn't specified).

For consistency, the value "report" has been added to the possible
values for error_policy as well.

QEMU 0.13 introduced cache=unsafe for -drive, this patch exposes
it in the libvirt layer.

  * Introduced a new QEMU capability flag ($prefix_CACHE_UNSAFE),
    as even if $prefix_CACHE_V2 is set, we can't know if unsafe
    is supported.

  * Improved the reliability of qemu cache type detection.

Newer QEMU introduced cache=directsync for -drive, this patchset
is to expose it in libvirt layer.

  * Introduced a new QEMU capability flag ($prefix_CACHE_DIRECTSYNC),
    As even $prefix_CACHE_V2 is set, we can't known if directsync
    is supported.

Prefer 'return EXIT_AM_SKIP' over 'exit(EXIT_AM_SKIP)'.

Prefer 'int main(void)' over 'int main(int argc, char **argv)'.

Fix mymain signature in commandtest and nodeinfotest.

When converting QEMU argv into a virDomainDefPtr, also extract
the pidfile, monitor character device config and the monitor
mode.

* src/qemu/qemu_command.c, src/qemu/qemu_command.h: Extract
  pidfile & monitor config from QEMU argv
* src/qemu/qemu_driver.c, tests/qemuargv2xmltest.c: Add extra
  params when calling qemuParseCommandLineString

Make virtTestLoadFile allocate the buffer to read the file into.

Fix logic error in virtTestLoadFile, stop reading on the first empty line.

Use virFileReadLimFD in virtTestCaptureProgramOutput to avoid manual
buffer handling.

A few of the tests were missing basic sanity checks, while most
of them were doing copy-and-paste initialization (in fact, some
of them pasted the argc > 1 check more than once!).  It's much
nicer to do things in one common place, and minimizes the size of
the next patch that fixes getcwd usage.

* tests/testutils.h (EXIT_AM_HARDFAIL): New define.
(progname, abs_srcdir): Define for all tests.
(VIRT_TEST_MAIN): Change callback signature.
* tests/testutils.c (virtTestMain): Do more common init.
* tests/commandtest.c (mymain): Simplify.
* tests/cputest.c (mymain): Likewise.
* tests/esxutilstest.c (mymain): Likewise.
* tests/eventtest.c (mymain): Likewise.
* tests/hashtest.c (mymain): Likewise.
* tests/networkxml2xmltest.c (mymain): Likewise.
* tests/nodedevxml2xmltest.c (myname): Likewise.
* tests/nodeinfotest.c (mymain): Likewise.
* tests/nwfilterxml2xmltest.c (mymain): Likewise.
* tests/qemuargv2xmltest.c (mymain): Likewise.
* tests/qemuhelptest.c (mymain): Likewise.
* tests/qemuxml2argvtest.c (mymain): Likewise.
* tests/qemuxml2xmltest.c (mymain): Likewise.
* tests/qparamtest.c (mymain): Likewise.
* tests/sexpr2xmltest.c (mymain): Likewise.
* tests/sockettest.c (mymain): Likewise.
* tests/statstest.c (mymain): Likewise.
* tests/storagepoolxml2xmltest.c (mymain): Likewise.
* tests/storagevolxml2xmltest.c (mymain): Likewise.
* tests/virbuftest.c (mymain): Likewise.
* tests/virshtest.c (mymain): Likewise.
* tests/vmx2xmltest.c (mymain): Likewise.
* tests/xencapstest.c (mymain): Likewise.
* tests/xmconfigtest.c (mymain): Likewise.
* tests/xml2sexprtest.c (mymain): Likewise.
* tests/xml2vmxtest.c (mymain): Likewise.

Remove the <stdbool.h> header from all source files / headers
and just put it into internal.h

* src/internal.h: Add <stdbool.h>

The new names comply more with the fact that they are all members of
enum qemuCapsFlags.

QEMU supports serving VNC over a unix domain socket rather than traditional
TCP host/port. This is specified with:

<graphics type='vnc' socket='/foo/bar/baz'/>

This provides better security access control than VNC listening on
127.0.0.1, but will cause issues with tools that rely on the lax security
(virt-manager in fedora runs as regular user by default, and wouldn't be
able to access a socket owned by 'qemu' or 'root').

Also not currently supported by any clients, though I have patches for
virt-manager, and virt-viewer should be simple to update.

v2:
    schema: Make listen vs. socket a <choice>

* qemuargv2xmltest.c, qemuhelptest.c,
  qemuxml2argvtest.c, testutilsqemu.c: Update includes and
  cope with API renames

Signed-off-by: NJosh Durgin <joshd@hq.newdream.net>

* src/qemu/qemu_conf.c (qemuParseCommandLineSmp): Distinguish
between vcpus and maxvcpus, for new enough qemu.
* tests/qemuargv2xmltest.c (mymain): Add new test.
* tests/qemuxml2argvtest.c (mymain): Likewise.
* tests/qemuxml2xmltest.c (mymain): Likewise.
* tests/qemuxml2argvdata/qemuxml2argv-smp.args: New file.

Before this patch, the testsuite was noisy:

TEST: qemuargv2xmltest
      ........................................ 40
      ................20:41:28.046: warning : qemuParseCommandLine:6565 : unknown QEMU argument '-unknown', adding to the qemu namespace
20:41:28.046: warning : qemuParseCommandLine:6565 : unknown QEMU argument 'parameter', adding to the qemu namespace
.                        57  OK
PASS: qemuargv2xmltest

It's not a real failure (which is why the test was completing
successfully), so much as an intentional warning to the user that use
of the qemu namespace has the potential for undefined effects that
leaked through the default logging behavior.  After this patch series,
all tests can access any logged data, and this particular test can
explicitly check for the presence or absence of the warning, such that
the test output becomes:

TEST: qemuargv2xmltest
      ........................................ 40
      .................                        57  OK
PASS: qemuargv2xmltest

* tests/testutils.h (virtTestLogContentAndReset): New prototype.
* tests/testutils.c (struct virtTestLogData): New struct.
(virtTestLogOutput, virtTestLogClose, virtTestLogContentAndReset):
New functions.
(virtTestMain): Always capture log data emitted during tests.
* tests/qemuargv2xmltest.c (testCompareXMLToArgvHelper, mymain):
Use flag to mark which tests expect noisy stderr.
(testCompareXMLToArgvFiles): Add parameter to test whether stderr
was appropriately silent.

Since commit 107a7bd0, the extraFlags argument was unused.

* tests/qemuargv2xmltest.c (DO_TEST): Drop extraFlags argument.
Adjust all callers.

Thanks to DV for knocking together the Relax-NG changes
quickly for me.

Changes since v1:
 - Change the domain.rng to correspond to the new schema
 - Don't allocate caps->ns in testQemuCapsInit since it is a static table

Changes since v2:
 - Change domain.rng to add restrictions on allowed environment names

Changes since v3:
 - Remove a bogus comment in the tests
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* Dan Kenigsberg requested explicit support for the qemu default disk error policy which is enospace

* Fixes per feedback from Dan and Daniel
* Added test datafiles
* Re-disabled JSON flags
* Added code to print the error policy attribute when generating XML
* Re-add empty tag

* global: patch created by running:
for f in $(git ls-files '*.[ch]') ; do
    cppi $f > $f.t && mv $f.t $f
done

The QEMU flags are commonly stored as a signed or unsigned int,
allowing only 31 flags. This limit is rather close, so to aid
future patches, change it to a 64-bit int

* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h, src/qemu/qemu_driver.c,
  tests/qemuargv2xmltest.c, tests/qemuhelptest.c, tests/qemuxml2argvtest.c:
  Use 'unsigned long long' for QEMU flags

The current QEMU code allocates PCI addresses incrementally starting
at 4. This is not satisfactory because the user may have given some
addresses in their XML config, which need to be skipped over when
allocating addresses to remaining devices.

It is thus neccessary to maintain a list of already allocated PCI
addresses and then only allocate ones that remain unused. This is
also required for domain device hotplug to work properly later.

* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h: Add APIs for creating
  list of existing PCI addresses, and allocating new addresses.
  Refactor address assignment to use this code
* src/qemu/qemu_driver.c: Pull PCI address assignment up into the
  qemuStartVMDaemon() method, as a prelude to moving it into the
  'define' method. Update list of allocated addresses when connecting
  to a running VM at daemon startup.
* tests/qemuxml2argvtest.c, tests/qemuargv2xmltest.c,
  tests/qemuxml2xmltest.c: Remove USB product test since all
  passthrough is done based on address
* tests/qemuxml2argvdata/qemuxml2argv-hostdev-usb-product.args,
  tests/qemuxml2argvdata/qemuxml2argv-hostdev-usb-product.xml: Kil
  unused data files

QEMU always configures a VGA card. If no video card is included in
the libvirt XML, it is neccessary to explicitly turn off the default
using -vga none

* src/qemu/qemu_conf.c: Pass -vga none if no video card is configured
* tests/qemuargv2xmltest.c, tests/qemuxml2argvtest.c: Test for
  handling -vga none.
* tests/qemuxml2argvdata/qemuxml2argv-nographics-vga.args,
  tests/qemuxml2argvdata/qemuxml2argv-nographics-vga.xml: Test
  data files

This appeases a new gnulib-provided "syntax-check".
* daemon/libvirtd.c (main): Use EXIT_FAILURE, not 1.
* proxy/libvirt_proxy.c (main): Likewise, and EXIT_SUCCESS, not 0.
* tests/conftest.c (main): Likewise.
* tests/reconnect.c (main): Likewise.
* tests/testutils.h (EXIT_AM_SKIP): Define.
* tests/nodeinfotest.c (mymain): Use EXIT_AM_SKIP, not 77.
* tests/qemuargv2xmltest.c: Likewise.
* tests/qemuxml2xmltest.c: Likewise.
* tests/virshtest.c (mymain): Likewise.

Nearly all of the methods in src/util/util.h have error codes that
must be checked by the caller to correct detect & report failure.
Add ATTRIBUTE_RETURN_CHECK to ensure compile time validation of
this

* daemon/libvirtd.c: Add explicit check on return value of virAsprintf
* src/conf/domain_conf.c: Add missing check on virParseMacAddr return
  value status & report error
* src/network/bridge_driver.c: Add missing OOM check on virAsprintf
  and report error
* src/qemu/qemu_conf.c: Add missing check on virParseMacAddr return
  value status & report error
* src/security/security_selinux.c: Remove call to virRandomInitialize
  that's done in libvirt.c already
* src/storage/storage_backend_logical.c: Add check & log on virRun
  return status
* src/util/util.c: Add missing checks on virAsprintf/Run status
* src/util/util.h: Annotate all methods with ATTRIBUTE_RETURN_CHECK
  if they return an error status code
* src/vbox/vbox_tmpl.c: Add missing check on virParseMacAddr
* src/xen/xm_internal.c: Add missing checks on virAsprintf
* tests/qemuargv2xmltest.c: Remove bogus call to virRandomInitialize()

This adds simple support for configuring a guest with a QEMU/KVM
virtual hardware watchdog device.

* src/qemu_conf.c, src/qemu_conf.h, src/qemu_driver.c,
  src/qemu_driver.h: Move to src/qemu/
* daemon/qemud.c, src/Makefile.am, tests/qemuargv2xmltest.c,
  tests/qemuhelptest.c, tests/qemuxml2argvtest.c,
  tests/qemuxml2xmltest.c: Adapt for changed paths

* src/qemu_conf.c: Fix leak of values upon OOM
* src/xend_internal.c: Fix missing check for OOM failure
* tests/qemuargv2xmltest.c, tests/qemuxml2argvtest.c: Free
  stateDir upon exit to avoid leak

PCI device assignment is only supported in KVM's fork of qemu, so we
should really detect its availability and give a nice error if its
not supported.

* src/qemu_conf.[ch]: introduce QEMUD_CMD_FLAG_PCIDEVICE indicating
  that the -pcidevice command line option is available

* tests/*: update the tests

* tests/qemuxml2argvtest.c: Revert the change,
"tests: diagnose open failure" of 2009-01-30.
* tests/testutils.c (virtTestLoadFile): Diagnose failure here.