This reverts commit 7b345b69.

Conflicts:

	tests/qemuxml2argvdata/qemuxml2argv-disk-scsi-vscsi.xml

This reverts commit c9abfadf.

Conflicts:

	tests/qemuxml2argvdata/qemuxml2argv-disk-scsi-virtio-scsi.xml

The auto-generated WWN comply with the new addressing schema of WWN:

<quote>
the first nibble is either hex 5 or 6 followed by a 3-byte vendor
identifier and 36 bits for a vendor-specified serial number.
</quote>

We choose hex 5 for the first nibble. And for the 3-bytes vendor ID,
we uses the OUI according to underlying hypervisor type, (invoking
virConnectGetType to get the virt type). e.g. If virConnectGetType
returns "QEMU", we use Qumranet's OUI (00:1A:4A), if returns
ESX|VMWARE, we use VMWARE's OUI (00:05:69). Currently it only
supports qemu|xen|libxl|xenapi|hyperv|esx|vmware drivers. The last
36 bits are auto-generated.

Some tools, such as virt-manager, prefers having the default USB
controller explicit in the XML document. This patch makes sure there
is one. With this patch, it is now possible to switch from USB1 to
USB2 from the release 0.9.1 of virt-manager.

Fix tests to pass with this change.

In case the caller specifies that confined guests are required but the
security driver turns out to be 'none', we should return an error since
this driver clearly cannot meet that requirement.  As a result of this
error, libvirtd fails to start when the host admin explicitly sets
confined guests are required but there is no security driver available.

Since security driver 'none' cannot create confined guests, we override
default confined setting so that hypervisor drivers do not thing they
should create confined guests.

Security label type 'none' requires relabel to be set to 'no' so there's
no reason to output this extra attribute.  Moreover, since relabel is
internally stored in a negative from (norelabel), the default value for
relabel would be 'yes' in case there is no <seclabel> element in domain
configuration.  In case VIR_DOMAIN_SECLABEL_DEFAULT turns into
VIR_DOMAIN_SECLABEL_NONE, we would incorrectly output relabel='yes' for
seclabel type 'none'.

Commit b170eb99 introduced a bug: domains that had an explicit
<seclabel type='none'/> when started would not be reparsed if
libvirtd restarted.  It turns out that our testsuite was not
exercising this because it never tried anything but inactive
parsing.  Additionally, the live XML for such a domain failed
to re-validate.  Applying just the tests/ portion of this patch
will expose the bugs that are fixed by the other two files.

* docs/schemas/domaincommon.rng (seclabel): Allow relabel under
type='none'.
* src/conf/domain_conf.c (virSecurityLabelDefParseXML): Per RNG,
presence of <seclabel> with no type implies dynamic.  Don't
require sub-elements for type='none'.
* tests/qemuxml2xmltest.c (mymain): Add test.
* tests/qemuxml2argvtest.c (mymain): Likewise.
* tests/qemuxml2argvdata/qemuxml2argv-seclabel-none.xml: Add file.
* tests/qemuxml2argvdata/qemuxml2argv-seclabel-none.args: Add file.
Reported by Ansis Atteka.

Unlike .cvsignore under CVS, git allows for ignoring nested
names.  We weren't very consistent where new tests were
being ignored (some in .gitignore, some in tests/.gitignore),
and I found it easier to just consolidate everything.

* .gitignore: Subsume entries from subdirectories.
* daemon/.gitignore: Delete.
* docs/.gitignore: Likewise.
* docs/devhelp/.gitignore: Likewise.
* docs/html/.gitignore: Likewise.
* examples/dominfo/.gitignore: Likewise.
* examples/domsuspend/.gitignore: Likewise.
* examples/hellolibvirt/.gitignore: Likewise.
* examples/openauth/.gitignore: Likewise.
* examples/domain-events/events-c/.gitignore: Likewise.
* include/libvirt/.gitignore: Likewise.
* src/.gitignore: Likewise.
* src/esx/.gitignore: Likewise.
* tests/.gitignore: Likewise.
* tools/.gitignore: Likewise.

The bulk of this patch was done with:

sed -i 's/\(\bfree *(/VIR_FREE(/g' tests/*.c

followed by fixing the few compile errors that resulted.

* cfg.mk (exclude_file_name_regexp--sc_prohibit_raw_allocation):
Remove tests from exemption.
* tests/testutils.h: Add common header.
* tests/commandhelper.c: Fix offenders.
* tests/cputest.c: Likewise.
* tests/domainsnapshotxml2xmltest.c: Likewise.
* tests/interfacexml2xmltest.c: Likewise.
* tests/networkxml2argvtest.c: Likewise.
* tests/networkxml2xmltest.c: Likewise.
* tests/nodedevxml2xmltest.c: Likewise.
* tests/nodeinfotest.c: Likewise.
* tests/nwfilterxml2xmltest.c: Likewise.
* tests/qemuargv2xmltest.c: Likewise.
* tests/qemuxml2argvtest.c: Likewise.
* tests/qemuxml2xmltest.c: Likewise.
* tests/qemuxmlnstest.c: Likewise.
* tests/qparamtest.c: Likewise.
* tests/sexpr2xmltest.c: Likewise.
* tests/storagepoolxml2xmltest.c: Likewise.
* tests/storagevolxml2xmltest.c: Likewise.
* tests/testutils.c: Likewise.
* tests/virshtest.c: Likewise.
* tests/xencapstest.c: Likewise.
* tests/xmconfigtest.c: Likewise.
* tests/xml2sexprtest.c: Likewise.

Our syntax checker missed all-lower-case variables (this will
be fixed by the next .gnulib update).  Additionally, anywhere
that we mix in-tree files with generated files, automake recommends
listing builddir prior to srcdir for VPATH builds.

* src/Makefile.am (*_la_CFLAGS): Favor $(top_srcdir).
(INCLUDES): Likewise, and follow automake recommendations on
builddir before srcdir.
* python/Makefile.am (INCLUDES): Swap directory order.
* tests/Makefile.am (INCLUDES): Likewise.
* tools/Makefile.am (INCLUDES): Likewise.
* daemon/Makefile.am (INCLUDES): Likewise.
(libvirtd.init, libvirtd.service): Favor $().
* examples/hellolibvirt/Makefile.am (hellolibvirt_LDADD):
Likewise.
* examples/openauth/Makefile.am (openauth_LDADD): Likewise.
* examples/dominfo/Makefile.am (INCLUDES): Drop dead include.
* examples/domsuspend/Makefile.am (INCLUDES): Likewise.

Sometimes, its easier to run children with 2>&1 in shell notation,
and just deal with stdout and stderr interleaved.  This was already
possible for fd handling; extend it to also work when doing string
capture of a child process.

* docs/internals/command.html.in: Document this.
* src/util/command.c (virCommandSetErrorBuffer): Likewise.
(virCommandRun, virExecWithHook): Implement it.
* tests/commandtest.c (test14): Test it.
* daemon/remote.c (remoteDispatchAuthPolkit): Use new command
feature.

Curently security labels can be of type 'dynamic' or 'static'.
If no security label is given, then 'dynamic' is assumed. The
current code takes advantage of this default, and avoids even
saving <seclabel> elements with type='dynamic' to disk. This
means if you temporarily change security driver, the guests
can all still start.

With the introduction of sVirt to LXC though, there needs to be
a new default of 'none' to allow unconfined LXC containers.

This patch introduces two new security label types

 - default:  the host configuration decides whether to run the
             guest with type 'none' or 'dynamic' at guest start
 - none:     the guest will run unconfined by security policy

The 'none' label type will obviously be undesirable for some
deployments, so a new qemu.conf option allows a host admin to
mandate confined guests. It is also possible to turn off default
confinement

  security_default_confined = 1|0  (default == 1)
  security_require_confined = 1|0  (default == 0)

* src/conf/domain_conf.c, src/conf/domain_conf.h: Add new
  seclabel types
* src/security/security_manager.c, src/security/security_manager.h:
  Set default sec label types
* src/security/security_selinux.c: Handle 'none' seclabel type
* src/qemu/qemu.conf, src/qemu/qemu_conf.c, src/qemu/qemu_conf.h,
  src/qemu/libvirtd_qemu.aug: New security config options
* src/qemu/qemu_driver.c: Tell security driver about default
  config

The path to the dnsmasq binary can be configured while in the test data
the path is hard-coded to /usr/bin/. This break the test suite if a the
binary is located in a different location, like /usr/local/sbin/.

Replace the hard coded path in the test data by a token, which is
dynamically replaced in networkxml2argvtest with the configured path
after the test data has been loaded.

(Another option would have been to modify configure.ac to generate the
 test data during configure, but I do not know of an easy way do trick
 configure into mass-generate those test files without listing every
 single one, which I consider less flexible.)

- unit-test the unit-test:
  #include <assert.h>
  #define TEST(in,token,rep,out) { char *buf = strdup(in); assert(!replaceTokens(&buf, token, rep) && !strcmp(buf, out)); free(buf); }
  TEST("", "AA", "B", "");
  TEST("A", "AA", "B", "A");
  TEST("AA", "AA", "B", "B");
  TEST("AAA", "AA", "B", "BA");
  TEST("AA", "AA", "BB", "BB");
  TEST("AA", "AA", "BBB", "BBB");
  TEST("<AA", "AA", "B", "<B");
  TEST("<AA", "AA", "BB", "<BB");
  TEST("<AA", "AA", "BBB", "<BBB");
  TEST("AA>", "AA", "B", "B>");
  TEST("AA>", "AA", "BB", "BB>");
  TEST("AA>", "AA", "BBB", "BBB>");
  TEST("<AA>", "AA", "B", "<B>");
  TEST("<AA>", "AA", "BB", "<BB>");
  TEST("<AA>", "AA", "BBB", "<BBB>");
  TEST("<AA|AA>", "AA", "B", "<B|B>");
  TEST("<AA|AA>", "AA", "BB", "<BB|BB>");
  TEST("<AA|AA>", "AA", "BBB", "<BBB|BBB>");
  TEST("<AAAA>", "AA", "B", "<BB>");
  TEST("<AAAA>", "AA", "BB", "<BBBB>");
  TEST("<AAAA>", "AA", "BBB", "<BBBBBB>");
  TEST("AAAA>", "AA", "B", "BB>");
  TEST("AAAA>", "AA", "BB", "BBBB>");
  TEST("AAAA>", "AA", "BBB", "BBBBBB>");
  TEST("<AAAA", "AA", "B", "<BB");
  TEST("<AAAA", "AA", "BB", "<BBBB");
  TEST("<AAAA", "AA", "BBB", "<BBBBBB");
  alarm(1); /* no infinite loop */
  TEST("A", "A", "A", "A");
  TEST("AA", "A", "A", "AA");
  alarm(0);
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

virnettlscontexttest uses gnutls_x509_crt_set_subject_alt_name() and
GNUTLS_FSAN_APPEND, which - according to
<http://www.gnu.org/software/gnutls/manual/gnutls.html> - are only
available since 2.6.0.

Since libvirt still works fine with gnutls-1.0.25 from RHEL5, only
enable the test when the version of GNUTLS is at least 2.6.0.
Signed-off-by: NPhilipp Hahn <hahn@univention.de>
Signed-off-by: NEric Blake <eblake@redhat.com>

This patch adds a new element <title> to the domain XML. This attribute
can hold a short title defined by the user to ease the identification of
domains. The title may not contain newlines and should be reasonably short.

 *docs/formatdomain.html.in
 *docs/schemas/domaincommon.rng
        - add schema grammar for the new element and documentation
  *src/conf/domain_conf.c
  *src/conf/domain_conf.h
        - add field to hold the new attribute
        - add code to parse and create XML with the new attribute

 This patch adds a new attribute "rawio" to the "disk" element
 of domain XML. Valid values of "rawio" attribute are "yes"
 and "no".
 rawio='yes' indicates the disk is desirous of CAP_SYS_RAWIO.

 If you specify the following XML:

 <disk type='block' device='lun' rawio='yes'>
  ...
 </disk>

 the domain will be granted CAP_SYS_RAWIO.
 (of course, the domain have to be executed with root privilege)

NOTE:
   - "rawio" attribute is only valid when device='lun'
   - At the moment, any other disks you won't use rawio can use rawio.
Signed-off-by: NTaku Izumi <izumi.taku@jp.fujitsu.com>

This patch addresses: https://bugzilla.redhat.com/show_bug.cgi?id=781562

Along with the "rombar" option that controls whether or not a boot rom
is made visible to the guest, qemu also has a "romfile" option that
allows specifying a binary file to present as the ROM BIOS of any
emulated or passthrough PCI device. This patch adds support for
specifying romfile to both passthrough PCI devices, and emulated
network devices that attach to the guest's PCI bus (just about
everything other than ne2k_isa).

One example of the usefulness of this option is described in the
bugzilla report: 82576 sriov network adapters don't provide a ROM BIOS
for the cards virtual functions (VF), but an image of such a ROM is
available, and with this ROM visible to the guest, it can PXE boot.

In libvirt's xml, the new option is configured like this:

   <hostdev>
     ...
     <rom file='/etc/fake/boot.bin'/>
     ...
   </hostdev

(similarly for <interface>).

When support for the rombar option was added, it was only added for
PCI passthrough devices, configured with <hostdev>. The same option is
available for any network device that is attached to the guest's PCI
bus. This patch allows setting rombar for any PCI network device type.

After adding cases to test this to qemuxml2argv-hostdev-pci-rombar.*,
I decided to rename those files (to qemuxml2argv-pci-rom.*) to more
accurately reflect the additional tests, and also noticed that up to
now we've only been performing a domainschematest for that case, so I
added the "pci-rom" test to both qemuxml2argv and qemuxml2xml (and in
the process found some bugs whose fixes I squashed into previous
commits of this series).

Since these two items are now in the virDomainDeviceInfo struct, it
makes sense to parse/format them in the functions written to
parse/format that structure. Not all types of devices allow them, so
two internal flags are added to indicate when it is appropriate to do
so.

I was lucky - only one test case needed to be re-ordered!

Otherwise, a failed test gives misleading output.

* tests/commandtest.c (test13, test14, test16): Pass arguments in
correct order.

QEMU supports a bunch of CPUID features that are tied to the kvm CPUID
nodes rather than the processor's.  They are "kvmclock",
"kvm_nopiodelay", "kvm_mmu", "kvm_asyncpf".  These are not known to
libvirt and their CPUID leaf might move if (for example) the Hyper-V
extensions are enabled. Hence their handling would anyway require some
special-casing.

However, among these the most useful is kvmclock; an additional
"property" of this feature is that a <timer> element is a better model
than a CPUID feature.  Although, creating part of the -cpu command-line
from something other than the <cpu> XML element introduces some
ugliness.
Reviewed-by: NJiri Denemark <jdenemar@redhat.com>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Recently (or not so recently) QEMU added the kvm32 and kvm64
architectures, representing a least common denominator of all
hosts that can run KVM.  Add them to the machine map.

Also, some features that TCG supports were added to qemu64.
Add them to the cpu_map.xml whenever KVM is guaranteed to support
those.  We still have to leave some out, because they would not
be available to guests running on older hosts.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

The qemu developers have made it clear that modern qemu will no
longer guarantee human monitor command stability; furthermore,
some features, such as async events, are only supported via qmp.
If we are compiled without support for handling JSON, we cannot
expect to sanely interact with modern qemu.

However, things must continue to build on RHEL 5, where qemu
is stuck at 0.10, and where yajl is not available.

Another benefit of this patch: future additions of new monitor
commands need only focus on qemu_monitor_json.c, instead of
also wasting time with qemu_monitor_text.c.

* src/qemu/qemu_capabilities.c (qemuCapsComputeCmdFlags): Report
error if yajl is missing but qemu requires qmp.
(qemuCapsParseHelpStr): Propagate error.
(qemuCapsExtractVersionInfo): Update caller.
* tests/qemuhelptest.c (testHelpStrParsing): Likewise.

I'm getting tired of remembering to backport RHEL-specific
patches when building upstream libvirt on RHEL 6.x or CentOS.
All the affected versions of RHEL qemu-kvm have backported
enough patches to a) make JSON useful, and b) modify the
-help text to mention libvirt as the preferred interface;
which means this string in the help output is a reliable
indicator that we can outsmart a strict version check,
even when upstream qemu 0.12 lacked the needed features.

* src/qemu/qemu_capabilities.c (qemuCapsComputeCmdFlags):
Recognize particular help string present when enough features were
backported to be worth using JSON.
* tests/qemuhelptest.c (mymain): Update tests accordingly.

In preparation for the patch to include Murmurhash3, which
introduces a virhashcode.h and virhashcode.c files, rename
the existing hash.h and hash.c to virhash.h and virhash.c
respectively.

In preparation for conversion over to use the Murmurhash3
algorithm, convert various virHash APIs to use size_t or
uint32 for their return values/parameters, instead of the
variable size 'unsigned long' or 'int' types

The old virRandom() API was not generating good random numbers.
Replace it with a new API virRandomBits which instead of being
told the upper limit, gets told the number of bits of randomness
required.

* src/util/virrandom.c, src/util/virrandom.h: Add virRandomBits,
  and move virRandomInitialize
* src/util/util.h, src/util/util.c: Delete virRandom and
  virRandomInitialize
* src/libvirt.c, src/security/security_selinux.c,
  src/test/test_driver.c, src/util/iohelper.c: Update for
  changes from virRandom to virRandomBits
* src/storage/storage_backend_iscsi.c: Remove bogus call
  to virRandomInitialize & convert to virRandomBits

It's better to group all the metadata together.  This is a
cosmetic output change; since the RNG allows interleave, it
doesn't matter where the user stuck it on input, and an XPath
query will find the same information when parsing the output.

* src/conf/domain_conf.c (virDomainDefFormatInternal): Output
metadata earlier.
* docs/formatdomain.html.in: Update documentation.
* tests/domainsnapshotxml2xmlout/metadata.xml: Update test.
* tests/qemuxml2xmloutdata/qemuxml2xmlout-metadata.xml: Likewise.

Applications can now insert custom nodes and hierarchies into domain
configuration XML. Although currently not enforced, applications are
required to use their own namespaces on every custom node they insert,
with only one top-level element per namespace.

One of latest patches (b7bcb22c) enhanced testing for virHashEqual.
However, hash2 variable might be used uninitialized.

Add a test case to test the virHashEqual function.

Although this is a public API break, it only affects users that
were compiling against *_LAST values, and can be trivially
worked around without impacting compilation against older
headers, by the user defining VIR_ENUM_SENTINELS before using
libvirt.h.  It is not an ABI break, since enum values do not
appear as .so entry points.  Meanwhile, it prevents users from
using non-stable enum values without explicitly acknowledging
the risk of doing so.

See this list discussion:
https://www.redhat.com/archives/libvir-list/2012-January/msg00804.html

* include/libvirt/libvirt.h.in: Hide all sentinels behind
LIBVIRT_ENUM_SENTINELS, and add missing sentinels.
* src/internal.h (VIR_DEPRECATED): Allow inclusion after
libvirt.h.
(LIBVIRT_ENUM_SENTINELS): Expose sentinels internally.
* daemon/libvirtd.h: Use the sentinels.
* src/remote/remote_protocol.x (includes): Don't expose sentinels.
* python/generator.py (enum): Likewise.
* tests/cputest.c (cpuTestCompResStr): Silence compiler warning.
* tools/virsh.c (vshDomainStateReasonToString)
(vshDomainControlStateToString): Likewise.

There was missing capability for blkiotune and thus specifying these
settings caused libvirt to run qemu with invalid parameters and then
reporting qemu error instead of the standard libvirt one. The support
for blkiotune setting was added in upstream qemu repo under commit
0563e191516289c9d2f282a8c50f2eecef2fa773.

This introduces new attribute wrpolicy with only supported
value as immediate. This will be an optional
attribute with no defaults. This helps specify whether
to skip the host page cache.

When wrpolicy is specified, meaning when wrpolicy=immediate
a writeback is explicitly initiated for the dirty pages in
the host page cache as part of the guest file write operation.

Usage:
<filesystem type='mount' accessmode='passthrough'>
  <driver type='path' wrpolicy='immediate'/>
  <source dir='/export/to/guest'/>
  <target dir='mount_tag'/>
</filesystem>

Currently this only works with type='mount' for the QEMU/KVM driver.
Signed-off-by: NDeepak C Shetty <deepakcs@linux.vnet.ibm.com>

qemuxml2argvtest sanitizes PATH to just /bin, but on at least
Fedora 16, dirname lives in /usr/bin instead.  Regression
introduced in commit e7201afd.

* tests/qemuxml2argvdata/qemu.sh: Avoid forking a dirname call,
since dirname might not be in PATH after test sanitization.
* tests/qemuxml2argvdata/qemu-supported-cpus.sh: Likewise.
Diagnosed by Michal Privoznik.

This adds support for host-model and host-passthrough CPU modes to qemu
driver. The host-passthrough mode is mapped to -cpu host.

VIR_DOMAIN_XML_UPDATE_CPU flag for virDomainGetXMLDesc may be used to
get updated custom mode guest CPU definition in case it depends on host
CPU. This patch implements the same behavior for host-model and
host-passthrough CPU modes.

The mode can be either of "custom" (default), "host-model",
"host-passthrough". The semantics of each mode is described in the
following examples:

- guest CPU is a default model with specified topology:
    <cpu>
      <topology sockets='1' cores='2' threads='1'/>
    </cpu>

- guest CPU matches selected model:
    <cpu mode='custom' match='exact'>
      <model>core2duo</model>
    </cpu>

- guest CPU should be a copy of host CPU as advertised by capabilities
  XML (this is a short cut for manually copying host CPU specification
  from capabilities to domain XML):
    <cpu mode='host-model'/>

  In case a hypervisor does not support the exact host model, libvirt
  automatically falls back to a closest supported CPU model and
  removes/adds features to match host. This behavior can be disabled by
    <cpu mode='host-model'>
      <model fallback='forbid'/>
    </cpu>

- the same as previous returned by virDomainGetXMLDesc with
  VIR_DOMAIN_XML_UPDATE_CPU flag:
    <cpu mode='host-model' match='exact'>
      <model fallback='allow'>Penryn</model>       --+
      <vendor>Intel</vendor>                         |
      <topology sockets='2' cores='4' threads='1'/>  + copied from
      <feature policy='require' name='dca'/>         | capabilities XML
      <feature policy='require' name='xtpr'/>        |
      ...                                          --+
    </cpu>

- guest CPU should be exactly the same as host CPU even in the aspects
  libvirt doesn't model (such domain cannot be migrated unless both
  hosts contain exactly the same CPUs):
    <cpu mode='host-passthrough'/>

- the same as previous returned by virDomainGetXMLDesc with
  VIR_DOMAIN_XML_UPDATE_CPU flag:
    <cpu mode='host-passthrough' match='minimal'>
      <model>Penryn</model>                        --+ copied from caps
      <vendor>Intel</vendor>                         | XML but doesn't
      <topology sockets='2' cores='4' threads='1'/>  | describe all
      <feature policy='require' name='dca'/>         | aspects of the
      <feature policy='require' name='xtpr'/>        | actual guest CPU
      ...                                          --+
    </cpu>

In case a hypervisor doesn't support the exact CPU model requested by a
domain XML, we automatically fallback to a closest CPU model the
hypervisor supports (and make sure we add/remove any additional features
if needed). This patch adds 'fallback' attribute to model element, which
can be used to disable this automatic fallback.