Signed-off-by: NChris Lalancette <clalance@redhat.com>

* qemuxml2argvdata/qemuxml2argv-graphics-vnc-sasl.args,
  qemuxml2argvdata/qemuxml2argv-graphics-vnc-tls.args,
  qemuxml2argvdata/qemuxml2argv-graphics-vnc.args,
  qemuxml2argvdata/qemuxml2argv-input-xen.args: Add in
  QEMU_AUDIO_DRV=none env variable

A given domain XML gets converted to a VMX config, uploaded to the host
and registered as new virtual machine.

* src/esx/esx_driver.c: refactor datastore related path parsing into
  esxUtil_ParseDatastoreRelatedPath()
* src/esx/esx_util.[ch]: add esxUtil_ParseDatastoreRelatedPath()
* src/esx/esx_vi.[ch]: add esxVI_Context_UploadFile(), add datastores to
  the traversal in esxVI_BuildFullTraversalSpecList(), add
  esxVI_LookupDatastoreByName()
* src/esx/esx_vi_methods.[ch]: add esxVI_RegisterVM_Task()
* src/esx/esx_vi_types.c: make some error message more verbose
* src/esx/esx_vmx.[ch]: add esxVMX_AbsolutePathToDatastoreRelatedPath()
  to convert a path into a datastore related path, add esxVMX_ParseFileName()
  to convert from VMX path format to domain XML path format, extend the other
  parsing function to be datastore aware, add esxVMX_FormatFileName() to
  convert from domain XML path format to VMX path format, fix VMX ethernet
  entry formating
* tests/esxutilstest.c: add test for esxUtil_ParseDatastoreRelatedPath()
* tests/vmx2xmldata/*: update domain XML files to use datastore related paths
* tests/xml2vmxdata/*: update domain XML files to use datastore related paths,
  update VMX files to use absolute paths

* tests/.gitignore: ignore new test binaries
* tests/Makefile.am: add new tests
* tests/esxutilstest.c: test esxVMX_IndexToDiskName()
* tests/vmx2xmldata/*: config files for the VMX to domain XML test
* tests/vmx2xmltest.c: test the VMX to domain XML mapping
* tests/xml2vmxdata/*: config files for the domain XML to VMX test
* tests/xml2vmxtest.c: test the domain XML to VMX mapping

Currently, libvirtd will start a dnsmasq process for the virtual
network, but (aside from killing the dnsmasq process and replacing it),
there's no way to define tftp boot options.

This change introduces the appropriate tags to the dhcp configuration:

 <network>
   <name>default</name>
   <bridge name="virbr%d" />
   <forward/>
   <ip address="192.168.122.1" netmask="255.255.255.0">
     <tftp root="/var/lib/tftproot" />
     <dhcp>
       <range start="192.168.122.2" end="192.168.122.254" />
       <bootp file="pxeboot.img"/>
     </dhcp>
   </ip>
 </network>

When the attributes are present, these are passed to the
arguments to dnsmasq:

 dnsmasq [...] --enable-tftp --tftp-root /srv/tftp --dhcp-boot pxeboot.img
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^
                      from <tftp />                     from <bootp />

At present, only local tftp servers are supported (ie, dnsmasq runs as
the tftp server), but we could improve this in future by adding a
server= attribute.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

2009-09-21  Paolo Bonzini  <pbonzini@redhat.com>
            Jeremy Kerr  <jk@ozlabs.org>

	* docs/formatnetwork.html.in: Document new tags.
	* docs/formatnetwork.html: Regenerate.
	* docs/schemas/network.rng: Update.
	* src/network_conf.c (virNetworkDefFree): Free new fields.
	(virNetworkDHCPRangeDefParseXML): Parse <bootp>.
	(virNetworkIPParseXML): New, parsing <dhcp> and <tftp>.
	(virNetworkDefParseXML): Use virNetworkIPParseXML instead of
	virNetworkDHCPRangeDefParseXML.
	(virNetworkDefFormat): Pretty print new fields.
	* src/network_conf.h (struct _virNetworkDef): Add netboot fields.
	* src/network_driver.c (networkBuildDnsmasqArgv): Add
	TFTP and BOOTP arguments.

	* tests/Makefile.am (EXTRA_DIST): Add networkschemadata.
	* tests/networkschematest: Look in networkschemadata.
	* tests/networkschemadata/netboot-network.xml: New.

Latest upstream QEMU can be built with Xen support, which introduces
a -xen-domid argument. This was  mistakenly detected as -domid due
to old Xenner support. Adapt to cope with both syntax. Also only
set domid if the virt type is xen, or the guest type is xen

* src/qemu_conf.c, src/qemu_conf.h: Detect new -xen-domid flag in
  preference to -domid.
* tests/qemuxml2argvdata/qemuxml2argv-bootloader.args,
  tests/qemuxml2argvdata/qemuxml2argv-input-xen.args: Add missing
  -domid param
* tests/qemuxml2argvdata/qemuxml2argv-misc-uuid.args: Remove bogus
  -boot param.
* tests/qemuxml2argvtest.c: Add missing QEMUD_CMD_FLAG_DOMID params

* docs/storage/: Move to examples/xml/storage/
* docs/test*.xml: Move to examples/xml/test/
* docs/Makefile.am: Remove example XML files from dist
* Makefile.am: Add examples/xml to EXTRA_DIST
* tests/virshtest.c: Update for moved test XML
* libvirt.spec.in: Include example XML files as docs
* tests/int-overflow: UPdate for moved XML

* tests/Makefile.am: Add missing data file to extra dist

* src/bridge.c, src/bridge.h, src/buf.c, src/buf.h, src/cgroup.c,
  src/cgroup.h, src/conf.c, src/conf.h, src/event.c, src/event.h,
  src/hash.c, src/hash.h, src/hostusb.c, src/hostusb.h,
  src/iptables.c, src/iptables.h, src/logging.c, src/logging.h,
  src/memory.c, src/memory.h, src/pci.c, src/pci.h, src/qparams.c,
  src/qparams.h, src/stats_linux.c, src/stats_linux.h,
  src/threads-pthread.c, src/threads-pthread.h, src/threads-win32.c,
  src/threads-win32.h, src/threads.c, src/threads.h, src/util.c,
  src/util.h, src/uuid.c, src/uuid.h, src/virterror.c,
  src/virterror_internal.h, src/xml.c, src/xml.h: Move all files
  into src/util/
* daemon/Makefile.am: Add -Isrc/util/ to build flags
* src/Makefile.am: Add -Isrc/util/ to build flags and update for
  moved files
* src/libvirt_private.syms: Export cgroup APIs since they're now
  in util rather than linking directly to drivers
* src/xen/xs_internal.c: Disable bogus virEventRemoveHandle call
  when built under PROXY
* proxy/Makefile.am: Update for changed file locations. Remove
  bogus build of event.c
* tools/Makefile.am, tests/Makefile.am: Add -Isrc/util/ to build flags

* src/capabilities.c, src/capabilities.h, src/domain_conf.c,
  src/domain_conf.h, src/domain_event.c, src/domain_event.h,
  src/interface_conf.c, src/interface_conf.h,
  src/network_conf.c, src/network_conf.h, src/node_device_conf.c,
  src/node_device_conf.h, src/secret_conf.c, src/secret_conf.h,
  src/storage_conf.c, src/storage_conf.h, src/storage_encryption_conf.c,
  src/storage_encryption_conf.h: Move to src/conf/
* src/Makefile.am: Add -Isrc/conf to the individual build targets
  which need to use XML config APIs. Remove LIBXML_CFLAGS, LIBSSH2_CFLAGS
  and SELINUX_CFLAGS from global INCLUDES and only have them in build
  targets which actually need them.  Create a libvirt_conf.la
  convenience library for all config parsers
* src/hostusb.h: Remove bogus include of domain_conf.h
* tests/Makefile.am: Add -Isrc/conf. Remove bogus -I$builddir/src
  since it never has any generated header files
* daemon/Makefile.am: Add -Isrc/conf
* proxy/Makefile.am: Add -Isrc/conf and cope with renamed files
* src/hash.c: Remove bogus include of libxml/threads.h

* daemon/default-network.xml: Move to src/network/default.xml
* daemon/libvirtd_qemu.aug, daemon/test_libvirtd_qemu.aug: Move
  to src/qemu/
* src/qemu.conf: Move to src/qemu/qemu.conf
* daemon/Makefile.am: Remove rules for default-nmetwork.xml and
  libvirtd_qemu.aug and test_libvirtd_qemu.aug. Fix typo in
  uninstall-local that would install polkit again.
* src/Makefile.am: Add rules for installing network/default.xml
  and the qemu/*.aug files. Add test case for QEMU augeas files.
  Add uninstall-local rule for files/directories created during
  install. Rename install-exec-local to install-data-local.
  Only install qemu.conf if WITH_QEMU is set.
* tests/networkschematest: Update for XML location move

Move the virsh tool and its man page into the tools directory

* Makefile.am: Remove rules for virsh.1 man page
* virsh.1: Remove auto-generated file
* docs/Makefile.am: Remove rules for virsh.pod man page
* docs/virsh.pod: Move to tools/ directory
* src/Makefile.am, src/.gitignore: Remove rules for virsh
* src/console.c, src/console.h, src/*.ico, src/virsh_win_icon.rc,
  src/virsh.c: Move into tools/ directory
* tools/Makefile.am: Add rules for building virsh
* tools/.gitignore: Ignore virsh built files
* tests/virshtest.c, tests/int-overflow: Update for new
  virsh location

* src/Makefile.am, src/qemu/qemu_conf.h, src/qemu/qemu_driver.c,
  tests/seclabeltest.c: Adapt for changed paths
* src/security.c: Rename to src/security/security_driver.c
* src/security.h: Rename to src/security/security_driver.h
* src/security_selinux.c, src/security_selinux.h: Move to src/security/

* src/qemu_conf.c, src/qemu_conf.h, src/qemu_driver.c,
  src/qemu_driver.h: Move to src/qemu/
* daemon/qemud.c, src/Makefile.am, tests/qemuargv2xmltest.c,
  tests/qemuhelptest.c, tests/qemuxml2argvtest.c,
  tests/qemuxml2xmltest.c: Adapt for changed paths

* src/Makefile.am, src/proxy_internal.c, src/proxy_internal.h
  src/sexpr.c, src/sexpr.h, src/xen_unified.c, src/xen_unified.h,
  src/xen_internal.c, src/xen_internal.h, src/xen_inotify.c,
  src/xen_inotify.h, src/xend_internal.c, src/xend_internal.h,
  src/xm_internal.c, src/xm_internal.h, src/xs_internal.c,
  src/xs_internal.h: Move to src/xen/ directory
* proxy/Makefile.am, proxy/libvirt_proxy.c, src/Makefile.am,
  src/libvirt.c, tests/sexpr2xmltest.c, tests/statstest.c,
  tests/xencapstest.c, tests/xmconfigtest.c, tests/xml2sexprtest.c:
  Adapt to changed xen location
* src/stats_linux.h, src/stats_linux.c: Remove xen specific block
  stats APIs
* src/qemu_driver.c, src/uml_driver.c: Add missing sys/un.h include
  uncovered after change to stats_linux.h
* src/xen/block_stats.h, src/xen/block_stats.c: Add xen specific
  block stats APIs

* qemud/: Rename to daemon/
* Makefile.am, configure.in, src/Makefile.am, src/remote_internal.c,
  tests/Makefile.am, tests/eventtest.c: s/qemud/daemon/ where needed

* src/qemu_conf.c: Fix leak of values upon OOM
* src/xend_internal.c: Fix missing check for OOM failure
* tests/qemuargv2xmltest.c, tests/qemuxml2argvtest.c: Free
  stateDir upon exit to avoid leak

Fix all thread locking bugs reported by object-locking test
case.

NB, some of the driver locking is getting too coarse. Driver
mutexes really need to be turned into RW locks instead to
significantly increase concurrency.

* src/lxc_driver.c: Fix useof driver when unlocked in the methods
  lxcDomainGetInfo, lxcSetSchedulerParameters, and
  lxcGetSchedulerParameters
* src/opennebula/one_driver.c: Fix missing unlock in oneDomainUndefine.
  Fix use of driver when unlocked in oneDomainGetInfo,
  oneGetOSType, oneDomainShutdown
* src/qemu_driver.c: Fix use of driver when unlocked in
  qemudDomainSavem, qemuGetSchedulerType, qemuSetSchedulerParameters
  and qemuGetSchedulerParameters
* src/storage_driver.c: Re-work storagePoolCreate to avoid bogus
  lock checking warning. Re-work storageVolumeCreateXMLFrom to
  remove a potential NULL de-reference & avoid bogus lock check
  warnings
* src/test.c: Remove testDomainAssignDef since it break lock chekc
  warnings.
* tests/object-locking.ml: Add oneDriverLock, oneDriverUnlock
  and one_driver_t methods/types to allow lock checking on the
   OpenNebula drivers

* tests/testutilsqemu.c: add a machine types list for /usr/bin/kvm
  which doesn't have any aliases, while the guest has aliases

* tests/qemuxml2argvdata/qemuxml2argv-machine-aliases2.*,
  tests/qemuxml2argvtest.c: add a test using /usr/bin/kvm and make
  sure that 'pc' machine type doesn't get canonicalized using the
  aliases in the guest machine type list

* tests/testutilsqemu.c: make 'pc' an alias for qemu-system-x86_64

* tests/qemuxml2argvdata/qemuxml2argv-machine-aliases1.*,
  tests/qemuxml2argvtest.c: add a test which uses qemu-system-x86_64
  and make sure the machine type is canonicalized.

* test/testutilsqemu.c: split out code to testQemuAllocMachines()
  and make use of the ARRAY_CARDINALITY macro

This doesn't have any affect on the current tests because we don't have
any machine aliases in the current test data.

* src/qemu_conf.h, src/qemu_driver.c: expose qemudCanonicalizeMachine()
  for the tests

* tests/qemuxml2argvtest.c: canonicalize the machine type

* src/testutils.[ch]: make testDebug externally available

* src/testutilsqemu.c: if VIR_TEST_DEBUG is set, dump the qemu
  driver capabilities to stderr

* tests/qemuhelpdata/qemu-kvm-0.11.0-rc2: add data

* tests/qemuhelptest.c: add expected output

Things added include

  - x86_64 host
  - Migration features
  - NUMA topology
  - Security model
  - Canonical machine types
  - Domain machine types
  - An arm guest
  - More machine types for mips, mipsel, sparc and ppc
  - An x86_64 xenner guest

This file was generated on a Fedora 12 machine using latest libvirt
and qemu-kvm-0.11.0-rc2

* tests/capabilityschemadata/caps-qemu-kvm.xml: add more features
  to test

* tests/eventtest.c (mymain): Exit nonzero upon write failure.
This also avoids several dead stores of the form ret = safewrite...

If a file descriptor with events=0 was added to the libvirtd
event loop, it would still be added to the poll() fds' array.
While it wouldn't see any POLLIN/OUT events, it'd still get
triggered for HANGUP/ERROR events which was not in compliance
with the libvirt events API contract.

* qemud/event.c: Don't poll on FDs with events=0
* tests/eventtest.c: Add test case to validate fix to event.c

* docs/schemas/domain.rng: Add <serial> element to disks
* src/domain_conf.h, src/domain_conf.c: XML parsing and
  formatting for disk serial numbers
* src/qemu_conf.c: Set serial number when launching guests
* tests/qemuxml2argvdata/qemuxml2argv-disk-drive-shared.args,
  tests/qemuxml2argvdata/qemuxml2argv-disk-drive-shared.xml: Add
  serial number to XML test

Add option to domain XML for

     <memoryBacking>
        <hugepages/>
     </memoryBacking>

* configure.in: Add check for mntent.h
* qemud/libvirtd_qemu.aug, qemud/test_libvirtd_qemu.aug, src/qemu.conf
  Add 'hugetlbfs_mount' config parameter
* src/qemu_conf.c, src/qemu_conf.h: Check for -mem-path flag in QEMU,
  and pass it when hugepages are requested.
  Load hugetlbfs_mount config parameter, search for mount if not given.
* src/qemu_driver.c: Free hugetlbfs_mount/path parameter in driver shutdown.
  Create directory for QEMU hugepage usage, chowning if required.
* docs/formatdomain.html.in: Document memoryBacking/hugepages elements
* docs/schemas/domain.rng: Add memoryBacking/hugepages elements to schema
* src/util.c, src/util.h, src/libvirt_private.syms: Add virFileFindMountPoint
  helper API
* tests/qemuhelptest.c: Add -mem-path constants
* tests/qemuxml2argvtest.c, tests/qemuxml2xmltest.c: Add tests for hugepage
  handling
* tests/qemuxml2argvdata/qemuxml2argv-hugepages.xml,
  tests/qemuxml2argvdata/qemuxml2argv-hugepages.args: Data files for
  hugepage tests

* tests/testutils.c: Run test function twice, once to prime it for
  static allocations, once to count the non-static allocations.
* tests/testutilsqemu.c: Initialize variable correctl
* src/capabilities.c: Don't free machines variable upon failure
  since caller must do that
* src/xm_internal.c: Add missing check for OOM in building VIF
  config param

Calling qsort() on the disks array causes disk to be
unneccessarily re-ordered, potentially breaking the
ability to boot if the boot disk gets moved later in
the list. The new algorithm will insert a new disk as
far to the end of the list as possible, while being
ordered correctly wrt other disks on the same bus.

* src/domain_conf.c, src/domain_conf.h: Remove disk sorting
  routines. Add API to insert a disk into existing list at
  the optimal position, without resorting disks
* src/libvirt_private.syms: Export virDomainDiskInsert
* src/xend_internal.c, src/xm_internal.c: Remove calls to
  qsort, use virDomainDiskInsert instead.
* src/qemu_driver.c: Remove calls to qsort, use virDoaminDiskInsert
  instead. Fix reordering bugs when hotunplugging disks and
  networks. Fix memory leak in disk/net unplug

The XML allows <encryption format='unencrypted'/>, this implementation
canonicalizes the internal representation so that "vol->encryption" is
non-NULL iff the volume is encrypted.

Note that partial encryption information (e.g. specifying an encryption
format, but not the key/passphrase) is valid, libvirt will automatically
choose value for the missing information during volume creation.  The
user can read the volume XML, and use the unmodified <encryption> tag in
future operations (without having to be able to understand) its contents.

* docs/formatstorage.html, docs/formatstorage.html.in: Document
  storage volume encryption options
* src/storage_conf.c, src/storage_conf.h: Hook up storage
  encryption XML handling
* tests/storagevolschemadata/vol-qcow2.xml: Test case for encryption
  schema changes

Test case for the fix applied in

  commit 14435163
  Author: Daniel Veillard <veillard@redhat.com>
  Date:   Fri Jun 26 18:14:16 2009 +0000

* tests/sexpr2xmldata/sexpr2xml-pv-vfb-type-crash.sexpr,
  tests/sexpr2xmldata/sexpr2xml-pv-vfb-type-crash.xml: Data
  files exhibiting the crash
* tests/sexpr2xmltest.c: Process new data files

PCI device assignment is only supported in KVM's fork of qemu, so we
should really detect its availability and give a nice error if its
not supported.

* src/qemu_conf.[ch]: introduce QEMUD_CMD_FLAG_PCIDEVICE indicating
  that the -pcidevice command line option is available

* tests/*: update the tests

* src/logging.c src/logging.h src/libvirt_private.syms:
  define new functions virLogSetFromEnv and virLogParseDefaultPriority
* qemud/qemud.c src/libvirt.c tests/eventtest.c: cleanup to use the
  unified functions

* tests/eventtest.c: don't covert high priority levels to debug level.
  Consider an invalid priority level setting a setup failure.

* src/xen_internal.c: Don't free memory now owned by the virCapsPtr
   structure
* tests/testutilsxen.c: Fix up for change in internal API

A subsequent commit will add a "canonical" field to this structure,
this patch basically just prepares the way for that.

The new type is added, along with virCapabilitiesAlloc/FreeMachines()
helpers and a whole bunch of code to make the transition.

One quirk is that virCapabilitiesAddGuestDomain() and
virCapabilitiesAddGuest() take ownership of the machine list rather
than duping it. This makes sense to avoid needless copying.

* src/capabilities.h: add the virCapsGuestMachine struct and use it
  in virCapsGuestDomainInfo, add prototypes for new functions and
  update the AddGuest() prototypes

* src/capabilities.c: add code for allocating and freeing the new
  type, change the machines parameter to AddGuest() etc.

* src/libvirt_private.syms: export the new helpers

* src/qemu_conf.c: update all the machine type code to use the new
  struct

* src/xen_internal.c: ditto

* tests/testutilsqemu.c: ditto

* tests/Makefile.am (EXTRA_DIST): Add interfaceschemadata.

We need these so that we can remove the devices via the monitor.

* src/domain_conf.h: add nic_name and hostnet_name to virDomainNetDef

* src/domain_conf.c: free nic_name and hostnet_name

* src/qemu_conf.c: add qemuAssignNetNames(), use it if qemu has
  support for the param and pass the names on the command line

* tests/qemuxml2argv*: add a test for this