A given domain XML gets converted to a VMX config, uploaded to the host
and registered as new virtual machine.

* src/esx/esx_driver.c: refactor datastore related path parsing into
  esxUtil_ParseDatastoreRelatedPath()
* src/esx/esx_util.[ch]: add esxUtil_ParseDatastoreRelatedPath()
* src/esx/esx_vi.[ch]: add esxVI_Context_UploadFile(), add datastores to
  the traversal in esxVI_BuildFullTraversalSpecList(), add
  esxVI_LookupDatastoreByName()
* src/esx/esx_vi_methods.[ch]: add esxVI_RegisterVM_Task()
* src/esx/esx_vi_types.c: make some error message more verbose
* src/esx/esx_vmx.[ch]: add esxVMX_AbsolutePathToDatastoreRelatedPath()
  to convert a path into a datastore related path, add esxVMX_ParseFileName()
  to convert from VMX path format to domain XML path format, extend the other
  parsing function to be datastore aware, add esxVMX_FormatFileName() to
  convert from domain XML path format to VMX path format, fix VMX ethernet
  entry formating
* tests/esxutilstest.c: add test for esxUtil_ParseDatastoreRelatedPath()
* tests/vmx2xmldata/*: update domain XML files to use datastore related paths
* tests/xml2vmxdata/*: update domain XML files to use datastore related paths,
  update VMX files to use absolute paths

Add esxVI_Occurence enum to describe expected occurence of items
* src/esx/esx_driver.c: update the use of esxVI_LookupVirtualMachineByUuid()
* src/esx/esx_vi.c: add an esxVI_Occurence parameter to
  esxVI_LookupVirtualMachineByUuid() and take care if esxVI_FindByUuid()
  can't find anything for a given uuid
* src/esx/esx_vi.h: add esxVI_Occurence enum
* src/esx/esx_vi_methods.c: expect null or more items to be returned
  from esxVI_FindByUuid()

* src/esx/esx_driver.c: add esxSupportsLongMode() and update esxCapsInit()
* src/esx/esx_vi.[ch]: Add AnyType handling for lists
* src/esx/esx_vi_types.c: bind VI type HostCpuIdInfo

Extend and cleanup the VMX to domain XML mapping. Add the domain XML to
VMX mapping functions.

* src/esx/esx_driver.c: add esxDomainXMLToNative()
* src/esx/esx_vmx.[ch]: add esxVMX_SCSIDiskNameToControllerAndID(),
  esxVMX_IDEDiskNameToControllerAndID(), esxVMX_FloppyDiskNameToController(),
  esxVMX_GatherSCSIControllers(), add basic handling for the VMX guestOS entry
  to distinguish between i686 and x86_64, make SCSI virtualDev VMX entry
  optional as it should be, map the VMX networkName entry to the domain XML
  interface bridge name, add basic mapping for serial devices in pipe mode,
  add several esxVMX_Format*() functions

This enables the auth callback to automatically distinguish between
requests for ESX host and vCenter credentials.

* src/esx/esx_util.[ch]: set challenge for auth callback to hostname

* src/esx/esx_driver.c: add esxNodeGetFreeMemory(), cache IP address
* src/esx/esx_vi.[ch]: refactor resource pool query into esxVI_GetResourcePool()
* src/esx/esx_vi_types.[ch]: bind VI type ResourcePoolResourceUsage

Currently, libvirtd will start a dnsmasq process for the virtual
network, but (aside from killing the dnsmasq process and replacing it),
there's no way to define tftp boot options.

This change introduces the appropriate tags to the dhcp configuration:

 <network>
   <name>default</name>
   <bridge name="virbr%d" />
   <forward/>
   <ip address="192.168.122.1" netmask="255.255.255.0">
     <tftp root="/var/lib/tftproot" />
     <dhcp>
       <range start="192.168.122.2" end="192.168.122.254" />
       <bootp file="pxeboot.img"/>
     </dhcp>
   </ip>
 </network>

When the attributes are present, these are passed to the
arguments to dnsmasq:

 dnsmasq [...] --enable-tftp --tftp-root /srv/tftp --dhcp-boot pxeboot.img
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^
                      from <tftp />                     from <bootp />

At present, only local tftp servers are supported (ie, dnsmasq runs as
the tftp server), but we could improve this in future by adding a
server= attribute.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

2009-09-21  Paolo Bonzini  <pbonzini@redhat.com>
            Jeremy Kerr  <jk@ozlabs.org>

	* docs/formatnetwork.html.in: Document new tags.
	* docs/formatnetwork.html: Regenerate.
	* docs/schemas/network.rng: Update.
	* src/network_conf.c (virNetworkDefFree): Free new fields.
	(virNetworkDHCPRangeDefParseXML): Parse <bootp>.
	(virNetworkIPParseXML): New, parsing <dhcp> and <tftp>.
	(virNetworkDefParseXML): Use virNetworkIPParseXML instead of
	virNetworkDHCPRangeDefParseXML.
	(virNetworkDefFormat): Pretty print new fields.
	* src/network_conf.h (struct _virNetworkDef): Add netboot fields.
	* src/network_driver.c (networkBuildDnsmasqArgv): Add
	TFTP and BOOTP arguments.

	* tests/Makefile.am (EXTRA_DIST): Add networkschemadata.
	* tests/networkschematest: Look in networkschemadata.
	* tests/networkschemadata/netboot-network.xml: New.

Add the virStrncpy function, which takes a dst string, source string,
the number of bytes to copy and the number of bytes available in the
dest string.  If the source string is too large to fit into the
destination string, including the \0 byte, then no data is copied and
the function returns NULL.  Otherwise, this function copies n bytes
from source into dst, including the \0, and returns a pointer to the
dst string.  This function is intended to replace all unsafe uses
of strncpy in the code base, since strncpy does *not* guarantee that
the buffer terminates with a \0.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Latest upstream QEMU can be built with Xen support, which introduces
a -xen-domid argument. This was  mistakenly detected as -domid due
to old Xenner support. Adapt to cope with both syntax. Also only
set domid if the virt type is xen, or the guest type is xen

* src/qemu_conf.c, src/qemu_conf.h: Detect new -xen-domid flag in
  preference to -domid.
* tests/qemuxml2argvdata/qemuxml2argv-bootloader.args,
  tests/qemuxml2argvdata/qemuxml2argv-input-xen.args: Add missing
  -domid param
* tests/qemuxml2argvdata/qemuxml2argv-misc-uuid.args: Remove bogus
  -boot param.
* tests/qemuxml2argvtest.c: Add missing QEMUD_CMD_FLAG_DOMID params

The commit cb51aa48 "Fix up connection
reference counting." changed the driver closing and virConnectPtr
unref-logic in virConnectClose().

Before this commit virConnectClose() closed all drivers of the given
virConnectPtr and virUnrefConnect()'ed it afterwards. After this
commit the driver-closing is done in virUnrefConnect() if and only if
the ref-count of the virConnectPtr dropped to zero.

This change in execution order leads to a virConnectPtr leak, at least
for connections to Xen.

The relevant call sequences:

virConnectOpen() -> xenUnifiedOpen() ...

... xenInotifyOpen() -> virConnectRef(conn)

... xenStoreOpen() -> xenStoreAddWatch() -> conn->refs++

virConnectClose() -> xenUnifiedClose() ...

... xenInotifyClose() -> virUnrefConnect(conn)

... xenStoreClose() -> xenStoreRemoveWatch() -> virUnrefConnect(conn)

Before the commit this additional virConnectRef/virUnrefConnect calls
where no problem, because virConnectClose() closed the drivers
explicitly and the additional refs added by the Xen subdrivers were
removed properly. After the commit this additional refs result in a
virConnectPtr leak (including a leak of the hypercall file handle;
that's how I noticed this problem), because now the drivers are only
close if and only if the ref-count drops to zero, but this cannot
happen anymore, because the additional refs from the Xen subdrivers
would only be removed if the drivers get closed, but that doesn't
happen because the ref-count cannot drop to zero.

The fix for this problem is simple: remove the
virConnectRef/virUnrefConnect calls from the Xen subdrivers (see
attached patch). Maybe someone could explain why the Xen Inotify and
Xen Store driver do this extra ref-counting, but none of the other Xen
subdrivers. It seems unnecessary to me and can be removed without
problems.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* src/conf/domain_conf.c: Don't assume all virDomainObjPtr have
  a non-NULL monitor_chr field in virDomainObjFormat.
* src/lxc/lxc_driver.c: Implement suspend/resume driver APis
* src/util/cgroup.c, src/util/cgroup.h: Support the 'freezer'
  cgroup controller
* src/libvirt_private.syms: Export virCgroupSetFreezerState
  and virCgroupGetFreezerState

* src/openvz/openvz_driver.c: try to autogenerate a non-conflicting
  id for the domain and use the name field as the name not as id.

This was prevent virDomainSave from working properly by default,
which is a bad thing.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

When making changes to the remote protocol, src/ is always built
first, so rpcgen should live there, to avoid having to run make
in the 'daemon/' directory before building src/

* src/Makefile.am: Add rules for rpcgen, and drop -I../daemon from
  remote client build
* daemon/Makefile.am: Add -I../src/remote/ to libvirtd build
  and remove rpcgen rules
* daemon/libvirtd.c: Adapt include of remote_driver.h taking
  into account new -I flag
* daemon/remote_protocol.c, daemon/remote_protocol.h,
  daemon/remote_protocol.x: Move to src/remote/
* daemon/rpcgen_fix.pl: Move to src/remote/rpcgen_fix.pl

* src/README: New file

* src/bridge.c, src/bridge.h, src/buf.c, src/buf.h, src/cgroup.c,
  src/cgroup.h, src/conf.c, src/conf.h, src/event.c, src/event.h,
  src/hash.c, src/hash.h, src/hostusb.c, src/hostusb.h,
  src/iptables.c, src/iptables.h, src/logging.c, src/logging.h,
  src/memory.c, src/memory.h, src/pci.c, src/pci.h, src/qparams.c,
  src/qparams.h, src/stats_linux.c, src/stats_linux.h,
  src/threads-pthread.c, src/threads-pthread.h, src/threads-win32.c,
  src/threads-win32.h, src/threads.c, src/threads.h, src/util.c,
  src/util.h, src/uuid.c, src/uuid.h, src/virterror.c,
  src/virterror_internal.h, src/xml.c, src/xml.h: Move all files
  into src/util/
* daemon/Makefile.am: Add -Isrc/util/ to build flags
* src/Makefile.am: Add -Isrc/util/ to build flags and update for
  moved files
* src/libvirt_private.syms: Export cgroup APIs since they're now
  in util rather than linking directly to drivers
* src/xen/xs_internal.c: Disable bogus virEventRemoveHandle call
  when built under PROXY
* proxy/Makefile.am: Update for changed file locations. Remove
  bogus build of event.c
* tools/Makefile.am, tests/Makefile.am: Add -Isrc/util/ to build flags

* src/capabilities.c, src/capabilities.h, src/domain_conf.c,
  src/domain_conf.h, src/domain_event.c, src/domain_event.h,
  src/interface_conf.c, src/interface_conf.h,
  src/network_conf.c, src/network_conf.h, src/node_device_conf.c,
  src/node_device_conf.h, src/secret_conf.c, src/secret_conf.h,
  src/storage_conf.c, src/storage_conf.h, src/storage_encryption_conf.c,
  src/storage_encryption_conf.h: Move to src/conf/
* src/Makefile.am: Add -Isrc/conf to the individual build targets
  which need to use XML config APIs. Remove LIBXML_CFLAGS, LIBSSH2_CFLAGS
  and SELINUX_CFLAGS from global INCLUDES and only have them in build
  targets which actually need them.  Create a libvirt_conf.la
  convenience library for all config parsers
* src/hostusb.h: Remove bogus include of domain_conf.h
* tests/Makefile.am: Add -Isrc/conf. Remove bogus -I$builddir/src
  since it never has any generated header files
* daemon/Makefile.am: Add -Isrc/conf
* proxy/Makefile.am: Add -Isrc/conf and cope with renamed files
* src/hash.c: Remove bogus include of libxml/threads.h

* daemon/default-network.xml: Move to src/network/default.xml
* daemon/libvirtd_qemu.aug, daemon/test_libvirtd_qemu.aug: Move
  to src/qemu/
* src/qemu.conf: Move to src/qemu/qemu.conf
* daemon/Makefile.am: Remove rules for default-nmetwork.xml and
  libvirtd_qemu.aug and test_libvirtd_qemu.aug. Fix typo in
  uninstall-local that would install polkit again.
* src/Makefile.am: Add rules for installing network/default.xml
  and the qemu/*.aug files. Add test case for QEMU augeas files.
  Add uninstall-local rule for files/directories created during
  install. Rename install-exec-local to install-data-local.
  Only install qemu.conf if WITH_QEMU is set.
* tests/networkschematest: Update for XML location move

Move the virsh tool and its man page into the tools directory

* Makefile.am: Remove rules for virsh.1 man page
* virsh.1: Remove auto-generated file
* docs/Makefile.am: Remove rules for virsh.pod man page
* docs/virsh.pod: Move to tools/ directory
* src/Makefile.am, src/.gitignore: Remove rules for virsh
* src/console.c, src/console.h, src/*.ico, src/virsh_win_icon.rc,
  src/virsh.c: Move into tools/ directory
* tools/Makefile.am: Add rules for building virsh
* tools/.gitignore: Ignore virsh built files
* tests/virshtest.c, tests/int-overflow: Update for new
  virsh location

* src/Makefile.am, src/qemu/qemu_conf.h, src/qemu/qemu_driver.c,
  tests/seclabeltest.c: Adapt for changed paths
* src/security.c: Rename to src/security/security_driver.c
* src/security.h: Rename to src/security/security_driver.h
* src/security_selinux.c, src/security_selinux.h: Move to src/security/

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/secret_driver.c, src/secret_driver.h: Move to src/secret/

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/interface_driver.c: Rename to src/interface/netcf_driver.c
* src/interface_driver.h: Rename to src/interface/netcf_driver.h

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/network_driver.c: Rename to src/network/bridge_driver.c
* src/network_driver.h: Rename to src/network/bridge_driver.h

* daemon/mdns.c: Remove bogus include
* daemon/qemud.c, src/Makefile.am, src/libvirt.c: Adapt for
  changed paths
* src/remote_internal.c: Rename to src/remote/remote_driver.c
* src/remote_internal.h: Rename to src/remote/remote_driver.h

* src/Makefile.am, src/libvirt.c: Adjust for changed paths
* src/test.c: Rename to src/test/test_driver.c,
* src/test.h: Rename to src/test/test_driver.h

* daemon/qemud.c, src/Makefile.am: Update for changed paths
* src/node_device*.{h,c}: Move to src/node_device/
* src/storage/storage_backend.c: Remove bogus import of node_device.c

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/storage*.c, src/storage/*.h, src/parthelpre.c: Move
  to src/storage/

* src/openvz_conf.c, src/openvz_conf.h, src/openvz_driver.c,
  src/openvz_driver.h: Move to src/openvz/
* src/Makefile.am, src/libvirt.c: Adjust for changed paths

* src/uml_conf.c, src/uml_conf.h, src/uml_driver.c,
  src/uml_driver.h: Move to src/uml/
* daemon/qemud.c, src/Makefile.am: Adjust for changed paths

* src/qemu_conf.c, src/qemu_conf.h, src/qemu_driver.c,
  src/qemu_driver.h: Move to src/qemu/
* daemon/qemud.c, src/Makefile.am, tests/qemuargv2xmltest.c,
  tests/qemuhelptest.c, tests/qemuxml2argvtest.c,
  tests/qemuxml2xmltest.c: Adapt for changed paths

* src/lxc_conf.c, src/lxc_conf.h, src/lxc_container.c,
  src/lxc_container.h, src/lxc_controller.c, src/lxc_driver.c,
  src/lxc_driver.h, src/veth.c, src/veth.h: Move to src/lxc/
* src/opennebula/one_driver.c: Remove bogus veth.h include
* src/Makefile.am: Adjust for lxc paths
* daemon/qemud.c: Adjust include for lxc

* src/Makefile.am, src/proxy_internal.c, src/proxy_internal.h
  src/sexpr.c, src/sexpr.h, src/xen_unified.c, src/xen_unified.h,
  src/xen_internal.c, src/xen_internal.h, src/xen_inotify.c,
  src/xen_inotify.h, src/xend_internal.c, src/xend_internal.h,
  src/xm_internal.c, src/xm_internal.h, src/xs_internal.c,
  src/xs_internal.h: Move to src/xen/ directory
* proxy/Makefile.am, proxy/libvirt_proxy.c, src/Makefile.am,
  src/libvirt.c, tests/sexpr2xmltest.c, tests/statstest.c,
  tests/xencapstest.c, tests/xmconfigtest.c, tests/xml2sexprtest.c:
  Adapt to changed xen location
* src/stats_linux.h, src/stats_linux.c: Remove xen specific block
  stats APIs
* src/qemu_driver.c, src/uml_driver.c: Add missing sys/un.h include
  uncovered after change to stats_linux.h
* src/xen/block_stats.h, src/xen/block_stats.c: Add xen specific
  block stats APIs

* qemud/: Rename to daemon/
* Makefile.am, configure.in, src/Makefile.am, src/remote_internal.c,
  tests/Makefile.am, tests/eventtest.c: s/qemud/daemon/ where needed

* src/qemu_driver.c: sync the hostdev hot-unplug code with the disk/net
  code.