In commit 3c80fac2 'vmxnet3' handling
was added to esxVMX_ParseEthernet(), but not to the inverse function
esxVMX_FormatEthernet().

* docs/drivers.html.in: list the ESX driver
* docs/drvesx.html.in: the new ESX driver documentation
* docs/hvsupport.html.in: add the ESX driver to the matrix
* docs/index.html.in, docs/sitemap.html.in: list the ESX driver
* src/esx/esx_driver.c: fix and cleanup some comments

There is currently no way to determine the libvirt version of a remote
libvirtd we are connected to. This is a useful piece of data to enable
feature detection.

This implements the virConnectIsSecure, virConnectIsEncrypted,
virDomainIsPersistent, virDomainIsActive, virNetworkIsActive,
virNetworkIsPersistent, virStoragePoolIsActive,
virStoragePoolIsPersistent, virInterfaceIsActive APIs in
(nearly) all drivers. Exceptions are:

 phyp: missing domainIsActive/Persistent
 esx: missing domainIsPersistent
 opennebula: missing domainIsActive/Persistent

* src/remote/remote_protocol.x: Define remote wire ABI for newly
  added APIs.
* daemon/remote_dispatch*.h: Re-generated from remote_protocol.x
* src/esx/esx_driver.c, src/lxc/lxc_driver.c, src/network/bridge_driver.c,
  src/opennebula/one_driver.c, src/openvz/openvz_conf.c,
  src/openvz/openvz_driver.c, src/phyp/phyp_driver.c,
  src/remote/remote_driver.c, src/storage/storage_driver.c,
  src/test/test_driver.c, src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c, src/xen/xen_driver.h, src/xen/xen_inotify.c,
  src/xen/xen_inotify.h: Implement all the new APIs where possible

Introduce a number of new APIs to  expose some boolean properties
of objects, which cannot otherwise reliably determined, nor are
aspects of the XML configuration.

 * virDomainIsActive: Checking virDomainGetID is not reliable
   since it is not possible to distinguish between error condition
   and inactive domain for ID of -1.
 * virDomainIsPersistent: Check whether a persistent config exists
   for the domain

 * virNetworkIsActive: Check whether the network is active
 * virNetworkIsPersistent: Check whether a persistent config exists
   for the network

 * virStoragePoolIsActive: Check whether the storage pool is active
 * virStoragePoolIsPersistent: Check whether a persistent config exists
   for the storage pool

 * virInterfaceIsActive: Check whether the host interface is active

 * virConnectIsSecure: whether the communication channel to the
   hypervisor is secure
 * virConnectIsEncrypted: whether any network based commnunication
   channels are encrypted

NB, a channel can be secure, even if not encrypted, eg if it does
not involve the network, like a UNIX socket, or pipe.

 * include/libvirt/libvirt.h.in: Define public API
 * src/driver.h: Define internal driver API
 * src/libvirt.c: Implement public API entry point
 * src/libvirt_public.syms: Export API symbols
 * src/esx/esx_driver.c, src/lxc/lxc_driver.c,
   src/interface/netcf_driver.c, src/network/bridge_driver.c,
   src/opennebula/one_driver.c, src/openvz/openvz_driver.c,
   src/phyp/phyp_driver.c, src/qemu/qemu_driver.c,
   src/remote/remote_driver.c, src/test/test_driver.c,
   src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
   src/xen/xen_driver.c: Stub out driver tables

A character device's target (it's interface in the guest) had only a
single property: port. This patch is in preparation for adding targets
which require other properties.
Since this changes the conf type for character devices this affects
a number of drivers:

* src/conf/domain_conf.[ch] src/esx/esx_vmx.c src/qemu/qemu_conf.c
  src/qemu/qemu_driver.c src/uml/uml_conf.c src/uml/uml_driver.c
  src/vbox/vbox_tmpl.c src/xen/xend_internal.c src/xen/xm_internal.c:
  target properties are moved into a union in virDomainChrDef, and a
  targetType field is added to identify which union member should be
  used. All current code which touches a virDomainChrDef is updated both
  to use the new union field, and to populate targetType if necessary.

If an error occurs between the allocation of an item and appending it
to the list, the item leaks. Free such orphaned items in error cases.

* src/esx/esx_vi.c: free orphaned items in error cases

The default transport for the VI API is HTTPS. If the server redirects
from HTTPS to HTTP the driver would silently follow that redirection.
The user assumes to communicate with the server over a secure transport
but isn't.

This patch disables automatical redirection following. The driver reports
an error if the server tries to redirect.

* src/esx/esx_vi.c: refactor the call to curl_easy_perform() into a
  function and do error handling there, disable automatical redirection
  following for curl
* src/esx/esx_vi.h: change the type of responseCode to int

Unified function naming scheme:
- 'lookup' functions query the ESX or vCenter for information
- 'get' functions return information from a local object

* src/esx/esx_driver.c, src/esx/esx_vi.[ch]: unify function naming

In order to register a new virtual machine the ESX driver needs to upload
a VMX file to a datastore. Try to put this file beside the main VMDK file
of the virtual machine. Change the disk selection for datastore detection
to choose the first file-based harddisk instead of just the first disk.
The first disk may be a CDROM disk and ISO images are normaly not located
in the virtual machine's directory.

* src/esx/esx_driver.c: change disk selection for datastore detection

This allows to use domain-xml-from-native with VMX files that reference
unavailable datastores.

* src/esx/esx_vmx.c: fallback to the preliminary name if the datastore
  cannot be found

Implementation of tunnelled migration, using a Unix Domain Socket
on the qemu backend.  Note that this requires very new versions of
qemu (0.10.7 at least) in order to get the appropriate bugfixes.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

An inaccessible datastore has no valid URL property so don't
access its URI property.

* src/esx/esx_vi.c: esxVI_LookupDatastoreByName(): check if datastore is
  accessible before accessing its URL property
* src/esx/esx_vmx.c: update to changed datastore properties

* src/esx/esx_driver.c: remove phantom mode

* src/esx/esx_util.[ch]: remove esxUtil_EqualSuffix()
* src/esx/esx_driver.c, src/esx/esx_vmx.c: replace esxUtil_EqualSuffix()
  with virFileHasSuffix()

A given domain XML gets converted to a VMX config, uploaded to the host
and registered as new virtual machine.

* src/esx/esx_driver.c: refactor datastore related path parsing into
  esxUtil_ParseDatastoreRelatedPath()
* src/esx/esx_util.[ch]: add esxUtil_ParseDatastoreRelatedPath()
* src/esx/esx_vi.[ch]: add esxVI_Context_UploadFile(), add datastores to
  the traversal in esxVI_BuildFullTraversalSpecList(), add
  esxVI_LookupDatastoreByName()
* src/esx/esx_vi_methods.[ch]: add esxVI_RegisterVM_Task()
* src/esx/esx_vi_types.c: make some error message more verbose
* src/esx/esx_vmx.[ch]: add esxVMX_AbsolutePathToDatastoreRelatedPath()
  to convert a path into a datastore related path, add esxVMX_ParseFileName()
  to convert from VMX path format to domain XML path format, extend the other
  parsing function to be datastore aware, add esxVMX_FormatFileName() to
  convert from domain XML path format to VMX path format, fix VMX ethernet
  entry formating
* tests/esxutilstest.c: add test for esxUtil_ParseDatastoreRelatedPath()
* tests/vmx2xmldata/*: update domain XML files to use datastore related paths
* tests/xml2vmxdata/*: update domain XML files to use datastore related paths,
  update VMX files to use absolute paths

Add esxVI_Occurence enum to describe expected occurence of items
* src/esx/esx_driver.c: update the use of esxVI_LookupVirtualMachineByUuid()
* src/esx/esx_vi.c: add an esxVI_Occurence parameter to
  esxVI_LookupVirtualMachineByUuid() and take care if esxVI_FindByUuid()
  can't find anything for a given uuid
* src/esx/esx_vi.h: add esxVI_Occurence enum
* src/esx/esx_vi_methods.c: expect null or more items to be returned
  from esxVI_FindByUuid()

* src/esx/esx_driver.c: add esxSupportsLongMode() and update esxCapsInit()
* src/esx/esx_vi.[ch]: Add AnyType handling for lists
* src/esx/esx_vi_types.c: bind VI type HostCpuIdInfo

Extend and cleanup the VMX to domain XML mapping. Add the domain XML to
VMX mapping functions.

* src/esx/esx_driver.c: add esxDomainXMLToNative()
* src/esx/esx_vmx.[ch]: add esxVMX_SCSIDiskNameToControllerAndID(),
  esxVMX_IDEDiskNameToControllerAndID(), esxVMX_FloppyDiskNameToController(),
  esxVMX_GatherSCSIControllers(), add basic handling for the VMX guestOS entry
  to distinguish between i686 and x86_64, make SCSI virtualDev VMX entry
  optional as it should be, map the VMX networkName entry to the domain XML
  interface bridge name, add basic mapping for serial devices in pipe mode,
  add several esxVMX_Format*() functions

This enables the auth callback to automatically distinguish between
requests for ESX host and vCenter credentials.

* src/esx/esx_util.[ch]: set challenge for auth callback to hostname

* src/esx/esx_driver.c: add esxNodeGetFreeMemory(), cache IP address
* src/esx/esx_vi.[ch]: refactor resource pool query into esxVI_GetResourcePool()
* src/esx/esx_vi_types.[ch]: bind VI type ResourcePoolResourceUsage

Add the virStrncpy function, which takes a dst string, source string,
the number of bytes to copy and the number of bytes available in the
dest string.  If the source string is too large to fit into the
destination string, including the \0 byte, then no data is copied and
the function returns NULL.  Otherwise, this function copies n bytes
from source into dst, including the \0, and returns a pointer to the
dst string.  This function is intended to replace all unsafe uses
of strncpy in the code base, since strncpy does *not* guarantee that
the buffer terminates with a \0.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

* src/esx/esx_vmx.c: add it to the list of accepted devices when
  parsing the Ethernet section.

* src/esx/esx_driver.c: also strip (C) and (TM) from the CPU model
  strings

* src/esx/esx_driver.c: reorder some function calls to avoid potential
  virDomainPtr leaks

* src/esx/esx_util.c: let esxUtil_GetConfigUUID() report an error if
  virUUIDParse() fails

* src/esx/esx_driver.c: add esxDomainUndefine() based on
  esxVI_UnregisterVM()
* src/esx/esx_vi_methods.[ch]: add esxVI_UnregisterVM()

* src/esx/esx_driver.c: add esxCapsInit() with default caps and add
  esxGetCapabilities()

* src/esx/esx_vi.[ch]: convert esxVI_RemoteRequest_Execute() to a
  simpler esxVI_Context_Execute() version, remove esxVI_RemoteRequest
  and convert esxVI_RemoteResponse to esxVI_Response
* src/esx/esx_vi_methods.c: update and simplify callers to use
  esxVI_Context_Execute() instead of esxVI_RemoteRequest_Execute()

* src/esx/esx_vi.[ch]: use virXPathNode*() in
  esxVI_RemoteRequest_Execute() and remove
  esxVI_RemoteResponse_DeserializeXPathObject*()
* src/esx/esx_vi_methods.c: update callers to use the new syntax of
  esxVI_RemoteRequest_Execute()

* src/esx/esx_driver.c: add configStatus to the requested properties
  to check it in esxVI_GetVirtualMachineIdentity()
* src/esx/esx_vi.[ch]: add esxVI_GetManagedEntityStatus()
  and use it in esxVI_GetVirtualMachineIdentity()
* src/esx/esx_vi_types.[ch]: add VI type esxVI_ManagedEntityStatus

* src/esx/esx_driver.c: handle spaces in VMX file path and use a
  virBuffer to encode spaces correctly in the resulting URL
* src/esx/esx_vi.c: include the URL in the error message in case
  of a download error

* src/esx/esx_vi.c (esxVI_Enumeration_Deserialize): Fix
reversed goto and result=-1 statements.

* src/esx/esx_util.c: esxUtil_ParseQuery() warns if a known query
  parameter should be ignored due to the corresponding char/int pointer
  being NULL, instead of silently ignoring it. Fix the control flow.

* src/esx/esx_vmx.c: add an extra type of addressType beside 'static'
  and 'generated', 'vpx' indicates that the MAC address was generated
  by a vCenter.

While trying to remove uses of unsafe strncpy in the tree, I came
across a couple of usages in the ESX driver.  To my eyes, the snprintf
replacements do the same thing in less code, and are also safer.
Signed-off-by: NChris Lalancette <clalance@redhat.com>
Tested-by: NMattias Bolte <matthias.bolte@googlemail.com>