Signed-off-by: NChris Lalancette <clalance@redhat.com>

We should always be using virGetHostname in place of
gethostname; thus add in a new syntax-check rule to make
sure no new uses creep in.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* src/libvirt.c: remove reverences to non existent dconn parameter
  when using that entry point.

Normally, when you migrate a domain from host A to host B,
the domain on host A remains defined but shutoff and the domain
on host B remains running but is a "transient".  Add a new
flag to virDomainMigrate() to allow the original domain to be
undefined on source host A, and a new flag to virDomainMigrate() to
allow the new domain to be persisted on the destination host B.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Introduces several new public API options for migration

 - VIR_MIGRATE_PEER2PEER: With this flag the client only
   invokes the virDomainMigratePerform method, expecting
   the source host driver to do whatever is required to
   complete the entire migration process.
 - VIR_MIGRATE_TUNNELLED: With this flag the actual data
   for migration will be tunnelled over the libvirtd RPC
   channel. This requires that VIR_MIGRATE_PEER2PEER is
   also set.
 - virDomainMigrateToURI: This is variant of the existing
   virDomainMigrate method which does not require any
   virConnectPtr for the destination host. Given suitable
   driver support, this allows for all the same modes as
   virDomainMigrate()

The URI for VIR_MIGRATE_PEER2PEER must be a valid libvirt
URI. For non-p2p migration a hypervisor specific migration
URI is used.

virDomainMigrateToURI without a PEER2PEER flag is only
support for Xen currently, and it involves XenD talking
directly to XenD, no libvirtd involved at all.

* include/libvirt/libvirt.h.in: Add VIR_MIGRATE_PEER2PEER
  flag for migration
* src/libvirt_internal.h: Add feature flags for peer to
  peer migration (VIR_FEATURE_MIGRATE_P2P) and direct
  migration (VIR_MIGRATE_PEER2PEER mode)
* src/libvirt.c: Implement support for VIR_MIGRATE_PEER2PEER
  and virDomainMigrateToURI APIs.
* src/xen/xen_driver.c: Advertise support for DIRECT migration
* src/xen/xend_internal.c: Add TODO item for p2p migration
* src/libvirt_public.syms: Export virDomainMigrateToURI
  method
* src/qemu/qemu_driver.c: Add support for PEER2PEER and
  migration, and adapt TUNNELLED migration.
* tools/virsh.c: Add --p2p and --direct args and use the
  new virDomainMigrateToURI method where possible.

The code for tunnelled migration wierdly required the app to pass
a  NULL 'dconn' parameter, only to have to use virConnectOpen
itself shortly thereafter to get a 'dconn' object. Remove this
bogus check & require the app to always pas 'dconn' as before

* src/libvirt.c: Require 'dconn' for virDomainMigrate calls again
  and remove call to virConnectOpen

Since virMigratePrepareTunnel() is used for migration over the
native libvirt connection, there is never any need to pass the
target URI to this method.

* daemon/remote.c, src/driver.h, src/libvirt.c, src/libvirt_internal.h,
  src/qemu/qemu_driver.c, src/remote/remote_driver.c,
  src/remote/remote_protocol.c, src/remote/remote_protocol.h,
  src/remote/remote_protocol.x: Remove 'uri_in' parameter from
  virMigratePrepareTunnel() method

Implementation of tunnelled migration, using a Unix Domain Socket
on the qemu backend.  Note that this requires very new versions of
qemu (0.10.7 at least) in order to get the appropriate bugfixes.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Fix a few mispellings :-) of "successfully" and regenerate
docs/libvirt-*.xml.
* src/libvirt.c: Fix typos.
* src/secret/secret_driver.c: Fix typos.
* docs/libvirt-api.xml: Regenerate.
* docs/libvirt-refs.xml: Regenerate.

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Function comments for virStreamEvent{Add,Update,Remove}Callback() are
missing a trailing ':'. Therefore apibuild.py fails to parse the comment
and warns about the missing ':'.

* docs/libvirt-api.xml, docs/libvirt-refs.xml: updated by apibuild.py
* src/libvirt.c: add missing ':' in function comments

* src/libvirt.c: Remove hand-crafted UUID parsers in favour of
  calling virParseUUID

* include/libvirt/libvirt.h.in: Public API contract for
  virStreamPtr object
* src/libvirt_public.syms: Export data stream APIs
* src/libvirt_private.syms: Export internal helper APIs
* src/libvirt.c: Data stream API driver dispatch
* src/datatypes.h, src/datatypes.c: Internal helpers for virStreamPtr
  object
* src/driver.h: Define internal driver API for streams
* .x-sc_avoid_write: Ignore src/libvirt.c because it trips
  up on comments including write()
* python/Makefile.am: Add libvirt-override-virStream.py
* python/generator.py: Add rules for virStreamPtr class
* python/typewrappers.h, python/typewrappers.c: Wrapper
  for virStreamPtr
* docs/libvirt-api.xml, docs/libvirt-refs.xml: Regenerate
  with new APIs

The python method help docs are copied across from the C
funtion comments, but in the process all line breaks and
indentation was being lost. This made the resulting text
and code examples completely unreadable. Both the API
doc extractor and the python generator were destroying
whitespace & this fixes them to preserve it exactly.

* docs/apibuild.py: Preserve all whitespace when extracting
  function comments. Print function comment inside a <![CDATA[
  section to fully preserve all whitespace. Look for the
  word 'returns' to describe return values, instead of 'return'
  to avoid getting confused with code examples including the
  C 'return' statement.
* python/generator.py: Preserve all whitespace when printing
  function help docs
* src/libvirt.c: Change any return parameter indicated by
  'return' to be 'returns', to avoid confusing the API extractor
* docs/libvirt-api.xml: Re-build for fixed descriptions

Signed-off-by: NChris Lalancette <clalance@redhat.com>

* daemon/mdns.c: Remove bogus include
* daemon/qemud.c, src/Makefile.am, src/libvirt.c: Adapt for
  changed paths
* src/remote_internal.c: Rename to src/remote/remote_driver.c
* src/remote_internal.h: Rename to src/remote/remote_driver.h

* src/Makefile.am, src/libvirt.c: Adjust for changed paths
* src/test.c: Rename to src/test/test_driver.c,
* src/test.h: Rename to src/test/test_driver.h

* src/openvz_conf.c, src/openvz_conf.h, src/openvz_driver.c,
  src/openvz_driver.h: Move to src/openvz/
* src/Makefile.am, src/libvirt.c: Adjust for changed paths

* src/Makefile.am, src/proxy_internal.c, src/proxy_internal.h
  src/sexpr.c, src/sexpr.h, src/xen_unified.c, src/xen_unified.h,
  src/xen_internal.c, src/xen_internal.h, src/xen_inotify.c,
  src/xen_inotify.h, src/xend_internal.c, src/xend_internal.h,
  src/xm_internal.c, src/xm_internal.h, src/xs_internal.c,
  src/xs_internal.h: Move to src/xen/ directory
* proxy/Makefile.am, proxy/libvirt_proxy.c, src/Makefile.am,
  src/libvirt.c, tests/sexpr2xmltest.c, tests/statstest.c,
  tests/xencapstest.c, tests/xmconfigtest.c, tests/xml2sexprtest.c:
  Adapt to changed xen location
* src/stats_linux.h, src/stats_linux.c: Remove xen specific block
  stats APIs
* src/qemu_driver.c, src/uml_driver.c: Add missing sys/un.h include
  uncovered after change to stats_linux.h
* src/xen/block_stats.h, src/xen/block_stats.c: Add xen specific
  block stats APIs

* include/libvirt/libvirt.h, include/libvirt/libvirt.h.in: Add
  virSecretGetUsageType, virSecretGetUsageID and virLookupSecretByUsage
* python/generator.py: Mark virSecretGetUsageType, virSecretGetUsageID
  as not throwing exceptions
* qemud/remote.c: Implement dispatch for virLookupSecretByUsage
* qemud/remote_protocol.x: Add usage type & ID as attributes of
  remote_nonnull_secret. Add RPC calls for new public APIs
* qemud/remote_dispatch_args.h, qemud/remote_dispatch_prototypes.h,
  qemud/remote_dispatch_ret.h, qemud/remote_dispatch_table.h,
  qemud/remote_protocol.c, qemud/remote_protocol.h: Re-generate
* src/datatypes.c, src/datatypes.h: Add usageType and usageID as
  properties of virSecretPtr
* src/driver.h: Add virLookupSecretByUsage driver entry point
* src/libvirt.c: Implement virSecretGetUsageType, virSecretGetUsageID
  and virLookupSecretByUsage
* src/libvirt_public.syms: Export virSecretGetUsageType, virSecretGetUsageID
  and virLookupSecretByUsage
* src/remote_internal.c: Implement virLookupSecretByUsage entry
* src/secret_conf.c, src/secret_conf.h: Remove the
  virSecretUsageType enum, now in public API. Make volume
  path mandatory when parsing XML
* src/secret_driver.c: Enforce usage uniqueness when defining secrets.
  Implement virSecretLookupByUsage api method
* src/virsh.c: Include usage for secret-list command

Convert all the secret/storage encryption APIs / wire format to
handle UUIDs in raw format instead of non-canonical printable
format. Guarentees data format correctness.

* docs/schemas/storageencryption.rng: Make UUID mandatory for a secret
  and validate fully
* docs/schemas/secret.rng: Fully validate UUID
* include/libvirt/libvirt.h, include/libvirt/libvirt.h.in, Add
  virSecretLookupByUUID and virSecretGetUUID. Make
  virSecretGetUUIDString follow normal API design pattern
* python/generator.py: Skip generation of virSecretGetUUID,
  virSecretGetUUIDString and virSecretLookupByUUID
* python/libvir.c, python/libvirt-python-api.xml: Manual impl
  of virSecretGetUUID,virSecretGetUUIDString and virSecretLookupByUUID
* qemud/remote.c: s/virSecretLookupByUUIDString/virSecretLookupByUUID/
  Fix get_nonnull_secret/make_nonnull_secret to use unsigned char
* qemud/remote_protocol.x: Fix remote_nonnull_secret to use a
  remote_uuid instead of remote_nonnull_string for UUID field.
  Rename REMOTE_PROC_SECRET_LOOKUP_BY_UUID_STRING to
  REMOTE_PROC_SECRET_LOOKUP_BY_UUID_STRING and make it take an
  remote_uuid  value
* qemud/remote_dispatch_args.h, qemud/remote_dispatch_prototypes.h,
  qemud/remote_dispatch_ret.h, qemud/remote_dispatch_table.h,
  qemud/remote_protocol.c, qemud/remote_protocol.h: Re-generate
* src/datatypes.h, src/datatypes.c: Store UUID in raw format instead
  of printable. Change virGetSecret to use raw format UUID
* src/driver.h: Rename virDrvSecretLookupByUUIDString to
  virDrvSecretLookupByUUID and use raw format UUID
* src/libvirt.c: Add virSecretLookupByUUID and virSecretGetUUID
  and re-implement virSecretLookupByUUIDString and
  virSecretGetUUIDString in terms of those
* src/libvirt_public.syms: Add virSecretLookupByUUID and
  virSecretGetUUID
* src/remote_internal.c: Rename remoteSecretLookupByUUIDString
  to remoteSecretLookupByUUID. Fix typo in args for
  remoteSecretDefineXML impl. Use raw UUID format for
  get_nonnull_secret and make_nonnull_secret
* src/storage_encryption_conf.c, src/storage_encryption_conf.h:
  Storage UUID in raw format, and require it to be present in
  XML. Use UUID parser to validate.
* secret_conf.h, secret_conf.c: Generate a UUID if none is provided.
  Storage UUID in raw format.
* src/secret_driver.c: Adjust to deal with raw UUIDs. Save secrets
  in a filed with printable UUID, instead of base64 UUID.
* src/virsh.c: Adjust for changed public API contract of
  virSecretGetUUIDString.
* src/storage_Backend.c: DOn't undefine secret we just generated
  upon successful volume creation. Fix to handle raw UUIDs. Generate
  a non-clashing UUID
* src/qemu_driver.c: Change to use lookupByUUID instead of
  lookupByUUIDString

Add a VIR_SECRET_GET_VALUE_INTERNAL_CALL flag value, replacing the
originally separate libvirt_internal_call parameter.  The flag is used
to differentiate external virSecretGetValue() calls from internal calls
by libvirt drivers that need to use the secret even if it is private.

* src/libvirt_internal.h Remove VIR_DOMAIN_XML_FLAGS_MASK
* src/driver.h Add VIR_SECRET_GET_VALUE_FLAGS_MASK constant and
  VIR_SECRET_GET_VALUE_INTERNAL_CALL. Re-add the
  VIR_DOMAIN_XML_FLAGS_MASK constant
* src/libvirt.c (virSecretGetValue): Don't allow the user to specify
  internal flags.

* src/libvirt.c: fix some minor grammer (and one other) nits in comments
  that end up in generated API reference documentation

* src/libvirt.c (virStoragePoolSetAutostart): Return -1 if the pool
argument is invalid, rather than "goto error" where we could dereference
that possibly-NULL "pool".
(virConnectFindStoragePoolSources): Likewise.
(virConnectNumOfDomains): Likewise.
Daniel P. Berrange spotted that the two latter functions
needed the same treatment.

Paolo Bonzini points out that in my refactoring of the code for
virDomainMigrate(), I added a check for the return value from
virDomainMigratePerform().  The problem is that we don't want to
exit if we fail, we actually want to go on and do
virDomainMigrateFinish2() with a non-0 return code to clean things
up.  Remove the check.

While reproducing this issue, I also noticed that we wouldn't
always properly propagate an error message.  In particular, I
found that if you blocked off the migration ports (with iptables)
and then tried the migration, it would actually fail but we would
get no failure output from Qemu.  Therefore, we would think we
succeeded, and leave a huge mess behind us.  Execute the monitor
command "info migrate", and look for a failure string in there
as well.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* include/libvirt/virterror.h, src/virterror.c: Add VIR_ERR_INVALID_SECRET
  and VIR_FROM_SECRET
* src/libvirt.c: Define stubs for every new public API

* include/libvirt/virterror.h, src/virterror.c: Add VIR_WAR_NO_SECRET
* src/libvirt_private.syms, src/datatypes.h, src/datatypes.c: Type
  virSecret struct definition and helper APIs
* src/driver.h: Sub-driver API definitions for secrets
* src/libvirt.c: Define new sub-driver for secrets

The documentation for virNodeGetCellsFreeMemory claims the values
returned are in kilobytes, but that's actually wrong; the value
returned is actually in bytes.  Fix up the documentation to be
correct.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Currently the reference counting for connections is busted.  I
first noticed it while trying to use virConnectRef; it would
eventually cause a crash in the remote_internal driver, although
that was really just a victim.  Really, we should only call the
close callbacks on the methods when the references drop to 0.  To
accomplish this, move all of the close callbacks into
virUnrefConnect (since there are lots of internal users of that
function), and arrange for virConnectClose to call that.

V2: Make sure to drop the connection lock before we call the close
    callbacks, otherwise we could deadlock the daemon
V3: Fix up a crash when we got an error from one of the drivers
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Re-factor virDomainMigrate to split out the version 1 and version 2
protocols into their own functions.  In reality, the two versions share
very little in common, so forcing them together in the same function was
just confusing.  This will also make adding tunnelled migration easier.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* src/logging.c src/logging.h src/libvirt_private.syms:
  define new functions virLogSetFromEnv and virLogParseDefaultPriority
* qemud/qemud.c src/libvirt.c tests/eventtest.c: cleanup to use the
  unified functions

* src/libvirt.c src/logging.c: Don't convert high priority levels to the
  debug level. Don't parse LIBVIRT_LOG_FILTERS and LIBVIRT_LOG_OUTPUTS
  when they're set to the empty string. Warn when the user specifies an
  invalid value (empty string remains a noop).
* po/POTFILES.in: src/logging.c now include translatable strings

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Do the check in libvirt.c, to save drivers from the burden. This changes
behavior slightly in the qemu driver: we no longer explictly error if
passed an empty string. An error will still be thrown when the device
lookup fails.

Features supported:
- Connects to HMC/VIOS or IVM systems.
- Life cycle commands (resume and shutdown).
- dumpxml
- 'list' and 'list --all'

What is being implemented:
- better and centralized control for UUID
- definexml
- CPU management commands

* src/domain_conf.c src/domain_conf.h: first version of the driver
* configure.in src/Makefile.am include/libvirt/virterror.h
  src/domain_conf.[ch] src/libvirt.c src/virterror.c: glue the driver
  in the general framework

* src/esx/esx_*.[ch]: the driver, uses a remote minimal SOAP client
  to talk to the VI services on ESX nodes.
* configure.in include/libvirt/virterror.h src/Makefile.am src/driver.h
  src/libvirt.c src/virterror.c: glue in the new driver

* include/libvirt/libvirt.h include/libvirt/libvirt.h.in: adds the new
  flag VIR_MEMORY_PHYSICAL for virDomainMemoryPeek
* src/libvirt.c: update the front-end checking
* src/qemu_driver.c: extend the QEmu driver