The virConnectPtr is no longer required for error reporting since
that is recorded in a thread local. Remove use of virConnectPtr
from all APIs in storage_conf.{h,c} and storage_encryption_conf.{h,c}
and update all callers to match

When creating preallocated large raw files opening them with O_DSYNC
prevents long delays in reading because cache pages can be immediately
reused without writing them on a disk first.

It was used for error reporting only.

* src/storage/storage_backend.c (virStorageBackendRunProgRegex):
Don't close a negative (read-only) file descriptor.

In order to avoid problems trying to chown files that were created by
root on a root-squashing nfs server, fork a new process that setuid's
to the desired uid before creating the file. (It's only done this way
if the pool containing the new volume is of type 'netfs', otherwise
the old method of creating the file followed by chown() is used.)

This changes the semantics of the "create_func" slightly - previously
it was assumed that this function just created the file, then the
caller would chown it to the desired uid. Now, create_func does both
operations.

There are multiple functions that can take on the role of create_func:

createFileDir - previously called mkdir(), now calls virDirCreate().
virStorageBackendCreateRaw - previously called open(),
                             now calls virFileCreate().
virStorageBackendCreateQemuImg - use virRunWithHook() to setuid/gid.
virStorageBackendCreateQcowCreate - same.
virStorageBackendCreateBlockFrom - preserve old behavior (but attempt
                                   chown when necessary even if not root)

* src/storage/storage_backend.[ch] src/storage/storage_backend_disk.c
  src/storage/storage_backend_fs.c src/storage/storage_backend_logical.c
  src/storage/storage_driver.c: change the create_func implementations,
  also propagate the pool information to be able to detect NETFS ones.

Replace free(virBufferContentAndReset()) with virBufferFreeAndReset().
Update documentation and replace all remaining calls to free() with
calls to VIR_FREE(). Also add missing calls to virBufferFreeAndReset()
and virReportOOMError() in OOM error cases.

When building with --disable-nls, I got a few messages like this:

storage/storage_backend.c: In function 'virStorageBackendCreateQemuImg':
storage/storage_backend.c:571: warning: format not a string literal and no format arguments

Fix these up.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Rename virStorageVolFormatFileSystem to virStorageFileFormat and
move to src/util/storage_file.[ch]

* src/Makefile.am: add src/util/storage_file.[ch]

* src/conf/storage_conf.[ch]: move enum from here ...

* src/util/storage_file.[ch]: .. to here

* src/libvirt_private.syms: update To/FromString exports

* src/storage/storage_backend.c, src/storage/storage_backend_fs.c,
  src/vbox/vbox_tmpl.c: update for above changes

* daemon/qemud.c, src/Makefile.am: Update for changed paths
* src/node_device*.{h,c}: Move to src/node_device/
* src/storage/storage_backend.c: Remove bogus import of node_device.c

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/storage*.c, src/storage/*.h, src/parthelpre.c: Move
  to src/storage/

Convert all the secret/storage encryption APIs / wire format to
handle UUIDs in raw format instead of non-canonical printable
format. Guarentees data format correctness.

* docs/schemas/storageencryption.rng: Make UUID mandatory for a secret
  and validate fully
* docs/schemas/secret.rng: Fully validate UUID
* include/libvirt/libvirt.h, include/libvirt/libvirt.h.in, Add
  virSecretLookupByUUID and virSecretGetUUID. Make
  virSecretGetUUIDString follow normal API design pattern
* python/generator.py: Skip generation of virSecretGetUUID,
  virSecretGetUUIDString and virSecretLookupByUUID
* python/libvir.c, python/libvirt-python-api.xml: Manual impl
  of virSecretGetUUID,virSecretGetUUIDString and virSecretLookupByUUID
* qemud/remote.c: s/virSecretLookupByUUIDString/virSecretLookupByUUID/
  Fix get_nonnull_secret/make_nonnull_secret to use unsigned char
* qemud/remote_protocol.x: Fix remote_nonnull_secret to use a
  remote_uuid instead of remote_nonnull_string for UUID field.
  Rename REMOTE_PROC_SECRET_LOOKUP_BY_UUID_STRING to
  REMOTE_PROC_SECRET_LOOKUP_BY_UUID_STRING and make it take an
  remote_uuid  value
* qemud/remote_dispatch_args.h, qemud/remote_dispatch_prototypes.h,
  qemud/remote_dispatch_ret.h, qemud/remote_dispatch_table.h,
  qemud/remote_protocol.c, qemud/remote_protocol.h: Re-generate
* src/datatypes.h, src/datatypes.c: Store UUID in raw format instead
  of printable. Change virGetSecret to use raw format UUID
* src/driver.h: Rename virDrvSecretLookupByUUIDString to
  virDrvSecretLookupByUUID and use raw format UUID
* src/libvirt.c: Add virSecretLookupByUUID and virSecretGetUUID
  and re-implement virSecretLookupByUUIDString and
  virSecretGetUUIDString in terms of those
* src/libvirt_public.syms: Add virSecretLookupByUUID and
  virSecretGetUUID
* src/remote_internal.c: Rename remoteSecretLookupByUUIDString
  to remoteSecretLookupByUUID. Fix typo in args for
  remoteSecretDefineXML impl. Use raw UUID format for
  get_nonnull_secret and make_nonnull_secret
* src/storage_encryption_conf.c, src/storage_encryption_conf.h:
  Storage UUID in raw format, and require it to be present in
  XML. Use UUID parser to validate.
* secret_conf.h, secret_conf.c: Generate a UUID if none is provided.
  Storage UUID in raw format.
* src/secret_driver.c: Adjust to deal with raw UUIDs. Save secrets
  in a filed with printable UUID, instead of base64 UUID.
* src/virsh.c: Adjust for changed public API contract of
  virSecretGetUUIDString.
* src/storage_Backend.c: DOn't undefine secret we just generated
  upon successful volume creation. Fix to handle raw UUIDs. Generate
  a non-clashing UUID
* src/qemu_driver.c: Change to use lookupByUUID instead of
  lookupByUUIDString

If the <encryption format='qcow'> element does not specify a secret
during volume creation, generate a suitable secret and add it to the
<encryption> tag.  The caller can view the updated <encryption> tag
using virStorageVolGetXMLDesc().

Similarly, when <encryption format='default'/> is specified while
creating a qcow or qcow2-formatted volume, change the format to "qcow"
and generate a secret as described above.

* src/storage_encryption_conf.h (VIR_STORAGE_QCOW_PASSPHRASE_SIZE,
  virStorageGenerateQcowPasphrase),
  src/storage_encryption_conf.c (virStorageGenerateQcowPasphrase),
  src/libvirt_private.syms: Add virStorageGenerateQcowPasphrase().
* src/storage_backend.c (virStoragegenerateQcowEncryption,
  virStorageBackendCreateQemuImg): Generate a passphrase and
  <encryption> when creating a qcow-formatted encrypted volume and the
  user did not supply the information.

Remove the bogus dependancy between node_device.c & storage_backend.c
by moving the virWaitForDevices into util.h where it can be shared
safely

* src/storage_backend_disk.c, src/storage_backend_logical.c,
  src/storage_backend_mpath.c, src/storage_backend_scsi.c: Replace
  virStorageBackendWaitForDevices with virFileWaitForDevices
* src/storage_backend.c, src/storage_backend.h: Remove
  virStorageBackendWaitForDevices, virWaitForDevices
* src/util.h, src/util.c: Add virFileWaitForDevices
* configure.in: Move xmlrpc check further down after pkgconfig
  is detected
* src/Makefile.am: Add missing XMLRPC_CFLAGS/LIBS to opennebula
* src/libvirt_private.syms: Add many missing exports

* configure.in src/Makefile.am src/storage_backend.[ch]
  src/storage_conf.[ch] src/storage_backend_mpath.[ch] po/POTFILES.in:
  add a new module for storage multipath, it requires device-mapper

* src/storage_backend.c: Include "internal.h".
(virStorageBackendCopyToFD): Mark inputvol parameter as "nonnull".
Remove test for non-NULL inputvol.  Both callers ensure it's non-NULL.

Supports only virStorageVolCreateXML, not virStorageVolCreateXMLFrom.

Curiously, qemu-img does not need the passphrase for anything to create
an encrypted volume.  This implementation thus does not need to touch
any secrets to work with cooperating clients.  More generic passphrase
handling is added in the next patch.

* src/storage_backend.c: Request encryption when creating qcow/qcow2
  files
* src/storage_backend_disk.c, src/storage_backend_fs.c,
  src/storage_backend_logical.c: Refuse to create volumes with
  encryption params set.

* src/storage_backend.c: as the absence of units can be interpreted
  in diverging ways depending on the version

* src/datatypes.c src/domain_conf.c src/interface_conf.c
  src/lxc_driver.c src/qemu_driver.c src/storage_backend.c src/virsh.c:
  add bare %s format string to printf-derivatives called with no format
  string

We don't gain any space savings, so skip the detection to speed up
the cloning operation.

Add a 'CreateBlockFrom' in the global storage_backend, which sets up the
destination block device: CopyFromFD does the rest of the cloning.

The CreateRaw function has some 'file' only assumptions, so break the agnostic
cloning bits to a separate function.

Have storage building functions be definitions of
virStorageBackendBuildVolFrom: we will need to do this in the future anyways
if we ever support the flags attribute.

These will be used by other pool cloning implementations.

by running this command:
git ls-files -z | xargs -0 perl -pi -0777 -e 's/\n\n+$/\n/'
This is in preparation for a more strict make syntax-check
rule that will detect trailing blank lines.

* src/node_device.[ch] src/storage_backend.[ch]: fixes a
  module dependency problem, patch by Dave Allan
Daniel

* src/Makefile.am src/node_device.[ch] src/node_device_conf.[ch]
  src/node_device_hal.[ch] src/node_device_hal_linux.c
  src/qemu_driver.c src/remote_internal.c src/storage_backend.c
  src/virsh.c src/xen_unified.c tests/nodedevxml2xmltest.c
  po/POTFILES.in: implementation for node device create and destroy
  in NPIV support, patch by David Allan
Daniel

have udevsettle.  So change our tests to prefer udevadm if it exists, but
to fall back to udevsettle if it does not.
Signed-off-by: NChris Lalancette <clalance@redhat.com>