Remove the original VolOpen implementation, which is now only used in
one spot.

(cherry picked from commit fa5b5549)

(cherry picked from commit 847a9eb1)

Conflicts:
	src/storage/storage_backend.h
	src/storage/storage_backend_mpath.c
	src/storage/storage_backend_scsi.c

And drop the original UpdateVolInfo. Makes it a bit easier to follow
the function usage.

And change the int parameter to an explicit bool.

(cherry picked from commit 16d75d19)

Conflicts:
	src/storage/storage_backend.h

Without this, using /dev/mapper as a directory pool
fails in virStorageBackendUpdateVolTargetInfoFD:
cannot seek to end of file '/dev/mapper/control': Illegal seek

Skip over character devices by default.

https://bugzilla.redhat.com/show_bug.cgi?id=710866
(cherry picked from commit 0edfc9ef)

We are calling fstat() at least twice per storage volume in
a directory storage pool; this is rather wasteful.  Refactoring
this is also a step towards making code reusable for gluster,
where gluster can provide struct stat but cannot use fstat().

* src/storage/storage_backend.h
(virStorageBackendVolOpenCheckMode)
(virStorageBackendUpdateVolTargetInfoFD): Update signature.
* src/storage/storage_backend.c
(virStorageBackendVolOpenCheckMode): Pass stat results back.
(virStorageBackendUpdateVolTargetInfoFD): Use existing stats.
(virStorageBackendVolOpen, virStorageBackendUpdateVolTargetInfo):
Update callers.
* src/storage/storage_backend_fs.c (virStorageBackendProbeTarget):
Likewise.
* src/storage/storage_backend_scsi.c
(virStorageBackendSCSIUpdateVolTargetInfo): Likewise.
* src/storage/storage_backend_mpath.c
(virStorageBackendMpathUpdateVolTargetInfo): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>
(cherry picked from commit 9cac8639)

This allows us to create a test for the generated command line.

Add VIR_STORAGE_VOL_CREATE_PREALLOC_METADATA flag to virStorageVolCreateXML
and virStorageVolCreateXMLFrom. This flag requests metadata
preallocation when creating/cloning qcow2 images, resulting in creating
a sparse file with qcow2 metadata. It has only slightly larger disk usage
compared to new image with no allocation, but offers higher performance.

Rename the 'wait' parameter to 'loop'.
This silences the warning:
storage/storage_backend.c:1348:34: error: declaration of 'wait' shadows
a global declaration [-Werror=shadow]
and fixes the build with -Werror.
--
Note: loop is pool backwards.

virStorageVolLookupByPath is an API call that virt-manager uses
quite a bit when dealing with storage. This call use BackendStablePath
which has several usleep() heuristics that can be tripped up
and hang virt-manager for a while.

Current example: an empty mpath pool pointing to /dev/mapper makes
_any_ calls to virStorageVolLookupByPath take 5 seconds.

The sleep heuristics are actually only needed in certain cases
when we are waiting for new storage to appear, so let's skip the
timeout steps when calling from LookupByPath.

https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.

* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/;  If/.  If/

Otherwise, in locations like virobject.c where PROBE is used,
for certain configure options, the compiler warns:

util/virobject.c:110:1: error: 'intptr_t' undeclared (first use in this function)

As long as we are making this header always available, we can
clean up several other files.

* src/internal.h (includes): Pull in <stdint.h>.
* src/conf/nwfilter_conf.h: Rely on internal.h.
* src/storage/storage_backend.c: Likewise.
* src/storage/storage_backend.h: Likewise.
* src/util/cgroup.c: Likewise.
* src/util/sexpr.h: Likewise.
* src/util/virhashcode.h: Likewise.
* src/util/virnetdevvportprofile.h: Likewise.
* src/util/virnetlink.h: Likewise.
* src/util/virrandom.h: Likewise.
* src/vbox/vbox_driver.c: Likewise.
* src/xenapi/xenapi_driver.c: Likewise.
* src/xenapi/xenapi_utils.c: Likewise.
* src/xenapi/xenapi_utils.h: Likewise.
* src/xenxs/xenxs_private.h: Likewise.
* tests/storagebackendsheepdogtest.c: Likewise.

Per the FSF address could be changed from time to time, and GNU
recommends the following now: (http://www.gnu.org/licenses/gpl-howto.html)

  You should have received a copy of the GNU General Public License
  along with Foobar.  If not, see <http://www.gnu.org/licenses/>.

This patch removes the explicit FSF address, and uses above instead
(of course, with inserting 'Lesser' before 'General').

Except a bunch of files for security driver, all others are changed
automatically, the copyright for securify files are not complete,
that's why to do it manually:

  src/security/security_selinux.h
  src/security/security_driver.h
  src/security/security_selinux.c
  src/security/security_apparmor.h
  src/security/security_apparmor.c
  src/security/security_driver.c

Update the storage driver to use virReportError instead of
the virStorageReportError custom macro
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This removes all the per-file error reporting macros
from the code in src/conf/
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To make it easier to dynamically change the command line ARGV,
switch all storage code over to use virCommandPtr APIs for
running programs
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently only VIR_STORAGE_VOL_RESIZE_DELTA flag is supported.

If the regexes supported (?:pvs)?, then we could handle this by
optionally matching but not returning the initial command name.  But it
doesn't.  So add a new char* argument to
virStorageBackendRunProgRegex().  If that argument is NULL then we act
as usual.  Otherwise, if the string at that argument is found at the
start of a returned line, we drop that before running the regex.

With this patch, virt-manager shows me lvs with command_names 1 or 0.

The definitions of PVS_BASE etc may want to be moved into the configure
scripts (though given how PVS is found, IIUC that could only happen if
pvs was a link to pvs_real), but in any case no sense dealing with that
until we're sure this is an ok way to handle it.
Signed-off-by: NSerge Hallyn <serge.hallyn@canonical.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Since directories can be used for <filesystem> passthrough, they are
basically storage volumes.

v2:
    Skip ., .., lost+found dirs

v3:
    Use gnulib last_component

v4:
    Use gnulib "dirname.h", not system <dirname.h>
    Don't skip lost+found

Some clients overwrite the error from the regex helper, or do half-baked
error reporting with the exitstatus.

Remove the <stdbool.h> header from all source files / headers
and just put it into internal.h

* src/internal.h: Add <stdbool.h>

When libvirt starts up all storage pools default to the inactive
state, even if the underlying storage is already active on the
host. This introduces a new API into the internal storage backend
drivers that checks whether a storage pool is already active. If
the pool is active at libvirtd startup, the volume list will be
immediately populated.

* src/storage/storage_backend.h: New internal API for checking
  storage pool state
* src/storage/storage_driver.c: Check whether a pool is active
  upon driver startup
* src/storage/storage_backend_fs.c, src/storage/storage_backend_iscsi.c,
  src/storage/storage_backend_logical.c, src/storage/storage_backend_mpath.c,
  src/storage/storage_backend_scsi.c: Add checks for pool state

* src/storage/storage_backend.h (VIR_STORAGE_VOL_OPEN_DEFAULT):
Adjust s/#define/# define/, and align continued lines.

If a directory pool contains pipes or sockets, a pool start can fail or hang:

https://bugzilla.redhat.com/show_bug.cgi?id=589577

We already try to avoid these special files, but only attempt after
opening the path, which is where the problems lie. Unify volume opening
into helper functions, which use the proper open() flags to avoid error,
followed by fstat to validate storage mode.

Previously, virStorageBackendUpdateVolTargetInfoFD attempted to enforce the
storage mode check, but allowed callers to detect this case and silently
continue. In practice, only the FS backend was using this feature, the rest
were treating unknown mode as an error condition. Unfortunately the InfoFD
function wasn't raising an error message here, so error reporting was
busted.

This patch adds 2 functions: virStorageBackendVolOpen, and
virStorageBackendVolOpenModeSkip. The latter retains the original opt out
semantics, the former now throws an explicit error.

This patch maintains the previous volume mode checks: allowing specific
modes for specific pool types requires a bit of surgery, since VolOpen
is called through several different helper functions.

v2: Use ATTRIBUTE_NONNULL. Drop stat check, just open with
    O_NONBLOCK|O_NOCTTY.

v3: Move mode check logic back to VolOpen. Use 2 VolOpen functions with
    different error semantics.

v4: Make second VolOpen function more extensible. Didn't opt to change
    FS backend defaults, this can just be to fix the original bug.

v5: Prefix default flags with VIR_, use ATTRIBUTE_RETURN_CHECK

Volume detection in the scsi backend was duplicating code already
present in storage_backend.c. Let's drop the duplicate code.

Also, change the shared function name to be less generic, and remove
some error squashing in the other call site.

* global: patch created by running:
for f in $(git ls-files '*.[ch]') ; do
    cppi $f > $f.t && mv $f.t $f
done

The virConnectPtr is no longer required for error reporting since
that is recorded in a thread local. Remove use of virConnectPtr
from all APIs in storage_conf.{h,c} and storage_encryption_conf.{h,c}
and update all callers to match

In order to avoid problems trying to chown files that were created by
root on a root-squashing nfs server, fork a new process that setuid's
to the desired uid before creating the file. (It's only done this way
if the pool containing the new volume is of type 'netfs', otherwise
the old method of creating the file followed by chown() is used.)

This changes the semantics of the "create_func" slightly - previously
it was assumed that this function just created the file, then the
caller would chown it to the desired uid. Now, create_func does both
operations.

There are multiple functions that can take on the role of create_func:

createFileDir - previously called mkdir(), now calls virDirCreate().
virStorageBackendCreateRaw - previously called open(),
                             now calls virFileCreate().
virStorageBackendCreateQemuImg - use virRunWithHook() to setuid/gid.
virStorageBackendCreateQcowCreate - same.
virStorageBackendCreateBlockFrom - preserve old behavior (but attempt
                                   chown when necessary even if not root)

* src/storage/storage_backend.[ch] src/storage/storage_backend_disk.c
  src/storage/storage_backend_fs.c src/storage/storage_backend_logical.c
  src/storage/storage_driver.c: change the create_func implementations,
  also propagate the pool information to be able to detect NETFS ones.

* src/storage/storage_backend.h: Include <stdint.h>.

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/storage*.c, src/storage/*.h, src/parthelpre.c: Move
  to src/storage/

Remove the bogus dependancy between node_device.c & storage_backend.c
by moving the virWaitForDevices into util.h where it can be shared
safely

* src/storage_backend_disk.c, src/storage_backend_logical.c,
  src/storage_backend_mpath.c, src/storage_backend_scsi.c: Replace
  virStorageBackendWaitForDevices with virFileWaitForDevices
* src/storage_backend.c, src/storage_backend.h: Remove
  virStorageBackendWaitForDevices, virWaitForDevices
* src/util.h, src/util.c: Add virFileWaitForDevices
* configure.in: Move xmlrpc check further down after pkgconfig
  is detected
* src/Makefile.am: Add missing XMLRPC_CFLAGS/LIBS to opennebula
* src/libvirt_private.syms: Add many missing exports

* configure.in src/Makefile.am src/storage_backend.[ch]
  src/storage_conf.[ch] src/storage_backend_mpath.[ch] po/POTFILES.in:
  add a new module for storage multipath, it requires device-mapper

Have storage building functions be definitions of
virStorageBackendBuildVolFrom: we will need to do this in the future anyways
if we ever support the flags attribute.

These will be used by other pool cloning implementations.

* src/node_device.[ch] src/storage_backend.[ch]: fixes a
  module dependency problem, patch by Dave Allan
Daniel

There is some funkiness here, since we are either dealing with 2 different
pools (which means validation x 2) or the same pool, where we have to be
careful not to deadlock.

* src/libvirt_private.syms src/storage_backend.h
  src/storage_backend_fs.c src/storage_conf.h src/storage_driver.c:
  drop the pool lock when allocating fs volumes, patch by Cole Robinson
daniel

really should be synchronizing against udev.  This is generally done by a call
to udevsettle, which is exactly what this patch implements for the storage
backends that are likely to create new /dev nodes.  I believe I've read that
even after udevsettle, you are not guaranteed that devices are all the way
created, so we still need the polling in the rest of the sources, but this
should give us a much better chance of things existing as we expect.
Signed-off-by: NChris Lalancette <clalance@redhat.com>