Update the storage driver to use virReportError instead of
the virStorageReportError custom macro
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To make it easier to dynamically change the command line ARGV,
switch all storage code over to use virCommandPtr APIs for
running programs
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The previous storage patch missed an instance affected by the struct
member rename.  It also had some botched whitespace detected by
'make check'.

* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIFindPoolSources): Adjust to new struct.
* src/conf/storage_conf.c (virStoragePoolSourceFormat): Fix
indentation.

The current storage pools for NFS and iSCSI only require one host to
connect to. Future storage pools like RBD and Sheepdog will require
multiple hosts.

This patch allows multiple source hosts and rewrites the current
storage drivers.
Signed-off-by: NWido den Hollander <wido@widodh.nl>

The code is splattered with a mix of

  sizeof foo
  sizeof (foo)
  sizeof(foo)

Standardize on sizeof(foo) and add a syntax check rule to
enforce it
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The old virRandom() API was not generating good random numbers.
Replace it with a new API virRandomBits which instead of being
told the upper limit, gets told the number of bits of randomness
required.

* src/util/virrandom.c, src/util/virrandom.h: Add virRandomBits,
  and move virRandomInitialize
* src/util/util.h, src/util/util.c: Delete virRandom and
  virRandomInitialize
* src/libvirt.c, src/security/security_selinux.c,
  src/test/test_driver.c, src/util/iohelper.c: Update for
  changes from virRandom to virRandomBits
* src/storage/storage_backend_iscsi.c: Remove bogus call
  to virRandomInitialize & convert to virRandomBits

* .gnulib: Update to latest, for improved syntax-check.
* src/lxc/lxc_container.c (includes): Drop unused include.
* src/network/bridge_driver.c: Likewise.
* src/node_device/node_device_linux_sysfs.c: Likewise.
* src/openvz/openvz_driver.c: Likewise.
* src/qemu/qemu_conf.c: Likewise.
* src/storage/storage_backend_iscsi.c: Likewise.
* src/storage/storage_backend_mpath.c: Likewise.
* src/uml/uml_conf.c: Likewise.
* src/uml/uml_driver.c: Likewise.

Splitting into two functions allows the user to call the right
function, rather than having to remember that a *Free function is
an exception to the rule.

* src/conf/storage_conf.h (virStoragePoolSourceClear): New function.
* src/libvirt_private.syms (storage_conf.h): Export it.
* src/conf/storage_conf.c (virStoragePoolSourceFree): Split...
(virStoragePoolSourceClear): ...into new function.
(virStoragePoolDefFree, virStoragePoolDefParseSourceString):
Update callers.
* src/test/test_driver.c (testStorageFindPoolSources): Likewise.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemNetFindPoolSourcesFunc)
(virStorageBackendFileSystemNetFindPoolSources): Likewise.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIFindPoolSources): Likewise.
* src/storage/storage_backend_logical.c
(virStorageBackendLogicalFindPoolSources): Likewise.

Detected by Coverity.  virStoragePoolSourceFree does not free the
actual passed-in pointer.  A bigger patch would be to rename it
virStoragePoolSourceClear to match behavior, or even split it into
two functions depending on needed behavior; but this is the minimal
fix to the one location out of eight that leaked memory.

* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIFindPoolSources): Free memory.

If the regexes supported (?:pvs)?, then we could handle this by
optionally matching but not returning the initial command name.  But it
doesn't.  So add a new char* argument to
virStorageBackendRunProgRegex().  If that argument is NULL then we act
as usual.  Otherwise, if the string at that argument is found at the
start of a returned line, we drop that before running the regex.

With this patch, virt-manager shows me lvs with command_names 1 or 0.

The definitions of PVS_BASE etc may want to be moved into the configure
scripts (though given how PVS is found, IIUC that could only happen if
pvs was a link to pvs_real), but in any case no sense dealing with that
until we're sure this is an ok way to handle it.
Signed-off-by: NSerge Hallyn <serge.hallyn@canonical.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

In preparation for a future patch adding new virFile APIs.

* src/util/files.h, src/util/files.c: Move...
* src/util/virfile.h, src/util/virfile.c: ...here, and rename
functions to virFile prefix.  Macro names are intentionally
left alone.
* *.c: All '#include "files.h"' uses changed.
* src/Makefile.am (UTIL_SOURCES): Reflect rename.
* cfg.mk (exclude_file_name_regexp--sc_prohibit_close): Likewise.
* src/libvirt_private.syms: Likewise.
* docs/hacking.html.in: Likewise.
* HACKING: Regenerate.

* src/storage/storage_backend.c (virStorageBackendCreateBlockFrom)
(virStorageBackendCreateQemuImg)
(virStorageBackendCreateQcowCreate): Reject unknown flags.
* src/storage/storage_backend_disk.c (virStorageBackendDiskBuildPool)
(virStorageBackendDiskDeleteVol): Likewise.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemNetFindPoolSources)
(virStorageBackendFileSystemBuild)
(virStorageBackendFileSystemDelete, createFileDir)
(virStorageBackendFileSystemVolBuildFrom)
(virStorageBackendFileSystemVolDelete): Likewise.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIFindPoolSources): Likewise.
* src/storage/storage_backend_logical.c
(virStorageBackendLogicalFindPoolSources)
(virStorageBackendLogicalBuildPool)
(virStorageBackendLogicalDeletePool)
(virStorageBackendLogicalDeleteVol): Likewise.
* src/storage/storage_driver.c (storageOpen, storagePoolCreate)
(storagePoolDefine, storagePoolRefresh, storagePoolGetXMLDesc)
(storageVolumeCreateXML, storageVolumeCreateXMLFrom)
(storageVolumeGetXMLDesc): Likewise.

Coverity detected that we could crash on bogus input.  Meanwhile,
strtok_r is rather heavy compared to strchr.

* src/storage/storage_backend_iscsi.c (virStorageBackendIQNFound):
  Check for parse failure, and use lighter-weight functions.

Some clients overwrite the error from the regex helper, or do half-baked
error reporting with the exitstatus.

v2:
    Use virCommand auto-cleanup

clang didn't like the last increment to nargs.  But why even
track nargs ourselves, when virCommand does it for us?

* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIConnection): Switch to virCommand to avoid
a dead-store warning on nargs.

Even with -Wuninitialized (which is part of autobuild.sh
--enable-compile-warnings=error), gcc does NOT catch this
use of an uninitialized variable:

{
  if (cond)
    goto error;
  int a = 1;
error:
  printf("%d", a);
}

which prints 0 (supposing the stack started life wiped) if
cond was true.  Clang will catch it, but we don't use clang
as often.  Using gcc -Wjump-misses-init catches it, but also
gives false positives:

{
  if (cond)
    goto error;
  int a = 1;
  return a;
error:
  return 0;
}

Here, a was never used in the scope of the error block, so
declaring it after goto is technically fine (and clang agrees).
However, given that our HACKING already documents a preference
to C89 decl-before-statement, the false positive warning is
enough of a prod to comply with HACKING.

[Personally, I'd _really_ rather use C99 decl-after-statement
to minimize scope, but until gcc can efficiently and reliably
catch scoping and uninitialized usage bugs, I'll settle with
the compromise of enforcing a coding standard that happens to
reject false positives if it can also detect real bugs.]

* acinclude.m4 (LIBVIRT_COMPILE_WARNINGS): Add -Wjump-misses-init.
* src/util/util.c (__virExec): Adjust offenders.
* src/conf/domain_conf.c (virDomainTimerDefParseXML): Likewise.
* src/remote/remote_driver.c (doRemoteOpen): Likewise.
* src/phyp/phyp_driver.c (phypGetLparNAME, phypGetLparProfile)
(phypGetVIOSFreeSCSIAdapter, phypVolumeGetKey)
(phypGetStoragePoolDevice)
(phypVolumeGetPhysicalVolumeByStoragePool)
(phypVolumeGetPath): Likewise.
* src/vbox/vbox_tmpl.c (vboxNetworkUndefineDestroy)
(vboxNetworkCreate, vboxNetworkDumpXML)
(vboxNetworkDefineCreateXML): Likewise.
* src/xenapi/xenapi_driver.c (getCapsObject)
(xenapiDomainDumpXML): Likewise.
* src/xenapi/xenapi_utils.c (createVMRecordFromXml): Likewise.
* src/security/security_selinux.c (SELinuxGenNewContext):
Likewise.
* src/qemu/qemu_command.c (qemuBuildCommandLine): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeEjectableMedia):
Likewise.
* src/qemu/qemu_process.c (qemuProcessWaitForMonitor): Likewise.
* src/qemu/qemu_monitor_text.c (qemuMonitorTextGetPtyPaths):
Likewise.
* src/qemu/qemu_driver.c (qemudDomainShutdown)
(qemudDomainBlockStats, qemudDomainMemoryPeek): Likewise.
* src/storage/storage_backend_iscsi.c
(virStorageBackendCreateIfaceIQN): Likewise.
* src/node_device/node_device_udev.c (udevProcessPCI): Likewise.

When libvirt starts up all storage pools default to the inactive
state, even if the underlying storage is already active on the
host. This introduces a new API into the internal storage backend
drivers that checks whether a storage pool is already active. If
the pool is active at libvirtd startup, the volume list will be
immediately populated.

* src/storage/storage_backend.h: New internal API for checking
  storage pool state
* src/storage/storage_driver.c: Check whether a pool is active
  upon driver startup
* src/storage/storage_backend_fs.c, src/storage/storage_backend_iscsi.c,
  src/storage/storage_backend_logical.c, src/storage/storage_backend_mpath.c,
  src/storage/storage_backend_scsi.c: Add checks for pool state

Since the previous patch added support for parsing the output of
the 'sendtargets' command, it is now trivial to support the
storage pool discovery API.

Given a hostname and optional portnumber and initiator IQN,
the code can return a full list of storage pool source docs,
each one representing a iSCSI target.

* src/storage/storage_backend_iscsi.c: Wire up target
  auto-discovery

The Linux iSCSI initiator toolchain has the dubious feature that
if you ever run the 'sendtargets' command to merely query what
targets are available from a server, the results will be recorded
in /var/lib/iscsi. Any time the '/etc/init.d/iscsi' script runs
in the future, it will then automatically login to all those
targets. /etc/init.d/iscsi is automatically run whenever a NIC
comes online.

So from the moment you ask a server what targets are available,
your client will forever more automatically try to login to all
targets without ever asking if you actually want it todo this.

To stop this stupid behaviour, we need to run

  iscsiadm --portal $PORTAL --target $TARGET
   --op update --name node.startup --value manual

For every target on the server.

* src/storage/storage_backend_iscsi.c: Disable automatic login
  for targets found as a result of a 'sendtargets' command

The following series of patches are adding significant
extra functionality to the iSCSI driver. THe current
internal helper methods are not sufficiently flexible
to cope with these changes. This patch refactors the
code to avoid needing to have a virStoragePoolObjPtr
instance as a parameter, instead passing individual
target, portal and initiatoriqn parameters.

It also removes hardcoding of port 3260 in the portal
address, instead using the XML value if any.

* src/storage/storage_backend_iscsi.c: Refactor internal
  helper methods

Similarly to deprecating close(), I am now deprecating fclose() and
introduce VIR_FORCE_FCLOSE() and VIR_FCLOSE(). Also, fdopen() is replaced with
VIR_FDOPEN().

Most of the files are opened in read-only mode, so usage of
VIR_FORCE_CLOSE() seemed appropriate. Others that are opened in write
mode already had the fclose()<  0 check and I converted those to
VIR_FCLOSE()<  0.

I did not find occurrences of possible double-closed files on the way.

Using automated replacement with sed and editing I have now replaced all
occurrences of close() with VIR_(FORCE_)CLOSE() except for one, of
course. Some replacements were straight forward, others I needed to pay
attention. I hope I payed attention in all the right places... Please
have a look. This should have at least solved one more double-close
error.

- a pure typo error and a wrong command referenced in an error message.

The virConnectPtr is no longer required for error reporting since
that is recorded in a thread local. Remove use of virConnectPtr
from all APIs in storage_conf.{h,c} and storage_encryption_conf.{h,c}
and update all callers to match

It was used for error reporting only.

* src/qemu/qemu_monitor_text.c (qemuMonitorTextGetAllPCIAddresses):
Use %s.
* src/storage/storage_backend_iscsi.c (virStorageBackendCreateIfaceIQN):
Likewise.
* tools/virsh.c (cmdSecretSetValue): Likewise.

Allows the initiator to use a variety of IQNs rather than just the
system IQN when creating iSCSI pools.
* docs/schemas/storagepool.rng: extends the syntax with <iqn name="..."/>
* src/conf/storage_conf.[ch]: read and stores the iqn name
* src/storage/storage_backend_iscsi.[ch]: implement the IQN selection
  when detected

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/storage*.c, src/storage/*.h, src/parthelpre.c: Move
  to src/storage/

* src/uml_conf.c (VIR_FROM_THIS): Define to VIR_FROM_UML.
* src/xs_internal.c (VIR_FROM_THIS): Define to VIR_FROM_XEN.
* src/xml.c (VIR_FROM_THIS): Define to VIR_FROM_XML.
* src/stats_linux.c (VIR_FROM_THIS): Define to VIR_FROM_STATS_LINUX.
* src/datatypes.c (VIR_FROM_THIS): Define to VIR_FROM_NONE.
* src/lxc_conf.c (VIR_FROM_THIS): Define to VIR_FROM_LXC.
* src/libvirt.c (VIR_FROM_THIS): Define to VIR_FROM_NONE.
* src/node_device_conf.c (VIR_FROM_THIS): Define to VIR_FROM_NODEDEV.
* src/openvz_conf.c (VIR_FROM_THIS): Define to VIR_FROM_OPENVZ.
* src/openvz_driver.c (VIR_FROM_THIS): Define to VIR_FROM_OPENVZ.
* src/conf.c (VIR_FROM_THIS): Define to VIR_FROM_CONF.
Note: this loses config_filename:config_lineno diagnostics,
but that's ok.
* src/node_device.c (VIR_FROM_THIS): Define to VIR_FROM_NODEDEV.
* src/sexpr.c (VIR_FROM_THIS): Define to VIR_FROM_SEXPR.
* po/POTFILES.in: remove src/sexpr.c and src/lxc_conf.c

really should be synchronizing against udev.  This is generally done by a call
to udevsettle, which is exactly what this patch implements for the storage
backends that are likely to create new /dev nodes.  I believe I've read that
even after udevsettle, you are not guaranteed that devices are all the way
created, so we still need the polling in the rest of the sources, but this
should give us a much better chance of things existing as we expect.
Signed-off-by: NChris Lalancette <clalance@redhat.com>