Otherwise, in locations like virobject.c where PROBE is used,
for certain configure options, the compiler warns:

util/virobject.c:110:1: error: 'intptr_t' undeclared (first use in this function)

As long as we are making this header always available, we can
clean up several other files.

* src/internal.h (includes): Pull in <stdint.h>.
* src/conf/nwfilter_conf.h: Rely on internal.h.
* src/storage/storage_backend.c: Likewise.
* src/storage/storage_backend.h: Likewise.
* src/util/cgroup.c: Likewise.
* src/util/sexpr.h: Likewise.
* src/util/virhashcode.h: Likewise.
* src/util/virnetdevvportprofile.h: Likewise.
* src/util/virnetlink.h: Likewise.
* src/util/virrandom.h: Likewise.
* src/vbox/vbox_driver.c: Likewise.
* src/xenapi/xenapi_driver.c: Likewise.
* src/xenapi/xenapi_utils.c: Likewise.
* src/xenapi/xenapi_utils.h: Likewise.
* src/xenxs/xenxs_private.h: Likewise.
* tests/storagebackendsheepdogtest.c: Likewise.

This converts the following public API datatypes to use the
virObject infrastructure:

  virConnectPtr
  virDomainPtr
  virDomainSnapshotPtr
  virInterfacePtr
  virNetworkPtr
  virNodeDevicePtr
  virNWFilterPtr
  virSecretPtr
  virStreamPtr
  virStorageVolPtr
  virStoragePoolPtr

The code is significantly simplified, since the mutex in the
virConnectPtr object now only needs to be held when accessing
the per-connection virError object instance. All other operations
are completely lock free.

* src/datatypes.c, src/datatypes.h, src/libvirt.c: Convert
  public datatypes to use virObject
* src/conf/domain_event.c, src/phyp/phyp_driver.c,
  src/qemu/qemu_command.c, src/qemu/qemu_migration.c,
  src/qemu/qemu_process.c, src/storage/storage_driver.c,
  src/vbox/vbox_tmpl.c, src/xen/xend_internal.c,
  tests/qemuxml2argvtest.c, tests/qemuxmlnstest.c,
  tests/sexpr2xmltest.c, tests/xmconfigtest.c: Convert
  to use virObjectUnref/virObjectRef
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The access, birth, modification and change times are added to
storage volumes and corresponding xml representations.  This
shows up in the XML in this format:

<timestamps>
  <atime>1341933637.027319099</atime>
  <mtime>1341933637.027319099</mtime>
</timestamps>
Signed-off-by: NEric Blake <eblake@redhat.com>

The option 'srcSpec' to virsh command find-storage-pool-sources
is optional for logical type of storage pool, but mandatory for
netfs and iscsi type.
When missing the option for netfs and iscsi, libvirt reports XML
parsing error due to null string option srcSpec.

before
error: Failed to find any netfs pool sources
error: (storage_source_specification):1: Document is empty
(null)

after:
error: pool type 'iscsi' requires option --srcSpec for source discovery

Any time we have a string with no % passed through gettext, a
translator can inject a % to cause a stack overread.  When there
is nothing to format, it's easier to ask for a string that cannot
be used as a formatter, by using a trivial "%s" format instead.

In the past, we have used --disable-nls to catch some of the
offenders, but that doesn't get run very often, and many more
uses have crept in.  Syntax check to the rescue!

The syntax check can catch uses such as
virReportError(code,
               _("split "
                 "string"));
by using a sed script to fold context lines into one pattern
space before checking for a string without %.

This patch is just mechanical insertion of %s; there are probably
several messages touched by this patch where we would be better
off giving the user more information than a fixed string.

* cfg.mk (sc_prohibit_diagnostic_without_format): New rule.
* src/datatypes.c (virUnrefConnect, virGetDomain)
(virUnrefDomain, virGetNetwork, virUnrefNetwork, virGetInterface)
(virUnrefInterface, virGetStoragePool, virUnrefStoragePool)
(virGetStorageVol, virUnrefStorageVol, virGetNodeDevice)
(virGetSecret, virUnrefSecret, virGetNWFilter, virUnrefNWFilter)
(virGetDomainSnapshot, virUnrefDomainSnapshot): Add %s wrapper.
* src/lxc/lxc_driver.c (lxcDomainSetBlkioParameters)
(lxcDomainGetBlkioParameters): Likewise.
* src/conf/domain_conf.c (virSecurityDeviceLabelDefParseXML)
(virDomainDiskDefParseXML, virDomainGraphicsDefParseXML):
Likewise.
* src/conf/network_conf.c (virNetworkDNSHostsDefParseXML)
(virNetworkDefParseXML): Likewise.
* src/conf/nwfilter_conf.c (virNWFilterIsValidChainName):
Likewise.
* src/conf/nwfilter_params.c (virNWFilterVarValueCreateSimple)
(virNWFilterVarAccessParse): Likewise.
* src/libvirt.c (virDomainSave, virDomainSaveFlags)
(virDomainRestore, virDomainRestoreFlags)
(virDomainSaveImageGetXMLDesc, virDomainSaveImageDefineXML)
(virDomainCoreDump, virDomainGetXMLDesc)
(virDomainMigrateVersion1, virDomainMigrateVersion2)
(virDomainMigrateVersion3, virDomainMigrate, virDomainMigrate2)
(virStreamSendAll, virStreamRecvAll)
(virDomainSnapshotGetXMLDesc): Likewise.
* src/nwfilter/nwfilter_dhcpsnoop.c (virNWFilterSnoopReqLeaseDel)
(virNWFilterDHCPSnoopReq): Likewise.
* src/openvz/openvz_driver.c (openvzUpdateDevice): Likewise.
* src/openvz/openvz_util.c (openvzKBPerPages): Likewise.
* src/qemu/qemu_cgroup.c (qemuSetupCgroup): Likewise.
* src/qemu/qemu_command.c (qemuBuildHubDevStr, qemuBuildChrChardevStr)
(qemuBuildCommandLine): Likewise.
* src/qemu/qemu_driver.c (qemuDomainGetPercpuStats): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainAttachNetDevice): Likewise.
* src/rpc/virnetsaslcontext.c (virNetSASLSessionGetIdentity):
Likewise.
* src/rpc/virnetsocket.c (virNetSocketNewConnectUNIX)
(virNetSocketSendFD, virNetSocketRecvFD): Likewise.
* src/storage/storage_backend_disk.c
(virStorageBackendDiskBuildPool): Likewise.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemProbe)
(virStorageBackendFileSystemBuild): Likewise.
* src/storage/storage_backend_rbd.c
(virStorageBackendRBDOpenRADOSConn): Likewise.
* src/storage/storage_driver.c (storageVolumeResize): Likewise.
* src/test/test_driver.c (testInterfaceChangeBegin)
(testInterfaceChangeCommit, testInterfaceChangeRollback):
Likewise.
* src/vbox/vbox_tmpl.c (vboxListAllDomains): Likewise.
* src/xenxs/xen_sxpr.c (xenFormatSxprDisk, xenFormatSxpr):
Likewise.
* src/xenxs/xen_xm.c (xenXMConfigGetUUID, xenFormatXMDisk)
(xenFormatXM): Likewise.

Per the FSF address could be changed from time to time, and GNU
recommends the following now: (http://www.gnu.org/licenses/gpl-howto.html)

  You should have received a copy of the GNU General Public License
  along with Foobar.  If not, see <http://www.gnu.org/licenses/>.

This patch removes the explicit FSF address, and uses above instead
(of course, with inserting 'Lesser' before 'General').

Except a bunch of files for security driver, all others are changed
automatically, the copyright for securify files are not complete,
that's why to do it manually:

  src/security/security_selinux.h
  src/security/security_driver.h
  src/security/security_selinux.c
  src/security/security_apparmor.h
  src/security/security_apparmor.c
  src/security/security_driver.c

This patch brings support to manage sheepdog pools and volumes to libvirt.
It uses the "collie" command-line utility that comes with sheepdog for that.

A sheepdog pool in libvirt maps to a sheepdog cluster.
It needs a host and port to connect to, which in most cases
is just going to be the default of localhost on port 7000.

A sheepdog volume in libvirt maps to a sheepdog vdi.
To create one specify the pool, a name and the capacity.
Volumes can also be resized later.

In the volume XML the vdi name has to be put into the <target><path>.
To use the volume as a disk source for virtual machines specify
the vdi name as "name" attribute of the <source>.
The host and port information from the pool are specified inside the host tag.

  <disk type='network'>
    ...
    <source protocol="sheepdog" name="vdi_name">
      <host name="localhost" port="7000"/>
    </source>
  </disk>

To work right this patch parses the output of collie,
so it relies on the raw output option. There recently was a bug which caused
size information to be reported wrong. This is fixed upstream already and
will be in the next release.
Signed-off-by: NSebastian Wiedenroth <wiedi@frubar.net>

Update the storage driver to use virReportError instead of
the virStorageReportError custom macro
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When passing a const message string to the error reporting APIs
RBD forgot to use "%s" to avoid GCC format string warnings
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This removes all the per-file error reporting macros
from the code in src/conf/
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When calling 'lvcreate' if specifying both the '-L' and
'--virtualsize' options, the latter will be treated as
the capacity and the former as the allocation. This can
be used to support sparse volume creation. In addition,
when listing volumes it is necessary to include the 'size'
field in lvs output, so that we can detect sparse volume
allocation correctly.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To make it easier to dynamically change the command line ARGV,
switch all storage code over to use virCommandPtr APIs for
running programs
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Fix the virStorageBackendFileSystemVolDelete method to not use
unlink() unconditionally. It must use rmdir() for volumes which
are directories. It should also raise an error if given a volume
which has the network/block type.

All the callers of virDirCreate are updated incidentally.

Commit 122fa379 introduces option to
store more than one host entry in a storage pool source definition. That
commit causes a regression, where a check is added that only one host
entry should be present (that actualy is not present as the source
structure was just allocated and zeroed) instead of allocating memory
for the host entry.

As the storage pool sources are stored in a list of structs, the pointer
returned by virStoragePoolSourceListNewSource() shouldn't be freed as it
points in the middle of a memory block. This combined with a regression
that takes the error path every time on caused a double-free abort on
the src struct in question.

mnt_fsname can not be the same, as we check the duplicate pool
sources earlier before, means it can't be the same pool, moreover,
a pool can't be started if it's already active anyway. So no reason
to act as success.

We used to prefix 'rbd:' to volume names, this is not necessary.

Qemu takes RBD devices in this way, like: qemu -drive rbd:pool/image

When attaching a network disk like RBD to a guest we however do not use this prefix.

Currently you can't map a RBD volume name directly to a domain without removing the prefix.
Signed-off-by: NWido den Hollander <wido@widodh.nl>

Storage is one of the last domains in libvirt where we don't fully
utilize inactive and live XML. Okay, it might be because we don't
have support for that. So implement such support. However, we need
to fallback when talking to old daemon which doesn't support this
new flag called VIR_STORAGE_XML_INACTIVE.

Currently, we share the idea of old & new def with domains. Users can
*-edit an object (domain, pool) which spawns a new internal
representation for them. This is referenced via
{domainObj,poolObj}->newDef [compared to ->def]. However, for pool we
were never overwriting def with newDef. This must be done on
pool-destroy (like we do analogically in domain detroy).

The comment says:

/* Now create the final dir in the path with the uid/gid/mode
 * requested in the config. If the dir already exists, just set
 * the perms.
 */

However, virDirCreate is only invoked if the target path doesn't
exist yet (which is opposite with the comment), or the uid from
the config is not -1 (I don't understand why, think it's just
another mistake). And the result is the perms of the pool won't
be changed if one tries to build the pool with different perms
again.

Besides these logic error fix, if no uid and gid are specified in
the config, the practical used uid, gid are reflected.

Remove the uid param from virGetUserConfigDirectory,
virGetUserCacheDirectory, virGetUserRuntimeDirectory,
and virGetUserDirectory

These functions were universally called with the
results of getuid() or geteuid(). To make it practical
to port to Win32, remove the uid parameter and hardcode
geteuid()
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Remove a number of pointless checks against PATH_MAX and
add a syntax-check rule to prevent its use in future
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch adds support for a new storage backend with RBD support.

RBD is the RADOS Block Device and is part of the Ceph distributed storage
system.

It comes in two flavours: Qemu-RBD and Kernel RBD, this storage backend only
supports Qemu-RBD, thus limiting the use of this storage driver to Qemu only.

To function this backend relies on librbd and librados being present on the
local system.

The backend also supports Cephx authentication for safe authentication with
the Ceph cluster.

For storing credentials it uses the built-in secret mechanism of libvirt.
Signed-off-by: NWido den Hollander <wido@widodh.nl>

As defined in:
http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

This offers a number of advantages:
 * Allows sharing a home directory between different machines, or
sessions (eg. using NFS)
 * Cleanly separates cache, runtime (eg. sockets), or app data from
user settings
 * Supports performing smart or selective migration of settings
between different OS versions
 * Supports reseting settings without breaking things
 * Makes it possible to clear cache data to make room when the disk
is filling up
 * Allows us to write a robust and efficient backup solution
 * Allows an admin flexibility to change where data and settings are stored
 * Dramatically reduces the complexity and incoherence of the
system for administrators

The previous storage patch missed an instance affected by the struct
member rename.  It also had some botched whitespace detected by
'make check'.

* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIFindPoolSources): Adjust to new struct.
* src/conf/storage_conf.c (virStoragePoolSourceFormat): Fix
indentation.

The current storage pools for NFS and iSCSI only require one host to
connect to. Future storage pools like RBD and Sheepdog will require
multiple hosts.

This patch allows multiple source hosts and rewrites the current
storage drivers.
Signed-off-by: NWido den Hollander <wido@widodh.nl>

lvcreate want's the parent pool's name, not the pool path
lvchange and lvremove want lv specified as $vgname/$lvname

This largely worked before because these commands strip off a
starting /dev. But https://bugzilla.redhat.com/show_bug.cgi?id=714986
is from a user using a 'nested VG' that was having problems.

I couldn't find any info on nested LVM and the reporter never responded,
but I reproduced with XML that specified a valid source name, and
set target path to a symlink.

The code is splattered with a mix of

  sizeof foo
  sizeof (foo)
  sizeof(foo)

Standardize on sizeof(foo) and add a syntax check rule to
enforce it
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Lets say I got a volume with '1G' allocation and '10G' capacity. The
available space in the parent pool is '5G'. With the current check for
overcapacity, I can only try to resize to <= '6G'. You see the problem?

* src/storage/storage_driver.c (storageVolumeWipeInternal):
    s/ pfitzner33/pfitzner33/.

* src/storage/storage_driver.c (storageVolumeWipeInternal): s/shneier/schneier.

http://code.google.com/p/diskscrub/Signed-off-by: NAlex Jia <ajia@redhat.com>

Currently, if scrub (used for wiping algorithms) is not present
at compile time, we don't support any other wiping algorithms than
zeroing, even if it was installed later. Switch to runtime detection
instead.

Input to the volume cloning code is a source volume and an XML
descriptor for the new volume. It is possible for the new volume
to have a greater size than source volume, at which point libvirt
will just stick 0s on the end of the new image (for raw format
anyways).

Unfortunately a logic error messed up our tracking of the of the
excess amount that needed to be written: end result is that sparse
clones were made very much non-sparse, and cloning regular disk
images could end up excessively sized (though data unaltered).

Drop the 'remain' variable entriely here since it's redundant, and
track actual allocation directly against the desired 'total'.

virFileOpenAs previously would only try opening a file as the current
user, or as a different user, but wouldn't try both methods in a
single call. This made it cumbersome to use as a replacement for
open(2). Additionally, it had a lot of historical baggage that led to
it being difficult to understand.

This patch refactors virFileOpenAs in the following ways:

* reorganize the code so that everything dealing with both the parent
  and child sides of the "fork+setuid+setgid+open" method are in a
  separate function. This makes the public function easier to understand.

* Allow a single call to virFileOpenAs() to first attempt the open as
  the current user, and if that fails to automatically re-try after
  doing fork+setuid (if deemed appropriate, i.e. errno indicates it
  would now be successful, and the file is on a networkFS). This makes
  it possible (in many, but possibly not all, cases) to drop-in
  virFileOpenAs() as a replacement for open(2).

  (NB: currently qemuOpenFile() calls virFileOpenAs() twice, once
  without forking, then again with forking. That unfortunately can't
  be changed without at least some discussion of the ramifications,
  because the requested file permissions are different in each case,
  which is something that a single call to virFileOpenAs() can't deal
  with.)

* Add a flag so that any fchown() of the file to a different uid:gid
  is explicitly requested when the function is called, rather than it
  being implied by the presence of the O_CREAT flag. This just makes
  for less subtle surprises to consumers. (Commit
  b1643dc1 added the check for O_CREAT
  before forcing ownership. This patch just makes that restriction
  more explicit.)

* If either the uid or gid is specified as "-1", virFileOpenAs will
  interpret this to mean "the current [gu]id".

All current consumers of virFileOpenAs should retain their present
behavior (after a few minor changes to their setup code and
arguments).

Currently only VIR_STORAGE_VOL_RESIZE_DELTA flag is supported.

The old virRandom() API was not generating good random numbers.
Replace it with a new API virRandomBits which instead of being
told the upper limit, gets told the number of bits of randomness
required.

* src/util/virrandom.c, src/util/virrandom.h: Add virRandomBits,
  and move virRandomInitialize
* src/util/util.h, src/util/util.c: Delete virRandom and
  virRandomInitialize
* src/libvirt.c, src/security/security_selinux.c,
  src/test/test_driver.c, src/util/iohelper.c: Update for
  changes from virRandom to virRandomBits
* src/storage/storage_backend_iscsi.c: Remove bogus call
  to virRandomInitialize & convert to virRandomBits

Currently, we support only filling a volume with zeroes on wiping.
However, it is not enough as data might still be readable by
experienced and equipped attacker. Many technical papers have been
written, therefore we should support other wiping algorithms.

On F16 at least, empty volume groups don't have a directory under /dev.
The directory only appears once a logical volume is created.

This tickles some behavior in BackendStablePath which ends with
libvirt sleeping for 5 seconds while waiting for the directory to appear.
This causes all sorts of problems for the virStorageVolLookupByPath API
which virtinst uses, even if trying to resolve a path that is independent
of the logical pool.

In reality we don't even need to do that checking since logical pools
always have a stable target path. Short circuit the polling in that
case.

Fixes bug 782261