Separate the steps to create libvirt's volume metadata from the actual
volume building process. This is already done for regular file based
pools to allow job support for storage APIs.

Change code ordering to avoid the need for a forward declaration.

Most of the usage of getuid()/getgid() is in cases where we are
considering what privileges we have. As such the code should be
using the effective IDs, not real IDs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This should resolve:

  https://bugzilla.redhat.com/show_bug.cgi?id=924672

For BZ 924672 the problem stems from the fact that thin pool logical
volume devices show up in /sbin/lvs output just like normal logical
volumes do. Libvirt incorrectly assumes they are just normal logical
volumes and that they will have a corresponding /dev/vgname/lvname
device that has been created by udev and tries to use this device.

To illustrate here is an example of the /dev/vgname/ directory and
the lvs output for a normal lv, thin lv, and thin pool:

    LV     VG       Attr      LSize  Pool Origin Data%  Move Log Copy%  Convert
    lv     vgguests -wi-a----  1.00g
    pool   vgguests twi-a-tz- 11.00g               0.00
    thinlv vgguests Vwi-a-tz-  1.00g pool          0.00
total 0
lrwxrwxrwx. 1 root root 7 Oct  8 19:35 lv -> ../dm-7
lrwxrwxrwx. 1 root root 7 Oct  8 19:37 thinlv -> ../dm-6

This patch modifies virStorageBackendLogicalMakeVol() to ignore thin pool
devices.

Semantics of the libvirt helper are more clear. This change also allows
to clean up some pieces of code.

POSIX states that the preg parameter to regcomp() is undefined on
failure, so no need to call regfree() in these cases.

http://pubs.opengroup.org/onlinepubs/009695399/functions/regcomp.html

See also a discussion on the libvirt dev list

https://www.redhat.com/archives/libvir-list/2013-September/msg00262.html

When using logical pools, we had to trust the target->path provided.
This parameter, however, can be completely ommited and we can use
'/dev/<source.name>' safely and populate it to target.path.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=952973

Convert the type of loop iterators named 'i', 'j', k',
'ii', 'jj', 'kk', to be 'size_t' instead of 'int' or
'unsigned int', also santizing 'ii', 'jj', 'kk' to use
the normal 'i', 'j', 'k' naming
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If the volume is of a clustered volume group, and not active, the
related pool APIs fails on opening /dev/vg/lv. If the volume is
suspended, it hangs on open(2) the volume.

Though the best solution is to expose the volume status in volume
XML, and even better to provide API to activate/deactivate the volume,
but it's not the work I want to touch currently. Volume status in
other status is just fine to skip.

About the 5th field of lv_attr (from man lvs[8])
<quote>
 5 State: (a)ctive, (s)uspended, (I)nvalid snapshot, invalid
   (S)uspended snapshot, snapshot (m)erge failed,suspended
   snapshot (M)erge failed, mapped (d)evice present without
   tables,  mapped device present with (i)nactive table
</quote>

The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.

When logical pool has no PVs associated with itself (user-created),
virCommandFree(cmd) is called twice with the same pointer and that
causes a segfault in daemon.

This patch plugs two memory leaks, removes some useless and confusing
constructs and renames renames "cleanup" label as "error" since it is
only used for error path rather then being common for both success and
error paths.

When virStorageBackendLogicalCreateVol() creates a snapshot for a
logical volume with backingStore element, it fails with the message
below:

  2013-01-17 03:10:18.869+0000: 1967: error : virCommandWait:2345 :
  internal error Child process (/sbin/lvcreate --name lvm-snapshot -L 51200K
  -s=/dev/lvm-pool/lvm-volume) unexpected exit status 3: /sbin/lvcreate:
  invalid option -- '='  Error during parsing of command line.

This is because virCommandAddArgPair() uses '=' to connect the two
parameters, it's unsuitable for -s option of the lvcreate.
Signed-off-by: NAtsushi Kumagai <kumagai-atsushi@mxc.nes.nec.co.jp>

Commit-id 'afc4631b' added the regfree(reg) to free resources alloc'd
during regcomp; however, reg still needed to be VIR_FREE()'d. The call
to regfree() also didn't account for possible NULL value.  Reformatted
the call to be closer to usage.

Use regfree instead of VIR_FREE.

Commit 258e06c8 removed setting of the volume type to
VIR_STORAGE_VOL_BLOCK, which leads to failures in
storageVolumeCreateXMLFrom.

The type (and target.format) of the volume was set to zero. In
virStorageBackendGetBuildVolFromFunction, this gets interpreted as
VIR_STORAGE_FILE_NONE and the qemu-img tool is called with unknown
"none" format.

Bug: https://bugzilla.redhat.com/show_bug.cgi?id=879780

On F17 at least, this command fails:

$ sudo /usr/sbin/lvcreate --name sparsetest -L 0K --virtualsize 16384K vgvirt
  Unable to create new logical volume with no extents

Which is unfortunate since allocation=0 is what virt-manager tries to use
by default.

Rather than telling the user 'don't do that', let's just give them the
smallest allocation possible if alloc=0 is requested.

https://bugzilla.redhat.com/show_bug.cgi?id=866481

Before:
$ sudo virsh vol-create-as --pool vgvirt sparsetest --capacity 16M --allocation 0
error: Failed to create vol sparsetest
error: internal error Child process (/usr/sbin/lvchange -aln vgvirt/sparsetest) unexpected exit status 5:   One or more specified logical volume(s) not found.

After:
$ sudo virsh vol-create-as --pool vgvirt sparsetest --capacity 16M --allocation 0
error: Failed to create vol sparsetest
error: internal error Child process (/usr/sbin/lvcreate --name sparsetest -L 0K --virtualsize 16384K vgvirt) unexpected exit status 5:   Unable to create new logical volume with no extents

https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.

* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/;  If/.  If/

Per the FSF address could be changed from time to time, and GNU
recommends the following now: (http://www.gnu.org/licenses/gpl-howto.html)

  You should have received a copy of the GNU General Public License
  along with Foobar.  If not, see <http://www.gnu.org/licenses/>.

This patch removes the explicit FSF address, and uses above instead
(of course, with inserting 'Lesser' before 'General').

Except a bunch of files for security driver, all others are changed
automatically, the copyright for securify files are not complete,
that's why to do it manually:

  src/security/security_selinux.h
  src/security/security_driver.h
  src/security/security_selinux.c
  src/security/security_apparmor.h
  src/security/security_apparmor.c
  src/security/security_driver.c

Update the storage driver to use virReportError instead of
the virStorageReportError custom macro
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When calling 'lvcreate' if specifying both the '-L' and
'--virtualsize' options, the latter will be treated as
the capacity and the former as the allocation. This can
be used to support sparse volume creation. In addition,
when listing volumes it is necessary to include the 'size'
field in lvs output, so that we can detect sparse volume
allocation correctly.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To make it easier to dynamically change the command line ARGV,
switch all storage code over to use virCommandPtr APIs for
running programs
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

lvcreate want's the parent pool's name, not the pool path
lvchange and lvremove want lv specified as $vgname/$lvname

This largely worked before because these commands strip off a
starting /dev. But https://bugzilla.redhat.com/show_bug.cgi?id=714986
is from a user using a 'nested VG' that was having problems.

I couldn't find any info on nested LVM and the reporter never responded,
but I reproduced with XML that specified a valid source name, and
set target path to a symlink.

If the vol object is newly created, it increases the volumes count,
but doesn't decrease the volumes count when do cleanup. It can
cause libvirtd to crash when one trying to free the volume objects
like:
    for (i = 0; i < pool->volumes.count; i++)
        virStorageVolDefFree(pool->volumes.objs[i]);

It's more reliable if we add the newly created vol object in the
end.

Current "-ay | -an" has problems on pool starting/refreshing if
the volumes are clustered. Rommer has posted a patch to list 2
months ago.

https://www.redhat.com/archives/libvir-list/2011-October/msg01116.html

But IMO we shouldn't skip the inactived vols. So this is a squashed
patch by Rommer.
Signed-off-by: NRommer <rommer@active.by>

virStorageBackendLogicalDeleteVol() could not remove the lv with error
"could not remove open logical volume" sometimes. Generally it's caused
by the volume is still active, even if lvremove tries to remove it with
option "--force".

This patch is to fix it by disbale the lv first using "lvchange -aln"
and "lvremove -f" afterwards if the direct "lvremove -f" failed.

lvs outputs "[$lvname_vorigin]" for the virtual snapshot lv
(created with "--virtualsize"), and the original device pointed
by "$lvname_vorigin" is just for lvm internal use, one should
never use it.

Per lvm's nameing rules, "[" is not valid as part of the vg/lv name.
(man 8 lvm).

<quote>
VALID NAMES
The following characters are valid for VG and LV names: a-z A-Z 0-9 + _
. -

VG and LV names cannot begin with a hyphen.   There  are  also  various
reserved  names that are used internally by lvm that can not be used as
LV or VG names.  A VG cannot be called anything that exists in /dev/ at
the time of creation, nor can it be called '.' or '..'.  A LV cannot be
called '.' '..' 'snapshot' or 'pvmove'. The LV name may also  not  con‐
tain the strings '_mlog' or '_mimage'
</quote>

So we can skip the set the lv's backingStore by checking if the name
begins with a "[".

Splitting into two functions allows the user to call the right
function, rather than having to remember that a *Free function is
an exception to the rule.

* src/conf/storage_conf.h (virStoragePoolSourceClear): New function.
* src/libvirt_private.syms (storage_conf.h): Export it.
* src/conf/storage_conf.c (virStoragePoolSourceFree): Split...
(virStoragePoolSourceClear): ...into new function.
(virStoragePoolDefFree, virStoragePoolDefParseSourceString):
Update callers.
* src/test/test_driver.c (testStorageFindPoolSources): Likewise.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemNetFindPoolSourcesFunc)
(virStorageBackendFileSystemNetFindPoolSources): Likewise.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIFindPoolSources): Likewise.
* src/storage/storage_backend_logical.c
(virStorageBackendLogicalFindPoolSources): Likewise.

Detected by Coverity.  Present since commit 82c1740a.

* src/storage/storage_backend_logical.c
(virStorageBackendLogicalMakeVol): Fix leak.

I noticed a couple typos in recent commits, and fixed the remaining
instances of them.

* docs/internals/command.html.in: Fix spelling errors.
* include/libvirt/libvirt.h.in (virConnectDomainEventCallback):
Likewise.
* python/libvirt-override.py (virEventAddHandle): Likewise.
* src/lxc/lxc_container.c (lxcContainerChild): Likewise.
* src/util/hash.c (virHashCreateFull): Likewise.
* src/storage/storage_backend_logical.c
(virStorageBackendLogicalMakeVol): Likewise.
* src/esx/esx_driver.c (esxFormatVMXFileName): Likewise.
* src/vbox/vbox_tmpl.c (vboxIIDIsEqual_v3_x): Likewise.

* src/storage/storage_backend_logical.c:

If a logical vol is created as striped. (e.g. --stripes 3),
the "device" field of lvs output will have multiple fileds which are
seperated by comma. Thus the RE we write in the codes will not
work well anymore. E.g. (lvs output for a stripped vol, uses "#" as
seperator here):

test_stripes##fSLSZH-zAS2-yAIb-n4mV-Al9u-HA3V-oo9K1B#\
/dev/sdc1(10240),/dev/sdd1(0)#42949672960#4194304

The RE we use:

    const char *regexes[] = {
        "^\\s*(\\S+),(\\S*),(\\S+),(\\S+)\\((\\S+)\\),(\\S+),([0-9]+),?\\s*$"
    };

Also the RE doesn't match the "devices" field of striped vol properly,
it contains multiple "device path" and "offset".

This patch mainly does:
    1) Change the seperator into "#"
    2) Change the RE for "devices" field from "(\\S+)\\((\\S+)\\)"
       into "(\\S+)".
    3) Add two new options for lvs command, (segtype, stripes)
    4) Extend the RE to match the value for the two new fields.
    5) Parse the "devices" field seperately in virStorageBackendLogicalMakeVol,
       multiple "extents" info are generated if the vol is striped. The
       number of "extents" is equal to the stripes number of the striped vol.

A incidental fix: (virStorageBackendLogicalMakeVol)
    Free "vol" if it's new created and there is error.

Demo on striped vol with the patch applied:

% virsh vol-dumpxml /dev/test_vg/vol_striped2
<volume>
  <name>vol_striped2</name>
  <key>QuWqmn-kIkZ-IATt-67rc-OWEP-1PHX-Cl2ICs</key>
  <source>
    <device path='/dev/sda5'>
      <extent start='79691776' end='88080384'/>
    </device>
    <device path='/dev/sda6'>
      <extent start='62914560' end='71303168'/>
    </device>
  </source>
  <capacity>8388608</capacity>
  <allocation>8388608</allocation>
  <target>
    <path>/dev/test_vg/vol_striped2</path>
    <permissions>
      <mode>0660</mode>
      <owner>0</owner>
      <group>6</group>
      <label>system_u:object_r:fixed_disk_device_t:s0</label>
    </permissions>
  </target>
</volume>

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=727474

If the regexes supported (?:pvs)?, then we could handle this by
optionally matching but not returning the initial command name.  But it
doesn't.  So add a new char* argument to
virStorageBackendRunProgRegex().  If that argument is NULL then we act
as usual.  Otherwise, if the string at that argument is found at the
start of a returned line, we drop that before running the regex.

With this patch, virt-manager shows me lvs with command_names 1 or 0.

The definitions of PVS_BASE etc may want to be moved into the configure
scripts (though given how PVS is found, IIUC that could only happen if
pvs was a link to pvs_real), but in any case no sense dealing with that
until we're sure this is an ok way to handle it.
Signed-off-by: NSerge Hallyn <serge.hallyn@canonical.com>
Signed-off-by: NEric Blake <eblake@redhat.com>