==5306== 8 bytes in 1 blocks are definitely lost in loss record 24 of 277
 ==5306==    at 0x4C28B2F: calloc (vg_replace_malloc.c:593)
 ==5306==    by 0x5293CAF: virAllocN (viralloc.c:152)
 ==5306==    by 0x52DFEAE: virXPathNodeSet (virxml.c:611)
 ==5306==    by 0x5313DD9: virNetworkDefParseXML (network_conf.c:1408)
 ==5306==    by 0x53170F6: virNetworkObjUpdateParseFile (network_conf.c:2031)
 ==5306==    by 0x131DA63C: networkStartup (bridge_driver.c:279)
 ==5306==    by 0x53481DF: virStateInitialize (libvirt.c:822)
 ==5306==    by 0x40DF44: daemonRunStateInit (libvirtd.c:877)
 ==5306==    by 0x52D2FF5: virThreadHelper (virthreadpthread.c:161)
 ==5306==    by 0x5D00C52: start_thread (in /usr/lib64/libpthread-2.17.so)
 ==5306==    by 0x6410ECC: clone (in /usr/lib64/libc-2.17.so)

This patch fixes crash of the daemon that happens due to the following race
condition:

Let's have two threads in the libvirtd daemon's qemu driver:
A - thread executing undefine on the same domain
B - thread executing a API call to get information about a domain

Assume following serialization of operations done by the threads:
1) A has the lock on the domain object and is executing some code prior to
   virDomainObjListRemove()
2) B takes the lock on the domain object list, looks up the domain object
pointer and blocks in the attempt to lock the domain object as A is holding the
lock
3) A reaches virDomainObjListRemove() and unlocks the lock on the domain object
4) A blocks on the attempt to get the domain list lock
5) B is able to lock the domain object now and unlocks the domain list
6) A is now able to lock the domain list, and sheds the last reference on the
domain object, this triggers the freeing function.
6) B starts executing the code on the pointer that is being freed
7) The libvirtd daemon crashes while attempting to access invalid pointer in
thread B.

This patch fixes the race by acquiring a reference on the domain object before
unlocking it in virDomainObjListRemove() and re-locks the object prior to
removing and freeing it. This ensures that no thread holds a lock on the domain
object at the time it is removed from the list, and that doing a list lookup
will never find a domain that is about to vanish.

This is a minimal fix of the problem, but a better solution will be to switch to
full reference counting for domain objects.

Commit 9a3ff01d (which was ACKed at
the end of January, but for some reason didn't get pushed until during
the 1.0.4 freeze) fixed the logic in virPCIGetVirtualFunctions().
Unfortunately, a typo in the fix (replacing VIR_REALLOC_N with
VIR_ALLOC_N during code movement) caused not only a memory leak, but
also resulted in most of the elements of the result array being
replaced with NULL. virNetDevGetVirtualFunctions() assumed (and I think
rightly so) that virPCIGetVirtualFunctions() wouldn't return any NULL
elements in the array, so it ended up segfaulting.

This was found when attempting to use a virtual network with an
auto-created pool of SRIOV VFs, e.g.:

    <forward mode='hostdev' managed='yes'>
      <pf dev='eth4'/>
    </forward>

(the pool of PCI addresses is discovered by calling
virNetDevGetVirtualFunctions() on the PF dev).

https://bugzilla.redhat.com/show_bug.cgi?id=948821

The helper function to look up disk controller model may be used by scsi
hostdev. But it should be changed to use device info.
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>

This effectively reverts commit 539d73db as the
changes aren't needed after introduction of the XML post parse callbacks.

With the default model assigned in the parse callback, this code is now obsolete.

Commit a68d6726 breaks networking on s390 as it
changes the default network card model.

Even though http://libvirt.org/formatdomain.html#elementsMetadata
states that it requires RFC4122 compliance UUIDs that are generated
by virUUIDGenerate() are not. Following patch modifies generated
UUIDs to conform to rules described in RFC.
Signed-off-by: NMilos Vyletel <milos.vyletel@sde.cz>

If the user requests a mount for /run, this may hide any existing
mounts that are lower down in /run. The result is that the
container still sees the mounts in /proc/mounts, but cannot
access them

sh-4.2# df
df: '/run/user/501/gvfs': No such file or directory
df: '/run/media/berrange/LIVE': No such file or directory
df: '/run/media/berrange/SecureDiskA1': No such file or directory
df: '/run/libvirt/lxc/sandbox': No such file or directory
Filesystem                      1K-blocks      Used Available Use% Mounted on
/dev/mapper/vg_t500wlan-lv_root 151476396 135390200   8384900  95% /
tmpfs                             1970888      3204   1967684   1% /run
/dev/sda1                          194241    155940     28061  85% /boot
devfs                                  64         0        64   0% /dev
tmpfs                                  64         0        64   0% /sys/fs/cgroup
tmpfs                             1970888      1200   1969688   1% /etc/libvirt-sandbox/scratch

Before mounting any filesystem at a particular location, we
must recursively unmount anything at or below the target mount
point
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Ensure lxcContainerUnmountSubtree is at the top of the
lxc_container.c file so it is easily referenced from
any other method. No functional change
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This allows a container-type domain to have exclusive access to one of
the host's NICs.

Wire <hostdev caps=net> with the lxc_controller - when moving the newly
created veth devices into a new namespace, also look for any hostdev
devices that should be moved. Note: once the container domain has been
destroyed, there is no code that moves the interfaces back to the
original namespace. This does happen, though, probably due to default
cleanup on namespace destruction.
Signed-off-by: NBogdan Purcareata <bogdan.purcareata@freescale.com>

This updates the definitions and supporting structures in the XML
schema and domain configuration files.
Signed-off-by: NBogdan Purcareata <bogdan.purcareata@freescale.com>

The virCgroupMounted method is badly named, since a controller can be
mounted, but disabled in the current object. Rename the method to be
virCgroupHasController. Also make it tolerant to a  NULL virCgroupPtr
and out-of-range controller index, to avoid duplication of these
checks in all callers
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This allows one use block type volume as the disk source for device
'lun'.

Since the source is already translated before. This just adds the
checking. Move !disk->shared and !disk->src to improve the performance
a bit.

To support "shareable" for volume type disk, we have to translate
the source before trying to add the shared disk entry. To achieve
the goal, this moves the helper qemuTranslateDiskSourcePool into
src/qemu/qemu_conf.c, and introduce an internal only member (voltype)
for struct _virDomainDiskSourcePoolDef, to record the underlying
volume type for use when building the drive string.

Later patch will support "shareable" volume type disk.

"seclabels" is only valid for 'file' or 'block' type storage volume.

"startupPolicy" is only valid for file type storage volume, otherwise
it fails on starting the domain.

This adds a new helper qemuTranslateDiskSourcePool which uses the
storage pool/vol APIs to translate the disk source before building
the drive string. Network volume is not supported yet. Disk chain
for volume type disk may be supported later, but before I'm confident
it doesn't break anything, it's just disabled now.

With this patch, one can specify the disk source using libvirt
storage like:

  <disk type='volume' device='disk'>
    <driver name='qemu' type='raw' cache='none'/>
    <source pool='default' volume='fc18.img'/>
    <target dev='vdb' bus='virtio'/>
  </disk>

"seclabels" and "startupPolicy" are not supported for this new
disk type ("volume"). They will be supported in later patches.

docs/formatdomain.html.in:
  * Add documents for new XMLs
docs/schemas/domaincommon.rng:
  * Add rng for new XMLs;
src/conf/domain_conf.h:
  * New struct for 'volume' type disk source (virDomainDiskSourcePoolDef)
  * Add VIR_DOMAIN_DISK_TYPE_VOLUME for enum virDomainDiskType
src/conf/domain_conf.c:
  * New helper virDomainDiskSourcePoolDefParse to parse the 'volume'
    type disk source.
  * New helper virDomainDiskSourcePoolDefFree to free the source def
    if 'volume' type disk.
tests/qemuxml2argvdata/qemuxml2argv-disk-source-pool.xml:
tests/qemuxml2xmltest.c:
  * New test

The code to format disk source is long enough to have a helper.

This finds the parent for vHBA by iterating over all the HBA
which supports vport_ops capability on the host, and return
the first one which is online, not saturated (vports in use
is less than max_vports).

startPool creates the vHBA if it's not existed yet, stopPool destroys
the vHBA. Also to support autostart, checkPool will creates the vHBA
if it's not existed yet.

The helper iterates over sysfs, to find out the matched scsi host
name by comparing the wwnn,wwpn pair. It will be used by checkPool
and refreshPool of storage scsi backend. New helper getAdapterName
is introduced in storage_backend_scsi.c, which uses the new util
helper virGetFCHostNameByWWN to get the fc_host adapter name.

It's possible to support fc_host adapter for phyp driver too, but
at this stage I'd like to not allow it when I'm not that clear
how it works.

It's only used by iscsi backend.

node device driver names the HBA like "scsi_host5", but storage
driver uses "host5", which could make the user confused. This
changes them to be consistent. However, for back-compat reason,
adapter name like "host5" is still supported.

This introduces 4 new attributes for storage pool source adapter.
E.g.

<adapter type='fc_host' parent='scsi_host5' wwnn='20000000c9831b4b' wwpn='10000000c9831b4b'/>

Attribute 'type' can be either 'scsi_host' or 'fc_host', and defaults
to 'scsi_host' if attribute 'name' is specified. I.e. It's optional
for 'scsi_host' adapter, for back-compat reason. However, mandatory
for 'fc_host' adapter and any new future adapter types. Attribute
'parent' is to specify the parent for the fc_host adapter.

* docs/formatstorage.html.in:
  - Add documents for the 4 new attrs
* docs/schemas/storagepool.rng:
  - Add RNG schema
* src/conf/storage_conf.c:
  - Parse and format the new XMLs
* src/conf/storage_conf.h:
  - New struct virStoragePoolSourceAdapter, replace "char *adapter" with it;
  - New enum virStoragePoolSourceAdapterType
* src/libvirt_private.syms:
  - Export TypeToString and TypeFromString
* src/phyp/phyp_driver.c:
  - Replace "adapter" with "adapter.data.name", which is member of the union
    of the new struct virStoragePoolSourceAdapter now. Later patch will
    add the checking, as "adapter.data.name" is only valid for "scsi_host"
    adapter.
* src/storage/storage_backend_scsi.c:
  - Like above
* tests/storagepoolxml2xmlin/pool-scsi-type-scsi-host.xml:
* tests/storagepoolxml2xmlin/pool-scsi-type-fc-host.xml:
  - New test for 'fc_host' and "scsi_host" adapter
* tests/storagepoolxml2xmlout/pool-scsi.xml:
  - Change the expected output, as the 'type' defaults to 'scsi_host' if 'name"
    specified now
* tests/storagepoolxml2xmlout/pool-scsi-type-scsi-host.xml:
* tests/storagepoolxml2xmlout/pool-scsi-type-fc-host.xml:
  - New test
* tests/storagepoolxml2xmltest.c:
  - Include the test

There are a number of places which generate cast alignment
warnings, which are difficult or impossible to address. Use
pragmas to disable the warnings in these few places

conf/nwfilter_conf.c: In function 'virNWFilterRuleDetailsParse':
conf/nwfilter_conf.c:1806:16: warning: cast increases required alignment of target type [-Wcast-align]
         item = (nwItemDesc *)((char *)nwf + att[idx].dataIdx);
conf/nwfilter_conf.c: In function 'virNWFilterRuleDefDetailsFormat':
conf/nwfilter_conf.c:3238:16: warning: cast increases required alignment of target type [-Wcast-align]
         item = (nwItemDesc *)((char *)def + att[i].dataIdx);

storage/storage_backend_mpath.c: In function 'virStorageBackendCreateVols':
storage/storage_backend_mpath.c:247:17: warning: cast increases required alignment of target type [-Wcast-align]
         names = (struct dm_names *)(((char *)names) + next);

nwfilter/nwfilter_dhcpsnoop.c: In function 'virNWFilterSnoopDHCPDecode':
nwfilter/nwfilter_dhcpsnoop.c:994:15: warning: cast increases required alignment of target type [-Wcast-align]
         pip = (struct iphdr *) pep->eh_data;
nwfilter/nwfilter_dhcpsnoop.c:1004:11: warning: cast increases required alignment of target type [-Wcast-align]
     pup = (struct udphdr *) ((char *) pip + (pip->ihl << 2));

nwfilter/nwfilter_learnipaddr.c: In function 'procDHCPOpts':
nwfilter/nwfilter_learnipaddr.c:327:33: warning: cast increases required alignment of target type [-Wcast-align]
                 uint32_t *tmp = (uint32_t *)&dhcpopt->value;
nwfilter/nwfilter_learnipaddr.c: In function 'learnIPAddressThread':
nwfilter/nwfilter_learnipaddr.c:501:43: warning: cast increases required alignment of target type [-Wcast-align]
                     struct iphdr *iphdr = (struct iphdr*)(packet +
nwfilter/nwfilter_learnipaddr.c:538:43: warning: cast increases required alignment of target type [-Wcast-align]
                     struct iphdr *iphdr = (struct iphdr*)(packet +
nwfilter/nwfilter_learnipaddr.c:544:48: warning: cast increases required alignment of target type [-Wcast-align]
                         struct udphdr *udphdr= (struct udphdr *)
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When reading the inotify FD, we get back a sequence of
struct inotify_event, each with variable length data following.
It is not safe to simply cast from the char *buf to the
struct inotify_event struct since this may violate data
alignment rules. Thus we must copy from the char *buf
into the struct inotify_event instance before accessing
the data.

uml/uml_driver.c: In function 'umlInotifyEvent':
uml/uml_driver.c:327:13: warning: cast increases required alignment of target type [-Wcast-align]
         e = (struct inotify_event *)tmp;
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The current way virObject instances are allocated using
VIR_ALLOC_N causes alignment warnings

util/virobject.c: In function 'virObjectNew':
util/virobject.c:195:11: error: cast increases required alignment of target type [-Werror=cast-align]

Changing to use VIR_ALLOC_VAR will avoid the need todo
the casts entirely.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virNetlinkCommand() method takes an 'unsigned char **'
parameter to be filled with the received netlink message.
The callers then immediately cast this to 'struct nlmsghdr',
triggering (bogus) warnings about increasing alignment
requirements

util/virnetdev.c: In function 'virNetDevLinkDump':
util/virnetdev.c:1300:12: warning: cast increases required alignment of target type [-Wcast-align]
     resp = (struct nlmsghdr *)*recvbuf;
            ^
util/virnetdev.c: In function 'virNetDevSetVfConfig':
util/virnetdev.c:1429:12: warning: cast increases required alignment of target type [-Wcast-align]
     resp = (struct nlmsghdr *)recvbuf;

Since all callers cast to 'struct nlmsghdr' we can avoid
the warning problem entirely by simply changing the
signature of virNetlinkCommand to return a 'struct nlmsghdr **'
instead of 'unsigned char **'. The way we do the cast inside
virNetlinkCommand does not have any alignment issues.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Playing games with field offsets in a struct causes all sorts
of alignment warnings on ARM platforms

util/virkeycode.c: In function '__virKeycodeValueFromString':
util/virkeycode.c:26:7: warning: cast increases required alignment of target type [-Wcast-align]
     (*(typeof(field_type) *)((char *)(object) + field_offset))
       ^
util/virkeycode.c:91:28: note: in expansion of macro 'getfield'
         const char *name = getfield(virKeycodes + i, const char *, name_offset);
                            ^
util/virkeycode.c:26:7: warning: cast increases required alignment of target type [-Wcast-align]
     (*(typeof(field_type) *)((char *)(object) + field_offset))
       ^
util/virkeycode.c:94:20: note: in expansion of macro 'getfield'
             return getfield(virKeycodes + i, unsigned short, code_offset);
                    ^
util/virkeycode.c: In function '__virKeycodeValueTranslate':
util/virkeycode.c:26:7: warning: cast increases required alignment of target type [-Wcast-align]
     (*(typeof(field_type) *)((char *)(object) + field_offset))
       ^
util/virkeycode.c:127:13: note: in expansion of macro 'getfield'
         if (getfield(virKeycodes + i, unsigned short, from_offset) == key_value)
             ^
util/virkeycode.c:26:7: warning: cast increases required alignment of target type [-Wcast-align]
     (*(typeof(field_type) *)((char *)(object) + field_offset))
       ^
util/virkeycode.c:128:20: note: in expansion of macro 'getfield'
             return getfield(virKeycodes + i, unsigned short, to_offset);

There is no compelling reason to use a struct for the keycode
tables. It can easily just use an array of arrays instead,
avoiding all alignment problems
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

For both "live" and "config" changes of vcpupin and emulatorpin, an
all clear bitmap doesn't make sense, and it can just cause corruptions.
E.g (similar for emulatorpin).

% virsh vcpupin hame 0 8,^8 --config

% virsh vcpupin hame
VCPU: CPU Affinity
----------------------------------
   0:
   1: 0-63
   2: 0-63
   3: 0-63

% virsh dumpxml hame | grep cpuset
    <vcpupin vcpu='0' cpuset=''/>

% virsh start hame
error: Failed to start domain hame
error: An error occurred, but the cause is unknown

This introduce a new attribute "num_queues" (same with the good name
QEMU uses) for virtio-scsi controller. An example of the XML:

<controller type='scsi' index='0' model='virtio-scsi' num_queues='8'/>

The corresponding QEMU command line:

-device virtio-scsi-pci,id=scsi0,num_queues=8,bus=pci.0,addr=0x3 \

By default, libtool builds two .o files for every .lo rule:
src/foo.o - static builds
src/.libs/foo.o - shared library builds

But since commit ad42b34b disabled static builds, src/foo.o is
no longer built by default.  On a fresh checkout, this means our
protocol check rules using pdwtags were testing a missing file,
and thanks to a lousy behavior of pdwtags happily giving no output
and 0 exit status (http://bugzilla.redhat.com/949034), we were
merely claiming that "dwarves is too old" and skipping the test.

However, if you swap between branches and do incremental builds,
such as building v0.10.2-maint and then switching back to master,
you end up with src/foo.o being leftover from its 0.10.2 state,
and then 'make check' fails because the .o file does not match
the protocol-structs file due to API additions in the meantime.

A simpler fix would be to always look in .libs for the .o to
be parsed; but since it is possible to pass ./configure options
to tell libtool to do a static-only build with no shared .o,
I went with the approach of finding the newest of the two files,
whenever both exist.

* src/Makefile.am (PDWTAGS): Ensure we test just-built file.