As there are two possible approaches to define a domain's memory size -
one used with legacy, non-NUMA VMs configured in the <memory> element
and per-node based approach on NUMA machines - the user needs to make
sure that both are specified correctly in the NUMA case.

To avoid this burden on the user I'd like to replace the NUMA case with
automatic totaling of the memory size. To achieve this I need to replace
direct access to the virDomainMemtune's 'max_balloon' field with
two separate getters depending on the desired size.

The two sizes are needed as:
1) Startup memory size doesn't include memory modules in some
hypervisors.
2) After startup these count as the usable memory size.

Note that the comments for the functions are future aware and document
state that will be present after a few later patches.

While qemu may be prepared to do this libvirt is not. Forbid the block
ops until we fix our code.

Surprisingly we did not grab a VM job when a block job finished and we'd
happily rewrite the backing chain data. This made it possible to crash
libvirt when queueing two backing chains tightly and other badness.

To fix it, add yet another handler to the helper thread that handles
monitor events that require a job.

We interpret port values as signed int (convert them from char *),
so if a negative value is provided in network disk's configuration,
we accept it as valid, however there's an 'unknown cause' error raised later.
This error is only accidental because we return the port value in the return code.
This patch adds just a minor tweak to the already existing check so we
reject negative values the same way as we reject non-numerical strings.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1163553

Valgrind detected a leak:

==17820== 102 (56 direct, 46 indirect) bytes in 1 blocks are definitely lost in loss record 479 of 646
==17820==    at 0x4A08946: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==17820==    by 0x508521A: virAllocVar (viralloc.c:560)
==17820==    by 0x50D9FCA: virObjectNew (virobject.c:193)
==17820==    by 0x50A4FD9: dnsmasqCapsNewEmpty (virdnsmasq.c:784)
==17820==    by 0x50A514E: dnsmasqCapsNewFromBinary (virdnsmasq.c:830)
==17820==    by 0x1B508287: networkStateInitialize (bridge_driver.c:666)

It looks like commit 172acef4 introduced the problem, because
networkGetDnsmasqCaps() increments the reference count but an
early exit never does a matching decrement.

* src/network/bridge_driver.c (networkStateCleanup): Plug leak.
Signed-off-by: NEric Blake <eblake@redhat.com>

Valgrind complained:

==3770== Syscall param ioctl(SIOCETHTOOL) points to uninitialised byte(s)
==3770==    at 0x919D407: ioctl (syscall-template.S:81)
==3770==    by 0x530FE7E: rpl_ioctl (ioctl.c:42)
==3770==    by 0x50CB433: virNetDevFeatureAvailable (virnetdev.c:2764)
==3770==    by 0x50CB6A7: virNetDevGetFeatures (virnetdev.c:2830)
==3770==    by 0x1F0E5347: udevProcessNetworkInterface (node_device_udev.c:722)
==3770==    by 0x1F0E689F: udevGetDeviceDetails (node_device_udev.c:1300)
==3770==    by 0x1F0E6E06: udevAddOneDevice (node_device_udev.c:1422)
==3770==    by 0x1F0E6FB8: udevProcessDeviceListEntry (node_device_udev.c:1464)
==3770==    by 0x1F0E70CF: udevEnumerateDevices (node_device_udev.c:1494)
==3770==    by 0x1F0E7BB4: nodeStateInitialize (node_device_udev.c:1806)
==3770==    by 0x51B4303: virStateInitialize (libvirt.c:777)
==3770==    by 0x11DEE7: daemonRunStateInit (libvirtd.c:906)
==3770==  Address 0x228e38d4 is on thread 12's stack
==3770==  in frame #2, created by virNetDevFeatureAvailable (virnetdev.c:2750)

* src/util/virnetdev.c (virNetDevFeatureAvailable): Initialize all
bytes of ifr.
Signed-off-by: NEric Blake <eblake@redhat.com>

Introduced by f6a2f97e

Problem Description:
After multiple times of migrating a domain, which has an ovs interface with no portData set,
with non-shared disk, nbd ports got overflowed.

The steps to reproduce the problem:
1 define and start a domain with its network configured as:
    <interface type='bridge'>
          <source bridge='br0'/>
          <virtualport type='openvswitch'>
          </virtualport>
          <model type='virtio'/>
          <driver name='vhost' queues='4'/>
    </interface>
2 do not set the network's portData.
3 migrate(ToURI2) it with flag 91(1011011), which means:
  VIR_MIGRATE_LIVE
  VIR_MIGRATE_PEER2PEER
  VIR_MIGRATE_PERSIST_DEST
  VIR_MIGRATE_UNDEFINE_SOURCE
  VIR_MIGRATE_NON_SHARED_DISK
4 migrate success, but we got an error log in libvirtd.log:
  error : virCommandWait:2423 : internal error: Child process (ovs-vsctl --timeout=5 get Interface
  vnet1 external_ids:PortData) unexpected exit status 1: ovs-vsctl: no key "PortData" in Interface
  record "vnet1" column external_ids
5 migrate it back, migrate it , migrate it back, .......
6 nbd port got overflowed.

The reasons for the problem is :
1 virNetDevOpenvswitchGetMigrateData() takes it as wrong if no portData is available for  the ovs
 interface of a domain. (We think it's not appropriate, as portData is just OPTIONAL)
2 in func qemuMigrationBakeCookie(), it fails in qemuMigrationCookieAddNetwork(), and returns with -1.
 qemuMigrationCookieAddNBD() is not called thereafter, and mig->nbd is still NULL.
3 However, qemuMigrationRun() just *WARN* if qemuMigrationBakeCookie() fails, migration still successes.
 cookie is NULL, it's not baked on the src side.
4 On the destination side, it would alloc a port first and then free the nbd port in COOKIE.
 But the cookie is NULL due to qemuMigrationCookieAddNetwork() failure at src side. thus the nbd port
 is not freed.

In this patch, we add "--if-exists" option to make ovs-vsctl not raise error if there's no portData available.
Further more, because portData may be NULL in the cookie at the dest side, check it before setting portData.
Signed-off-by: NZhou Yimin <zhouyimin@huawei.com>
Signed-off-by: NZhang Bo <oscar.zhangbo@huawei.com>

It will not be possible to detach such device later. Also improve
logging in such cases.
Signed-off-by: NMarek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

virDomainNetFindIdx no longer returns info whether device was not found,
or there was multiple matches. Additionally it already handle error
reporting. Introduce virDomainHasNet which does a simple task, without
implicit error reporting.
Signed-off-by: NMarek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

In Xen>=4.3, libxl supports new syntax for USB devices:
usbdevice=[ "DEVICE", "DEVICE", ... ]
Add support for that in xenconfig driver. When only one device is
defined, keep using old syntax for backward compatibility.

Adjust tests for changed options order.
Signed-off-by: NMarek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Signed-off-by: NMarek Marczykowski-Górecki <marmarek@invisiblethingslab.com>

Now that the network driver lock is ash heap of history,
we can use more of networkObjFromNetwork().
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

While in previous commits there were some places that relied on
the big lock, in this file there's no such place and the big
driver lock can be dropped completely. Yay!
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Well, if 'everywhere' is defined as that part of the driver code
that serves virNetwork* APIs. Again, we lower layers already have
their locks, so there's no point doing big lock.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Now that we have fine grained locks, there's no need to
lock the whole driver. We can rely on self-locking APIs.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

In order to drop network driver lock, lets annotate which
structure items are immutable, which have self-locking
APIs and so on.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is not an immutable pointer and can change during lifetime.
Therefore, in order to drop network driver lock, we must use an
internal accessor which does not lock the network driver yet, but
it will soon. Now it merely returns an referenced object.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Well, network driver code has the driver accessible as a global
variable. This makes any rework hard, as it's unclear where the
variable is accessed and/or modified. Lets just pass the driver
as a parameter to all functions where needed.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

A helper that never returns an error and treats bits out of bitmap range
as false.

Use it everywhere we use ignore_value on virBitmapGetBit, or loop over
the bitmap size.

for memory limits since unset ones are no longer zero
Signed-off-by: NMaxim Nestratov <mnestratov@parallels.com>

This NIC model is supported on hardware version 8 and newer and libvirt
ESX driver does support those.

For VBOX it's most likely that the connection is vbox:///session and it
runs with local non-root account. This caused permission denied when
LOCALSTATEDIR was used to create temp file. This patch makes use of the
virGetUserCacheDirectory to address this problem for non-root users.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Commit c9027d8f added a detection of NIC HW features, but some of them
are not available in old kernel.  Very old kernels lack enum
ethtool_flags and even if this enum is present, not all values are
available for all kernels.  To be sure that we have everything in kernel
that we need, we must check for existence of most of that flags, because
only few of them were defined at first.

Also to successfully build libvirt with older kernel we need to include
<linux/types.h> before <linux/ethtool.h> to have __u32 and friends
defined.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

as soon as x32 architecture is also supported

We've never formatted them on the qemu command line.

https://bugzilla.redhat.com/show_bug.cgi?id=1164053

Coverity notes in xenapiDomainGetXMLDesc that 'vms' is dereferenced
a few times before a "if (vms) xen_vm_set_free(vms);" call is made.
Since we'd exit out much sooner if the fetch of the vms failed, just
remove the unnecessary "if (vms)" check.

Coverity complains that "net_set" is compared to NULL before calling
xen_network_set_free, but used rather liberally before that.  While
I was looking at the code I also noted that if the virAsprintfQuiet
fails, then we leak our structures - so I added those too.

Coverity points out that the return from virDomainDefParseString is
not checked in xenapiDomainCreateXML like it should be which could
end up in a NULL pointer dereference

Coverity points out that check (def->uuid) has no effect since it's not
a pointer, rather an array of characters.   Just remove the extranous check.

Since inception.  Coverity complains that the code checks "(record ==
NULL && !session->ok)", but doesn't check (record != NULL) before
dereferencing at "record->is_a_template"

Commit id 'c9027d8f' added parsing of the CapNet for offload SRIOV NIC
discovery, but forgot to free the nodes

Add qemuDomainPinIOThread to handle setting the CPU affinity
for a specific IOThread

https://bugzilla.redhat.com/show_bug.cgi?id=1135491

More or less a virtual copy of the existing virDomainVcpuPin{Add|Del} API's.

NB: The IOThreads implementation "reused" the virDomainVcpuPinDefPtr
since it provided everything necessary - an "id" and a "map" for each
thread id configured.

Implement the remote plumbing.

Add virDomainPinIOThread to allow setting the CPU affinity for a specific
IOThread based on the output generated from virDomainGetIOThreadsInfo

The API supports updating both the --live domain and the --config data

This patch turns both virNetworkObjFindByUUID() and
virNetworkObjFindByName() to return an referenced object so that
even if caller unlocks it, it's for sure that object won't
disappear meanwhile. Especially if the object (in general) is
locked and unlocked during the caller run.
Moreover, this commit is nicely small, since the object unrefing
can be done in virNetworkObjEndAPI().
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Every API that touches internal structure of the object must lock
the object first. Not every API that has the object as an
argument needs to do that though. Some APIs just pass the object
to lower layers which, however, must lock the object then. Look
at the code, you'll get my meaning soon.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is going to be needed later, when some functions already
have the virNetworkObjList object already locked and need to
lookup a object to work on. As an example of such function is
virNetworkAssignDef(). The other use case might be in
virNetworkObjListForEach() callback.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Later we can turn APIs to lock the object if needed instead of
relying on caller to mutually exclude itself (probably done by
locking a big lock anyway).
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>