Until now the virDomainListAllDomains API would lock the domain list and
then every single domain object to access and filter it. This would
potentially allow a unresponsive VM to block the whole daemon if a
*listAllDomains call would get stuck.

To avoid this problem this patch collects a list of referenced domain
objects first from the list and then unlocks it right away. The
expensive operation requiring locking of the domain object is executed
after the list lock is dropped. While a single blocked domain will still
lock up a listAllDomains call, the domain list won't be held locked and
thus other APIs won't be blocked.

Additionally this patch also fixes the lookup code, where we'd ignore
the vm->removing flag and thus potentially return domain objects that
would be deleted very soon so calling any API wouldn't make sense.

As other clients also could benefit from operating on a list of domain
objects rather than the public domain descriptors a new intermediate
API - virDomainObjListCollect - is introduced by this patch.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1181074

The passed function is meant to filter domains according to ACL match.

Separate the code to simplify future refactors.

Extend it to a universal helper used for clearing lists of any objects.
Note that the argument type is specifically void * to allow implicit
typecasting.

Additionally add a helper that works on non-NULL terminated arrays once
we know the length.

My commit 747761a7 (v1.2.15 only) dropped this bit of logic when filling
in a default arch in the XML:

-    /* First try to find one matching host arch */
-    for (i = 0; i < caps->nguests; i++) {
-        if (caps->guests[i]->ostype == ostype) {
-            for (j = 0; j < caps->guests[i]->arch.ndomains; j++) {
-                if (caps->guests[i]->arch.domains[j]->type == domain &&
-                    caps->guests[i]->arch.id == caps->host.arch)
-                    return caps->guests[i]->arch.id;
-            }
-        }
-    }

That attempt to match host.arch is important, otherwise we end up
defaulting to i686 on x86_64 host for KVM, which is not intended.
Duplicate it in the centralized CapsLookup function.

Additionally add some testcases that would have caught this.

https://bugzilla.redhat.com/show_bug.cgi?id=1219191

https://bugzilla.redhat.com/show_bug.cgi?id=1176020

We had a check for the vcpu count total number in <numa>
before, however this check is not good enough. There are
some examples:

1. one of cpu id is out of maxvcpus, can set success(cpu count = 5 < 10):

<vcpu placement='static'>10</vcpu>
<cell id='0' cpus='0-3,100' memory='512000' unit='KiB'/>

2. use the same cpu in 2 cell, can set success(cpu count = 8 < 10):
<vcpu placement='static'>10</vcpu>
<cell id='0' cpus='0-3' memory='512000' unit='KiB'/>
<cell id='1' cpus='0-3' memory='512000' unit='KiB'/>

3. use the same cpu in 2 cell, cannot set success(cpu count = 11 > 10):
<vcpu placement='static'>10</vcpu>
<cell id='0' cpus='0-6' memory='512000' unit='KiB'/>
<cell id='1' cpus='0-3' memory='512000' unit='KiB'/>

Add a check for numa cpus, check if duplicate use one cpu in more
than one cell.
Signed-off-by: NLuyao Huang <lhuang@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Some platforms, like aarch64, don't have APIC but GIC. So there's
no reason to have <apic/> feature turned on. However, we are
still missing <gic/> feature. This commit introduces the feature
to XML parser and formatter, adds documentation and updates RNG
schema.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The phyp driver stuffed it into a DomainDefPtr during its attachdevice
routine, but the value is never advertised via capabilities so it should
be safe to drop.

Have the phyp driver use OSTYPE_LINUX, which is what it advertises via
capabilities.

If the added memory device would exceed the maximum memory size, reject
it.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1216046

Resolve some Coverity errors with IOThread changes
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Build with clang fails with:

  CC       conf/libvirt_conf_la-domain_conf.lo
  conf/domain_conf.c:13377:9: error: variable 'cpumask' is used
  uninitialized whenever 'if' condition is true
  [-Werror,-Wsometimes-uninitialized]
      if (!(tmp = virXMLPropString(node, "cpuset"))) {
                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

and many other similar errors regarding the 'cpuset' variable.

Fix by explicitly initializing it with NULL.

If someone has updated a network to change its bridge name, but the
network is still active (so that bridge name hasn't taken effect yet),
we still want to disallow another network from taking that new name.

We already check that any auto-assigned bridge device name for a
virtual network (e.g. "virbr1") doesn't conflict with the bridge name
for any existing libvirt network (via virNetworkSetBridgeName() in
conf/network_conf.c).

We also want to check that the name doesn't conflict with any bridge
device created on the host system outside the control of libvirt
(history: possibly due to the ploriferation of references to libvirt's
bridge devices in HOWTO documents all around the web, it is not
uncommon for an admin to manually create a bridge in their host's
system network config and name it "virbrX"). To add such a check to
virNetworkBridgeInUse() (which is called by virNetworkSetBridgeName())
we would have to call virNetDevExists() (from util/virnetdev.c); this
function calls ioctl(SIOCGIFFLAGS), which everyone on the mailing list
agreed should not be done from an XML parsing function in the conf
directory.

To remedy that problem, this patch removes virNetworkSetBridgeName()
from conf/network_conf.c and puts an identically functioning
networkBridgeNameValidate() in network/bridge_driver.c (because it's
reasonable for the bridge driver to call virNetDevExists(), although
we don't do that yet because I wanted this patch to have as close to 0
effect on function as possible).

There are a couple of inevitable changes though:

1) We no longer check the bridge name during
   virNetworkLoadConfig(). Close examination of the code shows that
   this wasn't necessary anyway - the only *correct* way to get XML
   into the config files is via networkDefine(), and networkDefine()
   will always call networkValidate(), which previously called
   virNetworkSetBridgeName() (and now calls
   networkBridgeNameValidate()). This means that the only way the
   bridge name can be unset during virNetworkLoadConfig() is if
   someone edited the config file on disk by hand (which we explicitly
   prohibit).

2) Just on the off chance that somebody *has* edited the file by hand,
   rather than crashing when they try to start their malformed
   network, a check for non-NULL bridge name has been added to
   networkStartNetworkVirtual().

   (For those wondering why I don't instead call
   networkValidateBridgeName() there to set a bridge name if one
   wasn't present - the problem is that during
   networkStartNetworkVirtual(), the lock for the network being
   started has already been acquired, but the lock for the network
   list itself *has not* (because we aren't adding/removing a
   network). But virNetworkBridgeInuse() iterates through *all*
   networks (including this one) and locks each network as it is
   checked for a duplicate entry; it is necessary to lock each network
   even before checking if it is the designated "skip" network because
   otherwise some other thread might acquire the list lock and delete
   the very entry we're examining. In the end, permitting a setting of
   the bridge name during network start would require that we lock the
   entire network list during any networkStartNetwork(), which
   eliminates a *lot* of parallelism that we've worked so hard to
   achieve (it can make a huge difference during libvirtd startup). So
   rather than try to adjust for someone playing against the rules, I
   choose to instead give them the error they deserve.)

3) virNetworkAllocateBridge() (now removed) would leak any "template"
   string set as the bridge name. Its replacement
   networkFindUnusedBridgeName() doesn't leak the template string - it
   is properly freed.

Add qemuDomainAddIOThread and qemuDomainDelIOThread in order to add or
remove an IOThread to/from the host either for live or config optoins

The implementation for the 'live' option will use the iothreadpids list
in order to make decision, while the 'config' option will use the
iothreadids list.  Additionally, for deletion each may have to adjust
the iothreadpin list.

IOThreads are implemented by qmp objects, the code makes use of the existing
qemuMonitorAddObject or qemuMonitorDelObject APIs.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

We're about to allow IOThreads to be deleted, but an iothreadid may be
included in some domain thread sched, so add a new API to allow removing
an iothread from some entry.

Then during the writing of the threadsched data and an additional check
to determine whether the bitmap is all clear before writing it out.

With iothreadid's allowing any 'id' value for an iothread_id, the
iothreadsched code needs a slight adjustment to allow for "any"
unsigned int value in order to create the bitmap of ids that will
have scheduler adjustments. Adjusted the doc description as well.

Remove the iothreadspin array from cputune and replace with a cpumask
to be stored in the iothreadids list.

Adjust the test output because our printing goes in order of the iothreadids
list now.

Since it's only ever referenced in domain_conf.c, make the function
static, but also will need to move it to somewhere before it's referenced
rather than forward referencing it.

Add 'thread_id' to the virDomainIOThreadIDDef as a means to store the
'thread_id' as returned from the live qemu monitor data.

Remove the iothreadpids list from _qemuDomainObjPrivate and replace with
the new iothreadids 'thread_id' element.

Rather than use the default numbering scheme of 1..number of iothreads
defined for the domain, use the iothreadid's list for the iothread_id

Since iothreadids list keeps track of the iothread_id's, these are
now used in place of the many places where a for loop would "know"
that the ID was "+ 1" from the array element.

The new tests ensure usage of the <iothreadid> values for an exact number
of iothreads and the usage of a smaller number of <iothreadid> values than
iothreads that exist (and usage of the default numbering scheme).

Adding a new XML element 'iothreadids' in order to allow defining
specific IOThread ID's rather than relying on the algorithm to assign
IOThread ID's starting at 1 and incrementing to iothreads count.

This will allow future patches to be able to add new IOThreads by
a specific iothread_id and of course delete any exisiting IOThread.

Each iothreadids element will have 'n' <iothread> children elements
which will have attribute "id".  The "id" will allow for definition
of any "valid" (eg > 0) iothread_id value.

On input, if any <iothreadids> <iothread>'s are provided, they will
be marked so that we only print out what we read in.

On input, if no <iothreadids> are provided, the PostParse code will
self generate a list of ID's starting at 1 and going to the number
of iothreads defined for the domain (just like the current algorithm
numbering scheme).  A future patch will rework the existing algorithm
to make use of the iothreadids list.

On output, only print out the <iothreadids> if they were read in.

use virNetworkRouteDefFree() instead of VIR_FREE to free routes, otherwise
the element 'family' would not be freed.
Signed-off-by: NZhang Bo <oscar.zhangbo@huawei.com>

use cleanup instead of error, so that the allocated strings could also get freed
when there's no error.
Signed-off-by: NZhang Bo <oscar.zhangbo@huawei.com>

virBufferContentAndReset() doesn't free buf contents, we should use
virBufferFreeAndReset() to get buf freed.
Signed-off-by: NZhang Bo <oscar.zhangbo@huawei.com>

This hash table will contain the same data as already existing one.
The only difference is that while the first table uses domain uuid as
key, the new table uses domain name. This will allow much faster (and
lockless) lookups by domain name.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Every domain that grabs a domain object to work over should
reference it to make sure it won't disappear meanwhile.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This is basically turning qemuDomObjEndAPI into a more general
function. Other drivers which gets a reference to domain objects may
benefit from this function too.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

There's no point in duplicating virObjectFreeHashData() in a separate
function.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

It's a pointer after all. We should initialize it to NULL instead of
false.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

In one of my previous patches (b68a56bc) I made class_id to
format more frequently. Well, now it's formatting way too
frequent - even for regular active XML. Users don't need to see
it, so lets format it only for the status XML where it's really
needed.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This needs to specified in way too many places for a simple validation
check. The ostype/arch/virttype validation checks later in
DomainDefParseXML should catch most of the cases that this was covering.

This revealed that GuestDefaultEmulator was a bit buggy, capable
of returning an emulator that didn't match the passed domain type. Fix
up the test suite input to continue to pass.

The strange logic here is just to duplicate the previous behavior,
which parts of the test suite are currently relying on.

Just move this code to where we parse virtType. Lots of things depend
on os.type values so it should be handled early like virtType.

This is a helper function to look up all capabilities data for all
the OS bits that are relevant to <domain>. This is

- os type
- arch
- domain type
- emulator
- machine type

This will be used to replace several functions in later commits.

Rather than an opencoded string. This should be a no-op

But the internal API stays the same, and we just convert the value as
needed. Not useful yet, but this is the beginning step of using an enum
for ostype throughout the code.

When parsing XML, we validate the passed ostype + arch combo against
the detected hypervisor capabilities. This has led to the following
problem:

- Define x86 qemu guest
- qemu is inadvertently removed from the host
- libvirtd is restarted. fails to parse VM config since arch is removed
- 'virsh list --all' is now empty, user is wondering where their VMs went

Add a new internal flag VIR_DOMAIN_DEF_PARSE_SKIP_OSTYPE_CHECKS. Use
it when loading VM and snapshot configs from disk.

https://bugzilla.redhat.com/show_bug.cgi?id=1043572