In order to share as much virsh' logic as possible with upcomming
virt-admin client we need to split virsh logic into virsh specific and
client generic features.

Since majority of virsh methods should be generic enough to be used by
other clients, it's much easier to rename virsh specific data to virshX
than doing this vice versa. It moved generic virsh commands (including info
and opts structures) to generic module vsh.c.

Besides renaming methods and structures, this patch also involves introduction
of a client specific control structure being referenced as private data in the
original control structure, introduction of a new global vsh Initializer,
which currently doesn't do much, but there is a potential for added
functionality in the future.
Lastly it introduced client hooks which are especially necessary during
client connecting phase.

We just need to update the entry in the second hash table. Since commit 8728a565
we have two hash tables for the domain list so that we can do O(1) lookup
regardless of looking up by UUID or name. Since with renaming a domain UUID does
not change, we only need to update the second hash table, where domains are
referenced by their name.

We will call both functions from the qemuDomainRename().
Signed-off-by: NTomas Meszaros <exo@tty.sk>

This new subelement is used in PCI controllers: the toplevel
*attribute* "model" of a controller denotes what kind of PCI
controller is being described, e.g. a "dmi-to-pci-bridge",
"pci-bridge", or "pci-root". But in the future there will be different
implementations of some of those types of PCI controllers, which
behave similarly from libvirt's point of view (and so should have the
same model), but use a different device in qemu (and present
themselves as a different piece of hardware in the guest). In an ideal
world we (i.e. "I") would have thought of that back when the pci
controllers were added, and used some sort of type/class/model
notation (where class was used in the way we are now using model, and
model was used for the actual manufacturer's model number of a
particular family of PCI controller), but that opportunity is long
past, so as an alternative, this patch allows selecting a particular
implementation of a pci controller with the "name" attribute of the
<model> subelement, e.g.:

  <controller type='pci' model='dmi-to-pci-bridge' index='1'>
    <model name='i82801b11-bridge'/>
  </controller>

In this case, "dmi-to-pci-bridge" is the kind of controller (one that
has a single PCIe port upstream, and 32 standard PCI ports downstream,
which are not hotpluggable), and the qemu device to be used to
implement this kind of controller is named "i82801b11-bridge".

Implementing the above now will allow us in the future to add a new
kind of dmi-to-pci-bridge that doesn't use qemu's i82801b11-bridge
device, but instead uses something else (which doesn't yet exist, but
qemu people have been discussing it), all without breaking existing
configs.

(note that for the existing "pci-bridge" type of PCI controller, both
the model attribute and <model> name are 'pci-bridge'. This is just a
coincidence, since it turns out that in this case the device name in
qemu really is a generic 'pci-bridge' rather than being the name of
some real-world chip)

This function should return the greatest CPU number set in
/domain/cpu/numa/cell/@cpus. The idea is that we should compare
the returned value against /domain/vcpu value. Yes, there exist
users who think the following is a good idea:

  <vcpu placement='static'>4</vcpu>
  <cpu mode='host-model'>
    <model fallback='allow'/>
    <numa>
      <cell id='0' cpus='0-1' memory='1048576' unit='KiB'/>
      <cell id='1' cpus='9-10' memory='2097152' unit='KiB'/>
    </numa>
  </cpu>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Qemu reports physical size 0 for block devices. As 15fa84ac
changed the behavior of qemuDomainGetBlockInfo to just query the monitor
this created a regression since we didn't report the size correctly any
more.

This patch adds code to refresh the physical size of a block device by
opening it and seeking to the end and uses it both in
qemuDomainGetBlockInfo and also in qemuDomainGetStatsOneBlock that was
broken since it was introduced in this respect.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1250982

The nodeinfo is reporting incorrect number of cpus and incorrect host
topology on PPC64 KVM hosts. The KVM hypervisor on PPC64 needs only
the primary thread in a core to be online, and the secondaries offlined.
While scheduling a guest in, the kvm scheduler wakes up the secondaries to
run in guest context.

The host scheduling of the guests happen at the core level(as only primary
thread is online). The kvm scheduler exploits as many threads of the core
as needed by guest. Further, starting POWER8, the processor allows splitting
a physical core into multiple subcores with 2 or 4 threads each. Again, only
the primary thread in a subcore is online in the host. The KVM-PPC
scheduler allows guests to exploit all the offline threads in the subcore,
by bringing them online when needed.
(Kernel patches on split-core http://www.spinics.net/lists/kvm-ppc/msg09121.html)

Recently with dynamic micro-threading changes in ppc-kvm, makes sure
to utilize all the offline cpus across guests, and across guests with
different cpu topologies.
(https://www.mail-archive.com/kvm@vger.kernel.org/msg115978.html)

Since the offline cpus are brought online in the guest context, it is safe
to count them as online. Nodeinfo today discounts these offline cpus from
cpu count/topology calclulation, and the nodeinfo output is not of any help
and the host appears overcommited when it is actually not.

The patch carefully counts those offline threads whose primary threads are
online. The host topology displayed by the nodeinfo is also fixed when the
host is in valid kvm state.
Signed-off-by: NShivaprasad G Bhat <sbhat@linux.vnet.ibm.com>
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>

The new name makes it clear that the returned bitmap contains the
information about which CPUs are online, not eg. which CPUs are
present.

No behavioral change.

If one calls update-device with information that is not updatable,
libvirt reports success even though no data were updated.  The example
used in the bug linked below uses updating device with <boot order='2'/>
which, in my opinion, is a valid thing to request from user's
perspective.  Mainly since we properly error out if user wants to update
such data on a network device for example.

And since there are many things that might happen (update-device on disk
basically knows just how to change removable media), check for what's
changing and moreover, since the function might be usable in other
drivers (updating only disk path is a valid possibility) let's abstract
it for any two disks.

We can't possibly check for everything since for many fields our code
does not properly differentiate between default and unspecified values.
Even though this could be changed, I don't feel like it's worth the
complexity so it's not the aim of this patch.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1007228

Add virNetworkDefGetRouteByIndex() similarly to
virNetworkDefGetIpByIndex(), but for routes.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

A helper function for copying error objects.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

This is a self-locking wrapper around virHashTable. Only a limited set
of APIs are implemented now (the ones which are used in the following
patch) as more can be added on demand.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

There are multiple consumers for the domain condition and we should
always wake them all.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

While working in qemu_monitor_json, I repeatedly found myself
getting a value then checking if it was an object.  Add some
wrappers to make this task easier.

* src/util/virjson.c (virJSONValueObjectGetByType)
(virJSONValueObjectGetObject, virJSONValueObjectGetArray): New
functions.
(virJSONValueObjectGetString, virJSONValueObjectGetNumberInt)
(virJSONValueObjectGetNumberUint)
(virJSONValueObjectGetNumberLong)
(virJSONValueObjectGetNumberUlong)
(virJSONValueObjectGetNumberDouble)
(virJSONValueObjectGetBoolean): Simplify.
(virJSONValueIsNull): Change return type.
* src/util/virjson.h: Reflect changes.
* src/libvirt_private.syms (virjson.h): Export them.
* tests/jsontest.c (testJSONLookup): New test.
Signed-off-by: NEric Blake <eblake@redhat.com>

The wrapper is useful for calling qemuBlockJobEventProcess with the
event details stored in disk's privateData, which is the most likely
usage of qemuBlockJobEventProcess.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Complex jobs, such as migration, need to monitor several events at once,
which is impossible when each of the event uses its own condition
variable. This patch adds a single condition variable to each domain
object. This variable can be used instead of the other event specific
conditions.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Add multikey API:

 * virTypedParamsFilter that filters all the parameters with specified name.
 * virTypedParamsGetStringList that returns a list with all the values for
   specified name and string type.
Signed-off-by: NPavel Boldin <pboldin@mirantis.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

virDomainObjGetOneDef will help to retrieve the correct definition
pointer from @vm in cases where VIR_DOMAIN_AFFECT_LIVE and
VIR_DOMAIN_AFFECT_CONFIG are mutually exclusive. The function simply
returns the correct pointer. This similarly to virDomainObjGetDefs will
greatly simplify the code.

https://bugzilla.redhat.com/show_bug.cgi?id=1220527

This type of information defines attributes of a system
baseboard. With one exception: board type is yet not implemented
in qemu so it's not introduced here either.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This function gets all the PIDs listed in /proc/PID/task. This will be
needed at least to move all qmeu-nbd tasks to the container cgroup.

Move all the system_* fields into a separate struct. Not only this
simplifies the code a bit it also helps us to identify whether BIOS
info is present. We don't have to check all the four variables for
being not-NULL, but we can just check the pointer to the struct.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Move all the bios_* fields into a separate struct. Not only this
simplifies the code a bit it also helps us to identify whether BIOS
info is present. We don't have to check all the four variables for
being not-NULL, but we can just check the pointer to the struct.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Faster version of virJSONValueFromString(virJSONValueToString()).
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

virDomainLiveConfigHelperMethod that is used for this job now does
modify the flags but still requires the callers to extract the correct
definition objects.

In addition coverity and other static analyzers are usually unhappy as
they don't grasp the fact that @flags are upadted according to the
correct def to be present.

To work this issue around and simplify the calling chain let's add a new
helper that will work only on drivers that always copy the persistent
def to a transient at start of a vm. This will allow to drop a few
arguments. The new function syntax will also fill two definition
pointers rather than modifying the @flags parameter.

Since some functions can be optimized by reusing the buffers that they
already have instead of allocating and copying new ones, lets split
virBitmapToData to two functions where one only converts the data and
the second one is a wrapper that allocates the buffer if necessary.

Store the emulator pinning cpu mask as a pure virBitmap rather than the
virDomainPinDef since it stores only the bitmap and refactor
qemuDomainPinEmulator to do the same operations in a much saner way.

As a side effect virDomainEmulatorPinAdd and virDomainEmulatorPinDel can
be removed since they don't add any value.

Sometimes the only thing we need is the pointer to virDomainDiskDef and
having to call virDomainDiskIndexBy* APIs, storing the disk index, and
looking it up in the disks array is ugly. After this patch, we can just
call virDomainDiskBy* and get the pointer in one step.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Two new domain configuration XML elements are added to enable/disable
the protected key management operations for a guest:

    <domain>
      ...
      <keywrap>
        <cipher name='aes|dea' state='on|off'/>
      </keywrap>
      ...
    </domain>
Signed-off-by: NTony Krowiak <akrowiak@linux.vnet.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NDaniel Hansel <daniel.hansel@linux.vnet.ibm.com>
Reviewed-by: NBoris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Because there are multiple potential reasons for an error, this
function logs any errors before returning NULL (since the caller won't
have the information needed to determine which was the reason for
failure).

And move it to qemu_domain.[ch] because this API is QEMU-only.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The APIs take the memory value in KiB and we store it in KiB
internally, but we cannot parse the whole ULONG_MAX range
on 64-bit systems, because virDomainParseScaledValue
needs to fit the value in bytes in an unsigned long long.

https://bugzilla.redhat.com/show_bug.cgi?id=1176739

Add virDomainObjListConvert that will take a list of virDomains, apply
filters and return a list of virDomainObjs.

Until now the virDomainListAllDomains API would lock the domain list and
then every single domain object to access and filter it. This would
potentially allow a unresponsive VM to block the whole daemon if a
*listAllDomains call would get stuck.

To avoid this problem this patch collects a list of referenced domain
objects first from the list and then unlocks it right away. The
expensive operation requiring locking of the domain object is executed
after the list lock is dropped. While a single blocked domain will still
lock up a listAllDomains call, the domain list won't be held locked and
thus other APIs won't be blocked.

Additionally this patch also fixes the lookup code, where we'd ignore
the vm->removing flag and thus potentially return domain objects that
would be deleted very soon so calling any API wouldn't make sense.

As other clients also could benefit from operating on a list of domain
objects rather than the public domain descriptors a new intermediate
API - virDomainObjListCollect - is introduced by this patch.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1181074

Extend it to a universal helper used for clearing lists of any objects.
Note that the argument type is specifically void * to allow implicit
typecasting.

Additionally add a helper that works on non-NULL terminated arrays once
we know the length.

We already check that any auto-assigned bridge device name for a
virtual network (e.g. "virbr1") doesn't conflict with the bridge name
for any existing libvirt network (via virNetworkSetBridgeName() in
conf/network_conf.c).

We also want to check that the name doesn't conflict with any bridge
device created on the host system outside the control of libvirt
(history: possibly due to the ploriferation of references to libvirt's
bridge devices in HOWTO documents all around the web, it is not
uncommon for an admin to manually create a bridge in their host's
system network config and name it "virbrX"). To add such a check to
virNetworkBridgeInUse() (which is called by virNetworkSetBridgeName())
we would have to call virNetDevExists() (from util/virnetdev.c); this
function calls ioctl(SIOCGIFFLAGS), which everyone on the mailing list
agreed should not be done from an XML parsing function in the conf
directory.

To remedy that problem, this patch removes virNetworkSetBridgeName()
from conf/network_conf.c and puts an identically functioning
networkBridgeNameValidate() in network/bridge_driver.c (because it's
reasonable for the bridge driver to call virNetDevExists(), although
we don't do that yet because I wanted this patch to have as close to 0
effect on function as possible).

There are a couple of inevitable changes though:

1) We no longer check the bridge name during
   virNetworkLoadConfig(). Close examination of the code shows that
   this wasn't necessary anyway - the only *correct* way to get XML
   into the config files is via networkDefine(), and networkDefine()
   will always call networkValidate(), which previously called
   virNetworkSetBridgeName() (and now calls
   networkBridgeNameValidate()). This means that the only way the
   bridge name can be unset during virNetworkLoadConfig() is if
   someone edited the config file on disk by hand (which we explicitly
   prohibit).

2) Just on the off chance that somebody *has* edited the file by hand,
   rather than crashing when they try to start their malformed
   network, a check for non-NULL bridge name has been added to
   networkStartNetworkVirtual().

   (For those wondering why I don't instead call
   networkValidateBridgeName() there to set a bridge name if one
   wasn't present - the problem is that during
   networkStartNetworkVirtual(), the lock for the network being
   started has already been acquired, but the lock for the network
   list itself *has not* (because we aren't adding/removing a
   network). But virNetworkBridgeInuse() iterates through *all*
   networks (including this one) and locks each network as it is
   checked for a duplicate entry; it is necessary to lock each network
   even before checking if it is the designated "skip" network because
   otherwise some other thread might acquire the list lock and delete
   the very entry we're examining. In the end, permitting a setting of
   the bridge name during network start would require that we lock the
   entire network list during any networkStartNetwork(), which
   eliminates a *lot* of parallelism that we've worked so hard to
   achieve (it can make a huge difference during libvirtd startup). So
   rather than try to adjust for someone playing against the rules, I
   choose to instead give them the error they deserve.)

3) virNetworkAllocateBridge() (now removed) would leak any "template"
   string set as the bridge name. Its replacement
   networkFindUnusedBridgeName() doesn't leak the template string - it
   is properly freed.

Add qemuDomainAddIOThread and qemuDomainDelIOThread in order to add or
remove an IOThread to/from the host either for live or config optoins

The implementation for the 'live' option will use the iothreadpids list
in order to make decision, while the 'config' option will use the
iothreadids list.  Additionally, for deletion each may have to adjust
the iothreadpin list.

IOThreads are implemented by qmp objects, the code makes use of the existing
qemuMonitorAddObject or qemuMonitorDelObject APIs.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

We're about to allow IOThreads to be deleted, but an iothreadid may be
included in some domain thread sched, so add a new API to allow removing
an iothread from some entry.

Then during the writing of the threadsched data and an additional check
to determine whether the bitmap is all clear before writing it out.

Since it's only ever referenced in domain_conf.c, make the function
static, but also will need to move it to somewhere before it's referenced
rather than forward referencing it.

Adding a new XML element 'iothreadids' in order to allow defining
specific IOThread ID's rather than relying on the algorithm to assign
IOThread ID's starting at 1 and incrementing to iothreads count.

This will allow future patches to be able to add new IOThreads by
a specific iothread_id and of course delete any exisiting IOThread.

Each iothreadids element will have 'n' <iothread> children elements
which will have attribute "id".  The "id" will allow for definition
of any "valid" (eg > 0) iothread_id value.

On input, if any <iothreadids> <iothread>'s are provided, they will
be marked so that we only print out what we read in.

On input, if no <iothreadids> are provided, the PostParse code will
self generate a list of ID's starting at 1 and going to the number
of iothreads defined for the domain (just like the current algorithm
numbering scheme).  A future patch will rework the existing algorithm
to make use of the iothreadids list.

On output, only print out the <iothreadids> if they were read in.

This is basically turning qemuDomObjEndAPI into a more general
function. Other drivers which gets a reference to domain objects may
benefit from this function too.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>