So while working on my previous patches, I've noticed that
virDomainRestore implementation in qemu and test drivers has the
same problem as I am fixing.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

So far we have the following pattern occurring over and over
again:

  if (!vm->persistent)
      qemuDomainRemoveInactive(driver, vm);

It's safe to put the check into the function and save some LoC.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=871452

So, you want to create a domain from XML. The domain already
exists in libvirt's database of domains. It's okay, because name
and UUID matches. However, on domain startup, internal
representation of the domain is overwritten with your XML even
though we claim that the XML you've provided is a transient one.
The bug is to be found across nearly all the drivers.
Le sigh.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=871452

Okay, so we allow users to 'virsh create' an already existing
domain, providing completely different XML than the one stored in
Libvirt. Well, as long as name and UUID matches. However, in some
drivers the code that handles errors unconditionally removes the
domain that failed to start even though the domain might have
been persistent. Fortunately, the domain is removed just from the
internal list of domains and the config file is kept around.

Steps to reproduce:

1) virsh dumpxml $dom > /tmp/dom.xml
2) change XML so that it is still parse-able but won't boot, e.g.
change guest agent path to /foo/bar
3) virsh create /tmp/dom.xml
4) virsh dumpxml $dom
5) Observe "No such domain" error
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

As announced on the list [1], Cherrypop is a management
application based on libvirt. It's a decentralized cloud software
with nice scaling ability.

1: https://www.redhat.com/archives/libvir-list/2015-September/msg00670.htmlSigned-off-by: NMichal Privoznik <mprivozn@redhat.com>

I initially added this in order to keep the code more error-prone to
following additions, but it seems it's still frowned upon.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Now that virQEMUDriverCreateXMLConf is never called with NULL
(after 086f37e9) we can safely drop useless check in
qemuDomainDeviceDefPostParse as we are guaranteed to be always
called with the driver initialized. Therefore checking if driver
is NULL makes no sense. Moreover, if we mix it with direct driver
dereference. And after that, we are sure that nor @cfg will be
NULL, therefore we can drop checks for that too.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Although 0 length block jobs aren't entirely useful, the output of virsh
blockjob is empty due to the condition that suppresses the output for
migration jobs that did not start. Since the only place that actually
uses the condition that suppresses the output is in migration, let's
move the check there and thus add support for 0 of 0 equaling to 100%.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1196711

Qemu unfortunately doesn't update internal state right after migration
and so the actual balloon size as returned by 'query-balloon' are
invalid for a while after the CPUs are started after migration. If we'd
refresh our internal state at this point we would report invalid current
memory size until the next balloon event would arrive.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1242940

My original implementation was based on a qemu version that still did
not have all the checks in place. Using sizes that would align to odd
megabyte increments will produce the following error:

qemu-kvm: -device pc-dimm,node=0,memdev=memdimm0,id=dimm0: backend memory size must be multiple of 0x200000
qemu-kvm: -device pc-dimm,node=0,memdev=memdimm0,id=dimm0: Device 'pc-dimm' could not be initialized

Introduce an alignment retrieval function for memory devices and use it
to align the devices separately and modify a test case to verify it.

After my "client rpc: Report proper error for keepalive disconnections"
patch, virsh would no long print a warning when it closes a connection
to a daemon after a keepalive timeout. Although the warning

    virsh # 2015-09-15 10:59:26.729+0000: 642080: info :
    libvirt version: 1.2.19
    2015-09-15 10:59:26.729+0000: 642080: warning :
    virKeepAliveTimerInternal:143 : No response from client
    0x7efdc0a46730 after 1 keepalive messages in 2 seconds

was pretty ugly, it was still useful. This patch brings the useful part
back while making it much nicer:

virsh # error: Disconnected from qemu:///system due to keepalive timeout
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Even though we hit an error in client's IO loop, we still want to
process any pending data. So instead of reporting the error right away,
we can finish the current iteration and report the error once we're done
with it. Note that the error is stored in client->error by
virNetClientMarkClose so we don't need to worry about it being reset or
rewritten by any API we call in the meantime.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Whenever a connection was closed due to keepalive timeout, we would log
a warning but the interrupted API would return rather useless generic
error:

    internal error: received hangup / error event on socket

Let's report a proper keepalive timeout error and make sure it is
propagated to all pending APIs. The error should be better now:

    internal error: connection closed due to keepalive timeout

Based on an old patch from Martin Kletzander.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Just like e92e5ba1, this attribute was missed.
Signed-off-by: NLuyao Huang <lhuang@redhat.com>

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Build fail and error like this:

  CC       qemu/libvirt_driver_qemu_impl_la-qemu_command.lo
qemu/qemu_capabilities.c:46:27: fatal error: qemu_capspriv.h: No such file or directory
 #include "qemu_capspriv.h"

Add qemu_capspriv.h to source.
Signed-off-by: NLuyao Huang <lhuang@redhat.com>

$ rpmbuild -ba libvirt.spec
warning: Macro expanded in comment on line 5: # If neither fedora nor rhel was defined, try to guess them from %{dist}

warning: Macro %enable_autotools defined but not used within scope
warning: Macro %client_only defined but not used within scope
...

We use the function to create a virDomainXMLOption object that is
required for some functions. However, we don't pass the driver
pointer to the object anywhere - rather than pass NULL. This
causes trouble later when parsing a domain XML and calling post
parse callbacks:

  Program received signal SIGSEGV, Segmentation fault.
  0x000000000043fa3e in qemuDomainDefPostParse (def=0x7d36c0, caps=0x7caf10, opaque=0x0) at qemu/qemu_domain.c:1043
  1043        qemuCaps = virQEMUCapsCacheLookup(driver->qemuCapsCache, def->emulator);
  (gdb) bt
  #0  0x000000000043fa3e in qemuDomainDefPostParse (def=0x7d36c0, caps=0x7caf10, opaque=0x0) at qemu/qemu_domain.c:1043
  #1  0x00007ffff2928bf9 in virDomainDefPostParse (def=0x7d36c0, caps=0x7caf10, xmlopt=0x7c82c0) at conf/domain_conf.c:4269
  #2  0x00007ffff294de04 in virDomainDefParseXML (xml=0x7da8c0, root=0x7dab80, ctxt=0x7da980, caps=0x7caf10, xmlopt=0x7c82c0, flags=0) at conf/domain_conf.c:16400
  #3  0x00007ffff294e5b5 in virDomainDefParseNode (xml=0x7da8c0, root=0x7dab80, caps=0x7caf10, xmlopt=0x7c82c0, flags=0) at conf/domain_conf.c:16582
  #4  0x00007ffff294e424 in virDomainDefParse (xmlStr=0x0, filename=0x7c7ef0 "/home/zippy/work/libvirt/libvirt.git/tests/securityselinuxlabeldata/disks.xml", caps=0x7caf10, xmlopt=0x7c82c0, flags=0) at conf/domain_conf.c:16529
  #5  0x00007ffff294e4b2 in virDomainDefParseFile (filename=0x7c7ef0 "/home/zippy/work/libvirt/libvirt.git/tests/securityselinuxlabeldata/disks.xml", caps=0x7caf10, xmlopt=0x7c82c0, flags=0) at conf/domain_conf.c:16553
  #6  0x00000000004303ca in testSELinuxLoadDef (testname=0x53c929 "disks") at securityselinuxlabeltest.c:192
  #7  0x00000000004309e8 in testSELinuxLabeling (opaque=0x53c929) at securityselinuxlabeltest.c:313
  #8  0x0000000000431207 in virtTestRun (title=0x53c92f "Labelling \"disks\"", body=0x430964 <testSELinuxLabeling>, data=0x53c929) at testutils.c:211
  #9  0x0000000000430c5d in mymain () at securityselinuxlabeltest.c:373
  #10 0x00000000004325c2 in virtTestMain (argc=1, argv=0x7fffffffd7e8, func=0x430b4a <mymain>) at testutils.c:863
  #11 0x0000000000430deb in main (argc=1, argv=0x7fffffffd7e8) at securityselinuxlabeltest.c:381
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Even though usage of the lock is limited to a very few cases,
it's still needed. Therefore we should initialize it too.
Otherwise we may get some random test failures:

==1204== Conditional jump or move depends on uninitialised value(s)
==1204==    at 0xEF7F7CF: pthread_mutex_lock (in /lib64/libpthread-2.20.so)
==1204==    by 0x9CA89A5: virMutexLock (virthread.c:89)
==1204==    by 0x450B2A: qemuDriverLock (qemu_conf.c:83)
==1204==    by 0x45549C: virQEMUDriverGetConfig (qemu_conf.c:869)
==1204==    by 0x448E29: qemuDomainDeviceDefPostParse (qemu_domain.c:1240)
==1204==    by 0x9CC9B13: virDomainDeviceDefPostParse (domain_conf.c:4224)
==1204==    by 0x9CC9B91: virDomainDefPostParseDeviceIterator (domain_conf.c:4251)
==1204==    by 0x9CC7843: virDomainDeviceInfoIterateInternal (domain_conf.c:3440)
==1204==    by 0x9CC9C25: virDomainDefPostParse (domain_conf.c:4276)
==1204==    by 0x9CEEE03: virDomainDefParseXML (domain_conf.c:16400)
==1204==    by 0x9CEF5B4: virDomainDefParseNode (domain_conf.c:16582)
==1204==    by 0x9CEF423: virDomainDefParse (domain_conf.c:16529)
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This reverts commit 69b850fe.

This change broke the ability to "clear" or reset unfiltered back
to filtered.

For some machine types ppc64 machines now require that memory sizes are
aligned to 256MiB increments (due to the dynamically reconfigurable
memory). As now we treat existing configs reasonably in regards to
migration, we can round all the sizes unconditionally. The only drawback
will be that the memory size of a VM can potentially increase by
(256MiB - 1byte) * number_of_NUMA_nodes.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1249006

When we are starting a qemu process for an incomming migration or
snapshot reloading we should not modify the memory sizes in the domain
since we could potentially change the guest ABI that was tediously
checked before. Additionally the function now updates the initial memory
size according to the NUMA node size, which should not happen if we are
restoring state.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1252685

When implementing memory hotplug I've opted to recalculate the initial
memory size (contents of the <memory> element) as a sum of the sizes of
NUMA nodes when NUMA was enabled. This was based on an assumption that
qemu did not allow starting when the NUMA node size total didn't equal
to the initial memory size. Unfortunately the check was introduced to
qemu just lately.

This patch uses the new XML parser flag to decide whether it's safe to
update the memory size total from the NUMA cell sizes or not.

As an additional improvement we now report an error in case when the
size of hotplug memory would exceed the total memory size.

The rest of the changes assures that the function is called with correct
flags.

Add 'initial_memory' member to struct virDomainMemtune so that the
memory size can be pre-calculated once instead of inferring it always
again and again.

Separating of the fields will also allow finer granularity of decisions
in later patches where it will allow to keep the old initial memory
value in cases where we are handling incomming migration from older
versions that did not always update the size from NUMA as the code did
previously.

The change also requires modification of the qemu memory alignment
function since at the point where we are modifying the size of NUMA
nodes the total size needs to be recalculated too.

The refactoring done in this patch also fixes a crash in the hyperv
driver that did not properly initialize def->numa and thus
virDomainNumaGetMemorySize(def->numa) crashed.

In summary this patch should have no functional impact at this point.

The name of the variable was misleading. Rename it and it's setting
accessor before other fixes.

The post parse func is growing rather large. Since later patches will
introduce more logic in the memory post parse code, split it into a
separate handler.

Add a new parser flag that will mark code paths that parse XML files
wich will not be used with existing VM state so that post parse
callbacks can possibly do ABI incompatible changes if needed.

The flag was used only for formatting the XML and once the parser and
formatter flags were split in 0ecd6851
it doesn't make sense any more to have it.

Extract the size determination into a separate function and reuse it
across the memory device alignment functions. Since later we will need
to decide the alignment size according to architecture let's pass def to
the functions.

Add a simple helper so that the code doesn't have to rewrite the same
condition multiple times.

Commits 45697fe5 and f863ac80 used direct access to the variable instead
of the preferred accessor method.

Since test suite now correctly creates capabilities cache, the hack is not
needed any more.
Signed-off-by: NPavel Fedin <p.fedin@samsung.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Use the new API in order to correctly add capability sets to the cache
before parsing XML files
Signed-off-by: NPavel Fedin <p.fedin@samsung.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

The main purpose of this patch is to introduce test mode to
virQEMUCapsCacheLookup(). This is done by adding a global variable, which
effectively overrides binary name. This variable is supposed to be set by
test suite.

The second addition is qemuTestCapsCacheInsert() function which allows the
test suite to actually populate the cache.
Signed-off-by: NPavel Fedin <p.fedin@samsung.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Two utility functions are introduced for proper initialization and
cleanup of the driver.
Signed-off-by: NPavel Fedin <p.fedin@samsung.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

 Invalid read of size 4
    at 0x945CA30: __pthread_mutex_unlock_full (in /lib64/libpthread-2.20.so)
    by 0x4F0404B: virMutexUnlock (virthread.c:94)
    by 0x4F7161B: virStoragePoolObjUnlock (storage_conf.c:2603)
    by 0x4FE0476: testStoragePoolUndefine (test_driver.c:4328)
    by 0x4FCF086: virStoragePoolUndefine (libvirt-storage.c:656)
    by 0x15A7F5: cmdPoolUndefine (virsh-pool.c:1721)
    by 0x12F48D: vshCommandRun (vsh.c:1212)
    by 0x132AA7: main (virsh.c:943)
  Address 0xfda56a0 is 16 bytes inside a block of size 104 free'd
    at 0x4C2BA6C: free (vg_replace_malloc.c:473)
    by 0x4EA5C96: virFree (viralloc.c:582)
    by 0x4F70B69: virStoragePoolObjFree (storage_conf.c:412)
    by 0x4F7167B: virStoragePoolObjRemove (storage_conf.c:437)
    by 0x4FE0468: testStoragePoolUndefine (test_driver.c:4323)
    by 0x4FCF086: virStoragePoolUndefine (libvirt-storage.c:656)
    by 0x15A7F5: cmdPoolUndefine (virsh-pool.c:1721)
    by 0x12F48D: vshCommandRun (vsh.c:1212)
    by 0x132AA7: main (virsh.c:943)

Rather than inlining the code, use the common API.

Rather than inlining the code, use the common API.