Commit 973fcd8f introduced the ability for qemu to reject snapshot
reversion on an ABI incompatibility; but the very example that was
first proposed on-list[1] as a demonstration of an ABI incompatibility,
namely that of changing the max memory allocation, was not being
checked for, resulting in a cryptic failure when running with larger
max mem than what the snapshot was created with:
error: operation failed: Error -22 while loading VM state

This commit merely protects the three variables within mem that are
referenced by qemu_command.c, rather than all 7 (the other 4 variables
affect cgroup handling, but as far as I can tell, have no visible effect
to the qemu guest).  This also affects migration and save file handling,
which are other places where we perform ABI compatibility checks.

[1] https://www.redhat.com/archives/libvir-list/2010-December/msg00331.html

* src/conf/domain_conf.c (virDomainDefCheckABIStability): Add
memory sizing checks.

Commit 498d7833 cleans up some of virtual file names for parsing strings
in memory. This patch cleans up (hopefuly) the rest forgotten by the
first patch.

This patch also changes all of the previously modified "filenames" to
valid URI's replacing spaces for underscores.

Changes to v1:
- Replace all spaces for underscores, so that the strings form valid
  URI's
- Replace spaces in places changed by commit 498d7833

And fix argument @pid's type of virDomainQemuAttach.

Regression introduced in commit 3881a470, due to an improper rebase
of a cleanup written beforehand but only applied after a rebased of
a refactoring that created a new function in commit 25fb3ef1.

Also avoids passing NULL to printf %s.

* src/qemu/qemu_driver.c: In qemuDomainSnapshotForEachQcow2()
it free up the memory of qemu_driver->qemuImgBinary in the
cleanup tag which leads to the garbage value of qemuImgBinary
in qemu_driver struct and libvirtd crash when running
"virsh snapshot-create" command a second time.
Signed-off-by: NEric Blake <eblake@redhat.com>

So we can allow GSSAPI authentication for ssh.
Signed-off-by: NMatthias Witte <witte@netzquadrat.de>

If we fail setting label on a file and this file is on NFS share,
it is wise to advise user to set virt_use_nfs selinux boolean
variable.

'+' in strings get translated to ' ' when editing domains.
While xenDaemonDomainCreateXML() did URL-escape the sexpr,
xenDaemonDomainDefineXML() did not.

Remove the explicit urlencode() in xenDaemonDomainCreateXML() and add
the direct encoding calls to xend_op_ext() because it calls xend_post()
which uses "Content-Type: application/x-www-form-urlencoded". According
to <http://www.w3.org/TR/html4/interact/forms.html#h-17.13.4.1> this
requires all parameters to be url-encoded as specified in rfc1738.

Notice: virBufferAsprintf(..., "%s=%s", ...) is again replaced by three
calls to virBufferURIEncodeString() and virBufferAddChar() because '='
is a "reserved" character, which would get escaped by
virBufferURIEncodeString(), which - by the way - escapes anything not
c_isalnum().
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

While parsing XML strings from memory, the previous convention in
libvirt was to set the virtual file name to "domain.xml" or something
similar. This could potentialy trick the user into looking for a file
named domain.xml on the disk in an attempt to fix the error.

This patch changes these filenames to something that can't be as easily
confused for a valid filename.

Examples of error messages:
---------------------------
Error while loading file from disk:

15:07:59.015: 527: error : catchXMLError:709 : /path/to/domain.xml:1: StartTag: invalid element name
<domain type='kvm'><
--------------------^

Error while parsing definition in memory:

15:08:43.581: 525: error : catchXMLError:709 : (domain definition):2: error parsing attribute name
  <name>vm1</name>
--^

Regression introduced in commit d6f6b2d1.  Running
'virsh snapshot-create dom' would mistakenly report that
disks can only be specified for disk snapshots.

* src/conf/domain_conf.c (virDomainSnapshotDefParseString): Only
give error about no disk support when <disk> was found.

These functions access internals of the opaque object, and do
not need any rpc counterpart.  It could be argued that we should
have provided these when snapshot objects were first introduced,
since all the other vir*Ptr objects have at least a GetName accessor.

* include/libvirt/libvirt.h.in (virDomainSnapshotGetName)
(virDomainSnapshotGetDomain, virDomainSnapshotGetConnect): Declare.
* src/libvirt.c (virDomainSnapshotGetName)
(virDomainSnapshotGetDomain, virDomainSnapshotGetConnect): New
functions.
* src/libvirt_public.syms: Export them.

Xen PV domU's have no PCI bus. node_device_udev.c calls pci_system_init
which looks for /sys/bus/pci. If it does not find /sys/bus/pci (which it
won't in a Xen PV domU) it returns unsuccesfully (ENOENT), which libvirt
considers fatal. This makes libvirt unusable in this environment, even
though there are plenty of valid virtualisation options that work
there (LXC, UML, and QEmu spring to mind)

https://bugzilla.redhat.com/show_bug.cgi?id=709471Signed-off-by: NSoren Hansen <soren@linux2go.dk>

* src/rpc/virnettlscontext.c: fix memory leak on
  virNetTLSContextValidCertificate.

* Detected in valgrind run:

==25667==
==25667== 6,085 (44 direct, 6,041 indirect) bytes in 1 blocks are definitely
lost in loss record 326 of 351
==25667==    at 0x4005447: calloc (vg_replace_malloc.c:467)
==25667==    by 0x4F2791F3: _asn1_add_node_only (structure.c:53)
==25667==    by 0x4F27997A: _asn1_copy_structure3 (structure.c:421)
==25667==    by 0x4F276A50: _asn1_append_sequence_set (element.c:144)
==25667==    by 0x4F2743FF: asn1_der_decoding (decoding.c:1194)
==25667==    by 0x4F22B9CC: gnutls_x509_crt_import (x509.c:229)
==25667==    by 0x805274B: virNetTLSContextCheckCertificate
(virnettlscontext.c:1009)
==25667==    by 0x804DE32: testTLSSessionInit (virnettlscontexttest.c:693)
==25667==    by 0x804F14D: virtTestRun (testutils.c:140)
==25667==
==25667== 23,188 (88 direct, 23,100 indirect) bytes in 11 blocks are definitely
lost in loss record 346 of 351
==25667==    at 0x4005447: calloc (vg_replace_malloc.c:467)
==25667==    by 0x4F22B841: gnutls_x509_crt_init (x509.c:50)
==25667==    by 0x805272B: virNetTLSContextCheckCertificate
(virnettlscontext.c:1003)
==25667==    by 0x804DDD1: testTLSSessionInit (virnettlscontexttest.c:673)
==25667==    by 0x804F14D: virtTestRun (testutils.c:140)

* How to reproduce?
% cd libvirt && ./configure && make && make -C tests valgrind
or
% valgrind -v --leak-check=full ./tests/virnettlscontexttest
Signed-off-by: NAlex Jia <ajia@redhat.com>

Variable 'l_disk' initialized to a null pointer value, control jumps to 'case
VIR_DOMAIN_DISK_DEVICE_DISK and then taking false branch, Within the expansion
of the macro 'libxlError': Field access results in a dereference of a null
pointer (loaded from variable 'l_disk').

* src/libxl/libxl_driver.c: Field access results in a dereference of a null
  pointer (loaded from variable 'l_disk')
Signed-off-by: NAlex Jia <ajia@redhat.com>

Regression introduced in commit 89b6284f, due to an incorrect
conversion to the new means of converting disk names back to
the correct object.

* src/qemu/qemu_driver.c (qemuDomainGetBlockInfo): Avoid NULL deref.

Exit early if managed object is not found, instead of dereferencing
a NULL pointer and triggering a segfault.

Although we were initializing worker threads during pool creating,
we missed this during virThreadPoolSendJob. This bug led to segmenation
fault as worker thread free() given argument.

This patch enables modifying network device configuration using the
virUpdateDeviceFlags API method. Matching of devices is accomplished
using MAC addresses.

While updating live configuration of a running domain, the user is
allowed only to change link state of the interface. Additional
modifications may be added later. For now the code checks for
unsupported changes and thereafter changes the link state, if
applicable.

When updating persistent configuration of guest's network interface the
whole configuration (except for the MAC address) may be modified and
is stored for the next startup.

* src/qemu/qemu_driver.c   - Add dispatching of virUpdateDevice for
                             network devices update (live/config)
* src/qemu/qemu_hotplug.c  - add setting of initial link state on live
                             device addition
                           - add function to change network device
                             configuration. By now it supports only
                             changing of link state
* src/qemu/qemu_hotplug.h  - Headers to above functions
* src/qemu/qemu_process.c  - set link states before virtual machine
                             start. Qemu does not support setting of
                             this on the command line.

This patch adds handlers for modification of guest's interface
link state. Both HMP and QMP commands are supported, but as the
link state functionality is from the beginning supported in QMP
the HMP code will probably never be used.

A new element is introduced to XML that allows to control
state of virtual network interfaces in hypervisors.

Live modification of the link state allows networking tools
propagate topology changes to guest OS or testing of
scenarios in complex (virtual) networks.

This patch adds elements to XML grammars and parsing and generating
code.

This patch adds functions to compare structures containing network
device configuration for equality. They serve for the purpose of
disallowing unsupported changes to live network devices.

This patch modifies error handling function for the XML parser provided
by libxml2.

Originaly only a line number and error message were logged. With this
new error handler function, the user is provided with a more complex
description of the parsing error.

Context of the error is printed in libXML2 style and filename of the
file, that caused the error is printed. Example of an parse error:

13:41:36.262: 16032: error : catchXMLError:706 :
/etc/libvirt/qemu/rh_bad.xml:58: Opening and ending tag mismatch: name
line 2 and domain
</domain>
---------^

Context of the error gives the user hints that may help to quickly
locate a corrupt xml file.

fixes BZs:
----------
Bug 708735 - [RFE] Show column and line on XML parsing error
https://bugzilla.redhat.com/show_bug.cgi?id=708735

Bug 726771 - libvirt does not specify problem file if persistent xml is
invalid
https://bugzilla.redhat.com/show_bug.cgi?id=726771

It is important to be able to attach USB redirected devices to a
particular controller (one that supports USB2 for instance).
Without this patch, only the default bus was used.

     <redirdev bus='usb' type='spicevmc'>
       <address type='usb' bus='0' port='4'/>
     </redirdev>

The mainly changes are:

1) Update qemuMonitorGetBlockStatsInfo and it's children (Text/JSON)
   functions to return the value of new latency fields.
2) Add new function qemuMonitorGetBlockStatsParamsNumber, which is
   to count how many parameters the underlying QEMU supports.
3) Update virDomainBlockStats in src/qemu/qemu_driver.c to be
   compatible with the changes by 1).

If libvirt daemon gets restarted and there is (at least) one
unresponsive qemu, the startup procedure hangs up. This patch creates
one thread per vm in which we try to reconnect to monitor. Therefore,
blocking in one thread will not affect other APIs.

This patch creates an optional BeginJob queue size limit. When
active, all other attempts above level will fail. To set this
feature assign desired value to max_queued variable in qemu.conf.
Setting it to 0 turns it off.

This patch annotates APIs with low or high priority.
In low set MUST be all APIs which might eventually access monitor
(and thus block indefinitely). Other APIs may be marked as high
priority. However, some must be (e.g. domainDestroy).

For high priority calls (HPC), there are some high priority workers
(HPW) created in the pool. HPW can execute only HPC, although normal
worker can process any call regardless priority. Therefore, only those
APIs which are guaranteed to end in reasonable small amount of time
can be marked as HPC.

The size of this HPC pool is static, because HPC are expected to end
quickly, therefore jobs assigned to this pool will be served quickly.
It can be configured in libvirtd.conf via prio_workers variable.
Default is set to 5.

To mark API with low or high priority, append priority:{low|high} to
it's comment in src/remote/remote_protocol.x. This is similar to
autogen|skipgen. If not marked, the generator assumes low as default.

With this, it is now possible to create external snapshots even
when SELinux is enforcing, and to protect the new file with a
lock manager.

* src/qemu/qemu_driver.c
(qemuDomainSnapshotCreateSingleDiskActive): Create and register
new file with proper permissions and locks.
(qemuDomainSnapshotCreateDiskActive): Update caller.

Lots of earlier patches led up to this point - the qemu snapshot_blkdev
monitor command can now be controlled by libvirt!  Well, insofar as
SELinux doesn't prevent qemu from open(O_CREAT) on the files.  There's
still some followup work before things work with SELinux enforcing,
but this patch is big enough to post now.

There's still room for other improvements, too (for example, taking a
disk snapshot of an inactive domain, by using qemu-img for both internal
and external snapshots; wiring up delete and revert control, including
additional flags from my RFC; supporting active QED disk snapshots;
supporting per-storage-volume snapshots such as LVM or btrfs snapshots;
etc.).  But this patch is the one that proves the new XML works!

* src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Wire in
active disk snapshots.
(qemuDomainSnapshotDiskPrepare)
(qemuDomainSnapshotCreateDiskActive)
(qemuDomainSnapshotCreateSingleDiskActive): New functions.

No one uses this yet, but it will be important once
virDomainSnapshotCreateXML learns a VIR_DOMAIN_SNAPSHOT_DISK_ONLY
flag, and the xml allows passing in the new file names.

* src/qemu/qemu_monitor.h (qemuMonitorDiskSnapshot): New prototype.
* src/qemu/qemu_monitor_text.h (qemuMonitorTextDiskSnapshot):
Likewise.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONDiskSnapshot):
Likewise.
* src/qemu/qemu_monitor.c (qemuMonitorDiskSnapshot): New
function.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONDiskSnapshot):
Likewise.

Snapshots alter the set of disk image files opened by qemu, so
they must be audited.  But they don't involve a full disk definition
structure, just the new filename.  Make the next patch easier by
refactoring the audit routines to just operate on file name.

* src/conf/domain_audit.h (virDomainAuditDisk): Update prototype.
* src/conf/domain_audit.c (virDomainAuditDisk): Act on strings,
not definition structures.
(virDomainAuditStart): Update caller.
* src/qemu/qemu_hotplug.c (qemuDomainChangeEjectableMedia)
(qemuDomainAttachPciDiskDevice, qemuDomainAttachSCSIDisk)
(qemuDomainAttachUsbMassstorageDevice)
(qemuDomainDetachPciDiskDevice, qemuDomainDetachDiskDevice):
Likewise.

My RFC for snapshot support [1] proposes several rules for when it is
safe to delete or revert to an external snapshot, predicated on
the existence of new API flags.  These will be incrementally added
in future patches, but until then, blindly mishandling a disk
snapshot risks corrupting internal state, so it is better to
outright reject the attempts until the other pieces are in place,
thus incrementally relaxing the restrictions added in this patch.

[1] https://www.redhat.com/archives/libvir-list/2011-August/msg00361.html

* src/qemu/qemu_driver.c (qemuDomainSnapshotCountExternal): New
function.
(qemuDomainUndefineFlags, qemuDomainSnapshotDelete): Use it to add
safety valve.
(qemuDomainRevertToSnapshot, qemuDomainSnapshotCreateXML): Add safety
valve.

Prior to this patch, <domainsnapshot>/<disks> was ignored.  This
changes it to be an error unless an explicit disk snapshot is
requested (a future patch may relax things if it turns out to
be useful to have a <disks> specification alongside a system
checkpoint).

* include/libvirt/libvirt.h.in
(VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY): New flag.
* src/libvirt.c (virDomainSnapshotCreateXML): Document it.
* src/esx/esx_driver.c (esxDomainSnapshotCreateXML): Disk
snapshots not supported yet.
* src/vbox/vbox_tmpl.c (vboxDomainSnapshotCreateXML): Likewise.
* src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Likewise.

I got confused when 'virsh domblkinfo dom disk' required the
path to a disk (which can be ambiguous, since a single file
can back multiple disks), rather than the unambiguous target
device name that I was using in disk snapshots.  So, in true
developer fashion, I went for the best of both worlds - all
interfaces that operate on a disk (aka block) now accept
either the target name or the unambiguous path to the backing
file used by the disk.

* src/conf/domain_conf.h (virDomainDiskIndexByName): Add
parameter.
(virDomainDiskPathByName): New prototype.
* src/libvirt_private.syms (domain_conf.h): Export it.
* src/conf/domain_conf.c (virDomainDiskIndexByName): Also allow
searching by path, and decide whether ambiguity is okay.
(virDomainDiskPathByName): New function.
(virDomainDiskRemoveByName, virDomainSnapshotAlignDisks): Update
callers.
* src/qemu/qemu_driver.c (qemudDomainBlockPeek)
(qemuDomainAttachDeviceConfig, qemuDomainUpdateDeviceConfig)
(qemuDomainGetBlockInfo, qemuDiskPathToAlias): Likewise.
* src/qemu/qemu_process.c (qemuProcessFindDomainDiskByPath):
Likewise.
* src/libxl/libxl_driver.c (libxlDomainAttachDeviceDiskLive)
(libxlDomainDetachDeviceDiskLive, libxlDomainAttachDeviceConfig)
(libxlDomainUpdateDeviceConfig): Likewise.
* src/uml/uml_driver.c (umlDomainBlockPeek): Likewise.
* src/xen/xend_internal.c (xenDaemonDomainBlockPeek): Likewise.
* docs/formatsnapshot.html.in: Update documentation.
* tools/virsh.pod (domblkstat, domblkinfo): Likewise.
* docs/schemas/domaincommon.rng (diskTarget): Tighten pattern on
disk targets.
* docs/schemas/domainsnapshot.rng (disksnapshot): Update to match.
* tests/domainsnapshotxml2xmlin/disk_snapshot.xml: Update test.

Adds an optional element to <domainsnapshot>, which will be used
to give user control over external snapshot filenames on input,
and specify generated filenames on output.

For now, no driver accepts this element; that will come later.

<domainsnapshot>
  ...
  <disks>
    <disk name='vda' snapshot='no'/>
    <disk name='vdb' snapshot='internal'/>
    <disk name='vdc' snapshot='external'>
      <driver type='qcow2'/>
      <source file='/path/to/new'/>
    </disk>
  </disks>
  <domain>
    ...
    <devices>
      <disk ...>
        <driver name='qemu' type='raw'/>
        <target dev='vdc'/>
        <source file='/path/to/old'/>
      </disk>
    </devices>
  </domain>
</domainsnapshot>

* src/conf/domain_conf.h (_virDomainSnapshotDiskDef): New type.
(_virDomainSnapshotDef): Add new elements.
(virDomainSnapshotAlignDisks): New prototype.
* src/conf/domain_conf.c (virDomainSnapshotDiskDefClear)
(virDomainSnapshotDiskDefParseXML, disksorter)
(virDomainSnapshotAlignDisks): New functions.
(virDomainSnapshotDefParseString): Parse new fields.
(virDomainSnapshotDefFree): Clean them up.
(virDomainSnapshotDefFormat): Output them.
* src/libvirt_private.syms (domain_conf.h): Export new function.
* docs/schemas/domainsnapshot.rng (domainsnapshot, disksnapshot):
Add more xml.
* docs/formatsnapshot.html.in: Document it.
* tests/domainsnapshotxml2xmlin/disk_snapshot.xml: New test.
* tests/domainsnapshotxml2xmlout/disk_snapshot.xml: Update.