Though we prefer users to have SSH keys setup, virt-manager users still
depend on remote SSH connections to launch a password dialog. This fixes
launch ssh-askpass

Fix suggested by danpb

DMI table is Intel & Intel-compatible specific. Therefore other
architectures miss dmidecode command. So we always fail in searching
for that command on non-Intel architectures.

* src/rpc/virnetclient.c: Skip close attempt if virNetClientPtr
  is NULL

If a key purpose or usage field is marked as non-critical in the
certificate, then a data mismatch is not (ordinarily) a cause for
rejecting the connection

* src/rpc/virnettlscontext.c: Honour key usage/purpose criticality

If key usage or purpose data is not present in the cert, the
RFC recommends that access be allowed. Also fix checking of
key usage to include requirements for client/server certs,
and fix key purpose checking to treat data as a list of bits

* src/rpc/virnettlscontext.c: Fix mixed up error messages

DMI is Intel & Intel-compatible specific. Don't try to dump information
on non-compatible architectures, which results only in error message in
logs.

* src/libxl/libxl_driver.c (libxlDomainUndefineFlags): Use correct
enum value.
* src/remote_protocol-structs (remote_procedure): Likewise.

If the domain has managed save image, and --managed-save is
not specified, then it fails with an error telling the user
that a managed save image still exists.

If the domain has managed save image, and --managed-save is
specified, it invokes virDomainUndefineFlags. If
virDomainUndefineFlags fails, then it tries to remove the managed
save image using virDomainManagedSaveRemove first, with
invoking virDomainUndefine following. (For compatibility between
new virsh with this patch and older libvirt without this patch).

Similarly if the domain has no managed save image. See the codes for
detail.

NOTE: Have not removing the codes checking if the domain is running
in function "cmdUndefine", it will go along with qemu driver's fix
(allow to undefine a running domain).

* src/libxl/libxl_driver.c: New callback for libxl_driver,
new function libxlDomainUndefineFlags, and changes libxlDomainUndefine
as a wrapper of libxlDomainUndefineFlags.

* src/qemu/qemu_driver.c: New call back for qemu_driver,
New function qemudDomainUndefineFlags, and changes on
qemudDomainUndefine.

This introduces a new API virDomainUndefineFlags to control the
domain undefine process, as the existing API virDomainUndefine
doesn't support flags.

Currently only flag VIR_DOMAIN_UNDEFINE_MANAGED_SAVE is supported.
If the domain has a managed save image, including
VIR_DOMAIN_UNDEFINE_MANAGED_SAVE in @flags will also remove that
file, and omitting the flag will cause undefine process to fail.

This patch also changes the behavior of virDomainUndefine, if the
domain has a managed save image, the undefine will be refused.

* cfg.mk (sc_copyright_format): Add to rule.
* src/util/files.h: Fix offenders.
* src/util/files.c: Likewise.

Gnutls requires that certificates have basic constraints present
to be used as a CA certificate. OpenSSL doesn't add this data
by default, so add a sanity check to catch this situation. Also
validate that the key usage and key purpose constraints contain
correct data

* src/rpc/virnettlscontext.c: Add sanity checking of certificate
  constraints

If the libvirt daemon or libvirt client is configured with bogus
certificates, it is very unhelpful to only find out about this
when a TLS connection is actually attempted. Not least because
the error messages you get back for failures are incredibly
obscure.

This adds some basic sanity checking of certificates at the
time the virNetTLSContext object is created. This is at libvirt
startup, or when creating a virNetClient instance.

This checks that the certificate expiry/start dates are valid
and that the certificate is actually signed by the CA that is
loaded.

* src/rpc/virnettlscontext.c: Add certificate sanity checks

Starting/ending jobs when closing the connection may reset any
error which was reported earlier in p2p migration. We must
save the original error before doing so. This means we can also
just call virConnectClose as normal, instead of virUnrefConnect

* src/qemu/qemu_migration.c: Preserve errors in p2p migration

Since the I/O callback registered against virNetSocket will
hold a reference on the virNetClient, we can't rely on the
virNetClientFree to be able to close the network connection.
The last reference will only go away when the event callback
fires (likely due to EOF from the server).

This is sub-optimal and can potentially cause a leak of the
virNetClient object if the server were to not explicitly
close the socket itself

* src/remote/remote_driver.c: Explicitly close the client
  object when disconnecting
* src/rpc/virnetclient.c, src/rpc/virnetclient.h: Add a
  virNetClientClose method

When unregistering an I/O callback from a virNetSocket object,
there is still a chance that an event may come in on the callback.
In this case it is possible that the virNetSocket might have been
freed already. Make use of a virFreeCallback when registering
the I/O callbacks and hold a reference for the entire time the
callback is set.

* src/rpc/virnetsocket.c: Register a free function for the
  file handle watch
* src/rpc/virnetsocket.h, src/rpc/virnetserverservice.c,
  src/rpc/virnetserverclient.c, src/rpc/virnetclient.c: Add
  a free function for the socket I/O watches

Remove the need for a virNetSocket object to be protected by
locks from the object using it, by introducing its own native
locking and reference counting

* src/rpc/virnetsocket.c: Add locking & reference counting

* src/rpc/virnetclient.c: Add debugging of ref counts

Commit 8665f855 introduced a slight regression in doc generation,
since make only quits a rule on the first failed command ending
with a newline rather than a semicolon.

* docs/Makefile.am (html/index.html): Don't use xmllint unless
xsltproc succeeded.
* .gitignore: Ignore recently updated stamp file name.

* src/util/event_poll.c: avoid file descriptors leak when
  virEventPollAddHandle fail on virEventPollInit function.

This reverts commit 40143fb6.

The patch prevents future compatibility if migration ever learns
how to do tunnelled without p2p.

If we get an I/O error in the async event callback for an RPC
client, we might not have consumed all pending data off the
wire. This could result in the callback being immediately
invoked again. At which point the same I/O might occur. And
we're invoked again. And again...And again...

Unregistering the async event callback if an error occurs is
a good safety net. The real error will be seen when the next
RPC method is invoked

* src/rpc/virnetclient.c: Unregister event callback on error

The current API build scripts will continue and exit with a zero
status even if they find problems. This has been the cause of many
build problems, or hidden build errors, in the past. Change the
scripts so they always exit with a non-zero status for any problems
they do not understand. Also turn off all debug output by default
so they respect $(AM_V_GEN)

* docs/Makefile.am: Use $(AM_V_GEN) for API/HTML scripts
* docs/apibuild.py, python/generator.py: Exit with non-zero status
  if problems are found. Also be silent, not outputting any debug
  messages.
* src/Makefile.am: Use $(AM_V_GEN) for ESX generator
* python/Makefile.am: Tweak rule

Caused by the new changed "dominfo" command.

* tools/virsh.c: new column "Managed save" for "cmdDominfo".
* tools/virsh.pod: Update document of "managedsave" to tell one can
  use "dominfo" to query whether a domain has any managed save image.

The problems:
    * Duplicate documents for "iface-name"
    * Lacks of document for "iface-mac"
    * Inconsistent option names with virsh help strings.

We can make the virsh migrate UI friendlier by supplying the
missing bit automatically instead of erroring out when requesting
--tunnelled without --p2p.

* tools/virsh.c (doMigrate): Make --p2p optional when using
--tunnelled.
* tools/virsh.pod (migrate): Tweak wording accordingly.

There were two API in driver.c that were silently masking flags
bits prior to calling out to the drivers, and several others
that were explicitly masking flags bits.  This is not
forward-compatible - if we ever have that many flags in the
future, then talking to an old server that masks out the
flags would be indistinguishable from talking to a new server
that can honor the flag.  In general, libvirt.c should forward
_all_ flags on to drivers, and only the drivers should reject
unknown flags.

In the case of virDrvSecretGetValue, the solution is to separate
the internal driver callback function to have two parameters
instead of one, with only one parameter affected by the public
API.  In the case of virDomainGetXMLDesc, it turns out that
no one was ever mixing VIR_DOMAIN_XML_INTERNAL_STATUS with
the dumpxml path in the first place; that internal flag was
only used in saving and restoring state files, which happened
to be in functions internal to a single file, so there is no
mixing of the internal flag with a public flags argument.
Additionally, virDomainMemoryStats passed a flags argument
over RPC, but not to the driver.

* src/driver.h (VIR_DOMAIN_XML_FLAGS_MASK)
(VIR_SECRET_GET_VALUE_FLAGS_MASK): Delete.
(virDrvSecretGetValue): Separate out internal flags.
(virDrvDomainMemoryStats): Provide missing flags argument.
* src/driver.c (verify): Drop unused check.
* src/conf/domain_conf.h (virDomainObjParseFile): Delete
declaration.
(virDomainXMLInternalFlags): Move...
* src/conf/domain_conf.c: ...here.  Delete redundant include.
(virDomainObjParseFile): Make static.
* src/libvirt.c (virDomainGetXMLDesc, virSecretGetValue): Update
clients.
(virDomainMemoryPeek, virInterfaceGetXMLDesc)
(virDomainMemoryStats, virDomainBlockPeek, virNetworkGetXMLDesc)
(virStoragePoolGetXMLDesc, virStorageVolGetXMLDesc)
(virNodeNumOfDevices, virNodeListDevices, virNWFilterGetXMLDesc):
Don't mask unknown flags.
* src/interface/netcf_driver.c (interfaceGetXMLDesc): Reject
unknown flags.
* src/secret/secret_driver.c (secretGetValue): Update clients.
* src/remote/remote_driver.c (remoteSecretGetValue)
(remoteDomainMemoryStats): Likewise.
* src/qemu/qemu_process.c (qemuProcessGetVolumeQcowPassphrase):
Likewise.
* src/qemu/qemu_driver.c (qemudDomainMemoryStats): Likewise.
* daemon/remote.c (remoteDispatchDomainMemoryStats): Likewise.

* tools/virsh.c: avoid double free of domain, when weight value of blkiotune
  less than 0, codes will free domain and jump to cleanup section, however,
  cleanup will free domain again.

Detected in valgrind run:

==21297== ERROR SUMMARY: 20 errors from 20 contexts (suppressed: 69 from 8)
==21297==
==21297== 1 errors in context 1 of 20:
==21297== Invalid read of size 4
==21297==    at 0x40E209B: virDomainFree (libvirt.c:2096)
==21297==    by 0x8065274: cmdBlkiotune (virsh.c:3695)
==21297==    by 0x8054CC1: vshCommandRun (virsh.c:13135)
==21297==    by 0x806B967: main (virsh.c:14487)
==21297==  Address 0x446ad48 is 0 bytes inside a block of size 36 free'd
==21297==    at 0x4005B0A: free (vg_replace_malloc.c:325)
==21297==    by 0x406814D: virFree (memory.c:310)
==21297==    by 0x40D6635: virReleaseDomain (datatypes.c:243)
==21297==    by 0x40D6C5E: virUnrefDomain (datatypes.c:280)
==21297==    by 0x40E20B9: virDomainFree (libvirt.c:2101)
==21297==    by 0x8065297: cmdBlkiotune (virsh.c:3613)
==21297==    by 0x8054CC1: vshCommandRun (virsh.c:13135)
==21297==    by 0x806B967: main (virsh.c:14487)
==21297==
==21297==
==21297== 1 errors in context 2 of 20:
==21297== Invalid read of size 4
==21297==    at 0x40E1FE6: virDomainFree (libvirt.c:2092)
==21297==    by 0x8065274: cmdBlkiotune (virsh.c:3695)
==21297==    by 0x8054CC1: vshCommandRun (virsh.c:13135)
==21297==    by 0x806B967: main (virsh.c:14487)
==21297==  Address 0x446ad48 is 0 bytes inside a block of size 36 free'd
==21297==    at 0x4005B0A: free (vg_replace_malloc.c:325)
==21297==    by 0x406814D: virFree (memory.c:310)
==21297==    by 0x40D6635: virReleaseDomain (datatypes.c:243)
==21297==    by 0x40D6C5E: virUnrefDomain (datatypes.c:280)
==21297==    by 0x40E20B9: virDomainFree (libvirt.c:2101)
==21297==    by 0x8065297: cmdBlkiotune (virsh.c:3613)
==21297==    by 0x8054CC1: vshCommandRun (virsh.c:13135)
==21297==    by 0x806B967: main (virsh.c:14487)

* how to reproduce?
  % valgrind -v --leak-check=full virsh blkiotune guestname --weight -1

"optional" is not a very good meta-syntactic construct in our man
page.  I scrubbed this, and additionally improved some documentation
on mutually exclusive options.  For example,

[[--live] [--config] | [--current]]

implies a set of optional flags, where within the set you can have
either --current or a choice of 0, 1, or both --live and --config.

* tools/virsh.pod: Use "[name]" rather than "optional name" for
optional arguments.

$@ already included $(srcdir)

When libvirtd restarts it will attempt to reconnect to existing
LXC containers. If it loads a XML state file for the container
the container will appear running. If we fail to read the PID
file, or fail to connect to the LXC monitor, we should be killing
off the guest, but if the VMs cgroup does not exist any more,
cleanup will get skipped. Reading the PID file is also pointless
since the PID is in the XML statefile

In lxcReconnectVM we do not need to read the PID file. If part
of the reconnect process fails we need to run the VM terminate
code as a safety net.

In lxcVMTerminate, if we can't obtain the VM cgroup, we know
the process has died, but we must still run lxcVMCleanup to
clear out the virDomainObjPtr live state

* src/lxc/lxc_driver.c: Fix cleanup of dead VMs on restart

We missed ending tag for paragraph element

These typos are introduced by file renaming in commit b17b4afa.

src/remote/qemu_protocol.x \
src/remote/remote_protocol.x \
src/rpc/gendispatch.pl:
    s/remote_generator/gendispatch/

src/rpc/genprotocol.pl:
    s/remote\/remote_protocol/remote_protocol/

The regression is introduced by Commit da1eba6b, the new
codes with this commit doesn't reset "ret" to "-1" when
it fails on parsing the device XML (live device attachment)

This patch changes the codes to reset the "ret" and "-1",
and also changes the codes so that it don't modify "ret"
for condition checking.

How to reproduce:

% cat test.xml
<disk type='oops' device='disk'>
  <driver name='qemu' type='raw'/>
  <source file='/var/lib/libvirt/images/test.img'/>
  <target dev='vda' bus='virtio'/>
</disk>

% virsh attach-device $domain test.xml
Device attached successfully

The device attachment failed actually with error "unknown disk type 'oops'",
however, it reports success.

As long as we guarantee RPC struct layout stability, we might as
well also guarantee RPC enum value constancy.

* src/Makefile.am (r1, r2, PDWTAGS): Adjust rule to pick up named
and anonymous enums.
* src/remote_protocol-structs: Add enum values.
* src/qemu_protocol-structs: Likewise.
* src/virnetprotocol-structs: Likewise.