Upcoming patches for revert-and-clone branching of snapshots need
to be able to copy a domain definition; make this step reusable.

* src/conf/domain_conf.h (virDomainDefCopy): New prototype.
* src/conf/domain_conf.c (virDomainObjCopyPersistentDef): Split...
(virDomainDefCopy): ...into new function.
(virDomainObjSetDefTransient): Use it.
* src/libvirt_private.syms (domain_conf.h): Export it.
* src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Use it.

Allow bootindex to be specified for redirected USB devices and host USB
devices.

Bug: https://bugzilla.redhat.com/show_bug.cgi?id=805414

For now, disk migration via block copy job is not implemented in
libvirt.  But when we do implement it, we have to deal with the
fact that qemu does not yet provide an easy way to re-start a qemu
process with mirroring still intact.  Paolo has proposed an idea
for a persistent dirty bitmap that might make this possible, but
until that design is complete, it's hard to say what changes
libvirt would need.  Even something like 'virDomainSave' becomes
hairy, if you realize the implications that 'virDomainRestore'
would be stuck with recreating the same mirror layout.

But if we step back and look at the bigger picture, we realize that
the initial client of live storage migration via disk mirroring is
oVirt, which always uses transient domains, and that if a transient
domain is destroyed while a mirror exists, oVirt can easily restart
the storage migration by creating a new domain that visits just the
source storage, with no loss in data.

We can make life a lot easier by being cowards for now, forbidding
certain operations on a domain.  This patch guarantees that we
never get in a state where we would have to restart a domain with
a mirroring block copy, by preventing saves, snapshots, migration,
hot unplug of a disk in use, and conversion to a persistent domain
(thankfully, it is still relatively easy to 'virsh undefine' a
running domain to temporarily make it transient, run tests on
'virsh blockcopy', then 'virsh define' to restore the persistence).
Later, if the qemu design is enhanced, we can relax our code.

The change to qemudDomainDefine looks a bit odd for undoing an
assignment, rather than probing up front to avoid the assignment,
but this is because of how virDomainAssignDef combines both a
lookup and assignment into a single function call.

* src/conf/domain_conf.h (virDomainHasDiskMirror): New prototype.
* src/conf/domain_conf.c (virDomainHasDiskMirror): New function.
* src/libvirt_private.syms (domain_conf.h): Export it.
* src/qemu/qemu_driver.c (qemuDomainSaveInternal)
(qemuDomainSnapshotCreateXML, qemuDomainRevertToSnapshot)
(qemuDomainBlockJobImpl, qemudDomainDefine): Prevent dangerous
actions while block copy is already in action.
* src/qemu/qemu_hotplug.c (qemuDomainDetachDiskDevice): Likewise.
* src/qemu/qemu_migration.c (qemuMigrationIsAllowed): Likewise.

This resolves:

   https://bugzilla.redhat.com/show_bug.cgi?id=862515

which describes inconsistencies in dealing with duplicate mac
addresses on network devices in a domain.

(at any rate, it resolves *almost* everything, and prints out an
informative error message for the one problem that isn't solved, but
has a workaround.)

A synopsis of the problems:

1) you can't do a persistent attach-interface of a device with a mac
address that matches an existing device.

2) you *can* do a live attach-interface of such a device.

3) you *can* directly edit a domain and put in two devices with
matching mac addresses.

4) When running virsh detach-device (live or config), only MAC address
is checked when matching the device to remove, so the first device
with the desired mac address will be removed. This isn't always the
one that's wanted.

5) when running virsh detach-interface (live or config), the only two
items that can be specified to match against are mac address and model
type (virtio, etc) - if multiple netdevs match both of those
attributes, it again just finds the first one added and assumes that
is the only match.

Since it is completely valid to have multiple network devices with the
same MAC address (although it can cause problems in many cases, there
*are* valid use cases), what is needed is:

1) remove the restriction that prohibits doing a persistent add of a
netdev with a duplicate mac address.

2) enhance the backend of virDomainDetachDeviceFlags to check for
something that *is* guaranteed unique (but still work with just mac
address, as long as it yields only a single results.

This patch does three things:

1) removes the check for duplicate mac address during a persistent
netdev attach.

2) unifies the searching for both live and config detach of netdevices
in the subordinate functions of qemuDomainModifyDeviceFlags() to use the
new function virDomainNetFindIdx (which matches mac address and PCI
address if available, checking for duplicates if only mac address was
specified). This function returns -2 if multiple matches are found,
allowing the callers to print out an appropriate message.

Steps 1 & 2 are enough to fully fix the problem when using virsh
attach-device and detach-device (which require an XML description of
the device rather than a bunch of commandline args)

3) modifies the virsh detach-interface command to check for multiple
matches of mac address and show an error message suggesting use of the
detach-device command in cases where there are multiple matching mac
addresses.

Later we should decide how we want to input a PCI address on the virsh
commandline, and enhance detach-interface to take a --address option,
eliminating the need to use detach-device

* src/conf/domain_conf.c
* src/conf/domain_conf.h
* src/libvirt_private.syms
  * added new virDomainNetFindIdx function
  * removed now unused virDomainNetIndexByMac and
    virDomainNetRemoveByMac

* src/qemu/qemu_driver.c
  * remove check for duplicate max from qemuDomainAttachDeviceConfig
  * use virDomainNetFindIdx/virDomainNetRemove instead
    of virDomainNetRemoveByMac in qemuDomainDetachDeviceConfig
  * use virDomainNetFindIdx instead of virDomainIndexByMac
    in qemuDomainUpdateDeviceConfig

* src/qemu/qemu_hotplug.c
  * use virDomainNetFindIdx instead of a homespun loop in
    qemuDomainDetachNetDevice.

* tools/virsh-domain.c: modified detach-interface command as described
    above

We used to walk the backing file chain at least twice per disk,
once to set up cgroup device whitelisting, and once to set up
security labeling.  Rather than walk the chain every iteration,
which possibly includes calls to fork() in order to open root-squashed
NFS files, we can exploit the cache of the previous patch.

* src/conf/domain_conf.h (virDomainDiskDefForeachPath): Alter
signature.
* src/conf/domain_conf.c (virDomainDiskDefForeachPath): Require caller
to supply backing chain via disk, if recursion is desired.
* src/security/security_dac.c
(virSecurityDACSetSecurityImageLabel): Adjust caller.
* src/security/security_selinux.c
(virSecuritySELinuxSetSecurityImageLabel): Likewise.
* src/security/virt-aa-helper.c (get_files): Likewise.
* src/qemu/qemu_cgroup.c (qemuSetupDiskCgroup)
(qemuTeardownDiskCgroup): Likewise.
(qemuSetupCgroup): Pre-populate chain.

Technically, we should not be re-probing any file that qemu might
be currently writing to.  As such, we should cache the backing
file chain prior to starting qemu.  This patch adds the cache,
but does not use it until the next patch.

Ultimately, we want to also store the chain in domain XML, so that
it is remembered across libvirtd restarts, and so that the only
kosher way to modify the backing chain of an offline domain will be
through libvirt API calls, but we aren't there yet.  So for now, we
merely invalidate the cache any time we do a live operation that
alters the chain (block-pull, block-commit, external disk snapshot),
as well as tear down the cache when the domain is not running.

* src/conf/domain_conf.h (_virDomainDiskDef): New field.
* src/conf/domain_conf.c (virDomainDiskDefFree): Clean new field.
* src/qemu/qemu_domain.h (qemuDomainDetermineDiskChain): New
prototype.
* src/qemu/qemu_domain.c (qemuDomainDetermineDiskChain): New
function.
* src/qemu/qemu_driver.c (qemuDomainAttachDeviceDiskLive)
(qemuDomainChangeDiskMediaLive): Pre-populate chain.
(qemuDomainSnapshotCreateSingleDiskActive): Uncache chain before
snapshot.
* src/qemu/qemu_process.c (qemuProcessHandleBlockJob): Update
chain after block pull.

Actually use the enum in the domain conf structure.

* src/conf/domain_conf.h (_virDomainDiskDef): Store enum rather
than string for disk type.
* src/conf/domain_conf.c (virDomainDiskDefFree)
(virDomainDiskDefParseXML, virDomainDiskDefFormat)
(virDomainDiskDefForeachPath): Adjust users.
* src/xenxs/xen_sxpr.c (xenParseSxprDisks, xenFormatSxprDisk):
Likewise.
* src/xenxs/xen_xm.c (xenParseXM, xenFormatXMDisk): Likewise.
* src/vbox/vbox_tmpl.c (vboxAttachDrives): Likewise.
* src/libxl/libxl_conf.c (libxlMakeDisk): Likewise.

Hypervisors are starting to support HyperV Enlightenment features that
improve behavior of guests running Microsoft Windows operating systems.

This patch adds support for the "relaxed" feature that improves timer
behavior and also establishes a framework to add these features in
future.

The apic-eoi feature enum and implementation can be made more universal
to allow re-use of the enum for other features.

This does a shallow copy of all the bits, then strdups the two items
that are actually allocated separately.

The onlined vcpu pinning policy should inherit def->cpuset if
it's not specified explicitly, and the affinity should be set
in this case. Oppositely, the offlined vcpu pinning policy should
be free()'ed.

Save/restore with passed through USB devices currently only works if the
USB device can be found at the same USB address where it used to be
before saving a domain. This makes sense in case a user explicitly
configure the USB address in domain XML. However, if the device was
found automatically by vendor/product identification, we should try to
search for that device when restoring the domain and use any device we
find as long as there is only one available. In other words, the USB
device can now be removed and plugged again or the host can be rebooted
between saving and restoring the domain.

All USB device lookup functions emit an error when they cannot find the
requested device. With this patch, their caller can choose if a missing
device is an error or normal condition.

USB devices can disappear without OS being mad about it, which makes
them ideal for startupPolicy. With this attribute, USB devices can be
configured to be mandatory (the default), requisite (will disappear
during migration if they cannot be found), or completely optional.

Using this new element, one can configure an action that should be
performed when resource locks are lost.

While current on_{poweroff,reboot,crash} action configuration is about
configuring life cycle actions, they can all be considered events and
actions that need to be done on a particular event. Let's generalize the
code by renaming life cycle actions to event actions so that it can be
reused later for non-lifecycle events.

Many parts of virDomainDefPtr were using 'int' variables as
array length counts. Replace all these with size_t and update
various format strings & API signatures to adapt
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This allows the user to control labelling of each character device
separately (the default is to inherit from the VM).
Signed-off-by: NRichard W.M. Jones <rjones@redhat.com>

https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.

* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/;  If/.  If/

Sometimes when guest machine crashes, coredump can get huge due to the
guest memory. This can be limited using madvise(2) system call and is
being used in QEMU hypervisor. This patch adds an option for configuring
that in the domain XML and related documentation.

Whenever the guest machine fails to boot, new parameter (reboot-timeout)
controls whether it should reboot and after how many ms it should do so.

Docs included.

This patch removes all commas after *_LAST sentinels in the enums.

This patch cleans up building the "-boot" parameter and while on that
fixes one inconsistency by modifying these things:

 - I completed the unfinished virDomainBootMenu enum by specifying
   LAST, declaring it and also declaring the TypeFromString and
   TypeToString parameters.
 - Previously mentioned TypeFromString and TypeToString are used when
   parsing the XML.
 - Last, but not least, visible change is that the "-boot" parameter
   is built and parsed properly:
    - The "order=" prefix is used only when additional parameters are
      used (menu, etc.).
    - It's rewritten in a way that other parameters can be added
      easily in the future (used in following patch).
    - The "order=" parameter is properly parsed regardless to where it
      is placed in the string (e.g. "menu=on,order=nc").
    - The "menu=" parameter (and others in the future) are created
      when they should be (i.e. even when bootindex is supported and
      used, but not when bootloader is selected).

Validates the wwn while parsing, error out if it's malformed.

* src/util/util.h: Declare virValidateWWN
* src/util/util.c: Implement virValidateWWN
* src/libvirt_private.syms: Export virValidateWWN.
* src/conf/domain_conf.h: New member 'wwn' for disk def.
* src/conf/domain_conf.c: Parse and format disk <wwn>

virBitmap is recommanded to store cpuset info, and
virBitmapFormat/virBitmapParse can do the format/parse
jobs.

New options is added to support EOI (End of Interrupt) exposure for
guests. As it makes sense only when APIC is enabled, I added this into
the <apic> element in <features> because this should be tri-state
option (cannot be handled as standalone feature).

https://bugzilla.redhat.com/show_bug.cgi?id=795929
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=6af165892cf900291046f1d25f95416f379504c2

This patch define and parse the input XML of USB redirection filter.
<devices>
...
  <redirdev bus='usb' type='spicevmc'>
    <address type='usb' bus='0' port='4'/>
  </redirdev>
  <redirfilter>
    <usbdev class='0x08' vendor='0x1234' product='0xbeef' \
            version='2.00' allow='yes'/>
    <usbdev allow='no'/>
  </redirfilter>
...
</devices>

There is no 1:1 mapping between ports and redirected devices and
qemu and spicy client couldn't decide into which usbredir ports
the client can 'plug' redirected devices. So it make sense to apply
all of filter rules global to all existing usb redirection devices.
class attribute is USB Class codes. version is bcdDevice value
of USB device. vendor and product is USB vendorId and productId.
-1 can be used to allow any value for a field. Except allow attribute
the other four are optional, default value is -1.

After discussion with DB we decided to rename the new iolimit
element as it creates the impression it would be there to
limit (i.e. throttle) I/O instead of specifying immutable
characteristics of a block device.
This is also backed by the fact that the term I/O Limits has
vanished from newer storage admin documentation.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

There is a new <pm/> element implemented that can control what ACPI
sleeping states will be advertised by BIOS and allowed to be switched
to by libvirt. The default keeps defaults on hypervisor, otherwise
forces chosen setting.
The documentation of the pm element is added as well.

Introducing a new iolimits element allowing to override certain
properties of a guest block device like the physical and logical
block size.
This can be useful for platforms with 'non-standard' disk formats
like S390 DASD with its 4K block size.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

To avoid backward compatibility issues, this patch suppresses
auto-generated DAC labels from XML. This change affects commands such as
dumpxml and save.
Signed-off-by: NMarcelo Cerri <mhcerri@linux.vnet.ibm.com>

virDomainVcpuPinAdd does a realloc on vcpupin_list if the new vcpu pin
definition doesn't fit into the array. The list is an array of pointers
but the function definition didn't support returning the changed pointer
to the caller if it was realloced. This caused segfaults if realloc
would change the base pointer.

The name 'virDomainDiskSnapshot' didn't fit in with our normal
conventions of using a prefix hinting that it is related to a
virDomainSnapshotPtr.  Also, a future patch will reuse the
enum for declaring where the VM memory is stored.

* src/conf/snapshot_conf.h (virDomainDiskSnapshot): Rename...
(virDomainSnapshotLocation): ...to this.
(_virDomainSnapshotDiskDef): Update clients.
* src/conf/domain_conf.h (_virDomainDiskDef): Likewise.
* src/libvirt_private.syms (domain_conf.h): Likewise.
* src/conf/domain_conf.c (virDomainDiskDefParseXML)
(virDomainDiskDefFormat): Likewise.
* src/conf/snapshot_conf.c: (virDomainSnapshotDiskDefParseXML)
(virDomainSnapshotAlignDisks, virDomainSnapshotDefFormat):
Likewise.
* src/qemu/qemu_driver.c (qemuDomainSnapshotDiskPrepare)
(qemuDomainSnapshotCreateSingleDiskActive)
(qemuDomainSnapshotCreateDiskActive, qemuDomainSnapshotCreateXML):
Likewise.

This has several benefits:
1. Future snapshot-related code has a definite place to go (and I
_will_ be adding some)
2. Snapshot errors now use the VIR_FROM_DOMAIN_SNAPSHOT error
classification, which has been underutilized (previously only in
libvirt.c)

* src/conf/domain_conf.h, domain_conf.c: Split...
* src/conf/snapshot_conf.h, snapshot_conf.c: ...into new files.
* src/Makefile.am (DOMAIN_CONF_SOURCES): Build new files.
* po/POTFILES.in: Mark new file for translation.
* src/vbox/vbox_tmpl.c: Update caller.
* src/esx/esx_driver.c: Likewise.
* src/qemu/qemu_command.c: Likewise.
* src/qemu/qemu_domain.h: Likewise.

We were failing to react to allocation failure when initializing
a snapshot object list.  Changing things to store a pointer
instead of a complete object adds one more possible point of
allocation failure, but at the same time, will make it easier to
react to failure now, as well as making it easier for a future
patch to split all virDomainSnapshotPtr handling into a separate
file, as I continue to add even more snapshot code.

Luckily, there was only one client outside of domain_conf.c that
was actually peeking inside the object, and a new wrapper function
was easy.

* src/conf/domain_conf.h (_virDomainObj): Use a pointer.
(virDomainSnapshotObjListInit): Rename.
(virDomainSnapshotObjListFree, virDomainSnapshotForEach): New
declarations.
(_virDomainSnapshotObjList): Move definitions...
* src/conf/domain_conf.c: ...here.
(virDomainSnapshotObjListInit, virDomainSnapshotObjListDeinit):
Rename...
(virDomainSnapshotObjListNew, virDomainSnapshotObjListFree): ...to
these.
(virDomainSnapshotForEach): New function.
(virDomainObjDispose, virDomainListPopulate): Adjust callers.
* src/qemu/qemu_domain.c (qemuDomainSnapshotDiscard)
(qemuDomainSnapshotDiscardAllMetadata): Likewise.
* src/qemu/qemu_migration.c (qemuMigrationIsAllowed): Likewise.
* src/qemu/qemu_driver.c (qemuDomainSnapshotLoad)
(qemuDomainUndefineFlags, qemuDomainSnapshotCreateXML)
(qemuDomainSnapshotListNames, qemuDomainSnapshotNum)
(qemuDomainListAllSnapshots)
(qemuDomainSnapshotListChildrenNames)
(qemuDomainSnapshotNumChildren)
(qemuDomainSnapshotListAllChildren)
(qemuDomainSnapshotLookupByName, qemuDomainSnapshotGetParent)
(qemuDomainSnapshotGetXMLDesc, qemuDomainSnapshotIsCurrent)
(qemuDomainSnapshotHasMetadata, qemuDomainRevertToSnapshot)
(qemuDomainSnapshotDelete): Likewise.
* src/libvirt_private.syms (domain_conf.h): Export new function.

This patch introduces support of setting emulator's period and
quota to limit cpu bandwidth when the vm starts.  Also updates
XML Schema for new entries and docs.

Introduce 2 APIs to support emulator threads pin.
    1) virDomainEmulatorPinAdd: setup emulator threads pin with a given cpumap string.
    2) virDomainEmulatorPinDel: remove all emulator threads pin.
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>