The src/util/network.c file is a dumping ground for many different
APIs. Split it up into 5 pieces, along functional lines

 - src/util/virnetdevbandwidth.c: virNetDevBandwidth type & helper APIs
 - src/util/virnetdevvportprofile.c: virNetDevVPortProfile type & helper APIs
 - src/util/virsocketaddr.c: virSocketAddr and APIs
 - src/conf/netdev_bandwidth_conf.c: XML parsing / formatting
   for virNetDevBandwidth
 - src/conf/netdev_vport_profile_conf.c: XML parsing / formatting
   for virNetDevVPortProfile

* src/util/network.c, src/util/network.h: Split into 5 pieces
* src/conf/netdev_bandwidth_conf.c, src/conf/netdev_bandwidth_conf.h,
  src/conf/netdev_vport_profile_conf.c, src/conf/netdev_vport_profile_conf.h,
  src/util/virnetdevbandwidth.c, src/util/virnetdevbandwidth.h,
  src/util/virnetdevvportprofile.c, src/util/virnetdevvportprofile.h,
  src/util/virsocketaddr.c, src/util/virsocketaddr.h: New pieces
* daemon/libvirtd.h, daemon/remote.c, src/conf/domain_conf.c,
  src/conf/domain_conf.h, src/conf/network_conf.c,
  src/conf/network_conf.h, src/conf/nwfilter_conf.h,
  src/esx/esx_util.h, src/network/bridge_driver.c,
  src/qemu/qemu_conf.c, src/rpc/virnetsocket.c,
  src/rpc/virnetsocket.h, src/util/dnsmasq.h, src/util/interface.h,
  src/util/iptables.h, src/util/macvtap.c, src/util/macvtap.h,
  src/util/virnetdev.h, src/util/virnetdevtap.c,
  tools/virsh.c: Update include files

The virtual port profile parsing/formatting APIs do not
correctly handle unknown profile type strings/numbers.
They behave as a no-op, instead of raising an error

* src/util/network.c, src/util/network.h: Fix error
  handling of port profile APIs
* src/conf/domain_conf.c, src/conf/network_conf.c: Update
  for API changes

Rename the virVirtualPortProfileParams struct to be
virNetDevVPortProfile, and rename the APIs to match
this prefix.

* src/util/network.c, src/util/network.h: Rename port profile
  APIs
* src/conf/domain_conf.c, src/conf/domain_conf.h,
  src/conf/network_conf.c, src/conf/network_conf.h,
  src/network/bridge_driver.c, src/qemu/qemu_hotplug.c,
  src/util/macvtap.c, src/util/macvtap.h: Update for
  renamed APIs/structs

Rename virBandwidth to virNetDevBandwidth, and virRate to
virNetDevBandwidthRate.

* src/util/network.c, src/util/network.h: Rename bandwidth
  structs and APIs
* src/conf/domain_conf.c, src/conf/domain_conf.h,
  src/conf/network_conf.c, src/conf/network_conf.h,
  src/lxc/lxc_driver.c, src/network/bridge_driver.c,
  src/qemu/qemu_command.c, src/util/macvtap.c,
  src/util/macvtap.h, tools/virsh.c: Update for API changes.

The socket address APIs in src/util/network.h either take the
form  virSocketAddrXXX, virSocketXXX or virSocketXXXAddr.

Sanitize this so everything is virSocketAddrXXXX, and ensure
that the virSocketAddr parameter is always the first one.

* src/util/network.c, src/util/network.h: Santize socket
  address API naming
* src/conf/domain_conf.c, src/conf/network_conf.c,
  src/conf/nwfilter_conf.c, src/network/bridge_driver.c,
  src/nwfilter/nwfilter_ebiptables_driver.c,
  src/nwfilter/nwfilter_learnipaddr.c,
  src/qemu/qemu_command.c, src/rpc/virnetsocket.c,
  src/util/dnsmasq.c, src/util/iptables.c,
  src/util/virnetdev.c, src/vbox/vbox_tmpl.c: Update for
  API renaming

This causes libvirtd to crash when both <boot dev='...'/> and <boot
order='...'/> are used in one domain XML. Introduced by
5fa3d775

The default console type may vary based on the OS type. ie a Xen
paravirt guests wants a 'xen' console, while a fullvirt guests
wants a 'serial' console.

A plain integer default console type in the capabilities does
not suffice. Instead introduce a callback that is passed the
OS type.

* src/conf/capabilities.h: Use a callback for default console
  type
* src/conf/domain_conf.c, src/conf/domain_conf.h: Use callback
  for default console type. Add missing LXC/OpenVZ console types.
* src/esx/esx_driver.c, src/libxl/libxl_conf.c,
  src/lxc/lxc_conf.c, src/openvz/openvz_conf.c,
  src/phyp/phyp_driver.c, src/qemu/qemu_capabilities.c,
  src/uml/uml_conf.c, src/vbox/vbox_tmpl.c,
  src/vmware/vmware_conf.c, src/xen/xen_hypervisor.c,
  src/xenapi/xenapi_driver.c: Set default console type callback

When no <target> element was set at all, the default console
target type was not being honoured

* src/conf/domain_conf.c: Set default target type for consoles
  with no <target>

While Xen only has a single paravirt console, UML, and
QEMU both support multiple paravirt consoles. The LXC
driver can also be trivially made to support multiple
consoles. This patch extends the XML to allow multiple
<console> elements in the XML. It also makes the UML
and QEMU drivers support this config.

* src/conf/domain_conf.c, src/conf/domain_conf.h: Allow
  multiple <console> devices
* src/lxc/lxc_driver.c, src/xen/xen_driver.c,
  src/xenxs/xen_sxpr.c, src/xenxs/xen_xm.c: Update for
  internal API changes
* src/security/security_selinux.c, src/security/virt-aa-helper.c:
  Only label consoles that aren't a copy of the serial device
* src/qemu/qemu_command.c, src/qemu/qemu_driver.c,
  src/qemu/qemu_process.c, src/uml/uml_conf.c,
  src/uml/uml_driver.c: Support multiple console devices
* tests/qemuxml2xmltest.c, tests/qemuxml2argvtest.c: Extra
  tests for multiple virtio consoles. Set QEMU_CAPS_CHARDEV
  for all console /channel tests
* tests/qemuxml2argvdata/qemuxml2argv-channel-virtio-auto.args,
  tests/qemuxml2argvdata/qemuxml2argv-channel-virtio.args
  tests/qemuxml2argvdata/qemuxml2argv-console-virtio.args: Update
  for correct chardev syntax
* tests/qemuxml2argvdata/qemuxml2argv-console-virtio-many.args,
  tests/qemuxml2argvdata/qemuxml2argv-console-virtio-many.xml: New
  test file

If we are comparing storage pools we must skip comparing with
ourself, so that re-defining an existing pool works

* conf/storage_conf.c: Skip self when comparing

This patch does some cleanups to my previous startupPolicy patchset.

To support "managed" mode of host PCI device, we record the original
states (unbind_from_stub, remove_slot, and reprobe) so that could
reattach the device to host with original driver. But there is no XML
for theses attrs, and thus after daemon is restarted, we lose the
original states. It's easy to reproduce:

    1) virsh start domain
    2) virsh attach-device dom hostpci.xml (in 'managed' mode)
    3) service libvirtd restart
    4) virsh destroy domain

    You will see the device won't be bound to the original driver
if there was one.

This patch is to solve the problem by introducing internal XML
(won't be dumped to user, only dumped to status XML). The XML is:
    <origstates>
      <unbind/>
      <remove_slot/>
      <reprobe/>
    </origstates>

Which will be child node of <hostdev><source>...</souce></hostdev>.
(only for PCI device).

A new struct "virDomainHostdevOrigStates" is introduced for the XML,
and the according members are updated when preparing the PCI device.
And function "qemuUpdateActivePciHostdevs" is modified to honor
the original states. Use of qemuGetPciHostDeviceList is removed
in function "qemuUpdateActivePciHostdevs", and the "managed" value of
the device config is honored by the change. This fixes another problem
alongside:

    qemuGetPciHostDeviceList set the device as "managed" force
    regardless of whether the device is configured as "managed='yes'"
    or not in XML, which is not right.

In hand-applying Josh and Sage's patch, I missed out on a break.

* src/conf/secret_conf.c (virSecretDefFree): Fix my botch.

Add additional fields to let you specify the how to authenticate with a disk.
The secret to use may be referenced by a usage string or a UUID, i.e.:

<auth username='myuser'>
 <secret type='ceph' usage='secretname'/>
</auth>

or

<auth username='myuser'>
 <secret type='ceph' uuid='0a81f5b2-8403-7b23-c8d6-21ccc2f80d6f'/>
</auth>
Signed-off-by: NJosh Durgin <josh.durgin@dreamhost.com>

Add a new secret type to store a Ceph authentication key. The name
is simply an identifier for easy human reference.

The xml looks like this:

<secret ephemeral='no' private='no'>
 <uuid>0a81f5b2-8403-7b23-c8d6-21ccc2f80d6f</uuid>
 <usage type='ceph'>
   <name>mycluster_admin</name>
 </usage>
</secret>
Signed-off-by: NSage Weil <sage@newdream.net>
Signed-off-by: NJosh Durgin <josh.durgin@dreamhost.net>

Break some long lines, and use more efficient functions when possible,
such as relying on virBufferEscapeString to skip output on a NULL arg.
Ensure that output does not embed newlines, since auto-indent won't
work in those situations.

* src/conf/domain_conf.c (virDomainTimerDefFormat): Break output lines.
(virDomainDefFormatInternal, virDomainDiskDefFormat)
(virDomainActualNetDefFormat, virDomainNetDefFormat)
(virDomainHostdevDefFormat): Minor cleanups.

Use auto-indent in more places.

* src/conf/storage_encryption_conf.h (virStorageEncryptionFormat):
Drop parameter.
* src/conf/storage_encryption_conf.c (virStorageEncryptionFormat)
(virStorageEncryptionSecretFormat): Simplify with auto-indent.
* src/conf/domain_conf.c (virDomainDiskDefFormat): Adjust caller.
* src/conf/storage_conf.c (virStorageVolTargetDefFormat): Likewise.

Fixing this involved some refactoring of common code out of
domain_conf and nwfilter_conf into nwfilter_params.

* src/conf/nwfilter_params.h (virNWFilterFormatParamAttributes):
Adjust signature.
* src/conf/nwfilter_params.c (_formatParameterAttrs)
(virNWFilterFormatParamAttributes): Adjust indentation handling,
and handle filterref here.
(formatterParam): Delete unused struct.
* src/conf/domain_conf.c (virDomainNetDefFormat): Adjust caller.
* src/conf/nwfilter_conf.c (virNWFilterIncludeDefFormat): Likewise.

If a disk source gets dropped because it is not accessible,
mgmt application might want to be informed about this. Therefore
we need to emit an event. The event presented in this patch
is however a bit superset of what written above. The reason is simple:
an intention to be easily expanded, e.g. on 'user ejected disk
in guest' events. Therefore, callback gets source string and disk alias
(which should be unique among a domain) and reason (an integer);

This attribute says what to do with cdrom (or floppy) if
the source is missing. It accepts:
- mandatory - fail if missing for any reason (the default)
- requisite - fail if missing on boot up, drop if missing on
              migrate/restore/revert
- optional  - drop if missing at any start attempt.

However, this patch introduces only XML part of this new
functionality.

Splitting into two functions allows the user to call the right
function, rather than having to remember that a *Free function is
an exception to the rule.

* src/conf/storage_conf.h (virStoragePoolSourceClear): New function.
* src/libvirt_private.syms (storage_conf.h): Export it.
* src/conf/storage_conf.c (virStoragePoolSourceFree): Split...
(virStoragePoolSourceClear): ...into new function.
(virStoragePoolDefFree, virStoragePoolDefParseSourceString):
Update callers.
* src/test/test_driver.c (testStorageFindPoolSources): Likewise.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemNetFindPoolSourcesFunc)
(virStorageBackendFileSystemNetFindPoolSources): Likewise.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSIFindPoolSources): Likewise.
* src/storage/storage_backend_logical.c
(virStorageBackendLogicalFindPoolSources): Likewise.

	This patch adds support for "continue" and "return" actions
in filter rules.
Signed-off-by: NDavid L Stevens <dlstevens@us.ibm.com>

More simplifications possible due to auto-indent.  Also,
<bandwidth> within <actual> was only using 6 instead of 8 spaces.

* src/util/network.h (virVirtualPortProfileFormat)
(virBandwidthDefFormat): Alter signature.
* src/util/network.c (virVirtualPortProfileFormat)
(virBandwidthDefFormat): Alter indentation.
(virBandwidthChildDefFormat): Tweak to make use easier.
* src/conf/network_conf.c (virPortGroupDefFormat)
(virNetworkDefFormat): Adjust callers.
* src/conf/domain_conf.c (virDomainNetDefFormat): Likewise.
(virDomainActualNetDefFormat): Likewise, and fix bandwidth
indentation.

Auto-indent makes life a bit easier; this patch also drops unused
arguments and replaces a misspelled flag name with two entry points
instead, so that callers don't have to worry about how much spacing
is present when embedding cpu elements.

* src/conf/cpu_conf.h (virCPUFormatFlags): Delete.
(virCPUDefFormat): Drop unused argument.
(virCPUDefFormatBuf): Alter signature.
(virCPUDefFormatBufFull): New prototype.
* src/conf/cpu_conf.c (virCPUDefFormatBuf): Split...
(virCPUDefFormatBufFull): ...into new function.
(virCPUDefFormat): Adjust caller.
* src/conf/domain_conf.c (virDomainDefFormatInternal): Likewise.
* src/conf/capabilities.c (virCapabilitiesFormatXML): Likewise.
* src/cpu/cpu.c (cpuBaselineXML): Likewise.
* tests/cputest.c (cpuTestCompareXML): Likewise.

The improvements to virBuffer, along with a paradigm shift to pass
the original buffer through rather than creating a second buffer,
allow us to shave off quite a few lines of code.

* src/util/sysinfo.h (virSysinfoFormat): Alter signature.
* src/util/sysinfo.c (virSysinfoFormat, virSysinfoBIOSFormat)
(virSysinfoSystemFormat, virSysinfoProcessorFormat)
(virSysinfoMemoryFormat): Change indentation parameter.
* src/conf/domain_conf.c (virDomainSysinfoDefFormat): Adjust
caller.
* src/qemu/qemu_driver.c (qemuGetSysinfo): Likewise.

Add a test for the simple parts of my indentation changes, and
fix the fallout.

* tests/domainsnapshotxml2xmltest.c: New test.
* tests/Makefile.am (domainsnapshotxml2xmltest_SOURCES): Build it.
* src/conf/domain_conf.c (virDomainSnapshotDefFormat): Avoid NULL
deref, match documented order.
* src/conf/domain_conf.h (virDomainSnapshotDefFormat): Add const.
* tests/domainsnapshotxml2xmlout/all_parameters.xml: Tweak output.
* tests/domainsnapshotxml2xmlout/disk_snapshot.xml: Likewise.
* tests/domainsnapshotxml2xmlout/full_domain.xml: Likewise.
* .gitignore: Exempt new binary.

<domainsnapshot> is the first public instance of <domain> being
used as a sub-element, although we have two other private uses
(runtime state, and migration cookie).  Although indentation has
no effect on XML parsing, using it makes the output more consistent.

This uses virBuffer auto-indentation to obtain the effect, for all
but the portions of <domain> that are not generated a line at a
time into the same virBuffer.  Further patches will clean up the
remaining problems.

* src/conf/domain_conf.h (virDomainDefFormatInternal): New prototype.
* src/conf/domain_conf.c (virDomainDefFormatInternal): Export.
(virDomainObjFormat, virDomainSnapshotDefFormat): Update callers.
* src/libvirt_private.syms (domain_conf.h): Add new export.
* src/qemu/qemu_migration.c (qemuMigrationCookieXMLFormat): Use
new function.
(qemuMigrationCookieXMLFormatStr): Update caller.

Tested with multiple AHCI controllers and multiple disks attached
to a controller. E.g.,

    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/test/disk0.raw'/>
      <target dev='sda' bus='sata'/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/test/disk1.raw'/>
      <target dev='sdb' bus='sata'/>
      <address type='drive' controller='0' bus='0' unit='1'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/images/test/disk2.raw'/>
      <target dev='sdc' bus='sata'/>
      <address type='drive' controller='1' bus='0' unit='0'/>
    </disk>
    <controller type='sata' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <controller type='sata' index='1'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </controller>

Detected by Coverity.  Leak present since commit 874e65aa; and
while commit d50bb45b tried to fix the issue, it missed a path.

* src/conf/domain_conf.c (virDomainDefParseBootXML): Always clean
up useserial.

Currently, push & pop from event queue (both server & client side)
rely on lock from higher levels, e.g. on driver lock (qemu),
private_data (remote), ...; This alone is not sufficient as not
every function that interacts with this queue can/does lock,
esp. in client where we have a different approach, "passing
the buck".

Therefore we need a separate lock just to protect event queue.

For more info see:
https://bugzilla.redhat.com/show_bug.cgi?id=743817

With the recent refactoring of qemu snapshot relationships, it
is now trivial to filter on leaves.

* src/conf/domain_conf.c (virDomainSnapshotObjListCount)
(virDomainSnapshotObjListCopyNames): Handle new flag.
* src/qemu/qemu_driver.c (qemuDomainSnapshotListNames)
(qemuDomainSnapshotNum, qemuDomainSnapshotListChildrenNames)
(qemuDomainSnapshotNumChildren): Pass new flag through.

VirtFS allows the user to choose between path/handle based fs driver.
As of now, libvirt hardcoded path based driver only. This patch provides
a solution to allow user to choose between path/handle based fs driver.

Sample:

    <filesystem type='mount'>
      <driver type='handle'/>
      <source dir='/folder/to/share1'/>
      <target dir='mount_tag1'/>
    </filesystem>

    <filesystem type='mount'>
      <driver type='path'/>
      <source dir='/folder/to/share2'/>
      <target dir='mount_tag2'/>
    </filesystem>
Signed-off-by: NHarsh Prateek Bora <harsh@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

The previous optimizations lead to some follow-on cleanups.

* src/conf/domain_conf.c (virDomainSnapshotForEachChild)
(virDomainSnapshotForEachDescendant): Drop dead parameter.
(virDomainSnapshotActOnDescendant)
(virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): Update callers.
* src/qemu/qemu_driver.c (qemuDomainSnapshotNumChildren)
(qemuDomainSnapshotListChildrenNames, qemuDomainSnapshotDelete):
Likewise.
* src/conf/domain_conf.h: Update prototypes.

Among other improvements, virDomainSnapshotForEachDescendant is
changed from iterative O(n^2) to recursive O(n).  A bit better
than the O(n^3) implementation in virsh snapshot-list!

* src/conf/domain_conf.c (virDomainSnapshotObjListNum)
(virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjeListGetNames, virDomainSnapshotForEachChild)
(virDomainSnapshotForEachDescendant): Optimize.
(virDomainSnapshotActOnDescendant): Tweak.
(virDomainSnapshotActOnChild, virDomainSnapshotMarkDescendant):
Delete, now that they are unused.

No one was using virDomainSnapshotHasChildren, but that was an
O(n) function.  Exposing and tracking a bit more metadata for each
snapshot will allow the same query to be made with an O(1) query
of the member field.  For single snapshot operations (create,
delete), callers can be trusted to maintain the metadata themselves,
but for reloading, we can't compute parents as we go since there
is no guarantee that parents were parsed before children, so we also
provide a function to refresh the relationships, and which can
be used to detect if the user has ignored our warnings and been
directly modifying files in /var/lib/libvirt/qemu/snapshot.  This
patch only adds metadata; later patches will actually use it.

This layout intentionally hardcodes the size of each snapshot struct,
by tracking sibling pointers, rather than having to deal with the
headache of yet more memory management by directly sticking a
dynamically sized child[] on each parent.

* src/conf/domain_conf.h (_virDomainSnapshotObj)
(_virDomainSnapshotObjList): Add members.
(virDomainSnapshotUpdateRelations, virDomainSnapshotDropParent):
New prototypes.
(virDomainSnapshotHasChildren): Delete.
* src/conf/domain_conf.c (virDomainSnapshotSetRelations)
(virDomainSnapshotUpdateRelations, virDomainSnapshotDropParent):
New functions.
(virDomainSnapshotHasChildren): Drop unused function.
* src/libvirt_private.syms (domain_conf): Update exports.

Not too hard to wire up.  The trickiest part is realizing that
listing children of a snapshot cannot use SNAPSHOT_LIST_ROOTS,
and that we overloaded that bit to also mean SNAPSHOT_LIST_DESCENDANTS;
we use that bit to decide which iteration to use, but don't want
the existing counting/listing functions to see that bit.

* src/conf/domain_conf.h (virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): New prototypes.
* src/conf/domain_conf.c (virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): New functions.
* src/libvirt_private.syms (domain_conf.h): Export them.
* src/qemu/qemu_driver.c (qemuDomainSnapshotNumChildren)
(qemuDomainSnapshotListChildrenNames): New functions.

Redefining disk-only snapshot xml should work even if the user
did not explicitly pass VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY;
the flag is only required for conditions where the <state>
subelement is not already present in parsing (that is, defining
a new snapshot).

Also, fix the error code of some user-visible errors (the remaining
VIR_ERR_INTERNAL_ERROR should not be user-visible, since parsing
of <active> is only done from internal code).

* src/conf/domain_conf.c (virDomainSnapshotDefParseString): Allow
disks during redefinition of disk snapshot.

Previously libvirt's disk device XML only had a single attribute,
error_policy, to control both read and write error policy, but qemu
has separate options for controlling read and write. In one case
(enospc) a policy is allowed for write errors but not read errors.

This patch adds a separate attribute that sets only the read error
policy. If just error_policy is set, it will apply to both read and
write error policy (previous behavior), but if the new rerror_policy
attribute is set, it will override error_policy for read errors only.
Possible values for rerror_policy are "stop", "report", and "ignore"
("report" is the qemu-controlled default for rerror_policy when
error_policy isn't specified).

For consistency, the value "report" has been added to the possible
values for error_policy as well.

commit 12062abb set rerror=ignore when error_policy="enospace" was
selected (since the rerror option in qemu doesn't accept "enospc", as
the werror option does).

After that patch was already pushed, Paolo Bonzini noticed it and
commented that leaving rerror at the default ("report") would be a
better choice. This patch corrects the problem - if error_policy =
"enospace" is given, rerror is left off the qemu commandline,
effectively setting it to "report". For other values, rerror is still
set to match werror.

Additionally, the parsing of error_policy was changed to no longer
erroneously allow "default" as a choice - as with most other
attributes, if you want the default setting, just don't specify an
error_policy.

Finally, two ommissions in the first patch were corrected - a
long-dormant qemuxml2argv test for enospace was enabled, and fixed to
pass, and the argv2xml parser in qemu_command.c was updated to
recognize the different spelling on the qemu commandline.