Ever since the conversion to using only QMP for probing features
of qemu 1.2 and newer, we have been unable to detect features
that are added only by additional command line options.  For
example, we'd like to know if '-machine mem-merge=on' (added
in qemu 1.5) is present.  To do this, we will take advantage
of qemu 1.5's query-command-line-parameters QMP call [1].

This patch wires up the framework for probing the command results;
if the QMP command is missing, or if a particular command line
option does not output any parameters (for example, -net uses
a polymorphic parser, which showed up as no parameters as of qemu
1.5), we silently treat that command as having no results.

[1] https://lists.gnu.org/archive/html/qemu-devel/2013-04/msg05180.html

* src/qemu/qemu_monitor.h (qemuMonitorGetOptions)
(qemuMonitorSetOptions)
(qemuMonitorGetCommandLineOptionParameters): New functions.
* src/qemu/qemu_monitor_json.h
(qemuMonitorJSONGetCommandLineOptionParameters): Likewise.
* src/qemu/qemu_monitor.c (_qemuMonitor): Add cache field.
(qemuMonitorDispose): Clean it.
(qemuMonitorGetCommandLineOptionParameters): Implement new function.
* src/qemu/qemu_monitor_json.c
(qemuMonitorJSONGetCommandLineOptionParameters): Likewise.
(testQemuMonitorJSONGetCommandLineParameters): Test it.
Signed-off-by: NEric Blake <eblake@redhat.com>

No need to open code a string list cleanup, if we are nice
to the caller by guaranteeing a NULL-terminated result.

* src/qemu/qemu_monitor_json.c (qemuMonitorJSONGetCPUDefinitions)
(qemuMonitorJSONGetCommands, qemuMonitorJSONGetEvents)
(qemuMonitorJSONGetObjectTypes, qemuMonitorJSONGetObjectProps):
Use simpler cleanup.
Signed-off-by: NEric Blake <eblake@redhat.com>

Follows on the heels of other bool cleanups, such as commit 93002b98.

* src/qemu/qemu_monitor.h (qemuMonitorOpen, qemuMonitorOpenFD):
Update json parameter type.
* src/qemu/qemu_monitor.c (qemuMonitorOpen, qemuMonitorOpenFD):
Likewise.
(_qemuMonitor): Adjust field type.
* src/qemu/qemu_domain.h (_qemuDomainObjPrivate): Likewise.
* src/qemu/qemu_domain.c (qemuDomainObjPrivateXMLParse): Adjust
client.
* src/qemu/qemu_process.c (qemuProcessStart): Likewise.
* tests/qemumonitortestutils.c (qemuMonitorTestNew): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

In an upcoming patch, I need the way to safely transfer a nested
virJSON object out of its parent container for independent use,
even after the parent is freed.

* src/util/virjson.h (virJSONValueObjectRemoveKey): New function.
(_virJSONObject, _virJSONArray): Use correct type.
* src/util/virjson.c (virJSONValueObjectRemoveKey): Implement it.
* src/libvirt_private.syms (virjson.h): Export it.
* tests/jsontest.c (mymain): Test it.
Signed-off-by: NEric Blake <eblake@redhat.com>

network: static route support for <network>

This patch adds the <route> subelement of <network> to define a static
route.  the address and prefix (or netmask) attribute identify the
destination network, and the gateway attribute specifies the next hop
address (which must be directly reachable from the containing
<network>) which is to receive the packets destined for
"address/(prefix|netmask)".

These attributes are translated into an "ip route add" command that is
executed when the network is started. The command used is of the
following form:

  ip route add <address>/<prefix> via <gateway> \
               dev <virbr-bridge> proto static metric <metric>

Tests are done to validate that the input data are correct.  For
example, for a static route ip definition, the address must be a
network address and not a host address.  Additional checks are added
to ensure that the specified gateway is directly reachable via this
network (i.e. that the gateway IP address is in the same subnet as one
of the IP's defined for the network).

prefix='0' is supported for both family='ipv4' address='0.0.0.0'
netmask='0.0.0.0' or prefix='0', and for family='ipv6' address='::',
prefix=0', although care should be taken to not override a desired
system default route.

Anytime an attempt is made to define a static route which *exactly*
duplicates an existing static route (for example, address=::,
prefix=0, metric=1), the following error message will be sent to
syslog:

    RTNETLINK answers: File exists

This can be overridden by decreasing the metric value for the route
that should be preferred, or increasing the metric for the route that
shouldn't be preferred (and is thus in place only in anticipation that
the preferred route may be removed in the future).  Caution should be
used when manipulating route metrics, especially for a default route.

Note: The use of the command-line interface should be replaced by
direct use of libnl so that error conditions can be handled better.  But,
that is being left as an exercise for another day.
Signed-off-by: NGene Czarcinski <gene@czarc.net>
Signed-off-by: NLaine Stump <laine@laine.org>

On RHEL 6.4 (gcc 4.4.7), I got:

fdstreamtest.c: In function 'testFDStreamReadCommon':
fdstreamtest.c:44: error: declaration of 'tmpfile' shadows a global declaration [-Wshadow]

* tests/fdstreamtest.c (testFDStreamReadCommon)
(testFDStreamWriteCommon): Rename 'tmpfile' variable.
Signed-off-by: NEric Blake <eblake@redhat.com>

Currently we report a bogus error message when macvlan
creation fails:

error: Failed to start domain migtest
error: operation failed: Unable to create macvlan device

With this removed, we see the real error:

error: Failed to start domain migtest
error: Unable to get index for interface p31p1: No such device
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use of the select() system call is inherantly dangerous since
applications will hit a buffer overrun if any FD number exceeds
the size of the select set size (typically 1024). Replace the
two uses of select() with poll() and use cfg.mk to ban any
future use of select().

NB: This changes the phyp driver so that it uses an infinite
timeout, instead of busy-waiting for 1ms at a time.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This adds both attachment and detachment support for scsi host
device.
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat>

Found that I was unable to start existing domains after updating
to a kernel with no cgroups support

  # zgrep CGROUP /proc/config.gz
  # CONFIG_CGROUPS is not set
  # virsh start test
  error: Failed to start domain test
  error: Unable to initialize /machine cgroup: Cannot allocate memory

virCgroupPartitionNeedsEscaping() correctly returns errno (ENOENT) when
attempting to open /proc/cgroups on such a system, but it was being
dropped in virCgroupSetPartitionSuffix().

Change virCgroupSetPartitionSuffix() to propagate errors returned by
its callees.  Also check for ENOENT in qemuInitCgroup() when determining
if cgroups support is available.

It's better to put the usb related codes into qemuDomainAttachHostUsbDevice
instead of qemuDomainAttachHostDevice.

And in the old qemuDomainAttachHostDevice, just stealing the "usb" from
driver->activeUsbHostdevs leaks the memory.

Escaping a leading '.' with '_' in the cgroup names
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Although virtio-scsi supports SCSI PR (Persistent Reservations),
the device on host may do not support it. To avoid losing data,
Just like PCI and USB pass through devices, only one live guest
is allowed per SCSI host pass through device."
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>

The LXC driver can already configure <disk> or <filesystem>
devices to use the loop device. This extends it to also allow
for use of the NBD device, to support non-raw formats.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The <filesystem> element can now accept a <driver type='nbd'/>
as an alternative to 'loop'. The benefit of NBD is support
for non-raw disk image formats.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a virFileNBDDeviceAssociate method, which given a filename
will setup a NBD device, using qemu-nbd as the server.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To correctly handle errors from readdir() you must set 'errno'
to zero before invoking it & check its value afterwards to
distinguish error from EOF.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The current code for setting up loop devices to LXC disks first
does a switch() based on the disk format, then looks at the
disk driver name. Reverse this so it first looks at the driver
name, and then the disk format. This is more useful since the
list of supported disk formats depends on what driver is used.

The code for setting loop devices for LXC fs entries also needs
to have the same logic added, now the XML schema supports this.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Extend the <driver> element in filesystem devices to
allow a storage format to be set. The new attribute
uses 'format' to reflect the storage format. This is
different from the <driver> element in disk devices
which use 'type' to reflect the storage format. This
is because the 'type' attribute on filesystem devices
is already used for the driver backend, for which the
disk devices use the 'name' attribute. Arggggh.

Anyway for disks we have

   <driver name="qemu" type="raw"/>

And for filesystems this change means we now have

   <driver type="loop" format="raw"/>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

After introducing AppArmorSetSecuritySCSILabel() in 2691cd5f
we are using virSCSIDevicePtr type without proper include.

To not introduce more redundant code, helpers are added for
both "selinux", "dac", and "apparmor" backends.
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat>

v2.5 - v3:
  * Splitted from 8/10 of v2.5
  * Don't forget the other backends (DAC, and apparmor)

This adds the scsi-generic device into the device controller's
whitelist, so that it's allowed to used by the qemu process.
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat.com>

Describe the new cgroups layout, how to customize placement
of guests and what virsh commands are used to access the
parameters.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Since it's generic enough to be used by other types in future, I
put it in <hostdev> as sub-element, though now it's only used by
scsi host device.

Except the scsi host device's controller is "lsilogic", mapping
between the libvirt attributes and scsi-generic properties is:

  libvirt     qemu
-----------------------------------------
  controller  bus ($libvirt_controller.0)
  bus         channel
  target      scsi-id
  unit        lun

For scsi host device with "lsilogic" controller, the mapping is:
('target (libvirt)' must be 0, as it's not used; 'unit (libvirt)
must <= 7).

  libvirt            qemu
----------------------------------------------------------
  controller && bus  bus ($libvirt_controller.$libvirt_bus)
  unit               scsi-id

It's not good to hardcode/hard-check limits of these attributes,
and even worse, these limits are not documented, one has to find
out by either testing or reading the qemu code, I'm looking forward
to qemu expose limits like these one day). For example, exposing
"max_target", "max_lun" for megasas:

static const struct SCSIBusInfo megasas_scsi_info = {
    .tcq = true,
    .max_target = MFI_MAX_LD,
    .max_lun = 255,

    .transfer_data = megasas_xfer_complete,
    .get_sg_list = megasas_get_sg_list,
    .complete = megasas_command_complete,
    .cancel = megasas_command_cancel,
};

Example of the qemu command line (lsilogic controller):

  -drive file=/dev/sg2,if=none,id=drive-hostdev-scsi_host7-0-0-0 \
  -device scsi-generic,bus=scsi0.0,scsi-id=8,\
  drive=drive-hostdev-scsi_host7-0-0-0,id=hostdev-scsi_host7-0-0-0

Example of the qemu command line (virtio-scsi controller):

  -drive file=/dev/sg2,if=none,id=drive-hostdev-scsi_host7-0-0-0 \
  -device scsi-generic,bus=scsi0.0,channel=0,scsi-id=128,lun=128,\
  drive=drive-hostdev-scsi_host7-0-0-0,id=hostdev-scsi_host7-0-0-0
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat.com>

This patch adds util functions for scsi hostdev.
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat.com>

Adding two cap flags for scsi-generic:
  QEMU_CAPS_SCSI_GENERIC
  QEMU_CAPS_SCSI_GENERIC_BOOTINDEX
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat.com>

All of these macros are now unused, so remove.

An example of the scsi hostdev XML:

    <hostdev mode='subsystem' type='scsi'>
      <source>
        <adapter name='scsi_host0'/>
        <address bus='0' target='0' unit='0'/>
      </source>
      <address type='drive' controller='0' bus='0' target='4' unit='8'/>
    </hostdev>

Controller is implicitly added for scsi hostdev, though the scsi
controller's model defaults to "lsilogic", which might be not what
the user wants (same problem exists for virtio-scsi disk). It's
the existing problem, will be addressed later.

The device address must be specified manually. Later patch will let
libvirt generate it automatically.

This only introduces the generic XMLs for scsi hostdev, later patches
will add other elements, e.g. <readonly>, <shareable>.
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat.com>

Since the NPIV machine is not easy to get, it's very likely to
introduce regressions when doing changes on the existing code.
This patch dumps part of the sysfs files (the necessary ones)
of fc_host as test input data, to test the related util functions.
It could be extended for more fc_host related testing in future.

The helper works for default sysfs_prefix, but for user specified
prefix, it doesn't work. (Detected when writing test cases. A later
patch will add the test cases for fc_host).

The returned result is something like "host5" acutally.

Function name with "aIsB" generally means its return value is
in Bi-state (true/false).

In case of the caller can pass a "prefix" (or "sysfs_prefix")
without the trailing slash, and Unix-Like system always eats
up the redundant "slash" in the filepath, let's add it explicitly.

Which refactored the old code, and introduced new helper
virIsCapableVport, but the path for checking with access() is not
correctly constructed.

Introduced by commit 244ce462, which refactored the helper for wwn
reading, however, it forgot to change the old "strndup" and "sizeof(buf)",
"sizeof(buf)" operates on the fixed length array ("buf") in the old code,
but now "buf" is a pointer.

Before the fix:

% virsh nodedev-dumpxml scsi_host5
<device>
  <name>scsi_host5</name>
  <parent>pci_0000_04_00_1</parent>
  <capability type='scsi_host'>
    <host>5</host>
    <capability type='fc_host'>
      <wwnn>2001001b</wwnn>
      <wwpn>2101001b</wwpn>
      <fabric_wwn>2001000d</fabric_wwn>
    </capability>
  </capability>
</device>

With the fix:

% virsh nodedev-dumpxml scsi_host5
<device>
  <name>scsi_host5</name>
  <parent>pci_0000_04_00_1</parent>
  <capability type='scsi_host'>
    <host>5</host>
    <capability type='fc_host'>
      <wwnn>0x2001001b32a9da4e</wwnn>
      <wwpn>0x2101001b32a9da4e</wwpn>
      <fabric_wwn>0x2001000dec9877c1</fabric_wwn>
    </capability>
  </capability>
</device>