While the QEMU monitor/agent do not want JSON strings pretty
printed, other parts of libvirt might. Instead of hardcoding
QEMU's desired behaviour in virJSONValueToString(), add a
boolean flag to control pretty printing
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The qemuProcessAutoDestroyRun function was removed in an earlier
commit, but the header file declaration was not deleted

virtPortProfile is now used by 4 different types of network devices
(NETWORK, BRIDGE, DIRECT, and HOSTDEV), and it's getting cumbersome to
replicate so much code in 4 different places just because each type
has the virtPortProfile in a slightly different place. This patch puts
a single virtPortProfile in a common place (outside the type-specific
union) in both virDomainNetDef and virDomainActualNetDef, and adjusts
the parse and format code (and the few other places where it is used)
accordingly.

Note that when a <virtualport> element is found, the parse functions
verify that the interface is of a type that supports one, otherwise an
error is generated (CONFIG_UNSUPPORTED in the case of <interface>, and
INTERNAL in the case of <actual>, since the contents of <actual> are
always generated by libvirt itself).

As the consensus in:
https://www.redhat.com/archives/libvir-list/2012-July/msg01692.html,
this patch is to destroy conf/virdomainlist.[ch], folding the
helpers into conf/domain_conf.[ch].

* src/Makefile.am:
  - Various indention fixes incidentally
  - Add macro DATATYPES_SOURCES (datatypes.[ch])
  - Link datatypes.[ch] for libvirt_lxc

* src/conf/domain_conf.c:
  - Move all the stuffs from virdomainlist.c into it
  - Use virUnrefDomain and virUnrefDomainSnapshot instead of
    virDomainFree and virDomainSnapshotFree, which are defined
    in libvirt.c, and we don't want to link to it.
  - Remove "if" before "free" the object, as virObjectUnref
    is in the list "useless_free_options".

* src/conf/domain_conf.h:
  - Move all the stuffs from virdomainlist.h into it
  - s/LIST_FILTER/LIST_DOMAINS_FILTER/

* src/libxl/libxl_driver.c:
  - s/LIST_FILTER/LIST_DOMAINS_FILTER/
  - no (include "virdomainlist.h")

* src/libxl/libxl_driver.c: Likewise

* src/lxc/lxc_driver.c: Likewise

* src/openvz/openvz_driver.c: Likewise

* src/parallels/parallels_driver.c: Likewise

* src/qemu/qemu_driver.c: Likewise

* src/test/test_driver.c: Likewise

* src/uml/uml_driver.c: Likewise

* src/vbox/vbox_tmpl.c: Likewise

* src/vmware/vmware_driver.c: Likewise

* tools/virsh-domain-monitor.c: Likewise

* tools/virsh.c: Likewise

This patch introduces a new error code VIR_ERR_OPERATION_UNSUPPORTED to
mark error messages regarding operations that failed due to lack of
support on the hypervisor or other than libvirt issues.

The code is first used in reporting error if qemu does not support block
IO tuning variables yielding error message:
error: Unable to get block I/O throttle parameters
error: Operation not supported: block_io_throttle field
'total_bytes_sec' missing in qemu's output

instead of:
error: Unable to get block I/O throttle parameters
error: internal error cannot read total_bytes_sec

The meat of this patch is just moving the calls to
virNWFilterRegisterCallbackDriver from each hypervisor's "register"
function into its "initialize" function. The rest is just code
movement to allow that, and a new virNWFilterUnRegisterCallbackDriver
function to undo what the register function does.

The long explanation:

There is an array in nwfilter called callbackDrvArray that has
pointers to a table of functions for each hypervisor driver that are
called by nwfilter. One of those function pointers is to a function
that will lock the hypervisor driver. Entries are added to the table
by calling each driver's "register" function, which happens quite
early in libvirtd's startup.

Sometime later, each driver's "initialize" function is called. This
function allocates a driver object and stores a pointer to it in a
static variable that was previously initialized to NULL. (and here's
the important part...) If the "initialize" function fails, the driver
object is freed, and that pointer set back to NULL (but the entry in
nwfilter's callbackDrvArray is still there).

When the "lock the driver" function mentioned above is called, it
assumes that the driver was successfully loaded, so it blindly tries
to call virMutexLock on "driver->lock".

BUT, if the initialize never happened, or if it failed, "driver" is
NULL. And it just happens that "lock" is always the first field in
driver so it is also NULL.

Boom.

To fix this, the call to virNWFilterRegisterCallbackDriver for each
driver shouldn't be called until the end of its (*already guaranteed
successful*) "initialize" function, not during its "register" function
(which is currently the case). This implies that there should also be
a virNWFilterUnregisterCallbackDriver() function that is called in a
driver's "shutdown" function (although in practice, that function is
currently never called).

When entering "confirm" phase, we are interested in the value of
cancelled rather then ret variable which was interesting before "finish"
phase and didn't change since then.

Previously, qemu did not respond to monitor commands during migration if
the limit was too high. This prevented us from raising the limit
earlier. The qemu issue seems to be fixed (according to my testing) and
we may remove the 32Mb/s limit.

This patch refactors the JSON parsing function that extracts the block
IO tuning parameters from qemu's output. The most impacting change
concerns the error message that is returned if the reply from qemu does
not contain the needed data. The data for IO parameter tuning were added
in qemu 1.1 and the previous error message was confusing.

This patch also breaks long lines and extracts a multiple time used code
pattern to a macro.

Rename qemuDefaultScsiControllerModel to qemuCheckScsiControllerModel.
When scsi model is given explicitly in XML(model > 0) checking if the
underlying QEMU supports it or not first, raise an error on checking
failure.
When the model is not given(mode <= 0), return LSI by default, if
the QEMU doesn't support it, raise an error.

  QEMU_CAPS_SCSI_LSI
    set the flag when "lsi53c895a", bus PCI, alias "lsi" in
    the output of "qemu -device ?"
    -device lsi in qemu command line

  QEMU_CAPS_VIRTIO_SCSI_PCI
    set the flag when "name "virtio-scsi-pci", bus PCI" in
    the output of qemu devices query.
    -device virtio-scsi-pci in qemu command line

Make virSocket use the virObject APIs for reference counting
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Make qemuAgentPtr and qemuMonitorPtr types use the virObject APIs
for reference counting
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Switch virDomainObjPtr to use the virObject APIs for reference
counting. The main change is that virObjectUnref does not return
the reference count, merely a bool indicating whether the object
still has any refs left. Checking the return value is also not
mandatory.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This converts the following public API datatypes to use the
virObject infrastructure:

  virConnectPtr
  virDomainPtr
  virDomainSnapshotPtr
  virInterfacePtr
  virNetworkPtr
  virNodeDevicePtr
  virNWFilterPtr
  virSecretPtr
  virStreamPtr
  virStorageVolPtr
  virStoragePoolPtr

The code is significantly simplified, since the mutex in the
virConnectPtr object now only needs to be held when accessing
the per-connection virError object instance. All other operations
are completely lock free.

* src/datatypes.c, src/datatypes.h, src/libvirt.c: Convert
  public datatypes to use virObject
* src/conf/domain_event.c, src/phyp/phyp_driver.c,
  src/qemu/qemu_command.c, src/qemu/qemu_migration.c,
  src/qemu/qemu_process.c, src/storage/storage_driver.c,
  src/vbox/vbox_tmpl.c, src/xen/xend_internal.c,
  tests/qemuxml2argvtest.c, tests/qemuxmlnstest.c,
  tests/sexpr2xmltest.c, tests/xmconfigtest.c: Convert
  to use virObjectUnref/virObjectRef
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch adds the support to run the QEMU network helper
under unprivileged user. It also adds the support for
attach-interface option in virsh to run under unprivileged
user.
Signed-off-by: NRicha Marwaha <rmarwah@linux.vnet.ibm.com>
Signed-off-by: Corey Bryant<coreyb@linux.vnet.ibm.com>

This patch adds the capability in libvirt to check if
-netdev bridge option is supported or not.
Signed-off-by: NRicha Marwaha <rmarwah@linux.vnet.ibm.com>
Signed-off-by: Corey Bryant<coreyb@linux.vnet.ibm.com>

If there's a memory leak in qemu or qemu is exploited the host's
system will sooner or later start trashing instead of killing
the bad process. This however has impact on performance and other
guests as well. Therefore we should set a reasonable RSS limit
even when user hasn't set any. It's better to be secure by default.

* src/conf/domain_conf.c:
  - Add virDomainControllerFind to find controller device by type
    and index.
  - Add virDomainControllerRemove to remove the controller device
    from maintained controler list.

* src/conf/domain_conf.h:
  - Declare the two new helpers.

* src/libvirt_private.syms:
  - Expose private symbols for the two new helpers.

* src/qemu/qemu_driver.c:
  - Support attach/detach controller device persistently

* src/qemu/qemu_hotplug.c:
  - Use the two helpers to simplify the codes.

One of our latest commits fbe87126 introduced this nasty typo:
func(vmdef, ...); where func() dereference vmdef->ncontrollers,
and vmdef was initialized to NULL. This leaves us with unconditional
immediate segfault. It should be vm->def instead.

This patch enables the "none" USB controller for qemu guests and adds
valdiation on hot-plugged devices if the guest has USB disabled.

This patch also adds a set of tests to check parsing of domain XMLs that
use the "none" controller and some forbidden situations concerning it.

Libvirt adds a USB controller to the guest even if the user does not
specify any in the XML. This is due to back-compat reasons.

To allow disabling USB for a guest this patch adds a new USB controller
type "none" that disables USB support for the guest.

Pushed without ack, under the trivial rule.

Detected by Coverity.

* src/qemu/qemu_hotplug.c (qemuDomainAttachHostDevice): Avoid
double free of usb on failure.

Any time we have a string with no % passed through gettext, a
translator can inject a % to cause a stack overread.  When there
is nothing to format, it's easier to ask for a string that cannot
be used as a formatter, by using a trivial "%s" format instead.

In the past, we have used --disable-nls to catch some of the
offenders, but that doesn't get run very often, and many more
uses have crept in.  Syntax check to the rescue!

The syntax check can catch uses such as
virReportError(code,
               _("split "
                 "string"));
by using a sed script to fold context lines into one pattern
space before checking for a string without %.

This patch is just mechanical insertion of %s; there are probably
several messages touched by this patch where we would be better
off giving the user more information than a fixed string.

* cfg.mk (sc_prohibit_diagnostic_without_format): New rule.
* src/datatypes.c (virUnrefConnect, virGetDomain)
(virUnrefDomain, virGetNetwork, virUnrefNetwork, virGetInterface)
(virUnrefInterface, virGetStoragePool, virUnrefStoragePool)
(virGetStorageVol, virUnrefStorageVol, virGetNodeDevice)
(virGetSecret, virUnrefSecret, virGetNWFilter, virUnrefNWFilter)
(virGetDomainSnapshot, virUnrefDomainSnapshot): Add %s wrapper.
* src/lxc/lxc_driver.c (lxcDomainSetBlkioParameters)
(lxcDomainGetBlkioParameters): Likewise.
* src/conf/domain_conf.c (virSecurityDeviceLabelDefParseXML)
(virDomainDiskDefParseXML, virDomainGraphicsDefParseXML):
Likewise.
* src/conf/network_conf.c (virNetworkDNSHostsDefParseXML)
(virNetworkDefParseXML): Likewise.
* src/conf/nwfilter_conf.c (virNWFilterIsValidChainName):
Likewise.
* src/conf/nwfilter_params.c (virNWFilterVarValueCreateSimple)
(virNWFilterVarAccessParse): Likewise.
* src/libvirt.c (virDomainSave, virDomainSaveFlags)
(virDomainRestore, virDomainRestoreFlags)
(virDomainSaveImageGetXMLDesc, virDomainSaveImageDefineXML)
(virDomainCoreDump, virDomainGetXMLDesc)
(virDomainMigrateVersion1, virDomainMigrateVersion2)
(virDomainMigrateVersion3, virDomainMigrate, virDomainMigrate2)
(virStreamSendAll, virStreamRecvAll)
(virDomainSnapshotGetXMLDesc): Likewise.
* src/nwfilter/nwfilter_dhcpsnoop.c (virNWFilterSnoopReqLeaseDel)
(virNWFilterDHCPSnoopReq): Likewise.
* src/openvz/openvz_driver.c (openvzUpdateDevice): Likewise.
* src/openvz/openvz_util.c (openvzKBPerPages): Likewise.
* src/qemu/qemu_cgroup.c (qemuSetupCgroup): Likewise.
* src/qemu/qemu_command.c (qemuBuildHubDevStr, qemuBuildChrChardevStr)
(qemuBuildCommandLine): Likewise.
* src/qemu/qemu_driver.c (qemuDomainGetPercpuStats): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainAttachNetDevice): Likewise.
* src/rpc/virnetsaslcontext.c (virNetSASLSessionGetIdentity):
Likewise.
* src/rpc/virnetsocket.c (virNetSocketNewConnectUNIX)
(virNetSocketSendFD, virNetSocketRecvFD): Likewise.
* src/storage/storage_backend_disk.c
(virStorageBackendDiskBuildPool): Likewise.
* src/storage/storage_backend_fs.c
(virStorageBackendFileSystemProbe)
(virStorageBackendFileSystemBuild): Likewise.
* src/storage/storage_backend_rbd.c
(virStorageBackendRBDOpenRADOSConn): Likewise.
* src/storage/storage_driver.c (storageVolumeResize): Likewise.
* src/test/test_driver.c (testInterfaceChangeBegin)
(testInterfaceChangeCommit, testInterfaceChangeRollback):
Likewise.
* src/vbox/vbox_tmpl.c (vboxListAllDomains): Likewise.
* src/xenxs/xen_sxpr.c (xenFormatSxprDisk, xenFormatSxpr):
Likewise.
* src/xenxs/xen_xm.c (xenXMConfigGetUUID, xenFormatXMDisk)
(xenFormatXM): Likewise.

This patch cleans up coding style breaks some long lines and fixes error
message spelling on various places in domain_conf.c and in the qemu
driver.

Per the FSF address could be changed from time to time, and GNU
recommends the following now: (http://www.gnu.org/licenses/gpl-howto.html)

  You should have received a copy of the GNU General Public License
  along with Foobar.  If not, see <http://www.gnu.org/licenses/>.

This patch removes the explicit FSF address, and uses above instead
(of course, with inserting 'Lesser' before 'General').

Except a bunch of files for security driver, all others are changed
automatically, the copyright for securify files are not complete,
that's why to do it manually:

  src/security/security_selinux.h
  src/security/security_driver.h
  src/security/security_selinux.c
  src/security/security_apparmor.h
  src/security/security_apparmor.c
  src/security/security_driver.c

The previous check for YAJL would have many undesirable
consequences, the most important being that it caused the
capabilities XML to lose all <guest> elements. There is
no user visible feedback as to what is wrong in this respect,
merely a syslog message. The empty capabilities causes
libvirtd to then throw away all guest XML configs that are
stored.

This changes the code so that the check for YAJL is only
performed at the time we attempt to spawn a QEMU process

error: Failed to start domain vm-vnc
error: unsupported configuration: this qemu binary requires libvirt to be compiled with yajl
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Update the QEMU driver to use virReportError instead of
the qemuReportError custom macro
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch cleans up some missing "%s" before translation macros,
for strings which are const without format specifiers

When sending SIGHUP to libvirtd, it will trigger the virStateDriver
reload operation. This is intended to reload the configuration files
for guests. For unknown historical reasons this is also triggering
autostart of all guests. Autostart is generally expected to be
something that happens on OS startup. Starting VMs on SIGHUP will
violate that expectation and potentially cause dangerous scenarios
if the admin has explicitly shutdown a misbehaving VM that has
been marked as autostart
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Noticed during the recent error cleanups.

* src/network/bridge_driver.c (networkStartRadvd): Fix spacing.
* src/openvz/openvz_conf.c (openvzReadMemConf): Likewise.
* src/qemu/qemu_command.c (qemuNetworkIfaceConnect): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainDetachNetDevice): Likewise.
* src/qemu/qemu_process.c (qemuProcessStop): Likewise.
* src/security/virt-aa-helper.c (vah_add_file): Likewise.

Introduce new members in the virMacAddr 'class'
- virMacAddrSet: set virMacAddr from a virMacAddr
- virMacAddrSetRaw: setting virMacAddr from raw 6 byte MAC address buffer
- virMacAddrGetRaw: writing virMacAddr into raw 6 byte MAC address buffer
- virMacAddrCmp: comparing two virMacAddr
- virMacAddrCmpRaw: comparing a virMacAddr with a raw 6 byte MAC address buffer

then replace raw MAC addresses by replacing

- 'unsigned char *' with virMacAddrPtr
- 'unsigned char ... [VIR_MAC_BUFLEN]' with virMacAddr

and introduce usage of above functions where necessary.

When host CPU could not be properly detected, virConnectCompareCPU will
just report that any CPU is incompatible with host CPU instead of
failing.

Even though qemu-kvm binaries can be used in TCG mode, libvirt would
only detect them if /dev/kvm was available. Thus, one would need to make
a /usr/bin/qemu symlink to be able to use TCG mode with qemu-kvm in an
environment without KVM support.

And even though QEMU is able to make use of KVM, libvirt would not
advertise KVM support unless there was a qemu-kvm symlink available.

This patch fixes both issues.

If QEMU supports the BALLOON_EVENT QMP event, then we can
avoid invoking 'query-balloon' when returning XML or the
domain info.

* src/qemu/qemu_capabilities.c, src/qemu/qemu_capabilities.h:
  Add QEMU_CAPS_BALLOON_EVENT
* src/qemu/qemu_driver.c: Skip query-balloon in
  qemudDomainGetInfo and qemuDomainGetXMLDesc if we have
  QEMU_CAPS_BALLOON_EVENT set
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Check
  for BALLOON_EVENT at connect to monitor. Add callback
  for balloon change notifications
* src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h:
  Add handling of BALLOON_EVENT and impl 'query-events'
  check
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The s390(x) architecture doesn't feature a PCI bus. For the purpose of
supporting virtio devices a virtual bus called virtio-s390 is used.
A new address type VIR_DOMAIN_DEVICE_ADDRESS_TYPE_VIRTIO_S390 is used to
distinguish the virtio devices on s390 from PCI-based virtio devices.

V3 Change: updated QEMU_CAPS_VIRTIO_S390 to fit upstream.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

This is in preparation of the enablement of s390 guests with virtio devices.

The assignment of device addresses happens in different places, i.e. the
qemu driver and process modules as well as in the unit tests in slightly
different flavors. Currently, these are PPC spapr-vio and PCI
devices, virtio-s390 (not PCI based) will follow.

By optionally passing to qemuDomainAssignAddresses the domain
object and the capabilities it is now possible to call the function
from most of the places (except for hotplug) where address assignment
is done.
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

Commit 5e6ce1 moved down detection of the ACPI feature in
qemuParseCommandLine. However, when ACPI is detected, it clears
all feature flags in def->features to only set ACPI. This used to
be fine because this was the first place were def->features was set,
but after the move this is no longer necessarily true because this
block comes before the ACPI check:

if (strstr(def->emulator, "kvm")) {
    def->virtType = VIR_DOMAIN_VIRT_KVM;
    def->features |= (1 << VIR_DOMAIN_FEATURE_PAE);
}

Since def is allocated in qemuParseCommandLine using VIR_ALLOC, we
can always use |= when modifying def->features