To enable virsh console (or equivalent) to be used remotely
it is necessary to provide remote access to the /dev/pts/XXX
pseudo-TTY associated with the console/serial/parallel device
in the guest. The virStream API provide a bi-directional I/O
stream capability that can be used for this purpose. This
patch thus introduces a virDomainOpenConsole API that uses
the stream APIs.

* src/libvirt.c, src/libvirt_public.syms,
  include/libvirt/libvirt.h.in, src/driver.h: Define the
  new virDomainOpenConsole API
* src/esx/esx_driver.c, src/lxc/lxc_driver.c,
  src/opennebula/one_driver.c, src/openvz/openvz_driver.c,
  src/phyp/phyp_driver.c, src/qemu/qemu_driver.c,
  src/remote/remote_driver.c, src/test/test_driver.c,
  src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c, src/xenapi/xenapi_driver.c: Stub
  API entry point

Using automated replacement with sed and editing I have now replaced all
occurrences of close() with VIR_(FORCE_)CLOSE() except for one, of
course. Some replacements were straight forward, others I needed to pay
attention. I hope I payed attention in all the right places... Please
have a look. This should have at least solved one more double-close
error.

* src/driver.h (virDrvDomainSetVcpusFlags)
(virDrvDomainGetVcpusFlags): New typedefs.
(_virDriver): New callback members.
* src/esx/esx_driver.c (esxDriver): Add stub for driver.
* src/lxc/lxc_driver.c (lxcDriver): Likewise.
* src/opennebula/one_driver.c (oneDriver): Likewise.
* src/openvz/openvz_driver.c (openvzDriver): Likewise.
* src/phyp/phyp_driver.c (phypDriver): Likewise.
* src/qemu/qemu_driver.c (qemuDriver): Likewise.
* src/remote/remote_driver.c (remote_driver): Likewise.
* src/test/test_driver.c (testDriver): Likewise.
* src/uml/uml_driver.c (umlDriver): Likewise.
* src/vbox/vbox_tmpl.c (Driver): Likewise.
* src/xen/xen_driver.c (xenUnifiedDriver): Likewise.
* src/xenapi/xenapi_driver.c (xenapiDriver): Likewise.

 This is from a bug report and conversation on IRC where Soren reported that while a filter update is occurring on one or more VMs (due to a rule having been edited for example), a deadlock can occur when a VM referencing a filter is started.

The problem is caused by the two locking sequences of

qemu driver, qemu domain, filter             # for the VM start operation
filter, qemu_driver, qemu_domain            # for the filter update operation

that obviously don't lock in the same order. The problem is the 2nd lock sequence. Here the qemu_driver lock is being grabbed in qemu_driver:qemudVMFilterRebuild()

The following solution is based on the idea of trying to re-arrange the 2nd sequence of locks as follows:

qemu_driver, filter, qemu_driver, qemu_domain

and making the qemu driver recursively lockable so that a second lock can occur, this would then lead to the following net-locking sequence

qemu_driver, filter, qemu_domain

where the 2nd qemu_driver lock has been ( logically ) eliminated.

The 2nd part of the idea is that the sequence of locks (filter, qemu_domain) and (qemu_domain, filter) becomes interchangeable if all code paths where filter AND qemu_domain are locked have a preceding qemu_domain lock that basically blocks their concurrent execution

So, the following code paths exist towards qemu_driver:qemudVMFilterRebuild where we now want to put a qemu_driver lock in front of the filter lock.

-> nwfilterUndefine()   [ locks the filter ]
    -> virNWFilterTestUnassignDef()
        -> virNWFilterTriggerVMFilterRebuild()
            -> qemudVMFilterRebuild()

-> nwfilterDefine()
    -> virNWFilterPoolAssignDef() [ locks the filter ]
        -> virNWFilterTriggerVMFilterRebuild()
            -> qemudVMFilterRebuild()

-> nwfilterDriverReload()
    -> virNWFilterPoolLoadAllConfigs()
        ->virNWFilterPoolObjLoad()
            -> virNWFilterPoolAssignDef() [ locks the filter ]
                -> virNWFilterTriggerVMFilterRebuild()
                    -> qemudVMFilterRebuild()

-> nwfilterDriverStartup()
    -> virNWFilterPoolLoadAllConfigs()
        ->virNWFilterPoolObjLoad()
            -> virNWFilterPoolAssignDef() [ locks the filter ]
                -> virNWFilterTriggerVMFilterRebuild()
                    -> qemudVMFilterRebuild()

Qemu is not the only driver using the nwfilter driver, but also the UML driver calls into it. Therefore qemuVMFilterRebuild() can be exchanged with umlVMFilterRebuild() along with the driver lock of qemu_driver that can now be a uml_driver. Further, since UML and Qemu domains can be running on the same machine, the triggering of a rebuild of the filter can touch both types of drivers and their domains.

In the patch below I am now extending each nwfilter callback driver with functions for locking and unlocking the (VM) driver (UML, QEMU) and introduce new functions for locking all registered callback drivers and unlocking them. Then I am distributing the lock-all-cbdrivers/unlock-all-cbdrivers call into the above call paths. The last shown callpath starting with nwfilterDriverStart() is problematic since it is initialize before the Qemu and UML drives are and thus a lock in the path would result in a NULL pointer attempted to be locked -- the call to virNWFilterTriggerVMFilterRebuild() is never called, so we never lock either the qemu_driver or the uml_driver in that path. Therefore, only the first 3 paths now receive calls to lock and unlock all callback drivers. Now that the locks are distributed where it matters I can remove the qemu_driver and uml_driver lock from qemudVMFilterRebuild() and umlVMFilterRebuild() and not requiring the recursive locks.

For now I want to put this out as an RFC patch. I have tested it by 'stretching' the critical section after the define/undefine functions each lock the filter so I can (easily) concurrently execute another VM operation (suspend,start). That code is in this patch and if you want you can de-activate it. It seems to work ok and operations are being blocked while the update is being done.
I still also want to verify the other assumption above that locking filter and qemu_domain always has a preceding qemu_driver lock.

Adding parsing code for memory tunables in the domain xml file
also change the internal define structures used for domain memory
informations
Adds a new specific test

Public api to set/get memory tunables supported by the hypervisors.

dv:
* some cleanups in libvirt.c
* adding extra checks in libvirt.c new entry points

v4:
* Move exporting public API to this patch
* Add unsigned int flags to the public api for future extensions

v3:
* Add domainGetMemoryParamters and NULL in all the driver interface

v2:
* Initialize domainSetMemoryParameters to NULL in all the driver
  interface structure.

When nwfilter support was added to UML, I didn't realise the UML driver
needed instrumentation to make updating nwfilters on the fly work. This
patch adds this bit of glue.
Signed-off-by: NSoren Hansen <soren@linux2go.dk>

Extend user-mode-linux driver to support nwfilter.
Signed-off-by: NSoren Hansen <soren@linux2go.dk>

For privileged UML connections (uml:///system), we shouldn't use root's
home dir, but rather somewhere in /var/run/libvirt/uml-guest.

https://bugzilla.redhat.com/show_bug.cgi?id=499536Signed-off-by: NSoren Hansen <soren@linux2go.dk>

UML supports hot plugging and unplugging of various devices. This patch
exposes this functionality for disks.
Signed-off-by: NSoren Hansen <soren@linux2go.dk>

Like the comment suggested, we just open the file and pass the file
descriptor to uml. The input "stream" is set to "null", since I couldn't
find any useful way to actually use a file for input for a chardev and
this also mimics what e.g. QEmu does internally.
Signed-off-by: NSoren Hansen <soren@linux2go.dk>

* src/qemu/qemu_driver.c (qemudGetProcessInfo): Clean up.
* src/uml/uml_driver.c (umlGetProcessInfo): Likewise.
* src/xen/sexpr.c (_string2sexpr): Likewise.

* src/uml/uml_driver.c (umlMonitorCommand): Validate that enough
bytes were read to dereference both res.length, and that many
bytes from res.data.
Reported by Soren Hansen.

Add the library entry point for the new virDomainQemuMonitorCommand()
entry point.  Because this is not part of the "normal" libvirt API,
it gets its own header file, library file, and will eventually
get its own over-the-wire protocol later in the series.

Changes since v1:
 - Go back to using the virDriver table for qemuDomainMonitorCommand, due to
   linking issues
 - Added versioning information to the libvirt-qemu.so

Changes since v2:
 - None

Changes since v3:
 - Add LGPL header to libvirt-qemu.c
 - Make virLibConnError and virLibDomainError macros instead of function calls

Changes since v4:
 - Move exported symbols to libvirt_qemu.syms
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* src/uml/uml_driver.c (umlMonitorCommand): Correct flaw that would
cause unconditional "incomplete reply ..." failure, since "nbytes"
was always 0 or 1.

* src/esx/esx_driver.c (esxDomainCreate): Move guts...
(esxDomainCreateWithFlags): ...to new function.
(esxDriver): Trivially support the new API.
* src/lxc/lxc_driver.c (lxcDomainStart, lxcDomainStartWithFlags)
(lxcDriver): Likewise.
* src/opennebula/one_driver.c (oneDomainStart)
(oneDomainStartWithFlags, oneDriver): Likewise.
* src/openvz/openvz_driver.c (openvzDomainCreate)
(openvzDomainCreateWithFlags, openvzDriver): Likewise.
* src/qemu/qemu_driver.c (qemudDomainStart)
(qemudDomainStartWithFlags, qemuDriver): Likewise.
* src/test/test_driver.c (testDomainCreate)
(testDomainCreateWithFlags, testDriver): Likewise.
* src/uml/uml_driver.c (umlDomainStart, umlDomainStartWithFlags)
(umlDriver): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainCreate)
(vboxDomainCreateWithFlags, Driver): Likewise.
* src/xen/xen_driver.c (xenUnifiedDomainCreate)
(xenUnifiedDomainCreateWithFlags, xenUnifiedDriver): Likewise.
* src/xenapi/xenapi_driver.c (xenapiDomainCreate)
(xenapiDomainCreateWithFlags, xenapiDriver): Likewise.

Persistent domain creation needs the same features as transient
domains, but virDomainCreate lacks the flags argument present in
virDomainCreateXML.  virDomainCreateFlags is already claimed as
a public enum, so we have to break convention and expose
virDomainCreateWithFlags.

* include/libvirt/libvirt.h.in (virDomainCreateWithFlags): Add.
* src/driver.h (virDrvDomainCreateWithFlags): Internal API.
* src/libvirt.c (virDomainCreateWithFlags): Glue public API to
driver API.
* src/libvirt_public.syms (LIBVIRT_0.8.2): Expose public API.
* src/esx/esx_driver.c (esxDriver): Add stub for driver.
* src/lxc/lxc_driver.c (lxcDriver): Likewise.
* src/opennebula/one_driver.c (oneDriver): Likewise.
* src/openvz/openvz_driver.c (openvzDriver): Likewise.
* src/phyp/phyp_driver.c (phypDriver): Likewise.
* src/qemu/qemu_driver.c (qemuDriver): Likewise.
* src/remote/remote_driver.c (remote_driver): Likewise.
* src/test/test_driver.c (testDriver): Likewise.
* src/uml/uml_driver.c (umlDriver): Likewise.
* src/vbox/vbox_tmpl.c (Driver): Likewise.
* src/xen/xen_driver.c (xenUnifiedDriver): Likewise.
* src/xenapi/xenapi_driver.c (xenapiDriver): Likewise.

Otherwise, a malicious packet could cause a DoS via spurious
out-of-memory failure.

* src/uml/uml_driver.c (umlMonitorCommand): Validate that incoming
data is reliable before using it to allocate/dereference memory.
Don't report bogus errno on short read.
Reported by Jim Meyering.

* include/libvirt/libvirt.h.in (virDomainCreateFlags): Add
VIR_DOMAIN_START_PAUSED.
* src/libvirt.c (virDomainCreateXML): Update documentation.
* src/lxc/lxc_driver.c (lxcDomainCreateAndStart): Reject new flag
as unimplemented.
* src/opennebula/one_driver.c (oneDomainCreateAndStart):
Likewise.
* src/openvz/openvz_driver.c (openvzDomainCreateXML): Likewise.
* src/phyp/phyp_driver.c (phypDomainCreateAndStart): Likewise.
* src/qemu/qemu_driver.c (qemudDomainCreate): Likewise.
* src/test/test_driver.c (testDomainCreateXML): Likewise.
* src/uml/uml_driver.c (umlDomainCreate): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainCreateXML): Likewise.
* src/xen/xend_internal.c (xenDaemonCreateXML): Likewise.
* src/xenapi/xenapi_driver.c (xenapiDomainCreateXML): Likewise.

These changes avoid false-positive syntax-check failure,
and also make the resulting diagnostics more comprehensible.

Run this:
  git grep -l 'VIR_ERROR\s*("'|xargs perl -pi -e \
    's/(VIR_ERROR)\s*\((".*?"),/$1(_($2),/'

Run this:
  git grep -l 'VIR_ERROR0\s*("'|xargs perl -pi -e \
    's/(VIR_ERROR0)\s*\((".*?")\)/$1(_($2))/'

Approximately 60 messages were marked.  Since these diagnostics are
intended solely for developers and maintainers, encouraging translation
is deemed to be counterproductive:
http://thread.gmane.org/gmane.comp.emulators.libvirt/25050/focus=25052

Run this command:
  git grep -l VIR_WARN|xargs perl -pi -e \
    's/(VIR_WARN0?)\s*\(_\((".*?")\)/$1($2/'

* src/uml/uml_driver.c (umlAutostartDomain): Handle a NULL return
from virGetLastError.

81 patches to gnulib, picks up several new syntax checks.

* .gnulib: Update to latest.
* .x-sc_prohibit_always_true_header_tests: New file.
* bootstrap.conf (gnulib_modules): Add sched.
* src/util/processinfo.c (includes): <sched.h> is now guaranteed.
* src/network/bridge_driver.c (includes): Drop useless
<strings.h>.
* src/openvz/openvz_conf.c (includes): Likewise.
* src/openvz/openvz_driver.c (includes): Likewise.
* src/phyp/phyp_driver.c (includes): Likewise.
* src/qemu/qemu_driver.c (includes): Likewise.
* src/uml/uml_driver.c (includes): Likewise.

This defines the internal driver API and stubs out each driver

* src/driver.h: Define virDrvDomainGetBlockInfo signature
* src/libvirt.c, src/libvirt_public.syms: Glue public API to drivers
* src/esx/esx_driver.c, src/lxc/lxc_driver.c, src/opennebula/one_driver.c,
  src/openvz/openvz_driver.c, src/phyp/phyp_driver.c,
  src/test/test_driver.c, src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c, src/xenapi/xenapi_driver.c: Stub out driver

and domainIsPersistent.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

virDomainManagedSave() is to be run on a running domain. Once the call
complete, as in virDomainSave() the domain is stopped upon completion,
but there is no restore counterpart as any order to start the domain
from the API would load the state from the managed file, similary if
the domain is autostarted when libvirtd starts.
Once a domain has restarted his managed save image is destroyed,
basically managed save image can only exist for a stopped domain,
for a running domain that would be by definition outdated data.

* include/libvirt/libvirt.h.in src/libvirt.c src/libvirt_public.syms:
  adds the new entry points virDomainManagedSave(),
  virDomainHasManagedSaveImage() and virDomainManagedSaveRemove()
* src/driver.h src/esx/esx_driver.c src/lxc/lxc_driver.c
  src/opennebula/one_driver.c  src/openvz/openvz_driver.c
  src/phyp/phyp_driver.c src/qemu/qemu_driver.c src/vbox/vbox_tmpl.c
  src/remote/remote_driver.c src/test/test_driver.c src/uml/uml_driver.c
  src/xen/xen_driver.c: add corresponding new internal drivers entry
  points

virParseVersionString uses virStrToLong_ui instead of sscanf.

This also fixes a bug in the UML driver, that always returned 0
as version number.

Introduce STRSKIP to check if a string has a certain prefix and
to skip this prefix.

When a domain is defined on host1, migrated to host2 and then migrated
back to host1, its current configuration would overwrite the libvirtd's
in-memory copy of persistent configuration of that domain. This is not
desired as we want to preserve the persistent configuration untouched.

This patch introduces new 'live' parameter to virDomainAssignDef.
Passing 'true' for 'live' means the configuration passed to
virDomainAssignDef describes a configuration of live instance of the
domain. This applies for saved domains which are being restored or for
incoming domains during migration.

All callers have been changed to pass the appropriate value.

The current virDomainAttachDevice API can be (ab)used to change
the media of an existing CDROM/Floppy device. Going forward there
will be more devices that can be configured on the fly and overloading
virDomainAttachDevice for this is not too pleasant. This patch adds
a new virDomainUpdateDeviceFlags() explicitly just for modifying
existing devices.

* include/libvirt/libvirt.h.in: Add virDomainUpdateDeviceFlags
* src/driver.h: Internal API for virDomainUpdateDeviceFlags
* src/libvirt.c, src/libvirt_public.syms: Glue public API to
  driver API
* src/esx/esx_driver.c, src/lxc/lxc_driver.c, src/opennebula/one_driver.c,
  src/openvz/openvz_driver.c, src/phyp/phyp_driver.c, src/qemu/qemu_driver.c,
  src/remote/remote_driver.c, src/test/test_driver.c, src/uml/uml_driver.c,
  src/vbox/vbox_tmpl.c, src/xen/xen_driver.c, src/xenapi/xenapi_driver.c: Add
  stubs for new driver entry point

The current API for domain events has a number of problems

 - Only allows for domain lifecycle change events
 - Does not allow the same callback to be registered multiple times
 - Does not allow filtering of events to a specific domain

This introduces a new more general purpose domain events API

  typedef enum {
     VIR_DOMAIN_EVENT_ID_LIFECYCLE = 0,       /* virConnectDomainEventCallback */
      ...more events later..
  }

  int virConnectDomainEventRegisterAny(virConnectPtr conn,
                                       virDomainPtr dom, /* Optional, to filter */
                                       int eventID,
                                       virConnectDomainEventGenericCallback cb,
                                       void *opaque,
                                       virFreeCallback freecb);

  int virConnectDomainEventDeregisterAny(virConnectPtr conn,
                                         int callbackID);

Since different event types can received different data in the callback,
the API is defined with a generic callback. Specific events will each
have a custom signature for their callback. Thus when registering an
event it is neccessary to cast the callback to the generic signature

eg

  int myDomainEventCallback(virConnectPtr conn,
                            virDomainPtr dom,
                            int event,
                            int detail,
                            void *opaque)
  {
    ...
  }

  virConnectDomainEventRegisterAny(conn, NULL,
                                   VIR_DOMAIN_EVENT_ID_LIFECYCLE,
                                   VIR_DOMAIN_EVENT_CALLBACK(myDomainEventCallback)
                                   NULL, NULL);

The VIR_DOMAIN_EVENT_CALLBACK() macro simply does a "bad" cast
to the generic signature

* include/libvirt/libvirt.h.in: Define new APIs for registering
  domain events
* src/driver.h: Internal driver entry points for new events APIs
* src/libvirt.c: Wire up public API to driver API for events APIs
* src/libvirt_public.syms: Export new APIs
* src/esx/esx_driver.c, src/lxc/lxc_driver.c, src/opennebula/one_driver.c,
  src/openvz/openvz_driver.c, src/phyp/phyp_driver.c,
  src/qemu/qemu_driver.c, src/remote/remote_driver.c,
  src/test/test_driver.c, src/uml/uml_driver.c,
  src/vbox/vbox_tmpl.c, src/xen/xen_driver.c,
  src/xenapi/xenapi_driver.c: Stub out new API entries

Coverity detected a potential dereference of uninitialized memory
if recvfrom got cut short.

* src/uml/uml_driver.c (umlMonitorCommand): Validate complete read
prior to dereferencing res.

This provides the internal glue for the driver API

* src/driver.h: Internal API contract
* src/libvirt.c, src/libvirt_public.syms: Connect public API
  to driver API
* src/esx/esx_driver.c, src/lxc/lxc_driver.c, src/opennebula/one_driver.c,
  src/openvz/openvz_driver.c, src/phyp/phyp_driver.c,
  src/qemu/qemu_driver.c, src/remote/remote_driver.c,
  src/test/test_driver.c src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c: Stub out entry points

The internal glue layer for the new pubic API

* src/driver.h: Define internal driver API contract
* src/libvirt.c, src/libvirt_public.syms: Wire up public
  API to internal driver API
* src/esx/esx_driver.c, src/lxc/lxc_driver.c, src/opennebula/one_driver.c,
  src/openvz/openvz_driver.c, src/phyp/phyp_driver.c,
  src/qemu/qemu_driver.c, src/remote/remote_driver.c,
  src/test/test_driver.c, src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c: Stub new entry point

* src/uml/uml_driver.c (umlMonitorCommand): This function would
sometimes return -1, yet fail to free the "reply" it had allocated.
Hence, no caller would know to free the corresponding argument.
When returning -1, be sure to free all allocated resources.