Over root-squashing nfs, when virFileOperation() is called as uid==0,
it may fail with EACCES, but also with EPERM, due to
virFileOperationNoFork()'s failed attemp to chown a writable file.

qemudDomainSaveFlag() should expect this case, too.

qemudOpenAsUID is intended to open a file with the credentials of a
specified uid. Current implementation fails if the file is accessible to
one of uid's groups but not owned by uid.

This patch replaces the supplementary group list that the child process
inherited from libvirtd with the default group list of uid.

* docs/formatdomain.html.in: Add memtune element details, added min_guarantee
* src/libvirt.c: Update virDomainGetMemoryParameters api description, make
  it more clear that the user first needs to call the api to get the number
  of parameters supported and then call again to get the values.
* tools/virsh.pod: Add usage of new command memtune in virsh manpage

VMware uses a mix of percent-, pipe- and base64-encoding in
different combinations in different places.

Add a testcase for this.

Patch to prevent multiple nwfilters with different name but same UUID.

This introduces new attribute to filesystem element
to support customizable access mode for mount type.
Valid accessmode are: passthrough, mapped and squash.

Usage:
        <filesystem type='mount' accessmode='passthrough'>
          <source dir='/export/to/guest'/>
          <target dir='mount_tag'/>
        </filesystem>

passthrough is the default model if not specified, that's
also the current behaviour.

The following filter transition from a filter allowing incoming TCP connections

  <rule action='accept' direction='in' priority='401'>
    <tcp/>
  </rule>
  <rule action='accept' direction='out' priority='500'>
    <tcp/>
  </rule>

to one that does not allow them

  <rule action='drop' direction='in' priority='401'>
    <tcp/>
  </rule>
  <rule action='accept' direction='out' priority='500'>
    <tcp/>
  </rule>

did previously not cut off existing (ssh) connections but only prevented newly initiated ones. The attached patch allows to cut off existing connections as well, thus enforcing what the filter is showing.

I had only tested with a configuration where the physical interface is connected to the bridge where the filters are applied. This patch now also solves a filtering problem where the physical interface is not connected to the bridge, but the bridge is given an IP address and the host routes between bridge and physical interface. Here the filters drop non-allowed traffic on the outgoing side on the host.

Explicitly raising a nice error in the case user tries to migrate a
guest with assigned host devices is much better than waiting for a
mysterious error with no clue for the reason.

When only some host CPUs given to cpuBaseline contain <vendor> element,
baseline CPU should not contain it. Otherwise the result would not be
compatible with the host CPUs without vendor. CPU vendors are still
taken into account when computing baseline CPU, it's just removed from
the result.

Recent CPU models were specified using invalid vendor element
<vendor>NAME</vendor>, which was silently ignored due to a bug in the
code which was parsing it.

'make -C src rpcgen' is supposed to be idempotent.  But commit
f928f43b mistakently manually edited a generated file rather
than fixing the upstream file.

* src/remote/remote_protocol.x (remote_memory_param_value): Use
correct spelling of enum values.
* src/remote/remote_protocol.c: Regenerate.

This enables support for nested SVM using the regular CPU
model/features block. If the CPU model or features include
'svm', then the '-enable-nesting' flag will be added to the
QEMU command line. Latest out of tree patches for nested
'vmx', no longer require the '-enable-nesting' flag. They
instead just look at the cpu features. Several of the models
already include svm support, but QEMU was just masking out
the svm bit silently. So this will enable SVM on such
models

* src/qemu/qemu_conf.h: flag for -enable-nesting
* src/qemu/qemu_conf.c: Use -enable-nesting if VMX or SVM are in
  the CPUID
* src/cpu/cpu.h, src/cpu/cpu.c: API to check for a named feature
* src/cpu/cpu_x86.c: x86 impl of feature check
* src/libvirt_private.syms: Add cpuHasFeature
* src/qemuhelptest.c: Add nesting flag where required

* src/xen/sexpr.c: Ensure () are escaped in sexpr2string
* tests/sexpr2xmldata/sexpr2xml-boot-grub.sexpr,
  tests/sexpr2xmldata/sexpr2xml-boot-grub.xml,
  tests/xml2sexprdata/xml2sexpr-boot-grub.sexpr,
  tests/xml2sexprdata/xml2sexpr-boot-grub.xml: Data files to
  check escaping
* tests/sexpr2xmltest.c, tests/xml2sexprtest.c: Add boot-grub
  escaping test case

 This is from a bug report and conversation on IRC where Soren reported that while a filter update is occurring on one or more VMs (due to a rule having been edited for example), a deadlock can occur when a VM referencing a filter is started.

The problem is caused by the two locking sequences of

qemu driver, qemu domain, filter             # for the VM start operation
filter, qemu_driver, qemu_domain            # for the filter update operation

that obviously don't lock in the same order. The problem is the 2nd lock sequence. Here the qemu_driver lock is being grabbed in qemu_driver:qemudVMFilterRebuild()

The following solution is based on the idea of trying to re-arrange the 2nd sequence of locks as follows:

qemu_driver, filter, qemu_driver, qemu_domain

and making the qemu driver recursively lockable so that a second lock can occur, this would then lead to the following net-locking sequence

qemu_driver, filter, qemu_domain

where the 2nd qemu_driver lock has been ( logically ) eliminated.

The 2nd part of the idea is that the sequence of locks (filter, qemu_domain) and (qemu_domain, filter) becomes interchangeable if all code paths where filter AND qemu_domain are locked have a preceding qemu_domain lock that basically blocks their concurrent execution

So, the following code paths exist towards qemu_driver:qemudVMFilterRebuild where we now want to put a qemu_driver lock in front of the filter lock.

-> nwfilterUndefine()   [ locks the filter ]
    -> virNWFilterTestUnassignDef()
        -> virNWFilterTriggerVMFilterRebuild()
            -> qemudVMFilterRebuild()

-> nwfilterDefine()
    -> virNWFilterPoolAssignDef() [ locks the filter ]
        -> virNWFilterTriggerVMFilterRebuild()
            -> qemudVMFilterRebuild()

-> nwfilterDriverReload()
    -> virNWFilterPoolLoadAllConfigs()
        ->virNWFilterPoolObjLoad()
            -> virNWFilterPoolAssignDef() [ locks the filter ]
                -> virNWFilterTriggerVMFilterRebuild()
                    -> qemudVMFilterRebuild()

-> nwfilterDriverStartup()
    -> virNWFilterPoolLoadAllConfigs()
        ->virNWFilterPoolObjLoad()
            -> virNWFilterPoolAssignDef() [ locks the filter ]
                -> virNWFilterTriggerVMFilterRebuild()
                    -> qemudVMFilterRebuild()

Qemu is not the only driver using the nwfilter driver, but also the UML driver calls into it. Therefore qemuVMFilterRebuild() can be exchanged with umlVMFilterRebuild() along with the driver lock of qemu_driver that can now be a uml_driver. Further, since UML and Qemu domains can be running on the same machine, the triggering of a rebuild of the filter can touch both types of drivers and their domains.

In the patch below I am now extending each nwfilter callback driver with functions for locking and unlocking the (VM) driver (UML, QEMU) and introduce new functions for locking all registered callback drivers and unlocking them. Then I am distributing the lock-all-cbdrivers/unlock-all-cbdrivers call into the above call paths. The last shown callpath starting with nwfilterDriverStart() is problematic since it is initialize before the Qemu and UML drives are and thus a lock in the path would result in a NULL pointer attempted to be locked -- the call to virNWFilterTriggerVMFilterRebuild() is never called, so we never lock either the qemu_driver or the uml_driver in that path. Therefore, only the first 3 paths now receive calls to lock and unlock all callback drivers. Now that the locks are distributed where it matters I can remove the qemu_driver and uml_driver lock from qemudVMFilterRebuild() and umlVMFilterRebuild() and not requiring the recursive locks.

For now I want to put this out as an RFC patch. I have tested it by 'stretching' the critical section after the define/undefine functions each lock the filter so I can (easily) concurrently execute another VM operation (suspend,start). That code is in this patch and if you want you can de-activate it. It seems to work ok and operations are being blocked while the update is being done.
I still also want to verify the other assumption above that locking filter and qemu_domain always has a preceding qemu_driver lock.

We don't fail when we can't contact HAL so we shouldn't fail if we can't
contact D-Bus either.

* include/libvirt/libvirt.h.in: some of the function type description
  were broken so they could not be automatically documented
* src/util/event.c docs/apibuild.py: event.c exports one public API
  so it needs to be scanned too, avoid a few warnings

Make use of the existing <filesystem> element to support plan9fs
filesystem passthrough in the QEMU driver

    <filesystem type='mount'>
      <source dir='/export/to/guest'/>
      <target dir='/import/from/host'/>
    </filesystem>

NB, the target is not actually a directory, it is merely a arbitrary
string tag that is exported to the guest as a hint for where to mount
it.

Add proper documentation to the new VIR_DOMAIN_MEMORY_* macros in
libvirt.h.in to placate apibuild.py.

Mark args as unused in for libvirt_virDomain{Get,Set}MemoryParameters
in the Python bindings and add both to the libvirtMethods array.

Update remote_protocol-structs to placate make syntax-check.

Undo unintended modifications in vboxDomainGetInfo.

Update the function table of the VirtualBox and XenAPI drivers.

Driver interface for getting memory parameters, eg. hard_limit,
soft_limit and swap_hard_limit.

Add support in the lxc driver for various memory controllable parameters

libvirt-lxc now configures the hardlimit, softlimit and swaplimit, if
specified in the domain xml file or picks up the defaults.

QEmu startup will pick up the memory tunables specified in the domain
configuration file

Driver interface for getting memory parameters, eg. hard_limit,
soft_limit and swap_hard_limit based on cgroup support

Driver interface for setting memory hard_limit, soft_limit and swap
hard_limit based on cgroup support

Provides interfaces for setting/getting memory tunables like hard_limit,
soft_limit and swap_hard_limit

Adding parsing code for memory tunables in the domain xml file
also change the internal define structures used for domain memory
informations
Adds a new specific test

Public api to set/get memory tunables supported by the hypervisors.

dv:
* some cleanups in libvirt.c
* adding extra checks in libvirt.c new entry points

v4:
* Move exporting public API to this patch
* Add unsigned int flags to the public api for future extensions

v3:
* Add domainGetMemoryParamters and NULL in all the driver interface

v2:
* Initialize domainSetMemoryParameters to NULL in all the driver
  interface structure.

Some features provided by the recently added CPU models were mentioned
twice for each model. This was a result of automatic generation of the
XML from qemu's CPU configuration file without noticing this redundancy.

Commit 1fe2927a forgot to export a symbol.

* src/libvirt_private.syms (virHexToBin): Add.
* src/.gitignore: Ignore temporary file.

To enable the CPU XML from the capabilities to be pasted directly
into the guest XML with no editing, pick a sensible default for
match and feature policy. The CPU match will be exact and the
feature policy will be require. This should ensure safety for
migration and give DWIM semantics for users

* src/conf/cpu_conf.c: Default to exact match and require policy
* docs/formatdomain.html.in: Document new defaults

According to API documentation virDomain{At,De}tachDevice calls are
supposed to only work on active guests for device hotplug. For anything
beyond that, their *Flags variants have to be used.

Despite the variant which was acked on libvirt mailing list
(https://www.redhat.com/archives/libvir-list/2010-January/msg00385.html)
commit ed9c14a7 (by Jim Fehlig)
introduced automagic behavior of these API calls for xen driver. Since
January, these calls always change persistent configuration of a guest
and if the guest is currently active, they also hot(un)plug the device.

That change didn't follow API documentation and also broke device
hot(un)plug for older xend implementations which do not support changing
persistent configuration of a guest and hot(un)plugging in one step.

This patch should not break anything for active guests. On the other
hand, changing inactive guests is not supported any more.

xenXMDomain*DeviceFlags() silently ignores requests to modify live
configuration of an active guest while still touching its persistent
configuration.

When a user calls to virDomain{Attach,Detach,Update}DeviceFlags() with
flags == VIR_DOMAIN_DEVICE_MODIFY_LIVE on an inactive guest running on
an old Xen hypervisor (such as RHEL-5) xend_internal driver reports:

    Xend version does not support modifying persistent config

which is pretty confusing since no-one requested to modify persistent
config.

to avoid long timeouts waiting for DNS servers