Unfortunately libssh2 doesn't support all types of host keys that can be
saved in the known_hosts file. Also it does not report that parsing of
the file failed. This results into truncated known_hosts files where the
standard client stores keys also in other formats (eg.
ecdsa-sha2-nistp256).

This patch changes the default location of the known_hosts file into the
libvirt private configuration directory, where it will be only written
by the libssh2 layer itself. This prevents trashing user's known_host
file.

The libssh2 code wasn't supposed to create the known_hosts file, but
recent findings show, that we can't use the default created by OpenSSH
as libssh2 might damage it. We need to create a private known_hosts file
in the config path.

This patch adds support for skipping error if the known_hosts file is
not present and let libssh2 create a new one.

allow the user change/get emulator's period and quota when the vm is running.

This patch adds two macros: VIR_DOMAIN_SCHEDULER_EMULATOR_PERIOD,
VIR_DOMAIN_SCHEDULER_EMULATOR_QUOTA for controlling cpu bandwidth
for emulator activities not tied to vcpus

This patch introduces support of setting emulator's period and
quota to limit cpu bandwidth when the vm starts.  Also updates
XML Schema for new entries and docs.

This patch changes the behaviour of xml element cputune.period
and cputune.quota to limit cpu bandwidth only for vcpus, and no
longer limit cpu bandwidth for the whole guest.

The reasons to do this are:

  - This matches docs of cputune.period and cputune.quota.
  - The other parts excepting vcpus are treated as "emulator",
    and there are separate period/quota settings for emulator
    in the subsequent patches

This is a helper function to print vcpu pin info.

Introduce 2 APIs to support emulator threads in remote driver.
    1) remoteDomainPinEmulator: call driver api, such as qemudDomainPinEmulator.
    2) remoteDomainGetEmulatorPinInfo: call driver api, such as qemudDomainGetEmulatorPinInfo.
They are similar to remoteDomainPinVcpuFlags and remoteDomainGetVcpuPinInfo.
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

Introduce 2 APIs to support emulator threads pin in qemu driver.

    1) qemudDomainPinEmulator: setup emulator threads pin info.
    2) qemudDomainGetEmulatorPinInfo: get all emulator threads pin info.

They are similar to qemudDomainPinVcpuFlags and qemudDomainGetVcpuPinInfo.
And also, remoteDispatchDomainPinEmulatorFlags and remoteDispatchDomainGetEmulatorPinInfo
functions are introduced.
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

Introduce 2 APIs to support emulator threads pin.
    1) virDomainEmulatorPinAdd: setup emulator threads pin with a given cpumap string.
    2) virDomainEmulatorPinDel: remove all emulator threads pin.
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

Introduce 2 APIs to set/get physical cpu pinning info of emulator threads.
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

Emulator threads should also be pinned by sched_setaffinity(), just
the same as vcpu threads.
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

Introduce qemuSetupCgroupEmulatorPin() function to add emulator
threads pin info to cpuset cgroup, the same as vcpupin.
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

This patch adds a new xml element <emulatorpin>, which is a sibling
to the existing <vcpupin> element under the <cputune>, to pin emulator
threads to specified physical CPUs.
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

vcpu threads pin are implemented using sched_setaffinity(), but
not controlled by cgroup. This patch does the following things:

    1) enable cpuset cgroup
    2) reflect all the vcpu threads pin info to cgroup
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

1. add a new function virDomainVcpuPinDefCopy
2. make virDomainVcpuPinDefFree non-static

Create a new cgroup and move all emulator threads to the new cgroup.
And then we can do the other things:
1. limit only vcpu usage rather than the whole qemu
2. limit for emulator threads(include vhost-net threads)
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

Introduce a new API to move tasks of one controller from a cgroup to another cgroup
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

Introduce the function virCgroupForEmulator() to create sub directory
for simulator thread(include I/O thread, vhost-net thread)
Signed-off-by: NWen Congyang <wency@cn.fujitsu.com>
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>

Since the move to systemd libvirt-guests doesn't output this progress
information anymore. This patch brings back this feature.

It is helpful to show the admin what the system is waiting for and what
is left of the timeout (e.g. for calibrating the shutdown timing of a ups).

Rewriting the current line with \r doesn't work anymore in the context
of systemd. So always write new lines, but move to 5 second intervals
to avoid flooding the console.

Qemu command line generation for geometry override and testcases.
Signed-off-by: NJ.B. Joret <jb@linux.vnet.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

A hypervisor may allow to override the disk geometry of drives.
Qemu, as an example with cyls=,heads=,secs=[,trans=].
This patch extends the domain config to allow the specification of
disk geometry with libvirt.
Signed-off-by: NJ.B. Joret <jb@linux.vnet.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

Older automake 1.9.6 (hello there, RHEL 5) did not populate
$(builddir), which meant 'make check' failed with:

make[3]: *** No rule to make target `/.libs/libvirt.la', needed by `check-symfile'.  Stop.

For that matter, even newer automake doesn't directly emit rules
to build .libs/libvirt.la; we are better off basing our rules
on the public ./libvirt.la.

* src/Makefile.am (check-symfile): Delete useless variable.

Without this patch, RHEL 5 fails to compile, since the dbus
files lives under /usr/include/dbus-1.0/dbus/dbus.h, and
DBUS_CFLAGS contains -I/usr/include/dbus-1.0.

In file included from network/bridge_driver.c:67:
../src/util/virdbus.h:26:25: error: dbus/dbus.h: No such file or directory

* src/Makefile.am (libvirt_driver_network_impl_la_CFLAGS): Add
DBUS_CFLAGS.

When gcc atomic intrinsics are not available (such as on RHEL 5
with gcc 4.1.2), we were getting link errors due to multiple
definitions:

./.libs/libvirt_util.a(libvirt_util_la-virobject.o): In function `virAtomicIntXor':
/home/dummy/l,ibvirt/src/util/viratomoic.h:404: multiple definition of `virAtomicIntXor'
./.libs/libvirt_util.a(libvirt_util_la-viratomic.o):/home/dummy/libvirt/src/util/viratomic.h:404: first defined here

Solve this by conditionally marking the functions static (the
condition avoids falling foul of gcc warnings about unused
static function declarations).

* src/util/viratomic.h: When not using gcc intrinsics, use static
functions to avoid linker errors on duplicate functions.

Building on RHEL 5 warned:

nodeinfo.c: 305: warning: implicit declaration of function 'CPU_COUNT'

This extension macro in <sched.h> was not added until later glibc.

* src/nodeinfo.c (CPU_COUNT): Add fallback implementation.

We already skip out on building the LXC under RHEL 5, because the
kernel is too old (commits 4c18acff, 2dee8965); but commit 9612e4b2
moved some LXC-only code into common files, resulting in this
build failure:

util/virfile.c: In function 'virFileLoopDeviceAssociate':
util/virfile.c:580: error: 'LO_FLAGS_AUTOCLEAR' undeclared (first use in this function)

Unfortunately, the kernel folks only made it an enum, rather than
also a #define, so we have to modify configure.ac to record when
it is usable.

* configure.ac (with_lxc): Mark when LO_FLAGS_AUTOCLEAR was found.
* src/util/virfile.c (virFileLoopDeviceAssociate): Avoid
compilation when kernel is too old.

Fix possible double close in the child process after the fork in case
infd and outfd are equal, just like they are after being called from
virNetSocketNewConnectCommand.

This patch provides basic support for using firewalld's firewall-cmd
rather than then plain eb/ip(6)tables commands.

* configure.ac, spec file: firewalld defaults to enabled if dbus is
  available, otherwise is disabled. If --with_firewalld is explicitly
  requested and dbus is not available, configure will fail.

* bridge_driver: add dbus filters to get the FirewallD1.Reloaded
  signal and DBus.NameOwnerChanged on org.fedoraproject.FirewallD1.
  When these are encountered, reload all the iptables reuls of all
  libvirt's virtual networks (similar to what happens when libvirtd is
  restarted).

* iptables, ebtables: use firewall-cmd's direct passthrough interface
  when available, otherwise use iptables and ebtables commands. This
  decision is made once the first time libvirt calls
  iptables/ebtables, and that decision is maintained for the life of
  libvirtd.

* Note that the nwfilter part of this patch was separated out into
  another patch by Stefan in V2, so that needs to be revised and
  re-reviewed as well.

================

All the configure.ac and specfile changes are unchanged from Thomas'
V3.

V3 re-ran "firewall-cmd --state" every time a new rule was added,
which was extremely inefficient.  V4 uses VIR_ONCE_GLOBAL_INIT to set
up a one-time initialization function.

The VIR_ONCE_GLOBAL_INIT(x) macro references a static function called
vir(Ip|Eb)OnceInit(), which will then be called the first time that
the static function vir(Ip|Eb)TablesInitialize() is called (that
function is defined for you by the macro). This is
thread-safe, so there is no chance of any race.

IMPORTANT NOTE: I've left the VIR_DEBUG messages in these two init
functions (one for iptables, on for ebtables) as VIR_WARN so that I
don't have to turn on all the other debug message just to see
these. Even if this patch doesn't need any other modification, those
messages need to be changed to VIR_DEBUG before pushing.

This one-time initialization works well. However, I've encountered
problems with testing:

1) Whenever I have enabled the firewalld service, *all* attempts to
call firewall-cmd from within libvirtd end with firewall-cmd hanging
internally somewhere. This is *not* the case if firewall-cmd returns
non-0 in response to "firewall-cmd --state" (i.e. *that* command runs
and returns to libvirt successfully.)

2) If I start libvirtd while firewalld is stopped, then start
firewalld later, this triggers libvirtd to reload its iptables rules,
however it also spits out a *ton* of complaints about deletion failing
(I suppose because firewalld has nuked all of libvirt's rules). I
guess we need to suppress those messages (which is a more annoying
problem to fix than you might think, but that's another story).

3) I noticed a few times during this long line of errors that
firewalld made a complaint about "Resource Temporarily
unavailable. Having libvirtd access iptables commands directly at the
same time as firewalld is doing so is apparently problematic.

4) In general, I'm concerned about the "set it once and never change
it" method - if firewalld is disabled at libvirtd startup, causing
libvirtd to always use iptables/ebtables directly, this won't cause
*terrible* problems, but if libvirtd decides to use firewall-cmd and
firewalld is later disabled, libvirtd will not be able to recover.

Generating "Unable to add lockspace /lock/space/dir/__LIBVIRT__DISKS__:
No such file or directory" is correct but not exactly clear. This patch
changes the error message to "Unable to create lockspace
/lock/space/dir/__LIBVIRT__DISKS__: parent directory does not exist or
is not a directory".

When running libvirtd from a build directory, libvirtd would load lock
drivers from system directory unless explicitly overridden by setting
LIBVIRT_LOCK_MANAGER_PLUGIN_DIR environment variable. Since we already
autodetect driver directory if libvirt is build with driver modules, we
can use the same trick to automagically set lock driver directory.

Commit 1d22ba95 was complete at the time, but we have since
reintroduced a warning that is fixed in the same manner:

  CCLD   storagebackendsheepdogtest

*** Warning: Linking the executable storagebackendsheepdogtest against the loadable module
*** libvirt_driver_storage.so is not portable!

* src/Makefile.am (libvirt_driver_storage.la): Factor into new
convenience library libvirt_driver_storage_impl.la.
* tests/Makefile.am (storagebackendsheepdogtest_LDADD): Link to
convenience library, not shared library.

Our existing STRNEQ_NULLABLE() triggered a warning in gcc 4.7 when
used with a literal NULL argument:

qemumonitorjsontest.c: In function 'testQemuMonitorJSONGetMachines':
qemumonitorjsontest.c:289:5: error: null argument where non-null required (argument 1) [-Werror=nonnull]

even though the strcmp is provably dead when a null argument is
present.  Squelch the warning by refactoring things so that gcc
never sees strcmp() called with NULL arguments (we still compare
NULL as not equal to "", this rewrite merely aids gcc).

Next, gcc has a valid warning about a literal NULLSTR(NULL):

qemumonitorjsontest.c:289:5: error: invalid application of 'sizeof' to a void type [-Werror=pointer-arith]

Of course, you'd never write NULLSTR(NULL) directly, but it is
handy to use through macros.  But the entire part about verify_true()
is unnecessary - gcc already warns about type mismatch with ?:,
without needing to make it more complex.

* src/internal.h (STREQ_NULLABLE, STRNEQ_NULLABLE): Avoid gcc 4.7
stupidity.
(NULLSTR): Simplify, to allow passing compile-time constants.

The DAC security driver uses the virStrToLong_ui function to
parse the uid/gid out of the seclabel string. This works on
Linux where 'uid_t' is an unsigned int, but on Mingw32 it is
just an 'int'. This causes compiler warnings about signed/
unsigned int pointer mis-match.

To avoid this, use explicit 'unsigned int ouruid' local
vars to pass into virStrToLong_ui, and then simply assign
to the 'uid_t' type after parsing
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>