This commit is separate due to unusual paradigm compared to the
most source files.

Within whole vircgroup.c we 'return -errno', e.g. 'return -ENOMEM'.
However, in this specific function virCgroupAddTaskStrController
we weren't returning -ENOMEM but -1 despite fact that later in
the function we are returning one of errno values indeed.

In my previous patches I enabled the IFF_MULTI_QUEUE flag every
time the user requested multiqueue TAP device. However, this
works only at runtime. During build time the flag may be
undeclared.

In order to learn libvirt multiqueue several things must be done:

1) The '/dev/net/tun' device needs to be opened multiple times with
IFF_MULTI_QUEUE flag passed to ioctl(fd, TUNSETIFF, &ifr);

2) Similarly, '/dev/vhost-net' must be opened as many times as in 1)
in order to keep 1:1 ratio recommended by qemu and kernel folks.

3) The command line construction code needs to switch from 'fd=X' to
'fds=X:Y:...:Z' and from 'vhostfd=X' to 'vhostfds=X:Y:...:Z'.

4) The monitor handling code needs to learn to pass multiple FDs.

https://bugzilla.redhat.com/show_bug.cgi?id=965169 documents a
problem starting domains when cgroups are enabled; I was able
to reliably reproduce the race about 5% of the time when I added
hooks to domain startup by 3 seconds (as that seemed to be about
the length of time that qemu created and then closed a temporary
thread, probably related to aio handling of initially opening
a disk image).  The problem has existed since we introduced
virCgroupMoveTask in commit 91028296 (v0.10.0).

There are some inherent TOCTTOU races when moving tasks between
kernel cgroups, precisely because threads can be created or
completed in the window between when we read a thread id from the
source and when we write to the destination.  As the goal of
virCgroupMoveTask is merely to move ALL tasks into the new
cgroup, it is sufficient to iterate until no more threads are
being created in the old group, and ignoring any threads that
die before we can move them.

It would be nicer to start the threads in the right cgroup to
begin with, but by default, all child threads are created in
the same cgroup as their parent, and we don't want vcpu child
threads in the emulator cgroup, so I don't see any good way
of avoiding the move.  It would also be nice if the kernel were
to implement something like rename() as a way to atomically move
a group of threads from one cgroup to another, instead of forcing
a window where we have to read and parse the source, then format
and write back into the destination.

* src/util/vircgroup.c (virCgroupAddTaskStrController): Ignore
ESRCH, because a thread ended between read and write attempts.
(virCgroupMoveTask): Loop until all threads have moved.
Signed-off-by: NEric Blake <eblake@redhat.com>

The virfile.c file uses c_isalpha on Win32 platforms, so must
include c-ctype.h
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Resolves:https://bugzilla.redhat.com/show_bug.cgi?id=927620

 #kill -STOP `pidof qemu-kvm`
 #virsh destroy $guest --graceful
 error: Failed to destroy domain testVM
 error: An error occurred, but the cause is unknown

With --graceful, SIGTERM always is emitted to kill driver
process, but it won't success till burning out waiting time
in case of process being stopped.
But domain destroy without --graceful can work, SIGKILL will
be emitted to the stopped process after 10 secs which always
kills a process even one that is currently stopped.
So report an error after burning out waiting time in this case.

Change bbe97ae9 caused the
QEMU driver to ignore ENOENT errors from cgroups, in order
to cope with missing /proc/cgroups. This is not good though
because many other things can cause ENOENT and should not
be ignored. The callers expect to see ENXIO when cgroups
are not present, so adjust the code to report that errno
when /proc/cgroups is missing
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

E.g.

% sg_map
/dev/sg0  /dev/sda
/dev/sg1  /dev/sr0

What the helper gets for /dev/sg0 is /dev/sda, it will be used by
later patch.

stdlib.h header file needed for getenv was only transitively included
through libdevmapper.h.

Commit ccff335f added ATTRIBUTE_NONNULL for an attribute which is not
a pointer and made files including virnetdev.h not compilable, so fix
that.

In an upcoming patch, I need the way to safely transfer a nested
virJSON object out of its parent container for independent use,
even after the parent is freed.

* src/util/virjson.h (virJSONValueObjectRemoveKey): New function.
(_virJSONObject, _virJSONArray): Use correct type.
* src/util/virjson.c (virJSONValueObjectRemoveKey): Implement it.
* src/libvirt_private.syms (virjson.h): Export it.
* tests/jsontest.c (mymain): Test it.
Signed-off-by: NEric Blake <eblake@redhat.com>

network: static route support for <network>

This patch adds the <route> subelement of <network> to define a static
route.  the address and prefix (or netmask) attribute identify the
destination network, and the gateway attribute specifies the next hop
address (which must be directly reachable from the containing
<network>) which is to receive the packets destined for
"address/(prefix|netmask)".

These attributes are translated into an "ip route add" command that is
executed when the network is started. The command used is of the
following form:

  ip route add <address>/<prefix> via <gateway> \
               dev <virbr-bridge> proto static metric <metric>

Tests are done to validate that the input data are correct.  For
example, for a static route ip definition, the address must be a
network address and not a host address.  Additional checks are added
to ensure that the specified gateway is directly reachable via this
network (i.e. that the gateway IP address is in the same subnet as one
of the IP's defined for the network).

prefix='0' is supported for both family='ipv4' address='0.0.0.0'
netmask='0.0.0.0' or prefix='0', and for family='ipv6' address='::',
prefix=0', although care should be taken to not override a desired
system default route.

Anytime an attempt is made to define a static route which *exactly*
duplicates an existing static route (for example, address=::,
prefix=0, metric=1), the following error message will be sent to
syslog:

    RTNETLINK answers: File exists

This can be overridden by decreasing the metric value for the route
that should be preferred, or increasing the metric for the route that
shouldn't be preferred (and is thus in place only in anticipation that
the preferred route may be removed in the future).  Caution should be
used when manipulating route metrics, especially for a default route.

Note: The use of the command-line interface should be replaced by
direct use of libnl so that error conditions can be handled better.  But,
that is being left as an exercise for another day.
Signed-off-by: NGene Czarcinski <gene@czarc.net>
Signed-off-by: NLaine Stump <laine@laine.org>

Currently we report a bogus error message when macvlan
creation fails:

error: Failed to start domain migtest
error: operation failed: Unable to create macvlan device

With this removed, we see the real error:

error: Failed to start domain migtest
error: Unable to get index for interface p31p1: No such device
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use of the select() system call is inherantly dangerous since
applications will hit a buffer overrun if any FD number exceeds
the size of the select set size (typically 1024). Replace the
two uses of select() with poll() and use cfg.mk to ban any
future use of select().

NB: This changes the phyp driver so that it uses an infinite
timeout, instead of busy-waiting for 1ms at a time.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Found that I was unable to start existing domains after updating
to a kernel with no cgroups support

  # zgrep CGROUP /proc/config.gz
  # CONFIG_CGROUPS is not set
  # virsh start test
  error: Failed to start domain test
  error: Unable to initialize /machine cgroup: Cannot allocate memory

virCgroupPartitionNeedsEscaping() correctly returns errno (ENOENT) when
attempting to open /proc/cgroups on such a system, but it was being
dropped in virCgroupSetPartitionSuffix().

Change virCgroupSetPartitionSuffix() to propagate errors returned by
its callees.  Also check for ENOENT in qemuInitCgroup() when determining
if cgroups support is available.

Escaping a leading '.' with '_' in the cgroup names
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a virFileNBDDeviceAssociate method, which given a filename
will setup a NBD device, using qemu-nbd as the server.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To correctly handle errors from readdir() you must set 'errno'
to zero before invoking it & check its value afterwards to
distinguish error from EOF.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch adds util functions for scsi hostdev.
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat.com>

The helper works for default sysfs_prefix, but for user specified
prefix, it doesn't work. (Detected when writing test cases. A later
patch will add the test cases for fc_host).

The returned result is something like "host5" acutally.

Function name with "aIsB" generally means its return value is
in Bi-state (true/false).

In case of the caller can pass a "prefix" (or "sysfs_prefix")
without the trailing slash, and Unix-Like system always eats
up the redundant "slash" in the filepath, let's add it explicitly.

Which refactored the old code, and introduced new helper
virIsCapableVport, but the path for checking with access() is not
correctly constructed.

Introduced by commit 244ce462, which refactored the helper for wwn
reading, however, it forgot to change the old "strndup" and "sizeof(buf)",
"sizeof(buf)" operates on the fixed length array ("buf") in the old code,
but now "buf" is a pointer.

Before the fix:

% virsh nodedev-dumpxml scsi_host5
<device>
  <name>scsi_host5</name>
  <parent>pci_0000_04_00_1</parent>
  <capability type='scsi_host'>
    <host>5</host>
    <capability type='fc_host'>
      <wwnn>2001001b</wwnn>
      <wwpn>2101001b</wwpn>
      <fabric_wwn>2001000d</fabric_wwn>
    </capability>
  </capability>
</device>

With the fix:

% virsh nodedev-dumpxml scsi_host5
<device>
  <name>scsi_host5</name>
  <parent>pci_0000_04_00_1</parent>
  <capability type='scsi_host'>
    <host>5</host>
    <capability type='fc_host'>
      <wwnn>0x2001001b32a9da4e</wwnn>
      <wwpn>0x2101001b32a9da4e</wwpn>
      <fabric_wwn>0x2001000dec9877c1</fabric_wwn>
    </capability>
  </capability>
</device>

Commit bfe7721d introduced a regression, but only on platforms
like FreeBSD that lack posix_fallocate and where mmap serves as
a nice fallback for safezero.

util/virfile.c: In function 'safezero':
util/virfile.c:837: error: 'PROT_READ' undeclared (first use in this function)

* src/util/virutil.c (includes): Move use of <sys/mman.h>...
* src/util/virfile.c (includes): ...to the file that uses mmap.
Signed-off-by: NEric Blake <eblake@redhat.com>

Apps using libvirt will often have code like

   if (virXXXX() < 0) {
      virErrorPtr err = virGetLastError();
      fprintf(stderr, "Something failed: %s\n",
              err && err->message ? err->message :
              "unknown error");
      return -1;
   }

Checking for a NULL error object or message leads to very
verbose code. A virGetLastErrorMessage() helper from libvirt
can simplify this to

   if (virXXXX() < 0) {
      fprintf(stderr, "Something failed: %s\n",
              virGetLastErrorMessage());
      return -1;
   }
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

- provide virNetDevSetMAC() implementation based on SIOCSIFLLADDR
  ioctl.
- adjust virNetDevExists() to check for ENXIO error because
  FreeBSD throws it when device doesn't exist
Signed-off-by: NEric Blake <eblake@redhat.com>

These all existed before virfile.c was created, and for some reason
weren't moved.

This is mostly straightfoward, although the syntax rule prohibiting
write() had to be changed to have an exception for virfile.c instead
of virutil.c.

This movement pointed out that there is a function called
virBuildPath(), and another almost identical function called
virFileBuildPath(). They really should be a single function, which
I'll take care of as soon as I figure out what the arglist should look
like.

This resolves:

     https://bugzilla.redhat.com/show_bug.cgi?id=851411
     https://bugzilla.redhat.com/show_bug.cgi?id=955500

The first problem was that virFileOpenAs was returning fd (-1) in one
of the error cases rather than ret (-errno), so the caller thought
that the error was EPERM rather than ENOENT.

The second problem was that some log messages in the general purpose
qemuOpenFile() function would always say "Failed to create" even if
the caller hadn't included O_CREAT (i.e. they were trying to open an
existing file).

This fixes virFileOpenAs to jump down to the error return (which
returns ret instead of fd) in the previously mentioned incorrect
failure case of virFileOpenAs(), removes all error logging from
virFileOpenAs() (since the callers report it), and modifies
qemuOpenFile to appropriately use "open" or "create" in its log
messages.

NB: I seriously considered removing logging from all callers of
virFileOpenAs(), but there is at least one case where the caller
doesn't want virFileOpenAs() to log any errors, because it's just
going to try again (qemuOpenFile()). We can't simply make a silent
variation of virFileOpenAs() though, because qemuOpenFile() can't make
the decision about whether or not it wants to retry until after
virFileOpenAs() has already returned an error code.

Likewise, I also considered changing virFileOpenAs() to return -1 with
errno set on return, and may still do that, but only as a separate
patch, as it obscures the intent of this patch too much.

The individual hypervisor drivers were directly referencing
APIs in virnodesuspend.c in their virDriverPtr struct. Separate
these methods, so there is always a wrapper in the hypervisor
driver. This allows the unused virConnectPtr args to be removed
from the virnodesuspend.c file. Again this will ensure that
ACL checks will only be performed on invocations that are
directly associated with public API usage.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently the virGetHostname() API has a bogus virConnectPtr
parameter. This is because virtualization drivers directly
reference this API in their virDriverPtr tables, tieing its
API design to the public virConnectGetHostname API design.

This also causes problems for access control checks since
these must only be done for invocations from the public
API, not internal invocation.

Remove the bogus virConnectPtr parameter, and make each
hypervisor driver provide a dedicated function for the
driver API impl. This will allow access control checks
to be easily inserted later.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Since PIDs can be reused, polkit prefers to be given
a (PID,start time) pair. If given a PID on its own,
it will attempt to lookup the start time in /proc/pid/stat,
though this is subject to races.

It is safer if the client app resolves the PID start
time itself, because as long as the app has the client
socket open, the client PID won't be reused.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

There are various methods named "virXXXXSecurityContext",
which are specific to SELinux. Rename them all to
"virXXXXSELinuxContext". They will still raise errors at
runtime if SELinux is not compiled in
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

While reviewing proposed VIR_STRDUP conversions, I've already noticed
several places that do:

if (str && VIR_STRDUP(dest, str) < 0)

which can be simplified by allowing str to be NULL (something that
strdup() doesn't allow).  Meanwhile, code that wants to ensure a
non-NULL dest regardless of the source can check for <= 0.

Also, make it part of the VIR_STRDUP contract that macro arguments
are evaluated exactly once.

* src/util/virstring.h (VIR_STRDUP, VIR_STRDUP_QUIET, VIR_STRNDUP)
(VIR_STRNDUP_QUIET): Improve contract.
* src/util/virstring.c (virStrdup, virStrndup): Change return
conventions.
* docs/hacking.html.in: Document this.
* HACKING: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

VIR_APPEND_ELEMENT(array, size, elem) was not safe if the expression
for 'size' had side effects.  While no one in the current code base
was trying to pass side effects, we might as well be robust and
explicitly document our intentions.

* src/util/viralloc.c (virInsertElementsN): Add special case.
* src/util/viralloc.h (VIR_APPEND_ELEMENT): Use it.
(VIR_ALLOC, VIR_ALLOC_N, VIR_REALLOC_N, VIR_EXPAND_N)
(VIR_RESIZE_N, VIR_SHRINK_N, VIR_INSERT_ELEMENT)
(VIR_DELETE_ELEMENT, VIR_ALLOC_VAR, VIR_FREE): Document
which macros are safe in the presence of side effects.
* docs/hacking.html.in: Document this.
* HACKING: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Use virBufferAddLit or virBufferAddChar instead.

The code adaptation is not done right now, but in subsequent patches.
Hence I am not implementing syntax-check rule as it would break
compilation. Developers are strongly advised to use these new macros.
They are similar to VIR_ALLOC() logic: VIR_STRDUP(dst, src) returns zero
on success, -1 otherwise. In case you don't want to report OOM error,
use the _QUIET variant of a macro.