The default conf files, for example libvirtd.conf, virtlockd.conf, and
virtlogd.conf, should be located under the directory "/etc/libvirt" when
root as root, rather than "/etc". When run as non-root, the configuration
files should be located under "$XDG_CONFIG_HOME/libvirt/", rather than
"XDG_CONFIG_HOME".
Signed-off-by: NLily Zhu <lizhu@redhat.com>
Signed-off-by: NErik Skultety <eskultet@redhat.com>

The log and lock protocol don't have an extra handshake to close the
connection. Instead they just close the socket. Unfortunately that
resulted into a lot of spurious garbage logged to the system log files:

2017-03-17 14:00:09.730+0000: 4714: error : virNetSocketReadWire:1800 : End of file while reading data: Input/output error

or in the journal as:

Mar 13 16:19:33 xxxx virtlogd[32360]: End of file while reading data: Input/output error

Use the new facility in the netserverclient to suppress the IO error
report from the virNetSocket layer.

Linux still defaults to a 1024 open file handle limit. This causes
scalability problems for libvirtd / virtlockd / virtlogd on large
hosts which might want > 1024 guest to be running. In fact if each
guest needs > 1 FD, we can't even get to 500 guests. This is not
good enough when we see machines with 100's of physical cores and
TBs of RAM.

In comparison to other memory requirements of libvirtd & related
daemons, the resource usage associated with open file handles
is essentially line noise. It is thus reasonable to increase the
limits unconditionally for all installs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Now that virLog{Get,Set}DefaultOutput routines are introduced we can wire them
up to the daemon's logging initialization code. Also, change the order of
operations a bit so that we still strictly honor our precedence of settings:
cmdline > env > config now that outputs and filters are not appended anymore.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Along with an empty string, it should also be possible for users to pass
NULL to the public APIs which in turn would trigger a routine(future
work) responsible for defining an appropriate default logging output
given the current circumstances.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

'log_outputs' would be read into the variable for log_filters

Similar to outputs, parser should do parsing only, thus the 'define' logic
is going to be stripped from virLogParseAndDefineFilters by replacing calls to
this method to virLogSetFilters instead.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Since virLogParseAndDefineOutputs is going to be stripped from 'output defining'
logic, replace all relevant occurrences with virLogSetOutputs call to make the
change transparent to all original callers (daemons mostly).
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Right now virLogParse* functions are doing both parsing and defining of filters
and outputs which should be two separate operations. Since the naming is
apparently a bit poor this patch renames these functions to
virLogParseAndDefine* which eventually will be replaced by virLogSet*.
Additionally, virLogParse{Filter,Output} will be later (after the split) reused,
so that these functions do exactly what the their name suggests.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Commit 839a0608 tied the lifecycle of virtlogd more
closely to that of libvirtd. Unfortunately, while starting
virtlogd when libvirtd is started is definitely a good idea,
restarting virtlogd or shutting it down at any time outside
of system poweroff is not.

Revert part of that commit by removing the PartOf= lines,
meaning that only startup requests will be propagated from
libvirtd to virtlogd.

Resolves: https://bugzilla.redhat.com/1372576

We already guarantee that virtlogd.socket is enabled/disabled
along with libvirtd.service, but if libvirtd.service has just
been installed and is started before rebooting, then
virtlogd.socket will not be running and guest startup will
fail.

Add Requires=virtlogd.socket to libvirtd.service to make sure
virtlogd.socket is always started along with libvirtd.service,
and add Before=libvirtd.service to both virtlogd.socket and
virtlogd.service so that virtlogd never disappears before
libvirtd has exited.

Also add PartOf=libvirtd.service to both virtlogd.socket and
virtlogd.service, so that virtlogd can be shut down when not
needed.

Resolves: https://bugzilla.redhat.com/1372576

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

People debugging guest OS boot processes and reported that
the default 128 KB size is too small to capture an entire
boot up sequence. Increase the default size to 2 MB which
should allow capturing a full boot up even with verbose
debugging.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently virtlogd has a hardcoded max file size of 128kb
and max of 3 backups. This adds two new config parameters
to /etc/libvirt/virtlogd.conf to let these be customized.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Reorder code for setting default log level from cmdline prior
initialization of log outputs. Thus the --verbose option is reflected.

This resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1325072

Implement the RPC dispatcher and caller for the new API.

For logging one-shot entries to the VM log file it's quite a waste to
hold open the file descriptor for logging that is provided by the
current API.

This new API will be ideal for logging one-shot entries to the file
e.g. at the point when we shut the VM down rather than having to add the
whole file-descriptor infrastructure.

Additionally this will allow to add the messages even after restart of
libvirtd since virtlogd doesn't allow to obtain a regular context with
filedescriptors while the VM is still active.

Convert to virGetLastErrorMessage() in the rest of the code

Admin API needs a way of addressing specific clients. Unlike servers, which we
are happy to address by names both because its name reflects its purpose (to
some extent) and we only have two of them (so far), naming clients doesn't make
any sense, since a) each client is an anonymous, i.e. not recognized after a
disconnect followed by a reconnect, b) we can't predict what kind of requests
it's going to send to daemon, and c) the are loads of them comming and going,
so the only viable option is to use an ID which is of a reasonably wide data
type.
Signed-off-by: NErik Skultety <eskultet@redhat.com>

Format the text properly.

After this commit, all man pages are generated using the same two
steps:

  1. Process a source $command.pod file with pod2man(1) to obtain
     a valid man page in $command.$section.in

  2. Process $command.$section.in with sed(1) to obtain the final
     man page in $command.$section

Take setlocale/gettext error handling pattern from tools/virsh-*
and use it for all standalone binaries via a new shared
virGettextInitialize routine. The virsh* pattern differed slightly
from other callers. All users now consistently:

* Ignore setlocale errors. virsh has done this forever, presumably for
  good reason. This has been partially responsible for some bug reports:

  https://bugzilla.redhat.com/show_bug.cgi?id=1312688
  https://bugzilla.redhat.com/show_bug.cgi?id=1026514
  https://bugzilla.redhat.com/show_bug.cgi?id=1016158

* Report the failed function name
* Report strerror

https://bugzilla.redhat.com/show_bug.cgi?id=1303885

Since servers know their name, there is no need to supply such
information twice.  Also defeats inconsistencies.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

At first I did not want to do this, but after trying to implement some
newer feaures in the admin API I realized we need that to make our lives
easier.  On the other hand they are not saved redundantly and the
virNetServer objects are still kept in a hash table.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

The virtlogd daemon currently opens all files for append, but
in some cases the user may wish to discard existing data. Define
a new flag to indicate that logfiles should be truncated when
opening.

Not all callers of virLogManagerDomainOpenLogFile will
care about getting the current inode/offset, so we should
allow those parameters to be NULL
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Since the daemon can manage and add (at fresh start) multiple servers,
we also should be able to add them from a JSON state file in case of a
daemon restart, so post exec restart support for multiple servers is also
provided. Patch also updates virnetdaemontest accordingly.
Signed-off-by: NErik Skultety <eskultet@redhat.com>
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Since we pass dummy variables @fdout and @fdoutlen into
virNetClientProgramCall() we make it alloc @fdout array (even
though it's an array of 0 elements since vitlogd can hardly pass
us some FDs at this stage). Nevertheless, it's an allocation not
followed by free():

==29385== 0 bytes in 60 blocks are definitely lost in loss record 2 of 1,009
==29385==    at 0x4C2C070: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==29385==    by 0x54B99EF: virAllocN (viralloc.c:191)
==29385==    by 0x56821B1: virNetClientProgramCall (virnetclientprogram.c:359)
==29385==    by 0x563B304: virLogManagerDomainReadLogFile (log_manager.c:272)
==29385==    by 0x217CD613: qemuDomainLogContextRead (qemu_domain.c:2485)
==29385==    by 0x217EDC76: qemuProcessReadLog (qemu_process.c:1660)
==29385==    by 0x217EDE1D: qemuProcessReportLogError (qemu_process.c:1696)
==29385==    by 0x217EE8C1: qemuProcessWaitForMonitor (qemu_process.c:1957)
==29385==    by 0x217F6636: qemuProcessLaunch (qemu_process.c:4955)
==29385==    by 0x217F71A4: qemuProcessStart (qemu_process.c:5152)
==29385==    by 0x21846582: qemuDomainObjStart (qemu_driver.c:7396)
==29385==    by 0x218467DE: qemuDomainCreateWithFlags (qemu_driver.c:7450)
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Commit 2b6f6ad6 introduced the virxdrdefs.h header with
common definitions to be included in the protocol files,
but logging/log_protocol.x was missed, so add it there as well.

Hopefully this fixes build on OS X.

Commit b22344f3 mistakenly reordered
Default-* lines.  Thanks to that I noticed that we are very inconsistent
with our init scripts, so I took the liberty of synchronizing them,
updating them and making them all look shiny and new.  So apart from
fixing the LSB requirements, I also fixed the ordering, specified
runlevels and fix the link to the reference specification.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Currently pkg build of master branch fails:

[  300s] + /usr/lib/rpm/brp-boot-scripts
[  300s] E: File `virtlogd' is missing `Required-Start', please add even if empty!
[  300s] W: File `virtlogd' is missing `Required-Stop', please add even if empty!
[  300s] E: File `virtlogd' has empty `Default-Start', please specify default runlevel(s)!
[  300s] ERROR: found one or more broken init or boot scripts, please fix them.
[  300s]        For more information about LSB headers please read the manual
[  300s]        page of of insserv by executing the command `man 8 insserv'.
[  300s]        If you don't understand this, mailto=werner@suse.de
[  300s] error: Bad exit status from /var/tmp/rpm-tmp.44965 (%install)

Add the required tags, fix the existing tags.
Use soft dependency "Should-Start" because virtlogd may work without network.
Signed-off-by: NOlaf Hering <olaf@aepfle.de>

The virtlogd initscript's lock file should go in /var/lock/subsys/, not
(the nonexistent) /var/log/subsys/.
Signed-off-by: NMichael Chapman <mike@very.puzzling.org>

The virtlogd RPC messages all have a flags parameter. For
sake of future error reporting we should be verifying
these are all 0 for now.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The current virtlogd RPC protocol provides the ability to
handle log files associated with QEMU stdout/err. The log
protocol messages take the virt driver, domain name and
use that to form a log file path. This is quite restrictive
as it prevents us re-using the same RPC protocol messages
for logging to char device backends where the filename
can be arbitrarily user specified. It is also bad because
it means we have 2 separate locations which have to decide
on logfile name.

This change alters the RPC protocol so that we pass the
desired log file path along when opening the log file
initially. Now the virt driver is exclusively in charge
of deciding the log filename
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virt driver, dom name and uuid associated with a log
file are important pieces of metadata to keep around for
sake of future enhancements to virtlogd. Currently we
discard them after opening the log file, but we should
preserve them, even across restarts.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The problem is that in some mingw header DATADIR is used but
gnulib defines it too. This leads to the following compile error:

  CC       locking/libvirt_driver_la-lock_manager.lo
In file included from /usr/i686-w64-mingw32/sys-root/mingw/include/objbase.h:66:0,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/ole2.h:17,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/wtypes.h:12,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/winscard.h:10,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/windows.h:97,
                 from /usr/i686-w64-mingw32/sys-root/mingw/include/winsock2.h:23,
                 from ../gnulib/lib/unistd.h:48,
                 from ../../src/util/virutil.h:29,
                 from ../../src/logging/log_manager.c:30:
/usr/i686-w64-mingw32/sys-root/mingw/include/objidl.h:12275:2: error: expected identifier or '(' before string constant
 } DATADIR;
  ^
Makefile:7888: recipe for target 'logging/libvirt_driver_la-log_manager.lo' failed

The fix is to include configmake.h at the end of includes.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

With some versions of GLibC / GCC, a variable called 'daemon'
will result in a warning about clashing with the function also
named 'daemon'. Rename it to 'dmn' to avoid the clash.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Otherwise we fail on 32bit with:

CC     logging/virtlogd-log_daemon_dispatch.o
logging/log_daemon_dispatch.c: In function 'virLogManagerProtocolDispatchDomainReadLogFile':
logging/log_daemon_dispatch.c:120:9: error: format '%zu' expects argument of type 'size_t', but argument 7 has type 'uint64_t' [-Werror=format]

The virtlogd daemon is launched with a 30 second timeout for
unprivileged users. Unfortunately the timeout is only inhibited
while RPC clients are connected, and they only connect for a
short while to open the log file descriptor. We need to hold
an inhibition for as long as the log file descriptor itself
is open.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>