Introduce sysinfo profilers (journalctl & vmstat) [v5]

Sysinfo module now allows to define profilers, which are programs
defined in command line, to start as a background processes and log
the output in a proper directory. Example: vmstat 1.

The profilers are started before a job execution and then, stopped at
the end of the job. The output is stored inside `sysinfo/profile/*`
subdirectory, in job result directory.

The profilers can be customized in `avocado.conf`, section
`sysinfo.collect`.  The default configuration define
profilers as the following: `journalctl -f` and `vmstat 1`.

This feature is turned off by default.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Signed-off-by: NRudá Moura <rmoura@redhat.com>

Set _DEFAULT_COMMANDS_END_JOB as the same as DEFAULT_COMMANDS_START_JOB.
Set _DEFAULT_FILES_END_JOB as the same as _DEFAULT_FILES_END_JOB.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Define attributes .pre_dir and .post_dir (and for future usage, .profile_dir)
inside Sysinfo class, to handle the paths of those directories at the initialization.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Don't ignore stderr, redirect its output to stdout, in order to catch
commands not installed or malformed command line options.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Detect output to stdout as invalid at the beginning of execution
of the plugin, and not at the end of the html result execution.

Fix problem detected on test_output_incompatible_setup_3 when
sysinfo profile daemons are started and then left running, after
plugin exits with code 2.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Job standalone mode

Sometimes, exception classes might be written in a funky way,
and upon unserialization their __init__ methods might be
called. In this case, if the __init__ gets called, the
needed arguments won't be passed and then avocado will crash.

As we don't really need the actual class, just pick the
string representation of it.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The _DebugJob object was using '.' as the logdir, which meant
creating log files in the avocado source tree. Let's make
something more proper by setting the logdir to a tmpdir, and
by deleting the tmpdir at the end of the test.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Having avocado_loader_unittest as the prefix here was
basically a copy/paste mistake.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Some plugins don't really need to be started in paginator
mode (such as the test runner). Only plugins that need to
list things as opposed to execute test operations will
need it. Therefore:

1) Remove use_paginator=True from test runner, remote
   and vm plugins
2) Add use_paginator=True for the test lister plugin
3) Fix bug in paginator use that will make echo to
   be turned off if you use the paginator (causing
   input to not be displayed in your terminal after
   you use 'avocado list', for example).

The problem with the paginator bug is that it is
explicitly necessary to close the paginator once
we are done with it, as opposed to let it get
garbage collected. For that, we introduced the method
View.cleanup().
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

This makes avocado output a little easier in the eyes,
by adding appropriate padding for the messages in the
remote/vm plugins.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

It is easy to tell when avocado is simply not installed
in the remote machine. So instead of giving the generic
crash message to the user, present a better, specific
error that will help the user to proceed on fixing
the root cause.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The idea of the standalone mode is to be able to have a avocado test to
behave like a regular script, outputing to STDOUT the basic log (the
same as --show-job-log would) and return an meaningful status code). This
should help avocado tests to be run along any other test code or framework.

In this mode, a job will still be created, and the basic output files will
also be created, but will be hidden from the user, and removed after the
job finishes running.

This is a partial solution to the so called "jobless" mode, which would
allow a test to run without a job.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since we provide an utility library, let's make use of it. This is not
only for the sake of consistency, but to eventually allow the framework
to run in a non-writable mode.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

avocado.plugins.wrapper: Don't activate when not enabled

When no wrappers are activated this plugin still checks for
the compatibility with --gdb. Skip activate when no --wrapper set.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Better handling of docs/build

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

As far as I can see and test, the directory doesn't have to be present in the
repository, as it'll be created on-the-fly during the documentation generation.

As a bonus, avoid a dangerous "rm -rf $(VARIABLE)/*" in make clean (which is a
pattern for catastrophic results -- if $VARIABLE, whatever it is, is not set,
this will result in "rm -rf /*").

avocado.plugins.remote: Unify test locations and use absolute path [v5]

Default timeout for execution is 60s. This patch removes this for the
remote `avocado ...` run as there is no sensible default for it, nor
convinient way to set it on the cmdline. It should be relatively safe as
Avocado on remote machine should timeout using the same timeouts as on
host and in case the machine dies fabric fails the execution and
returns.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Executable bit is not set without mirror_local_mode or mode setting.
This is needed for remote test execution as they need to be executable.
Correct me if I'm wrong but for our purposes mirror_local_mode is the
most useful way without the need to make it optional.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Use flexmock to supplement remote machine and check booth
vm and remote plugins deeper.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This argument disables copying the tests and only executes the tests as
they are specified inside the remote or vm machine.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently remote plugin doubles the HumanTestResults. This removes
all shared parts and inherits from HumanTestResults. Results are
identical only the amount of code is decreased.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

No actual change, only:
1) imports are ordered global->local and alphabetically
2) 80 lines length (sorry guys, my debug screen is small)
3) added couple of docstrings.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

_setup_environment doesn't use `self` so it should be defined as
@staticmethod.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

All class variables should be defined in header or __init__ to avoid
AttributeError.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

No actual changes, only use UPPER_CASE to distinguish global variables
(PEP8)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado.multiplexer: Modify multiplexation mechanism [v4]

Using remote plugin results in very scary results when mixing different
ways to execute test. This patch translates urls into absolute paths
and copies each test's base directory into $remote_test_dir + the
original absolute path.

This approach is very deterministic, it avoids any clashes and as you
see the actual path it helps understanding of what happened where so
users can easily look into remote machine.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

vm plugin only extends the remote plugin and shares the test execution
part. This patch inherits vm plugin from remote one and adds only
the setup/cleanup part.

Additionally it removes the --vm argument and enables this plugin when
--vm-domain is set.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Check required params in plugin.activate() and set RemoteTestResults
urls during __init__.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

[V13] Test discovery and load (Phase II)

In case of custom tags like !using pyyaml requires space between the tag
and the ':'. This might be strange to people so this patch adds human
friendly message informing about this possibility.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>