Docs: document optional packages installation

Some pieces of Avocado are distributed as optional packages, let's
document that fact and how to install them using both RPM and source
based installs.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Recently I fixed a bug where test which asks for input hangs for
infinity but I forgot to add unittest for it. Let's add it now.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

* https://github.com/avocado-framework/avocado:
  Sysinfo: make journalctl quiet on cursor acquisition
  Functional Tests: sysinfo collection may trigger unexpected messages
  spell.ignore: add words to ignore list

On environments where there are no journals available, journalctl may
output error messages such as "No journal files were found", unless
`--quiet` is given as an option.  The other call to journalctl already
does that, but we missed in the first call (that does the cursor
acquisition).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It turns out that, due to other issues, sysinfo collection may trigger
unwanted output.

Since these changed tests are really about the jobscripts plugin
execution, and accordingly to the vast majority of other functional
tests, let's disable sysinfo to speed them up and avoid these noises.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

avocado.utils.git: Don't produce check output

avocado.utils.iso9660: Several improvements and cleanups [v3]

Docker improvements [v3]

Docs: describe OpenSUSE support in Avocado

This works great to hosts wich many containers

Example:
avocado run --docker debian-avocado-test /bin/true
DOCKER     : Container id '0f9ebae391183937fb6e98c50cc10f4927963140caf6e63fd108876a6ac7270f'
DOCKER     : Container name 'job-2016-10-02T07.14-a404c72.avocado'
..

Let's add dockefile example. This greatly simplify avocado test development because
allow to create hermetic devel images w/o breaking system instalation
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Docker run enviroments mostly configured by extra run options.
Let's allow runnter to pass it.
#Example:
Run test in remote docker with privileged commands allowed(mount, kvm, etc)
and bindmount local directory '/vol' to '/v' inside container
$ avocado run --docker-options="-H=tcp://remote:2375 --privileged -v /vol:/v" \
	      --docker dmonakhov/debian-avocado passtest.py

The OpenSUSE project ships the LTS versions of Avocado.  Let's
document that so users on that distro can get a head start using
official distro packages if they choose to do so.
Signed-off-by: NSergey Bronnikov <sergeyb@openvz.org>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

No actual changes, just style fixes and docstring updates.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Sometimes the iso9660/isofs is not loaded, let's try to modprobe it and
then re-check whether iso9660 is supported.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The Iso9660Mount requires root, but with this patch it can also use
normal user with sudo configured.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `avocado.utils.process` allows the use of sudo, but it might not be
always enabled/configured in the system. This method should allow
detecting whether it is or is not available.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The Iso9660Mount defines "mnt_dir" variable, which is frequently used in
tests to make the content of the image available. This is not supported
by other providers, which adds complexity to the code around mnt_dir.
This patch makes "mnt_dir" property part of the API and implements it in
all current providers.

Note that the "mnt_dir" feature is currently uses the "Iso9660Mount"
features, therefor the user must be root and the mount has to be
available, otherwise one gets exception when he tries to use "mnt_dir".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The usual use of the git library is to get latest (or some) version,
quite frequently into a unique tmp dir. Let's disable recording it's
output for --output-check.
Signed-off-by: NPhilipp Wagner <imphil@philipp-wagner.com>
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The main purpose of multiplexer is to create variants. This patch adds a
check which makes sure the expected number of tests were executed.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch avoids avocado crash on empty multiplex yaml file. Empty file
contains basically nothing and that is how it's handled now.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Recently we created separate `yaml2mux` plugin, which keeps using the
`--multiplex` argument. The connection between yaml2mux and multiplex is
not really obvious and the separation was meant to bring the order to
the avocado params system. This is another step which replaces the
`--multiplex|-m` with `--mux-yaml|-m` and makes it more obvious that
it's related to `yaml` files and it's also related to `multiplexation`.

You might notice that the old `-m` was kept. It's very convenient and
also allows coexisting of old and new versions (useful eg. in remote
executions).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently `replay` has hardcoded treating of `--multiplex`, which is not
systematic and even now allows some unhandled errors (like --mux-filter
along with replay-test-status). With this patch I propose to ignore all
multiplex related options and use the one from the job and process the
mux only when the user uses `--replay-ignore mux`. This means it does
not allow mixing of previous job's mux with new options, on the other
hand it's simple and easily understandable:

1. no `--replay-ignore mux` => all --mux options are ignored and the
   original Mux is used
2. `--replay-ignore mux` => old Mux is ignored and full new Mux is used
   including system-wide args, --mux options and the yaml parser (-m)

To make things clear this patch always logs a warning notifying the user
about the current situation.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The --replay-ignore is a list of arguments, but the default is None,
which requires special hendling in the code. Let's change it to list and
remove the special handling.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This complex patch replaces the custom `--multiplex` handling with a
brand new plugin called `yaml_to_mux`. This touches several place and
removes the `yaml` functionality from the avocado core and moves it into
the plugin code.

There are still 2 references from plugins to the `yaml_to_mux` plugin
which are not defined as a public API yet.

1. remote runner - which allows copying --multiplex files to remote
                   machine.
2. replay - which allows overriding of the job's mux variants with
            the --multiplex files. Note that by doing this you
            lose the system-wide options. It is described in the
            warning message.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The mux need to support debugging the options. This is a basic
implementation which allows requesting mux to use TreeNodeDebug. Then
it's the plugin's job to check whether mux uses debug (`mux.debug`) and
act accordingly.

In this basic implementation the debug is only enabled for `--multiplex`
and `@None` is used for other injects (like `--mux-inject`)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `--multiplex` is about to be changed into a separate plugin. It does
not sound fair to keep it as default positional argument of the
`multiplex` command. Let's sync the naming scheme and require
`--multiplex` for it.

This also means the `-m` is optional so one can run `avocado multiplex`
without arguments to get default multiplex values.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The option changed from --multiplex-files to `--multiplex` long time ago
but we kept using the original `multiplex_files` destination for it.
This is not systematic and can lead to problems in the future. Let's
change it once for all.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch is a 1st step to independent multiplex plugin. It unifies the
multiplexation, which is handled exclusively by the
avocado.core.multiplexer.Mux object, which is initialized directly in
argument parser as `args.mux`, instead of the
`parser.default_avocado_params`.

The idea is to provide unified database with simple interface:

 * data_inject - inject key/value[/path] data
 * data_merge - merge node/tree into the data

Additionally it supports internal API to generate the params:

 * is_parsed - whether it was already initialized/filtered/...
 * get_number_of_tests - reports number of test*variants
 * itertests - iterate through tests

All those changes are pickle-safe from the old object to the new object
(not vice versa), therefor it's possible to use old Mux objects stored
in `jobdata` to replay tests.

This API is not 100% stable yet, but we plan to keep the one sided
pickle compatibility into the future (no guarantee, but we'll do our
best).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

process.Popen returns the process ID, so let's use it in our
CmdResult class.

Fixes: #1514
Signed-off-by: NAmador Pahim <apahim@redhat.com>

It might be useful to also list the command which failed to run in
parallel selfcheck.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Docs: document unittest class level setUp and tearDown compatibility

scripts/avocado-journal-replay: Fix usage message