This patch avoids avocado crash on empty multiplex yaml file. Empty file
contains basically nothing and that is how it's handled now.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Recently we created separate `yaml2mux` plugin, which keeps using the
`--multiplex` argument. The connection between yaml2mux and multiplex is
not really obvious and the separation was meant to bring the order to
the avocado params system. This is another step which replaces the
`--multiplex|-m` with `--mux-yaml|-m` and makes it more obvious that
it's related to `yaml` files and it's also related to `multiplexation`.

You might notice that the old `-m` was kept. It's very convenient and
also allows coexisting of old and new versions (useful eg. in remote
executions).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently `replay` has hardcoded treating of `--multiplex`, which is not
systematic and even now allows some unhandled errors (like --mux-filter
along with replay-test-status). With this patch I propose to ignore all
multiplex related options and use the one from the job and process the
mux only when the user uses `--replay-ignore mux`. This means it does
not allow mixing of previous job's mux with new options, on the other
hand it's simple and easily understandable:

1. no `--replay-ignore mux` => all --mux options are ignored and the
   original Mux is used
2. `--replay-ignore mux` => old Mux is ignored and full new Mux is used
   including system-wide args, --mux options and the yaml parser (-m)

To make things clear this patch always logs a warning notifying the user
about the current situation.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The --replay-ignore is a list of arguments, but the default is None,
which requires special hendling in the code. Let's change it to list and
remove the special handling.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This complex patch replaces the custom `--multiplex` handling with a
brand new plugin called `yaml_to_mux`. This touches several place and
removes the `yaml` functionality from the avocado core and moves it into
the plugin code.

There are still 2 references from plugins to the `yaml_to_mux` plugin
which are not defined as a public API yet.

1. remote runner - which allows copying --multiplex files to remote
                   machine.
2. replay - which allows overriding of the job's mux variants with
            the --multiplex files. Note that by doing this you
            lose the system-wide options. It is described in the
            warning message.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The mux need to support debugging the options. This is a basic
implementation which allows requesting mux to use TreeNodeDebug. Then
it's the plugin's job to check whether mux uses debug (`mux.debug`) and
act accordingly.

In this basic implementation the debug is only enabled for `--multiplex`
and `@None` is used for other injects (like `--mux-inject`)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `--multiplex` is about to be changed into a separate plugin. It does
not sound fair to keep it as default positional argument of the
`multiplex` command. Let's sync the naming scheme and require
`--multiplex` for it.

This also means the `-m` is optional so one can run `avocado multiplex`
without arguments to get default multiplex values.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The option changed from --multiplex-files to `--multiplex` long time ago
but we kept using the original `multiplex_files` destination for it.
This is not systematic and can lead to problems in the future. Let's
change it once for all.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch is a 1st step to independent multiplex plugin. It unifies the
multiplexation, which is handled exclusively by the
avocado.core.multiplexer.Mux object, which is initialized directly in
argument parser as `args.mux`, instead of the
`parser.default_avocado_params`.

The idea is to provide unified database with simple interface:

 * data_inject - inject key/value[/path] data
 * data_merge - merge node/tree into the data

Additionally it supports internal API to generate the params:

 * is_parsed - whether it was already initialized/filtered/...
 * get_number_of_tests - reports number of test*variants
 * itertests - iterate through tests

All those changes are pickle-safe from the old object to the new object
(not vice versa), therefor it's possible to use old Mux objects stored
in `jobdata` to replay tests.

This API is not 100% stable yet, but we plan to keep the one sided
pickle compatibility into the future (no guarantee, but we'll do our
best).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

It might be useful to also list the command which failed to run in
parallel selfcheck.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Docs: document unittest class level setUp and tearDown compatibility

scripts/avocado-journal-replay: Fix usage message

This issue has come up repeatedly, so let's document how and why we
(don't) support `setUpClass` and `tearDownClass`.  While at it, let's
give some ideas on how to work around this, or better put, write tests
more aligned with Avocado's design of more isolated tests.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There's a missing space at the end of the line, that is, let's
replace "datato" with "data to".
Signed-off-by: NWei Jiangang <weijg.fnst@cn.fujitsu.com>

README.rst: mention make develop (fixes #1496)

Plugins: add mechanism to disable installed plugins

Handle result plugin failures / HTML result fix with older pystache

selftests.checkall: Adjust the behavior and add parallel check [v3]

Python 2.6 / EL6 fixes [v2]

Before you can run the script straight out of the run tree you need to
run a make develop step so let's make it clear.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

Using avocado.utils.process to run the journalctl commands makes avocado
to ignore the KeyboardInterrupt signal and consider the test as PASS due
to the later (sysinfo post) command executed successfully.

This patch changes the method to run the journalctl commands to
subprocess.check_output. Now Avocado seems to handle KeyboardInterrupt
nicely.

Reference: https://trello.com/c/dNgrjMOJSigned-off-by: NAmador Pahim <apahim@redhat.com>

Currently the fail-fast works per section (pylint style, unittests, ...)
but this patch enables it also inside the unittests, therefor it
finishes with the first failed unittest finishing even quicker than
before.
Note that this option is incompatible with the parallel check.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch allows running the tests in parallel, when
AVOCADO_PARALLEL_CHECK env variable is set. It spawns 2xno_cpu workers
(or less when there are not enough tests), captures the output and shows
only output of failed workers. Before the exit it also reports the
number of executed tests in the same fashion as normal unittest
execution (without the number of failed tests, though).

On my laptop this decreases the duration from 125s to 28s.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch adds summary at the end of the execution and uses colors to
mark start and finish of each section. This might produce some
unexpected characters on redirections/unsupported terminals at the
beginning and endings of the highlighted lines, but in most cases it
looks way better and those +3 chars can be ignored by humans.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The setuptools entry point namespace, currently being show in the
output of the plugin listing, is not that relevant to users.

Let's use the plugin types (as documented in the section about the
fully qualified name of plugins), which is more relevant to Avocado
users.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

While the behavior of having a plugin enabled once it's installed
has suited Avocado reasonably well, there are valid use cases that
require plugins to be disabled.

While adding this mechanism, let's define what the Avocado fully
qualified name for a given plugin is.

While not an urgent feature, enabling and disabling plugins is
actually one requirement for the implementation of the Job API.  While
this change allows for disabling plugins globally, a refinement of the
same logic will allow a given Job to enable/disable some of Avocado's
functionality.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The sleeptenmin example test contains `self.report_state` which should
not be called directly and it defines variable which is not used.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We have been running the selftests during the build of the Fedora
packages, but have skipped that on EL6 (and EL7).  The reason is that
there were a few packages missing from EPEL.

They're now present on the EPEL 6 repos so let's enable checks while
building EL6 packages.

Some functional tests are failing on EL7, and fixing those and
enabling checks for EL7 packages will be the scope of further work.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When no C compiler is available, these tests will produce errors.
Let's skip them when the requirements cannot be met.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Older python-sphinx cannot handle newer inventory files (objects.inv).
While this is not fail proof, as there can be newer python-sphinx on
older Python, but it seems to be good enough as it works on all system
setups I tested.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Some older versions of sphinx do not ship with `sphinx-apidoc`.
Instead of breaking the document generation and selftests, let's just
skip the generation of API docs.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And not based on Python version.  It was observed that platforms based
on Python 2.6, such as EL6, do ship with setuptools that has support
for `--user`.  This eases considerably the development environment
on these platforms.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This will change the behavior one gets with `make develop` so that
optional (but in-tree) plugins will be enabled by default.  For now,
this means enabling the HTML result plugin.

This also means that `make check` will also test those by default,
as it's already been done on Travis CI.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Commit 3154794d broke the HTML report
when older some versions of pystache is used.  The current status is
that the HTML report generated on those systems is a one line report
containing the literal `utf8` string.

This proposal does not try to add support Unicode based inputs with
that version of pystache, because it's internally not well supported.
Internal methods such as `Template.render_tags()` will blindly try to
do conversions without respecting the encoding passed on `render()`.

Also, it's true that we're missing Unicode support in many other
places, such as documented here in this card:

 https://trello.com/c/bfE9NBbl

For now, let's try properly generate the report, and keep the error
handling already present, that let's users know there have been
encoding errors.

References: https://trello.com/c/8j3aVUlISigned-off-by: NCleber Rosa <crosa@redhat.com>

The original idea of the custom `map_methods` implementation
in the Job Pre/Post dispatcher was to overload the existing
`map_method` method.   But, I wrote it in the plural, and there's
no reason to do so.

Let's put them back to the more standard name.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Previously, a crash in one result plugin could abort the execution
of other plugins and consequently the last parts of the avocado
command line application execution.

Let's handle exceptions that could be raised there, as it's already
being done with Job Pre/Post plugins.
Signed-off-by: NCleber Rosa <crosa@redhat.com>