Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The mux need to support debugging the options. This is a basic
implementation which allows requesting mux to use TreeNodeDebug. Then
it's the plugin's job to check whether mux uses debug (`mux.debug`) and
act accordingly.

In this basic implementation the debug is only enabled for `--multiplex`
and `@None` is used for other injects (like `--mux-inject`)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `--multiplex` is about to be changed into a separate plugin. It does
not sound fair to keep it as default positional argument of the
`multiplex` command. Let's sync the naming scheme and require
`--multiplex` for it.

This also means the `-m` is optional so one can run `avocado multiplex`
without arguments to get default multiplex values.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The option changed from --multiplex-files to `--multiplex` long time ago
but we kept using the original `multiplex_files` destination for it.
This is not systematic and can lead to problems in the future. Let's
change it once for all.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch is a 1st step to independent multiplex plugin. It unifies the
multiplexation, which is handled exclusively by the
avocado.core.multiplexer.Mux object, which is initialized directly in
argument parser as `args.mux`, instead of the
`parser.default_avocado_params`.

The idea is to provide unified database with simple interface:

 * data_inject - inject key/value[/path] data
 * data_merge - merge node/tree into the data

Additionally it supports internal API to generate the params:

 * is_parsed - whether it was already initialized/filtered/...
 * get_number_of_tests - reports number of test*variants
 * itertests - iterate through tests

All those changes are pickle-safe from the old object to the new object
(not vice versa), therefor it's possible to use old Mux objects stored
in `jobdata` to replay tests.

This API is not 100% stable yet, but we plan to keep the one sided
pickle compatibility into the future (no guarantee, but we'll do our
best).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

It might be useful to also list the command which failed to run in
parallel selfcheck.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Docs: document unittest class level setUp and tearDown compatibility

scripts/avocado-journal-replay: Fix usage message

This issue has come up repeatedly, so let's document how and why we
(don't) support `setUpClass` and `tearDownClass`.  While at it, let's
give some ideas on how to work around this, or better put, write tests
more aligned with Avocado's design of more isolated tests.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There's a missing space at the end of the line, that is, let's
replace "datato" with "data to".
Signed-off-by: NWei Jiangang <weijg.fnst@cn.fujitsu.com>

README.rst: mention make develop (fixes #1496)

Plugins: add mechanism to disable installed plugins

Handle result plugin failures / HTML result fix with older pystache

selftests.checkall: Adjust the behavior and add parallel check [v3]

Python 2.6 / EL6 fixes [v2]

Before you can run the script straight out of the run tree you need to
run a make develop step so let's make it clear.
Signed-off-by: NAlex Bennée <alex.bennee@linaro.org>

Using avocado.utils.process to run the journalctl commands makes avocado
to ignore the KeyboardInterrupt signal and consider the test as PASS due
to the later (sysinfo post) command executed successfully.

This patch changes the method to run the journalctl commands to
subprocess.check_output. Now Avocado seems to handle KeyboardInterrupt
nicely.

Reference: https://trello.com/c/dNgrjMOJSigned-off-by: NAmador Pahim <apahim@redhat.com>

Currently the fail-fast works per section (pylint style, unittests, ...)
but this patch enables it also inside the unittests, therefor it
finishes with the first failed unittest finishing even quicker than
before.
Note that this option is incompatible with the parallel check.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch allows running the tests in parallel, when
AVOCADO_PARALLEL_CHECK env variable is set. It spawns 2xno_cpu workers
(or less when there are not enough tests), captures the output and shows
only output of failed workers. Before the exit it also reports the
number of executed tests in the same fashion as normal unittest
execution (without the number of failed tests, though).

On my laptop this decreases the duration from 125s to 28s.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch adds summary at the end of the execution and uses colors to
mark start and finish of each section. This might produce some
unexpected characters on redirections/unsupported terminals at the
beginning and endings of the highlighted lines, but in most cases it
looks way better and those +3 chars can be ignored by humans.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The setuptools entry point namespace, currently being show in the
output of the plugin listing, is not that relevant to users.

Let's use the plugin types (as documented in the section about the
fully qualified name of plugins), which is more relevant to Avocado
users.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

While the behavior of having a plugin enabled once it's installed
has suited Avocado reasonably well, there are valid use cases that
require plugins to be disabled.

While adding this mechanism, let's define what the Avocado fully
qualified name for a given plugin is.

While not an urgent feature, enabling and disabling plugins is
actually one requirement for the implementation of the Job API.  While
this change allows for disabling plugins globally, a refinement of the
same logic will allow a given Job to enable/disable some of Avocado's
functionality.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The sleeptenmin example test contains `self.report_state` which should
not be called directly and it defines variable which is not used.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We have been running the selftests during the build of the Fedora
packages, but have skipped that on EL6 (and EL7).  The reason is that
there were a few packages missing from EPEL.

They're now present on the EPEL 6 repos so let's enable checks while
building EL6 packages.

Some functional tests are failing on EL7, and fixing those and
enabling checks for EL7 packages will be the scope of further work.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When no C compiler is available, these tests will produce errors.
Let's skip them when the requirements cannot be met.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Older python-sphinx cannot handle newer inventory files (objects.inv).
While this is not fail proof, as there can be newer python-sphinx on
older Python, but it seems to be good enough as it works on all system
setups I tested.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Some older versions of sphinx do not ship with `sphinx-apidoc`.
Instead of breaking the document generation and selftests, let's just
skip the generation of API docs.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And not based on Python version.  It was observed that platforms based
on Python 2.6, such as EL6, do ship with setuptools that has support
for `--user`.  This eases considerably the development environment
on these platforms.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This will change the behavior one gets with `make develop` so that
optional (but in-tree) plugins will be enabled by default.  For now,
this means enabling the HTML result plugin.

This also means that `make check` will also test those by default,
as it's already been done on Travis CI.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Commit 3154794d broke the HTML report
when older some versions of pystache is used.  The current status is
that the HTML report generated on those systems is a one line report
containing the literal `utf8` string.

This proposal does not try to add support Unicode based inputs with
that version of pystache, because it's internally not well supported.
Internal methods such as `Template.render_tags()` will blindly try to
do conversions without respecting the encoding passed on `render()`.

Also, it's true that we're missing Unicode support in many other
places, such as documented here in this card:

 https://trello.com/c/bfE9NBbl

For now, let's try properly generate the report, and keep the error
handling already present, that let's users know there have been
encoding errors.

References: https://trello.com/c/8j3aVUlISigned-off-by: NCleber Rosa <crosa@redhat.com>

The original idea of the custom `map_methods` implementation
in the Job Pre/Post dispatcher was to overload the existing
`map_method` method.   But, I wrote it in the plural, and there's
no reason to do so.

Let's put them back to the more standard name.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Previously, a crash in one result plugin could abort the execution
of other plugins and consequently the last parts of the avocado
command line application execution.

Let's handle exceptions that could be raised there, as it's already
being done with Job Pre/Post plugins.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `checkall` script should not try to develop/install/clean anything.
It's suppose to run the tests and it's upto the user to prepare the
environment they want to check.

This won't affect `make check` as it already cleans/develops before
executing checkall.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently the `make check` goes through all checks and then reports
overall results. This might take a while and is not really `git bisect`
friendly. This patch changes it to report early unless
SELF_CHECK_CONTINUOUS is specified. This is used in `travis` as we are
interested in all failures.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado.core.test: Always use "test" as methodName in dry-run

selftests: Use LC_ALL=C in tests which parse output

docs: Add note about args vs. positional args [v2]