It might be useful to also list the command which failed to run in
parallel selfcheck.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently the fail-fast works per section (pylint style, unittests, ...)
but this patch enables it also inside the unittests, therefor it
finishes with the first failed unittest finishing even quicker than
before.
Note that this option is incompatible with the parallel check.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch allows running the tests in parallel, when
AVOCADO_PARALLEL_CHECK env variable is set. It spawns 2xno_cpu workers
(or less when there are not enough tests), captures the output and shows
only output of failed workers. Before the exit it also reports the
number of executed tests in the same fashion as normal unittest
execution (without the number of failed tests, though).

On my laptop this decreases the duration from 125s to 28s.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch adds summary at the end of the execution and uses colors to
mark start and finish of each section. This might produce some
unexpected characters on redirections/unsupported terminals at the
beginning and endings of the highlighted lines, but in most cases it
looks way better and those +3 chars can be ignored by humans.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `checkall` script should not try to develop/install/clean anything.
It's suppose to run the tests and it's upto the user to prepare the
environment they want to check.

This won't affect `make check` as it already cleans/develops before
executing checkall.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently the `make check` goes through all checks and then reports
overall results. This might take a while and is not really `git bisect`
friendly. This patch changes it to report early unless
SELF_CHECK_CONTINUOUS is specified. This is used in `travis` as we are
interested in all failures.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

`.git` may hold content that ends up being checked by inspektor, such
as the `git-rerere` cache.  This may cause false positives, so let's
skip `.git` completetly from those checks.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This allows `AVOCADO_SELF_CHECK=1 ./selftests/checkall` to run
inside virtualenv environemnts.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Not all contrib content is created equal, so let's just move scripts
to a properly named directory, in anticipation for different types of
content.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This implements a "make selfcheck" that developers can use to run
the Avocado selftests in itself. It also keeps the old "make check"
(for a while) so that we can compare to the old approach if any
regressions occur in the near future.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The unittest module, and unittest2 backport on Python 2.6, is capable of
finding unittests and running them. So, effectively, we do not need nose
at all.

This patch removes the dependency on nose, replacing the run script
with a version based solely on the unittest module.

One change of functionality is that run now looks and runs tests in the
standard selftests directories (unit, functional, doc), and does not
accept command line arguments. If one wants to run a subset of them,
it's pretty easy to just use the unittest module for that:

 $ python -m unittest discover -s selftests/unit
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This check is being added to both to checkall and to travis.yaml,
s o that either in local and remote (CI) checks this can be quickly
flagged.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Make sure the checkall and coverageall scripts return
failures.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

One convenient command to run the checks Travis runs on
the code base (less verbose version).
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>