Inspektor 0.4.5 brings a number of updates, including
porting to the cliff command line application framework,
which makes inspekt commands to look more natural in
terms of arguments.  Let's update and reflect that
in the CI scripts.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

For couple of weeks the `inspekt lint` produces unwanted `Your code has
been rated at 10.00/10 (previous run: 10.00/10, +0.00)` messages
polluting the log too much for me to bear. I tried to remove those
messages, but so far I haven't found a solution, so let's just silence
it unless there is a lint failure.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

To make sure the seltests does not leave job results behind,
we introduced a check which compares the job-results directory
content from before the selftets with the content after the
selftests. But using `AVOCADO_SELF_CHECK=1 make check`, we
expect at least one extra result in the job-results directory,
making the content of the job-results to not be the same after
the tests.

This patch changes the avocado command used for self check
in `make check`, pointing the results to a temporary directory.

Reference: https://trello.com/c/y57xZSUSSigned-off-by: NAmador Pahim <apahim@redhat.com>

checkall runs `ls` twice in the job-results dir. Once before the
selftests and once again after the tests, comparing the outputs to make
sure there's no new results there after the selftests.

When the job-results does not exist in advance (the case in travis
environments), the message 'ls: cannot access /home/travis/avocado/job-results:
No such file or directory' is shown in the logs. The message is harmless
and does not affect the check. Let's just suppress the stderr from the
`ls` command in that check.

Reference: https://trello.com/c/IBoLAJjfSigned-off-by: NAmador Pahim <apahim@redhat.com>

It happened couple of times that running our selftests produced a new
entries in the default results directory. Let's use this optional check
to avoid such issue.

To enable this check one has to set "AVOCADO_RESULTSDIR_CHECK" env
variable. This check is enabled in travis as I do not expect travis to
run avocado tests while running the selfcheck, unlike on developer
computers.

Note the check is rather simple and only assumes the default results
directory. For selfcheck I think it's enough.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

So far we only executed the first argument, but we might want to execute
complex cmdlines instead just one command. Let's use $* instead of $1.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

pep8 v2.3.0 introduced the check for bare except clause (E722). Until
now, we don't explicitly disable the pep8 check we don't want because
inspect has a default exclude list (E501, E265, W601 and E402).

Let's make explicit the exclude list for the checks we don't want in our
selftests, adding to the list the new E722 check.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The signed-off-by check makes sure the latest commit contains correct
"Signed-off-by" message. As this check is executed per each commit in
travis, it should make sure all new commits contain this message which
is quite important as it states that the author agrees with the terms
from our contribution guide.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

It might be useful to also list the command which failed to run in
parallel selfcheck.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently the fail-fast works per section (pylint style, unittests, ...)
but this patch enables it also inside the unittests, therefor it
finishes with the first failed unittest finishing even quicker than
before.
Note that this option is incompatible with the parallel check.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch allows running the tests in parallel, when
AVOCADO_PARALLEL_CHECK env variable is set. It spawns 2xno_cpu workers
(or less when there are not enough tests), captures the output and shows
only output of failed workers. Before the exit it also reports the
number of executed tests in the same fashion as normal unittest
execution (without the number of failed tests, though).

On my laptop this decreases the duration from 125s to 28s.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch adds summary at the end of the execution and uses colors to
mark start and finish of each section. This might produce some
unexpected characters on redirections/unsupported terminals at the
beginning and endings of the highlighted lines, but in most cases it
looks way better and those +3 chars can be ignored by humans.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `checkall` script should not try to develop/install/clean anything.
It's suppose to run the tests and it's upto the user to prepare the
environment they want to check.

This won't affect `make check` as it already cleans/develops before
executing checkall.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently the `make check` goes through all checks and then reports
overall results. This might take a while and is not really `git bisect`
friendly. This patch changes it to report early unless
SELF_CHECK_CONTINUOUS is specified. This is used in `travis` as we are
interested in all failures.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

`.git` may hold content that ends up being checked by inspektor, such
as the `git-rerere` cache.  This may cause false positives, so let's
skip `.git` completetly from those checks.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This allows `AVOCADO_SELF_CHECK=1 ./selftests/checkall` to run
inside virtualenv environemnts.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Not all contrib content is created equal, so let's just move scripts
to a properly named directory, in anticipation for different types of
content.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This implements a "make selfcheck" that developers can use to run
the Avocado selftests in itself. It also keeps the old "make check"
(for a while) so that we can compare to the old approach if any
regressions occur in the near future.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The unittest module, and unittest2 backport on Python 2.6, is capable of
finding unittests and running them. So, effectively, we do not need nose
at all.

This patch removes the dependency on nose, replacing the run script
with a version based solely on the unittest module.

One change of functionality is that run now looks and runs tests in the
standard selftests directories (unit, functional, doc), and does not
accept command line arguments. If one wants to run a subset of them,
it's pretty easy to just use the unittest module for that:

 $ python -m unittest discover -s selftests/unit
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This check is being added to both to checkall and to travis.yaml,
s o that either in local and remote (CI) checks this can be quickly
flagged.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Make sure the checkall and coverageall scripts return
failures.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

One convenient command to run the checks Travis runs on
the code base (less verbose version).
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>