Currently all failures are ignorred, which makes it hard to actually
understand why our beloved test-reference is not a robot test. Let's
report !ROBOT line along with the exceptions details in such case.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

While debugging Avocado I noticed some crashes when attaching debugger.
It was caused by the debugger which force-flushes the STDOUT on some
occasions, which is usually supported, but is not supported by our
Paginator class. This commit adds the flush method which flushes the
output pipe (unless it's already closed)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently we provide a short mapping to notify the user why is this
test_reference not a test along with coloring it by a different color.
Let's also include a shor message to the test name to explain why is it
so to improve the user experience.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The Loader can report non-tests which are not intended for execution but
only for listing purposes. Let's avoid inheriting them from avocado Test
as it is not really necessary as the tests are basically dummy entities
which does not really do anything.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Make clean is missing the egg directories. This patch includes in make
clean a command to remove them.

Reference: https://trello.com/c/VgF0fkEaSigned-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

If state['params'] is nor present, let's generate the html report
anyway.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

* https://github.com/avocado-framework/avocado:
  spell: v50 fix

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

* https://github.com/avocado-framework/avocado:
  Journal plugin: only attempt to close it if it has been initialized
  Human plugin: only print job results when job is successful

Currently we hardcode the "avocado.app" and "avocado.test" loggers
everywhere in Avocado, let's use the newly defined
`avocado.core.output.LOG_{UI,JOB}` variables everywhere in Avocado
except of `avocado.utils`, which shouldn't have used this interface in
the first place and will be subject of change in the future.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

On many places we are directly getting "avocado.app" and "avocado.test"
loggers and interacting with them. This can become an issue when we try
to replace or modify our logging streams in the future as well as it
requires some hidden knowledge. Let's define LOG_UI and LOG_JOB in
`avocado.core.output` to be available for Avocado core (and silently
also for plugin) developers.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `exceptions` module is not imported even though it's suppose to be
used.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Sometimes we already have the logger so it makes a little sense to pass
logger name only to obtain the logger again in this function. Lets allow
both.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The python root logger is '""' and not "root".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Attempting to close a journal that wasn't initialized is certainly a
bug.  This will check for its initialization first.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The UI was showing information about the job, no matter if the job got
to run tests.

Even though the formal available statuses of a job are not too evolved
and documented, Avocado already sets a job status as "PASS" when it
finishes running the tests.  This involves most of the job phases.

Let's use that to check if the UI should print information about the
test results and job execution time.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Only one spell fix for this version.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

* https://github.com/avocado-framework/avocado:
  python-avocado.spec: include package ownership of "base dir" for "data dir"
  selftests: move InterruptTest.tests_well_behaved into level 1
  Travis-CI: run the extended set of tests, AKA "level 1"
  python-avocado.spec: only run "level 0" tests on package builds
  python-avocado.spec: also setup resultsdb plugin on %check
  selftests: introduce multiple levels of testing coverage

It's a fact that the Avocado package owns not only /var/lib/avocado/data,
but also /var/lib/avocado.  Let's fix that, so users doing something like:

 rpm -qf /var/lib/avocado

Would get the right information.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is time sensitive (currently has a 2s timeout) and has failed
during package builds.

By having it as level 1, it will still be run on Travis-CI.

Reference: https://kojipkgs.fedoraproject.org/work/tasks/4583/19204583/build.logSigned-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is basically to be explicit about selftest level run on check.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

A finer level of granularity may help to extend our testing coverage
while reducing the amount of false positives.  This introduces the
three different levels, mapped like this:

   * level 0, AKA "make check"
   * level 2, AKA "make check-full"

There are no changes of test assignment, that is, tests previously
being run under "check-full" will continue to be run only at that
target.  The same is true for tests that would run under "make check"
before this.

The big change is that there's now a middle ground, that can be
activated by manually setting the AVOCADO_CHECK_LEVEL variable.  Level
1 is intended to be used be used on environments that are halfway
between a dedicated machine and a really low powered environment.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

* https://github.com/avocado-framework/avocado:
  Human interface: do not show tests execution time
  Human Interface: show job execution time
  Job: add time accounting
  Job Pre/Post plugins: move them to the appropriate location

This is a UI tweak, as suggested during review, to remove the tests
time.

It's kept separately, because it can be easily dropped if any reason
to keep it arises.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

A long standing task is to show the job execution time, instead of the
sum of tests execution time.  After some major work, this is now
possible.

This change adds the job execution time to the human interface UI, in
addition to the test execution time.  Some users may prefer the job
execution time alone, while others may prefer all the time statistics
on a single line.

Since we cannot please everyone, this version just adds it.  I believe
UI tweaks can follow.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It may be useful to users of the Job class to have information on how
much time it took for it to run.  Since the goal for the Job class is
to be flexible when it comes to its usage, it may not be possible to
track the execution time if users follow a different path.

For now, the time accouting is done automatically if users use the
`run()` method.

If advanced users of the Job class choose to set the start time, end
time and elapsed time themselves, they are free to do it and `run()`
will never overwrite it.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The JobPre and JobPost plugin interfaces were intended to be run
before and after the job, but in reality they have been executed
at different points *within* the job execution.

This puts the execution of those plugins at their original intended
location:

 * Plugins implementing JobPre will be executed right after the Job
   instance is created, *before any other action*, such as the
   creation of the test suite.

 * Plugins implementing JobPost will be executed right after the Job
   instance `run()` method.

 * Plugins implementing `Result` will be executed after JobPost ones.

Users relying on the old location please take notice.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The tests seems to be stable after the series of improvements applied to
our selftests procedures. Time to restore this check in travis.

Reference: https://trello.com/c/yXtgDRjkSigned-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

* https://github.com/avocado-framework/avocado:
  Extended vendors_map dict for Power8

Added a new item for vendors_map dic to def get_cpu_vendor_name():
Signed-off-by: NVenkat R B <vrbagal1@linux.vnet.ibm.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

The recently added CANCEL color is not cleaned when we disable TERM
support.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

* https://github.com/avocado-framework/avocado:
  run-kvm-unit-test.sh: enable options for configuring kvm-unit-tests