Move remote/vm result functionality to runners [v1]

avocado.core.html: Add extensive debug on UnicodeDecodeError

Two bugfixes regarding the testplan

Currently we store job urls in job.urls, while letting people override
them in "job.run()". This is unnecessary and IMO wrong. Let's simply
set the "job.urls" in "job.__init__()" and use them throughout the
whole workflow.

This requires very minor change in standalone tests, everything else
works as before. Additionally we don't need the "preprocess_urls"
which was not actually used.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The functionality to run tests on remote machines and on VMs are
implemented using a mixture of TestRunner and TestResult classes.

In all fairness, it looks like the TestResult classes were abused
because non-standard attributes were used as a foundation for that
functionality. Now that `setup` and `tear_down` have been added
to the TestRunner interface, let's move those custom features from
the TestResult classes to the TestRunner ones.
Signed-off-by: NCleber Rosa <crosa@redhat.com>
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Some users are getting UnicodeDecodeError, but we're unable to reproduce
it. This commit lists the full template and repr() of the failure so we
can at least see what was rendered and identify the source.

No one should actually see this message and we should remove it once we
identify and fix this problem, but without such extensive debug we can't
do anything about it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The parsed (and possibly processed) list of urls given on the command
line is quite useful to other implementation of test runners (see
the upcoming changes on the remote/vm runners). So, let's add a proper
interface that can be used to customize/sanitize the urls given on
the command line, and keep that as a reference in the `urls` attribute.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Instead of getting the application arguments from the test result,
let's get them from from job object itself. The idea is that, once
fnctionality is moved from the test result class to the test runner
class, the same source of arguments can be used for the incoming
code.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

We're adding extra functionality to test runners, were they seem to
be more appropriate than in the test result classes. Let's add empty
implementations on the base class so that their re-implementation
is optional on inherited classes.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This implementation of __getattr__ has allowed users of the test
result proxy, most importantly the result test runner, to ask for
attributes, including important methods, that would only be available
on the RemoteTestResult implementation.

Things have been working more or less by pure chance, based on the
lucky fact that the RemoteTestResult would be at the right place on
the proxy `output_plugins`.

Let's forbid users of the test result proxy to look for attributes
that are not part of the test result "contract" by removing this
magical (and rather evil) method.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The "ps" binary is used in avocado.utils.process library.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This unittest patches find_command to return /usr/bin/sudo and then runs
it and expects some results. This works only with correctly set sudo,
but it does not work without sudo (as the binary is not found) or with
incorrectly set sudo. Let's use "/bin/true" instead.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado.core.runner: Support sigtstp to interrupt tests [v3]

Makefile: cleanup Python compiled files properly

When we abort the test and the test ignores SIGTERM, avocado hangs for
infinity. Let's wait a while and send SIGKILL.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

One of the clean steps executed prior to the deletion of the 'pyc' files
calls `$(PYTHON) setup.py develop --uninstall --user`, which ends up
loading modules and creating some of those again. Let's change the order
so that the Python compiled files are really gone after a `make clean`.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Sometimes it's useful for debugging purposes to pause the running test.
This is currently hard as tests run as subprocess. This patch adds
a special handler for SIGTSTP (ctrl+z) which sends SIGSTOP/SIGCONT
to the test process and it's children.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The "kill_process_tree" executes desired kill and then it sends
SIGCONT to the pid. In some cases (like sending SIGSTOP) this is
undesirable, but on a lot of places the code assumes it sends it.
So let's add "send_sigcont=True", which allows one to disable it
in his particular case.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently the timeout is obtained from Test.params in runner
when early_status is obtained. This might give invalid results as
some tests override the `self.params` with other params (avocado-vt).
This patch sets the `self.timeout` value instead and uses it in
runner to avoid such problems. One can modify this value in
Test.__init__ so it should be flexible enough.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently we don't support merging params provided by loader and
multiplex variants. The way it worked now was, that loader provided
params and they were silently overridden by multiplexer. This patch
raises an exception if params already exists (and multiplexer is used).

Additionally it removes the "avocado_inject_params" hack. Let's leave it
up to the loader to specify params using different keyword and let the
test __init__ to handle it properly (demonstrated in avocado-vt pending
pull request)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This function is useful, when one executes shell=True commands where
he sets values and then executes command. He can use this function
to get the actual binary path instead of the first variable=value
assignment.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Incremental smoke-check

This target checks very basic function of avocado and is used
incrementally in travis to verify this functionality for each
commit up to the origin/master.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Makefile: move the removal of avocado directories to the clean target

The check target have a couple of (relevant) directory cleaning up
commands that are, IMHO, out of place. Let's move them to the clean
target.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Last time it did not work, let's try it again to see whether it works.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado/core/settings.py: simple optimization

Reuse the value of "_config_path_intree" to set "_config_path_intree_extra".
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Test result small improvements

Avocado defines avocado.core.result.TestResult, so that alternative
test result classes can inherit from it. When a user writes a new
test result class, he's left wondering how to activate that.

First, for consistency, Avocado should define proper interfaces
for "test result plugins". This is on sight, and will ultimately
allow users to register new test result implementations simply by
registering the setuptools entry points, just as it's currently
done (on versions >= 0.31.0) for command line related plugins.

Until then, let's at least improve the current state of things.
These changes believe that the following improvements are given here:

* avocado.core.job.Job._set_output_plugins now won't look at every
  possible key in the application arguments, and won't silence
  possible errors while trying to instantiate test result classes.
* test result writers can now use a documented method for registering
  their test result classes.

The same approach was not chosen to be done with the test runner
classes because there can only by one test runner per Avocado job.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is really intended to be the same for all instances of a given
class. Every other (xunit, json, journal, html) result class already
does it like that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The code for executing binary is the same for booth, let's require
self._command to contain the binary and merge the code to avoid future
diversion.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado.core.runner: Force methodName on replay_map substitution

avocado.utils.software_manager: Bugfixes and cleanups

Test namespace cleanup [3rd round]