The subprocess.DEVNULL is not available on Python2.

This reverts commit e51acbfa.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The HTML plugin is using the `subprocess` library to fire up the
web browser that will show the report.  In theory, we could be
using our own `avocado.utils.process` for that, but given the
comments in that section, I believe there may be caveats.

As a fix for the immediate issue at hand (pylint catching a failure on
a Python 3 environment about using `file`), let's just use the
definitions from the `subprocess` module itself.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

As part of 0795831f, all occurrences of dict.iteritems() turned into
iteritems(dict).  The problem is that AvocadoParams doesn't implement
the items() method that six.iteritems() expects from Python 3.

So given that AvocadoParams doesn't provide it, we could either implement
items() just to satisfy iteritems(), or just use the existing interface.

This commit chooses the later, which means reverting a small part of
the changes (incorrect ones) from 0795831f.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This changes uses the six compatibility equivalents for iterkeys,
iteritems and itervalues.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `self.logf` is only used as output location, let's use
`astring.string_to_safe_path` to make sure it can be written. As this
makes sure the name is fs friendly, we don't need any hacks like
`replace(" ", "_")` anymore, which also affects the html plugin.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The location of the HTML report was moved from
`$RESULTS/html/results.html` to `$RESULTS/results.html`, while a link
was left on the first location.

We don't have to carry that compatibility link anymore.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

All other result plugins write (by default) to the job results dir.
The HTML plugin was different because it used to require a number of
files.  Now, it's a single file, and thus, could/should live in the
same place as other files.

To ease the transition of the location, let's add a symlink, so that
users can still rely on the old location.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently the results are spread into multiple files, which does not
work nicely when results are compressed and only the result.html is
uncompressed. Let's embed everything into `result.html` to make things
easier.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

If state['params'] is nor present, let's generate the html report
anyway.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Currently we hardcode the "avocado.app" and "avocado.test" loggers
everywhere in Avocado, let's use the newly defined
`avocado.core.output.LOG_{UI,JOB}` variables everywhere in Avocado
except of `avocado.utils`, which shouldn't have used this interface in
the first place and will be subject of change in the future.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Test ID has the test UID, the Test Name and the Variant. Let's split
those three information so users can order by any of them separately.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Convenience tooltip to expose the parameters available to the test.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

When --html is used the original in-job-dir results are removed just
before the second render. This was probably due to typo as the original
location was (probably) intended for removal. This patch doesn't do that
because the user-defined dir can be shared dir with other user files so
let's just produce the results and keep the dirs intact (while keeping
the refresh in job-dir as that is guaranteed to be our directory and we
do know it can be removed).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We have received many requests to support skip tests from inside the
test method. As we can do that without breaking our own concepts, we
decided to introduce the CANCEL status and the corresponding
self.cancel() method to the Test class.

Reference: https://trello.com/c/viBJIEwISigned-off-by: NAmador Pahim <apahim@redhat.com>

When long fail_reason is detected, we add a on_click popup with the full
description, but somehow we included string "fail_reason" instead of the
fail_reason variable.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

When running tests remotely, html plugin might be already present. Let's
remove the previously existing files to avoid failure and to always get
the latest results.

NOTE: We might want to re-consider whether to skip the html update, or
whether to always update results, but the current status is not optimal.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The html plugin can only cope with finished results. Currently this is
not problem, because `render` is not called, but following patches adds
the possibility to postprocess results in such cases so let's prepare
for it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Commit 3154794d broke the HTML report
when older some versions of pystache is used.  The current status is
that the HTML report generated on those systems is a one line report
containing the literal `utf8` string.

This proposal does not try to add support Unicode based inputs with
that version of pystache, because it's internally not well supported.
Internal methods such as `Template.render_tags()` will blindly try to
do conversions without respecting the encoding passed on `render()`.

Also, it's true that we're missing Unicode support in many other
places, such as documented here in this card:

 https://trello.com/c/bfE9NBbl

For now, let's try properly generate the report, and keep the error
handling already present, that let's users know there have been
encoding errors.

References: https://trello.com/c/8j3aVUlISigned-off-by: NCleber Rosa <crosa@redhat.com>

The "Result" plugin interface was defined a while ago, and we now have
three implementations (JSON, XUnit, HTML).

Let's include them in the plugin listing, that is, when a user calls
`avocado plugins`.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The method `logdir` is not used anywhere in the template, nor in the
plugin itself. I tried to find any occurrences and also setting a
breakpoint without any use whatsoever. Let's remove it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

No actual changes, just style fixes.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This step makes the HTML result a proper Python plugin, by living
outside the Avocado package namespace.

The RPM package is also adjusted: it now builds ands install both
Avocado and the HTML result plugin while keeping the two separate
installable packages.

Developers can use the usual `make link` and `make clean` to enable
and disable all plugins (including this one).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This change removes the knowledge, and dependency, that the current
Avocado core has of the HTML rendering code.

The only user visible change is that the UI message about the
generated HTML file now appears one line after it used to appear.  The
reason is not because of the HTML plugin itself, but because the
results are now generated outside the context of the job, and that is,
chronologically a bit later than it used to be.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The initial versions of Avocado focused on tests, and ommited a bit
the idea of jobs.  This had the consequence of some unfit names, such
as the TestResult class, which defines methods such as "start_tests".
It's clear that a "Test" starting "tests" is either about a nested
relationship or is badly named.

Instead of replacing the name of the TestResult related classes to
JobResult, which is currently our top level player in most of the
Avocado use cases, let's just consider that this class holds the
result of what Avocado runs.  It's an abstract name, but it's not
vague because the individual attributes refer to the exact type of
information that is being held.  For instance, `job_unique_id` is
clearly about a Job, and `tests_run` is clearly about tests.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Avocado currently does not define a set of APIs for plugin writers.
It's well understood that anyone writing plugins is really playing
under the hoods of core Avocado.

Because there's no such module namespace allocated for that purpose
alone (say, `avocado.plugin`) it was decided to put plugin interfaces
together with the "core plugin repository", that is the
"avocado/plugins" directory.  By no means the contents of
"avocado/plugins/", that is, the plugin implementations themselves,
need be the "avocado.plugins" module, it could just as well live in
"avocado-core-plugins" directory or repository.  This is also well
exemplified by the avocado-vt and avocado-virt plugins, which live in
completely separate repositories.

So, the first goal of these changes are, effectively, to make all
plugin implementations equal, always referring to
`avocado.core.plugins_interface`.  Then, the secondary goal is to pave
the way for a specific namespace to contain only the interfaces (which
should be non-core) which plugin writers should rely on.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The "View" concept was developed to abstract the messages and events.
The problem is it was only abused to pass the status to other places, to
colorize the messages and to allow creating paginated view.

There already is a nice class hooked to all important events, the
"TestResult".

For (not only human readable) messages this commit uses the standard
python logging as it's pretty well known, widely used and very scalable.
The colored output is handled by already existing
"ProgressStreamHandler", which maps: DEBUG,INFO,WARNING and >=ERROR
messages to previously existing: minor, message, warning and error event
types.

The paginator was unified and is initialized during logging reconfigure.
During reconfigure all previously logged messages are re-logged into the
output so one does not lose those messages. Another great difference is
that the Paginator is cleaned at exit by avocado and does not require
complex handling to avoid broken console. To use paginator one just
enables it in "args" and writes to any available stream/stdout/stderr
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This contains some import cleanups, few docstring updates and escaped
new line fixes.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

No actual changes, only reorder imports alphabetically (system, utils,
relative).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Avocado defines avocado.core.result.TestResult, so that alternative
test result classes can inherit from it. When a user writes a new
test result class, he's left wondering how to activate that.

First, for consistency, Avocado should define proper interfaces
for "test result plugins". This is on sight, and will ultimately
allow users to register new test result implementations simply by
registering the setuptools entry points, just as it's currently
done (on versions >= 0.31.0) for command line related plugins.

Until then, let's at least improve the current state of things.
These changes believe that the following improvements are given here:

* avocado.core.job.Job._set_output_plugins now won't look at every
  possible key in the application arguments, and won't silence
  possible errors while trying to instantiate test result classes.
* test result writers can now use a documented method for registering
  their test result classes.

The same approach was not chosen to be done with the test runner
classes because there can only by one test runner per Avocado job.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>