The newly introduced YAML Loader plugin is currently hosted on the
same plugin (directory and module) as the varianter one.  While they
share a lot of code, they have different purposes.

Let's move the YAML loader into its own plugin, so that its
installation is also optional and the separation is clearer.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The LoggingFile is used to forward sys.stdout|stderr messages into
expected loggers. Using "paramiko" or "fabric" seems like a bug to me.
Let's only forward them into TEST_LOG, which also propagates to job.log.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The argument logger actually expects multiple loggers in a list. Let's
reflect that to avoid possible confusion.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Use the new ability to report full mappings after the discovery and
allow using `test_reference_resolver_class` to specify the resolver
class intended for test discovery.

To allow sufficient flexibility also support
`test_reference_resolver_args` to override resolver arguments and
`test_reference_resolver_extra` to extend the resolver extra_params.

Note the overridden arguments apply only to the resolver, not to whole
Avocado so it's not possible to override wrappers or such just for a
single test execution. Anyway it is possible to set `vt_config` just for
that single loader, which is more-than useful.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Let's add yaml_testsuite loader which allows parsing YAML file and
producing testsuite out of it.

At this point due to Avocado limitation it only supports
FileLoader-based test discovery, but it should be possible with greater
changes to extend the scope of this loader for just about any Avocado
loader.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The location of the HTML report was moved from
`$RESULTS/html/results.html` to `$RESULTS/results.html`, while a link
was left on the first location.

We don't have to carry that compatibility link anymore.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

All other result plugins write (by default) to the job results dir.
The HTML plugin was different because it used to require a number of
files.  Now, it's a single file, and thus, could/should live in the
same place as other files.

To ease the transition of the location, let's add a symlink, so that
users can still rely on the old location.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently the results are spread into multiple files, which does not
work nicely when results are compressed and only the result.html is
uncompressed. Let's embed everything into `result.html` to make things
easier.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

New optional plugins are now available, and some in RPM package
format.  Let's include them where plugin RPM packages are listed.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The suite order means the order in which the variants will be applied to
tests. Right now we run all variants of first test, then the same for
the second, third, ... tests. The new option "--execution-order" allows
to alternatively specify "tests-per-variant" mode where we run first
variant of all tests, then second variant, third, ...
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently the yaml_to_mux plugin is a core plugin without properly
defined requirements. Let's separate it to optional_plugins, define
dependencies and also build it separately in RPM.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Instead of current, let's stick to the default of using the word
default, and also the option value from optparse itself.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently all failures are ignorred, which makes it hard to actually
understand why our beloved test-reference is not a robot test. Let's
report !ROBOT line along with the exceptions details in such case.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

If state['params'] is nor present, let's generate the html report
anyway.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Currently we hardcode the "avocado.app" and "avocado.test" loggers
everywhere in Avocado, let's use the newly defined
`avocado.core.output.LOG_{UI,JOB}` variables everywhere in Avocado
except of `avocado.utils`, which shouldn't have used this interface in
the first place and will be subject of change in the future.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `exceptions` module is not imported even though it's suppose to be
used.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Test ID has the test UID, the Test Name and the Variant. Let's split
those three information so users can order by any of them separately.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Convenience tooltip to expose the parameters available to the test.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

All optional plugins require the installation (or presence) of
"avocado-framework", so let's do that to the newly introduced
resultsdb plugin too.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The ResultsDB expects the Testcase, not the Test ID. Even so, the
Testcase can be considered the combination of the test name plus the
variant.

While on this, this patch adds the parameters information to the
ResultsDB 'data' section.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

When --html is used the original in-job-dir results are removed just
before the second render. This was probably due to typo as the original
location was (probably) intended for removal. This patch doesn't do that
because the user-defined dir can be shared dir with other user files so
let's just produce the results and keep the dirs intact (while keeping
the refresh in job-dir as that is guaranteed to be our directory and we
do know it can be removed).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Propagate tests results to resultsdb.

Reference: https://trello.com/c/AcruJdgS/906-result-plugin-for-resultsdbSigned-off-by: NAmador Pahim <apahim@redhat.com>

This commit modifies the optional_plugins to depend on VERSION files
located in their main directories and adds "propagate-version" target to
Makefile to update the VERSION files for optional as well as "linkable"
plugins, if VERSION file was present.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Simply because this file won't be available when being installed
from any method outside the source tree.

The downside is obvious: it requires extra steps when doing a new
release.  But that is still better than not being able to ship
these plugins to users using the standard Python mechanisms.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

So users will now be able to find Avocado plugins on PyPI, and install
them via `pip`.  Let's try to be consistent with the naming on those
packages, so that users don't fall into naming traps such as
installing Django related extensions.

Also, those packages map more closely to their (RPM) packaged versions.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Fixing pylint messages for the robot plugin module.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

When robot plugin cannot determine the tests from the test references we
should return an empty list instead of an exception.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

We have to consider that `avocado list` may be executed without the test
references.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

This patch creates an optional plugin to run Robot Framework tests
inside Avocado.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

This is a simple collection of syntatic fixes that work across
Python 2.7 and 3.x.
Signed-off-by: NCleber Rosa <crosa@redhat.com>