This change adds a new section to yaml to mux documentation about how to
inject values to the final multiplex tree using --mux-inject argument.
Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

According to the author of the publication, the authoritative source,
we were using an incorrect URL to the publication that backs the CIT
implementation.  Let's fix that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Implementation of the CIT varianter optional plugin.
Signed-off-by: NAmador Pahim <amador@apahim.org>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

I never tried this plugin but based on the description it seems to allow
using azure as client for Avocado (or even avocado-vt?) tests.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Now users can run in Avocado the tests created with the GLib Test
Framework with the same level of granularity as they have when executing
the tests directly.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Multiple tests executed by "avocado_loader_yaml" result in the same
name, which makes it hard to distinguish which one belongs to which
params. This commit allows one to specify "mux_suite_test_name_prefix"
which gets prefixed to the test name.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The "yaml_to_mux_loader" implements "special" keys that define the
tests, rather than their params. Let's document them on one place.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The remote execution is an optional plugin, let's just mention the ways
and link the description to the remote_runners there.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

It would be nice to have convinient way to upload results to centralized server.
This plugin superb resultsdb which support external log URL.
Proposed implementation is very simple, but it does it's job.
Signed-off-by: NDmitry Monakhov <dmonakhov@openvz.org>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

On `make rpm`, docs build fails with:

    docs/source/optional_plugins/varianter_pict.rst:33: WARNING: Could
    not lex literal_block as "ini".

This patch changes the way to define a block of code for a
proven-to-be-good one.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

PICT is a free software (MIT licensed) tool that implements
combinatorial testing.  Information about it can be found here:

   https://github.com/Microsoft/pict/

This is a very simple, but functional varianter implementation.  It
performs the basic tasks of creating variants and making parameters
accessible to tests.  To test this version, it's recommended to
download PICT, build it (it only takes a C compiler such as GCC or
clang and make) and put it in a location included in your PATH (or use
`--pict-binary`).

To list the variants generated with the default order (2, that is,
do a pairwise idenpendent combinatorial testing):

 $ avocado variants --pict-parameter-file=examples/params.pict
 Pict Variants (11):
 Variant amd-scsi-rtl8139-windows-rhel6-acff:    /run
 ...
 Variant amd-ide-e1000-linux-rhel6-eb43:    /run

To list the variants generated with a 3-way combination:

 $ avocado variants --pict-parameter-file=examples/params.pict \
   --pict-order-of-combinations=3

 Pict Variants (28):
 Variant intel-ide-virtio-windows-rhel7-aea5:    /run
 ...
 Variant intel-scsi-e1000-linux-rhel7-9f61:    /run

It's also possible to run tests with variants generated by the same
command line switches.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

golang ships a builtin test runner, which is called by
`go test <opts> <package or path>`. This patch introduces the golang
optional plugin, which can be used to discover and run golang tests
using Avocado.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Don't create nodes directly, but first attempt to use OrderedDict. If
advanced features (tags, ...) are detected, declare the current mapping
TreeNode and follow as before. In the end create mapping which should
turn the directly connected OrderedDicts into TreeNodes while keep the
hidden user dicts as OrderedDicts.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The keys are not smart-converted into detected types, but always used as
they are. Let's mention this in our documentation.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The newly introduced YAML Loader plugin is currently hosted on the
same plugin (directory and module) as the varianter one.  While they
share a lot of code, they have different purposes.

Let's move the YAML loader into its own plugin, so that its
installation is also optional and the separation is clearer.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Use the new ability to report full mappings after the discovery and
allow using `test_reference_resolver_class` to specify the resolver
class intended for test discovery.

To allow sufficient flexibility also support
`test_reference_resolver_args` to override resolver arguments and
`test_reference_resolver_extra` to extend the resolver extra_params.

Note the overridden arguments apply only to the resolver, not to whole
Avocado so it's not possible to override wrappers or such just for a
single test execution. Anyway it is possible to set `vt_config` just for
that single loader, which is more-than useful.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Let's add yaml_testsuite loader which allows parsing YAML file and
producing testsuite out of it.

At this point due to Avocado limitation it only supports
FileLoader-based test discovery, but it should be possible with greater
changes to extend the scope of this loader for just about any Avocado
loader.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Words like JSON and YAML are acronyms, and should be written
using capital letters only.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The examples/mux-* yaml files are intended for use in selftests, but
they lived for a long time directly in examples. Let's move them to
proper place to avoid unnecessary confusion to newcomers.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently the yaml_to_mux plugin is a core plugin without properly
defined requirements. Let's separate it to optional_plugins, define
dependencies and also build it separately in RPM.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Robot plugin was introduced without documentation. This PR adds the
basic information about the Robot plugin.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

We have a new section "optional_plugins". Let's create a "results"
category, move the "html" plugin documentation there and link to this
place from the "Result formats" section.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Propagate tests results to resultsdb.

Reference: https://trello.com/c/AcruJdgS/906-result-plugin-for-resultsdbSigned-off-by: NAmador Pahim <apahim@redhat.com>