The newly introduced YAML Loader plugin is currently hosted on the
same plugin (directory and module) as the varianter one.  While they
share a lot of code, they have different purposes.

Let's move the YAML loader into its own plugin, so that its
installation is also optional and the separation is clearer.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Use the new ability to report full mappings after the discovery and
allow using `test_reference_resolver_class` to specify the resolver
class intended for test discovery.

To allow sufficient flexibility also support
`test_reference_resolver_args` to override resolver arguments and
`test_reference_resolver_extra` to extend the resolver extra_params.

Note the overridden arguments apply only to the resolver, not to whole
Avocado so it's not possible to override wrappers or such just for a
single test execution. Anyway it is possible to set `vt_config` just for
that single loader, which is more-than useful.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Let's add yaml_testsuite loader which allows parsing YAML file and
producing testsuite out of it.

At this point due to Avocado limitation it only supports
FileLoader-based test discovery, but it should be possible with greater
changes to extend the scope of this loader for just about any Avocado
loader.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently the yaml_to_mux plugin is a core plugin without properly
defined requirements. Let's separate it to optional_plugins, define
dependencies and also build it separately in RPM.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently we hardcode the "avocado.app" and "avocado.test" loggers
everywhere in Avocado, let's use the newly defined
`avocado.core.output.LOG_{UI,JOB}` variables everywhere in Avocado
except of `avocado.utils`, which shouldn't have used this interface in
the first place and will be subject of change in the future.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The "avocado multiplex" command is misleading as multiplexation is only
the generic concept used by one of our varianter plugins
implementations. Let's change the command name to "avocado variants"
which better reflects what is it used for.

As people depend on "avocado multiplex" command let's keep both of them
for now to give people time till Avocado 52.0 to migrate.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This adds the support to define per-node filters, which take part of
environment and are evaluated during multiplexation.

The basic idea is to define relations ({could be only used | could
not be used} with other branch), which makes it a bit different from
what we know from `--mux-filter-{only|out}`:

1. They are applied per complete variant (because the filters are
   compound of per-leaf-filters)
2. They don't remove/keep the nodes, but they only evaluate whether
   this resulting variant makes sense (whether it passes the filters)

See the updated documentation for details.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The tree_node_from_values is already quite complex and is about to be
extended. Let's split the body into functions.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

That includes creating a new section for them, and also adding
name and description to the `yaml_to_mux` implementation.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The definition of AVOCADO_JOB_FAIL was "Something went wrong with the
Job itself, by explicit avocado.core.exceptions.JobError exception",
while AVOCADO_FAIL is defined as "Something else went wrong and
avocado failed (or crashed). Commonly used on command line validation
errors".

With those in mind, I support that AVOCADO_JOB_FAIL should be slightly
adjusted (see changes in the diff), to include other types of
failures, but limited to job *execution* failures.  AVOCADO_FAIL is
already generic enough, so there's no need to change its definition.
This is important, IMHO, because users could quickly understand if a
job was executed, and failed, or if some earlier type of failure (such
as command line validation) happened.

With this updated definition, I spotted a few places where AVOCADO_FAIL
and AVOCADO_JOB_FAIL usage should be changed.  Examples:

 * In `avocado.plugins.diff` the resultsdir is attempted to be read, but
   in my book that still doesn't qualify as a job failure, because a job
   is not being executed.

 * In `avocado.plugins.replay` while there's functionality to run a
   (replayed) job, many of the occurrences of AVOCADO_JOB_FAIL are
   similar to the `avocado.plugins.diff` and happen while trying to load
   jobdata from the previous job resultsdir.

 * In `avocado.plugins.multiplex` the situation is a bit more tricky.  The
   same code is executed when multiple avocado subcommands are executed.
   If the currently executed command is `run`, it would mean that a job is
   being attempted to be executed.  In fact, a job *is* already executing
   at this point, so a AVOCADO_JOB_FAIL is appropriate.  In other scenarios,
   such as when running `$ avocado multiplex`, these failures have nothing
   to do with jobs, so AVOCADO_FAIL is appropriate.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

All the plugin interfaces defined in `avocado.core.plugin_interfaces`
ommit the "plugin" term, simply because this is already qualified in
the module name.

Let's also standardize the Varianter interface to "plugin-less" name.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This patch replaces the hardcoded mux in Varianter for list of Varianter
plugins, iterating through all of them to produce all specified
variants. The interface is described in `avocado.core.plugin_interfaces`
and we also provide base-implementation for Mux-like plugins in
`avocado.core.mux`.

Mainly this PR is about defining the stevedore entry points, moving the
code around and wrapping the code into a class according the defined
stevedore interface. Doing so allowed us to simplify the `MuxTree`
object which does not need to keep track of the `root` of the tree,
because the root is stored in the pluggin's instance.

There is still one potential issue in Varianter which is probably even
more visible now, which is that we are pickling it in jobdata in order
to be able to reproduce the variants when re-running/diffing the job.
Before this was prone to code-changes but now it can be even worse as
multiple plugins can be enabled and can break loading of the previous
state. To avoid the potential breakage this version simply initializes
only the available extensions which should produce only the supported
variants.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

--filter-only/--filter-out are being deprecated by
--mux-filter-only/--mux-filter-out, but the deprecation
messages are pointing users to --mux-only/--mux-out, which
does not exist.

This patch fixes the deprecation messages, referencing the
correct new options.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Previously we were directly using the yaml.Loader, which is shared
across all `yaml` instances. This patch creates a `_BaseLoader` with the
basic set of constructors. That one is copied and updated of the
run-time data (debug/non-debug) during the `_create_from_yaml` to allow
loading debug and non-debug files in a single execution without
overriding the same loader.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

When we use args:--mux-yaml or --multiplex if the yaml file not existing it
will show more detail error messages

Reference: https://trello.com/c/hYCrS1oI/900-improve-yaml-to-mux-error-messagesSigned-off-by: Nmengalong <alongmeng@gmail.com>

The filtering is not useful for basic params, let's move it to mux and
while on it also remove the avocado arguments (options) to the
yaml_to_mux plugin.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `avocado.core.tree` contains multiplex/mux-related features like
multiplex and ctrl, which are not really useful for basic params. Let's
simplify the base tree object and put the extracted features in
`avocado.core.mux`.

This is a first step and the `avocado.core.mux` should contain most of
the generic parts useful for creating varianter plugins.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `_create_from_yaml` function allows specifying custom tree node
object, but it only works properly with `tree.TreeNode` inherited ones.
Let's use the custom class instead to make the code cope with different
use cases.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The concept of multiplexer is about to be split into an abstract API for
variants generation, multiplexer which will be in-core implementation of
one of the variants generation plugins and yaml_to_mux plugin which uses
this multiplexer implementation to create variants from a yaml file.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Multiplexer is about to extend the number of deprecated messages, let's
create a method to log these.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Since the test logger is not yet enabled when plugins are in action, we
should use avocado.app logger to warn user about the deprecated option
--multiplex.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

This patch avoids avocado crash on empty multiplex yaml file. Empty file
contains basically nothing and that is how it's handled now.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Recently we created separate `yaml2mux` plugin, which keeps using the
`--multiplex` argument. The connection between yaml2mux and multiplex is
not really obvious and the separation was meant to bring the order to
the avocado params system. This is another step which replaces the
`--multiplex|-m` with `--mux-yaml|-m` and makes it more obvious that
it's related to `yaml` files and it's also related to `multiplexation`.

You might notice that the old `-m` was kept. It's very convenient and
also allows coexisting of old and new versions (useful eg. in remote
executions).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This complex patch replaces the custom `--multiplex` handling with a
brand new plugin called `yaml_to_mux`. This touches several place and
removes the `yaml` functionality from the avocado core and moves it into
the plugin code.

There are still 2 references from plugins to the `yaml_to_mux` plugin
which are not defined as a public API yet.

1. remote runner - which allows copying --multiplex files to remote
                   machine.
2. replay - which allows overriding of the job's mux variants with
            the --multiplex files. Note that by doing this you
            lose the system-wide options. It is described in the
            warning message.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>