Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since the new replay command will not support this anymore and it is
already documented on our docs, lets remove this test.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Vmimage automatic documentation [v2]

This generates a documentation table with every tested distribution for
vmimage.
Signed-off-by: NJan Richter <jarichte@redhat.com>

The current runner loader/discovery method is a custom build and finds
modules/tests recursively on the current folder. With the new runner,
part of the python unittest discover was delegated to
'loadTestsFromName()' on the standard library that uses a different
logic for this (assumes that we need to import the test). Because of
that we need 1) create a custom base loader here, or 2) prepend the path
with the current dir so loadTestsFromName will return a valid test
suite.

This might not be the perfect solution but was a hot-fix found for LTS.
Fixes #3851

Reference: https://docs.python.org/3/library/unittest.html#unittest.TestLoader.loadTestsFromNameSigned-off-by: NBeraldo Leal <bleal@redhat.com>

Some plugins have custom configs we need to parse this as soon plugins
are loaded.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>
Signed-off-by: NPlamen Dimitrov <pdimitrov@pevogam.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Now that the old settings module is resting in peace, the new one
should take its place.

This is being done so that plugins can deal with the change in
implementation, but don't have to deal with multiple locations.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Since that we are using the new register option API and default values are
defined inside the source code at registration time, config files can be empty.
avocado config reference will print a list of all supported options. This will
reduce the number of places to update the documentation strings.

Reference: https://github.com/avocado-framework/avocado/issues/3654Signed-off-by: NBeraldo Leal <bleal@redhat.com>

This is removing settings.py old module in favor of future.settings.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

This introduces a new option for replay jobs, one that is expected to
replace the `avocado run --replay` option currently still kept
available.

This implementation has one very striking difference from the previous
one: it really runs a job with all of the source job configurations,
instead of picking individual configurations override.  That has the
obvious differences, but it's understood that it's much more honest
than the previous implementation.  The implementation itself is very
straightforward.

On top of this basic implementation, there could be future extensions
such as:

 * An universal configuration override option, that is, one that
   points to other Job configurations that should be overriden in
   the replayed job.

 * Modifying the source job configuration so that test resolution is
   not performed again, but reuses the resolutions (once also
   persisted) from the source job.  This would increase the job
   reproducibility and it's intended, say, for rerunning the same
   job on a different system with different resolver plugins or
   different tests;

 * Modifying the source job configuration so that test variants (and
   the parameters they contain) are not parsed and generated again,
   but retrieved from the source job in already processed form.  For
   example, when the original job variants plugins generated variants
   "A" and "B", but the environment replaying the job would, because
   of a different varianter version or behavior, would generated
   variants "A", "B" and "C".
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

So that the new implementation can live in a more permanent and
obvious location.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

This introduces the high level architecture of the N(ext) Runner
integration with an Avocado Job, based on Tasks' Life-Cycle.
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

We need to move the initialization steps even earlier to accommodate a
few options that are used during the import time. IMO, this is not the
best approach, but a temporary fix to #4071 and #4081 before LTS.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

docs: small fixes on documentation

Those are suggestions made by @pevogam on #4065.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

This is the initial support to test the Avocado features/options
using the Job API.
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Job API caused errors in some result events plugins. This was caused because
these plugins were loaded too early when job environment wasn't completely set
up. Let's load these plugins when they are needed.

Reference: #4034
Signed-off-by: NJan Richter <jarichte@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is waiting for #4062 to be solved.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Adding here some some possible use case of the current API. So far I
notice that we have two big cases: automatic job discovery vs
manual/custom job. I understand that there is room for improvements but
I would like to keep things consistent and intuitive here, because of
that this work is separating the logic behind the constructors and the
from_config() methods for both: Job() and TestSuite().
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

A lot of changes was made inside job and suite modules. This change will
update the docstrings to map with the current code.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Before this patch directories are not being created properly because
uuid1 was not being called. Took the oportunity to replace to uuid4.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

With .get() we don't need this, since we have default values.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

This is a natural move since with this flexibility would be nice to have
test_parameters per test suite.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

If job.config is present when creating a test suite then we will extend
this configuration. This will make it possible to passa a very long job
config and only small/specific configs for each test suite with the
changes.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

git://git.kernel.org/pub/scm/virt/kvm/kvm-unit-tests is not used
anymore, use https://gitlab.com/kvm-unit-tests/kvm-unit-tests instead.
Signed-off-by: NYihuang Yu <yihyu@redhat.com>

Add an OpenEuler Probe for openEuler system.
Signed-off-by: NWang Shengfang <wangshengfang@huawei.com>

Since we have one variant per suite now, this plugin needs to get the
list of variants instead of only one.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>