Currently, the only way to pass parameters to Avocado is by using the
variants layer, that is, one of the existing varianter plugin
implementations.  This was based on earlier design decisions, that
we're trying to fix without breaking expectation and compatibility.

While it'd makes sense to support parameters created by the varianter
layer itself, together with non-varianter parameters, it seems that we
can start by adding support for one or the other being used at a time.

This commit does exactly that: add support for a method of passing
parameters via the command line for all tests in a job, without
having to resort to the varianter.  When the varianter is used,
it takes precedence and the core test parameter passing support
is ignored.

This should solve a number of use cases that require one-time
executions of tests with different parameters, especially
executions during test development and debugging.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Implementation of the CIT varianter optional plugin.
Signed-off-by: NAmador Pahim <amador@apahim.org>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Our "other resources" section is fairly limited, let's add new items.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

I never tried this plugin but based on the description it seems to allow
using azure as client for Avocado (or even avocado-vt?) tests.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently TAP plugin includes test logs. This changed several times in
the history, but mainly is not needed, so let's disable it by default
and add argument to enable it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `avocado.Test.datadir` attribute has been deprecated, and it's now
time to have it removed for good.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

After the deprecation, comes the removal.  Hopefully, "workdir" is now
being used in tests by now.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It might happen that user runs process with encoding that is ritcher
than system one. This is fine by the process as internally it stores
encoded bytes object, but the drainer might fail attempting to log the
output.

Those failures should not be critical as, under described circumstances,
the unavailability of the output is expected and should not prevent the
execution. Let's simply replace those characters using 'replace' (the
'xmlcharrefreplace' might fail with ASCII coder) and keep processing the
data.

To simplify error handling new "astring.to_text" argument "errors" is
added.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Avocado is frequently used with Avocado-VT, so let's make it easier
for users to jump to Avocado-VT's documentation for a smoothier
installation experience.

This is based on real feedback from a long time user, after wasting time
attempting to find Avocado-VT references.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NMerlin Mathesius <mmathesi@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Some tests might produce too much logs which might result in too big
xunit results, that might be hard/impossible to publish. Let's allow
setting a limit to number of characters to those files and embed only
1/2 of the limit from the beginning and 1/2 of the limit from the end of
the log.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

These warnings are described as:

 W0612: Used when a variable is defined but not used.
 W0622: Used when a variable or function override a built-in.

After a suggestion from Andrei Stepanov, we decided to immediately
enable these.

Rerefence: https://pagure.io/standard-test-roles/pull-request/144
Reference: https://trello.com/c/Ohh7Q6qj/1183-enable-w-in-travis-make-checkSigned-off-by: NCleber Rosa <crosa@redhat.com>

Python already has a builtin MemoryError exception.  The description
of such an exception is different from the goal here, so let's just
use a different name.

Reference: https://docs.python.org/3/library/exceptions.html#MemoryErrorSigned-off-by: NCleber Rosa <crosa@redhat.com>

After much discussion, it was decided that `srcdir` must go, and
`workdir` should be used instead.  At this time, let's deprecate
it, and in the near future, remove it.

The deprecation approach chosen is to flag tests that use `srcdir`
with warning, so users will hopefully take notice.

Reference: https://github.com/avocado-framework/avocado/issues/1924
Reference: https://trello.com/c/U1aaJjPJ/1158-deprecate-testsrcdir
Reference: https://trello.com/c/dCcxhJc2/1267-remove-testsrcdirSigned-off-by: NCleber Rosa <crosa@redhat.com>

The skip() method has been deprecated for quite some time.  Let's
remove it for good and have users using the decorators and cancel()
instead.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

We don't do Python 2.6 anymore, so it's safe to drop this workaround.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Now users can run in Avocado the tests created with the GLib Test
Framework with the same level of granularity as they have when executing
the tests directly.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Depending on the number of parameters, generating the Variants can be
compute intensive. This patch adds the ability to dump/load the variants
to/from a JSON file so the Variants generation can be offloaded from the
machine under test.

Reference: https://trello.com/c/llZJKBdiSigned-off-by: NAmador Pahim <apahim@redhat.com>

Multiple tests executed by "avocado_loader_yaml" result in the same
name, which makes it hard to distinguish which one belongs to which
params. This commit allows one to specify "mux_suite_test_name_prefix"
which gets prefixed to the test name.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The "yaml_to_mux_loader" implements "special" keys that define the
tests, rather than their params. Let's document them on one place.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The "url" and "test" fields are just legacy and duplicated fields
that carry the same data as the "id" field.  Let's remove those.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There's a better maintained trinity test at:

   https://github.com/avocado-framework-tests/avocado-misc-tests/blob/master/fuzz/trinity.py

Let's drop this outdated example.

Reference: https://github.com/avocado-framework/avocado/issues/2285
Reference: https://trello.com/c/uCcng8dN/1168-drop-trinity-example-testSigned-off-by: NCleber Rosa <crosa@redhat.com>

HTTPS has been enabled on the avocado-project.org server, so let's use
it by default on docs and when downloading images from it.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently simple tests have a limited status API, being able to set only
the WARN status by generating an output string in a very specific and
hard-coded format.

This patch, while respects the current behaviour, creates configuration
keys, allowing users to provide regular expressions to search for in the
test outputs, aiming to set the final test status to either WARN or SKIP
when the test finishes with exit code 0.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Some tweaks here and there found during the upkeep.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The remote execution is an optional plugin, let's just mention the ways
and link the description to the remote_runners there.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Add the description of the recent changes of Avocado packaging to
LTS.next documentation.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

To remove some of the bloat (even if little) of the base
python-avocado package, let's move content that is not useful to
Python developers out of the way.

This should also make the experience of content more similar across
users of the Python module (say, from a `pip install`) and users of
the python-avocado package.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And move that information to the documentation directory, where people
will probably look for instructions if they feel like building the
docs.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Which is also supposed to be the most universal installation
method: pip based installs in the user's own directories.

Then, give information on virtual environment based installations.

The README file was missing the section on pip based installs, so
let's add it there.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>