Based on some reports of failures to behave properly on
unittest.TestCase's assertRaises methods, let's improve the test
coverage on these situations.

References: https://github.com/avocado-framework/avocado/issues/2791Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since the pystache project seems to be dead, without recent updates and
has not support planned for Python 3.7+, this change replace it by jinja
template engine.
Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

In an effort to make jobs more self contained, and given that dry-run
jobs created temporary content that is not supposed to be kept after
the job is run, this introduces the following changes:

 * Jobs with "dry-run" enabled will now, by default, cleanup their
   own data.  Basically, this means that the "base log directory"
   created during job initialization, and that for non "dry-run"
   jobs is usually "~/avocado/job-results", will also be removed
   at the job's "cleanup()" phase.

 * A new "--dry-run-no-cleanup" command line option, that only makes
   sense when jobs with dry-run enabled start to clean up data for
   themselves

 * Because some result plugins will, by default, write to the job
   results directory, results plugins are now run before a job cleanup
   is done.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There are at least two places in which the same functionality is
present.  Let's make it a single utility function instead.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Given that the setUp() phase is not useful for test_disk_space, and the
custom cleanup done in the test is better had at the tearDown() phase.

Additionally, the following changes where made to make it more reliable:

 * Avoid the busy loop giving modprobe some time between attempts
 * Use "modprobe -r" instead of rmmod, which handlers module dependencies
 * Do an explicit check for status
 * If "modprobe -r" doesn't succeed, it's not a test failure, but a test
   a test error (given that the get_diskspace() function already ran
   and succeeded).

Ideally, the module removal would be done by the linux_modules.unload_module()
but there are some issues with that function that needs addressing first
(such as returning a meaningful result).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

As we don't need to support Python < 2.7 anymore.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The newly released pylint version 2.2.0, changed its default behavior
(enabled) the following checks:

 * :unnecessary-pass (W0107): *Unnecessary pass statement*
   Used when a "pass" statement that can be avoided is encountered.

 * :unused-import (W0611): *Unused %s*
   Used when an imported module or variable is not used.

Let's apply fixes to those checks, and at the same time, pin the
pylint versions so that our CI environment is more stable (code that
test our code -- pylint -- doesn't change, while our code changes).

For Python 2, the latest release was 1.9.3, so let's use that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The correct name of the exception is TestAbortError, and not
TestAbortedError.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently, the only way to pass parameters to Avocado is by using the
variants layer, that is, one of the existing varianter plugin
implementations.  This was based on earlier design decisions, that
we're trying to fix without breaking expectation and compatibility.

While it'd makes sense to support parameters created by the varianter
layer itself, together with non-varianter parameters, it seems that we
can start by adding support for one or the other being used at a time.

This commit does exactly that: add support for a method of passing
parameters via the command line for all tests in a job, without
having to resort to the varianter.  When the varianter is used,
it takes precedence and the core test parameter passing support
is ignored.

This should solve a number of use cases that require one-time
executions of tests with different parameters, especially
executions during test development and debugging.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The lv_utils functional tests, which usually don't run because
they require privileged execution, revealed a number of broken
expectations when it comes to bytes and text handling.

Let's fix both the module and the test.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There was a functional test on avocado that depended on html plugin.
This change moves this test to proper plugin instead of run the test
only if the plugin is available during test execution.

Reference: https://trello.com/c/EVggIZzjSigned-off-by: NCaio Carrara <ccarrara@redhat.com>

There was a functional test on avocado that depended on remote plugin.
This change moves this test to proper plugin instead of run the test
only if the plugin is available during test execution.

Reference: https://trello.com/c/NPEUj0wZSigned-off-by: NCaio Carrara <ccarrara@redhat.com>

Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

I was trying to use the external runner, in a reasonably simple way:
from within a project (QEMU) build dir, I'd run:

 $ avocado run --external-runner=tests/qemu-iotests/check \
   --external-runner-chdir=runner -- 001 002

But, what should work, did not.  The problem is with the non-absolute
path to the external runner, that is needed because we're changing
directories.

Absolute paths won't hurt otherwise (when no chdir is done), so it's
always done here.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This pylint checks for possible redefining outer names. Some occurrences
was preserved with disable flag because they would broke some public
api, like method/functions names or parameters.
Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

The W0613 check for unnused arguments. The most of the current cases
seems legitmate so the proper flag was added in each line so pylint can
ignore it.
Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

To decode into the correct encoding and work properly across different
Python versions.

While at it, improve the docstring.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

To the relevant code block that can, for a reason we expect, raise
such exception.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

If the AVOCADO variable starts with the Python interpreter, the
command that will be logged by Avocado needs to be adjusted.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Instead of repeating those definitions on most test files, let's define
them once, and reuse them across tests.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When AVOCADO is set by the environment variable, it may or may not contain
a Python interpreter.  In any situation, Avocado itself won't know about
it, and will log the avocado test runner script in its logs.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Instead of trying locations on Python 2 and 3.  The same pattern is
used on other locations.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Instead of creating new temporary dirs and files, and handling their
removal in each test, let's just create additional dirs/files in the
already provided tmpdir, which will be removed by the test tearDown
method.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

`examples/tests/simplewarning.sh` calls a generic avocado command,
which gets added to the path by the test code. That generic avocado
command is `scripts/avocado`, from the source repository, which
contains the unversioned `/usr/bin/env python`.

Under some environments, such as Fedora >= 29, there may be no
unversioned Python binary.  Let's respect the UNITTEST_AVOCADO_CMD
environment variable, and add the the directory containting that
binary to the PATH.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NMerlin Mathesius <mmathesi@redhat.com>

Signed-off-by: NMerlin Mathesius <mmathesi@redhat.com>

Formatting bytes was only introduced on Python 3.5, so let's just
add the bytes. At the same time, there's no need for some of that
content to live in the module namespace, as only STDOUT and STDERR
are used later in the tests.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Let's double-check the recorded output is correct and not just it's
presence.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This logger is used by stdout/stderr recorder. We decided to keep it in
text-mode so let's treat the encoding errors by using "astring.to_text"
and "xmlcharrefreplace" error handling in case odd characters are
present.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

It has been reported that on some occasions python can report extra
lines when the pipe is broken:

    python2 -c 'print("Hello, World!")' | no-such-command
    -bash: no-such-command: command not found
    close failed in file object destructor:
    sys.excepthook is missing
    lost sys.stderr

Let's remove the check for the number of lines and only keep the check
for unexpected words.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Many arguments are provided by plugins, let's suggest checking the
installed plugins on unrecognized argument to raise awareness of the
need to install additional plugins which might help people that see
Avocado for the first time.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently TAP plugin includes test logs. This changed several times in
the history, but mainly is not needed, so let's disable it by default
and add argument to enable it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

AVOCADO_TEST_DATADIR is deprecated.
Removing them from the examples and selftests.
Signed-off-by: NNarasimhan V <sim@linux.vnet.ibm.com>

After the deprecation, comes the removal.  Hopefully, "workdir" is now
being used in tests by now.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NMerlin Mathesius <mmathesi@redhat.com>

Currently Avocado crashes on py2 when the exception contains unicode.
Let's use the new "astring.to_text" method to safely get the exception
details.

As this exception is part of the results, this commit also fixes the
result plugins to cope with unicodes in them.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

When a plugin is not installed, its command line options are not
registered with the optparse based code.  That is caught earlier than
Avocado's own handling of incompatible options (in this case, the replay
and the remote runner), and results in the errors such as:

  avocado run: error: unrecognized arguments: --remote-hostname

When building RPM packages for Python 3, the remote runner plugin is
not available, and this test should be skipped.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The 'avocado list -V' prints the list of tests as a matrix with
Type, Test and Tag(s) columns. Some lines may get trailing
whitespace chars due the method used to align the columns' value.

Changed the call to avocado.utils.astring.iter_tabular_output()
so that trailing whitespace are removed from each row.

Adjusted the PluginsTest.test_list_no_file_loader() test due lack
of whitespace on outputs to be compared.

Reference: https://trello.com/c/JWeay87z/1045-strip-tabularoutput-linesSigned-off-by: NWainer dos Santos Moschetta <wainersm@redhat.com>