This is similar to the change in 418bf92a, but now applied to the
functional tests that deal with xunit.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Automatically chosen on Python 3.  This mimmics the (proposed)
behavior in `avocado/core/runner.py`.

Reference: https://github.com/avocado-framework/avocado/pull/2451Signed-off-by: NCleber Rosa <crosa@redhat.com>

That's specific to Python 3, but works fine on Python 2.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Which includes the module name, and the different treatment of
exceptions that do not inherit from BaseExcpetion.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Avocado used to provide support for "test aliases", that is, one could
call "avocado run synctest" when the file containing the test was
named "synctest.py".  See e65ff614 for an example.

Since this "feature" has been long removed, let's remove the tests
also.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Remove variable assigments that are used just once and never actually
used.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And not a directory.  Tests were being always skipped becaused of that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The directory check is being performed, but without value to the test.
Let's turn it into an assertion.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently simple tests have a limited status API, being able to set only
the WARN status by generating an output string in a very specific and
hard-coded format.

This patch, while respects the current behaviour, creates configuration
keys, allowing users to provide regular expressions to search for in the
test outputs, aiming to set the final test status to either WARN or SKIP
when the test finishes with exit code 0.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Which is an optional plugin, and the test statuses feature is at the
innermost core of Avocado.  While the original test indeed saved a lot
of duplicated code, it's code that serves a specific testing purpose,
and it's better to be simpler and more verbose and with the right (no)
dependencies.

This issue came up during the Python 3 port work.  Now that all the
unittests (selftests/unit/*) were passing (locally, there's one other
PR pending to be accepted), the next logical step is getting the
"core" functional (selftests/functional/*) tests passing.  Because
the 'varianter_yaml_to_mux' optional plugins port is not done, the
"core" functional tests fail and the current work flow is broken.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Whiteboard was (probably accidentally) removed from the test attributes.
This patch recreates the whiteboard.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

In case on of the first cleanups fails the following ones are not
performed, leaving uncleaned files. Lets perform always all steps and
then check for occurred errors.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The prevailing coding style of Avocado (unittest.TestCase) tests is to
use the specialized assert methods, such as assertNotRegexpMatches.

Unfortunately, the regular expression related assertion methods have
been through a messy renaming.  In theory, assertNotRegexpMatches and
assertRegexpMatches should be available in Python 3.0 and 3.1, but not
in Python >= 3.2, where it got renamed to assertRegex and
assertNotRegex.

In practice, in my system with Python 3.6.3, I see:

>>> unittest.TestCase.assertNotRegexpMatches
<function TestCase._deprecate.<locals>.deprecated_func at 0x7f3b85b709d8>

But in Travis, with Python 3.4, it's not available:

ERROR: test_directives_regex (selftests.unit.test_safeloader.DocstringDirectives)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/avocado-framework/avocado/selftests/unit/test_safeloader.py", line 135, in test_directives_regex
    self.assertNotRegexpMatches(directive, safeloader.DOCSTRING_DIRECTIVE_RE)
AttributeError: 'DocstringDirectives' object has no attribute 'assertNotRegexpMatches'

Let's simplify things and just use a regex match and check for its
result.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The "realpath" is being deprecated in favor of "readlink -f", which
should mainly work the same way. This is required by Travis as it's
version of Ubuntu does not contain "realpath".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

In Travis more tests are failing when running in parallel, but still 20
failed tests in parallel re-ran in series are faster than all tests in
series.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

make spell is now covering more cases. This patch fixes the actual spell
errors and adds the required exceptions.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The docstring given on `_greedy_path()` has examples that don't match
the implementation.  To prevent the implementation from deviating
(again?) let's turn those into tests.

Also, to make the purpose of that function more clear, this
changes its name to `_greedy_path_to_re()`.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The order in which test classes and methods are found is relevant
to the loader result.  Currently, this ordering is not dependable
as the loader findings are stored on a regular dictionary.

This change makes the ordering dependable, and so happens to also
make the test pass on a Python 3 environment.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Which, different than standard Python xml.etree library, can do
schema validation.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

That's the only sane way to handle data, as bytes.  Let's not assume
any enconding on them.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Move tests to the plugin's own directory.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Move tests to the plugin's own directory.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And drop the notion of a "multiplex capable" Avocado.  If the plugin is
installed, then we assume the dependencies have been fulfilled.  The plugin
presence is used then for the creation of the test suite, including the
plugin's own tests.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This drops the notion of a "multiplex capable" Avocado.  If the plugin
is installed, then we assume the dependencies have been fulfilled.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

This change enables plugins tests to also be run in parallel, if
they exist.

Because the "selftests/{unit,functional,doc}" directories are valid
Python modules, that is, they contain `__init__.py` files and can
be imported by the unittest runner, no special handling is necessary.

But, since the complete path for directories which (can) hold plugins
tests are not valid Python modules (and shouldn't be), some special
handling of directories is necessary.

Because of that, it's not currently possible to run *all* of the
tests (from "core" selftests and from plugins) at the same time.
A future enhacement would be to better schedule processes that
can be loaded from the same path at the same time, but all of
them at once with global "test slices".
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Some versions of the unittest mock library cannot easily deal with
(builtin) open().  Let's skip these tests on those platforms, instead
of coming up with cumbersome code to support them on all (ancient)
versions.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This simple test, after the s390x version, checks the proper parsing
of and x86_64 /proc/cpuinfo.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The output in /proc/cpuinfo is different between s390x and other archs.
This is to support to parse the output on s390x.
Signed-off-by: NDan Zheng <dzheng@redhat.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `avocado.core.tree.TreeNode` is the foundation of the current
Parameter System (`avocado.core.parameters.AvocadoParams`), but it
lacks unittests that only deal with itself.

Some tests that exercise the Multiplexer version of the `TreeNodes`
(`MuxTreeNodes`) exist, but again, that's not what the core Parameter
System is based on.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The AvocadoParams, as a database of test parameters, has no need
to know information about the test.  In fact, the test id parameter
has never had any real use so far.

Let's remove it.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

A lot of the core Avocado code still refers to "mux" based terminology.
For instance, what was previously "mux_path" is now "parameter paths",
and what was previously "mux" is now really variants.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The sysinfo code replicates some of the process handling code
already in avocado.utils.process.  Let's use our library instead
and remove code duplication.

This also marks the sysinfo interruption tests as "time and resource
sensitive", and will not run on environments such as Travis-CI, in
a similar way that other interruption based tests are currently set.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is an attempt to catch most (if not all) usages of open
that do not follow the context manager pattern.

Sometimes, for better readability, our own `genio` library is
used.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

To avoid having W0102 (dangerous default arguments) enable W0102 in
`make check`.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Despite the reviews we have some W0102s that potentially corrupts the
arguments. Let's get rid of them.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The default parameters mechanism proved to be confusing, overlapping
and thus was scheduled to be removed.  This implements the removal
of the default_params API.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Our goal to completely separate the basic parameter interface and the
varianter pass through here: splitting these modules into their
reespective ones.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The avocado.utils.process APIs offer the `allow_output_check`
parameter which exists at a function call level, and thus should be
applied only during the scope of that function.

While it's questionable if a test process should be able to skip
some of its generated output (by means of an `avocado.utils.process`
function parameter), the fact is that the utility libraries offer
that at a lower level.
Signed-off-by: NCleber Rosa <crosa@redhat.com>