In situations where a user of the CmdResult class manually sets the
stdout/stderr attributes to a string, instead of the recommended and
documented content of type "bytes", it won't be possible to "decode"
the bytes into a string of a given encoding, and we'll end up with
a crash.

Since the goal of std{out,err}_text is to return a text version
of stdout, if itself already is of such a type, let's just return
it.

Additionally, if the data cannot be "decode()d", let's raise an
explicit TypeError on this location, rather than later in the code
when its value is attempted to be used.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Depending on the number of parameters, generating the Variants can be
compute intensive. This patch adds the ability to dump/load the variants
to/from a JSON file so the Variants generation can be offloaded from the
machine under test.

Reference: https://trello.com/c/llZJKBdiSigned-off-by: NAmador Pahim <apahim@redhat.com>

Looks like the Python 3 module import code behaves differently than
Python 2 with regards to where it looks modules by name:

   File "/home/cleber/src/avocado/avocado/avocado/core/test.py", line 932, in _run_avocado
      raise test_exception
   File "/home/cleber/src/avocado/avocado/avocado/core/test.py", line 819, in _run_avocado
      testMethod()
   File "/home/cleber/src/avocado/avocado/avocado/core/test.py", line 1371, in test
      self.error(self.exception)
   File "/home/cleber/src/avocado/avocado/avocado/core/test.py", line 1041, in error
      raise exceptions.TestError(message)\navocado.core.exceptions.TestError: Traceback (most recent call last):
   File "/usr/lib64/python3.6/imp.py", line 297, in find_module
      raise ImportError(_ERR_MSG.format(name), name=name)
   ImportError: No module named \'whiteboard\'\n\n'

Effectively, this means when running this test, on Python 3, on the
parallel mode, the change of directory implemented there
(on selftests/checkall) will make Python 3 fail to find this module,
and the test will fail.

This makes the reference to the module path absolute, so no matter
where this test is run from, it should be able to find the module.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This removes the exception of allowing failures on the Travis CI jobs
with Python 3, while, at the same time, temporary restricts Python 3
jobs to run only unittests and the documentation build for the Avocado
core.  That is, it excludes the functional tests, and the optional
plugin's tests.

While it may look like a step back, it's actually a step forward in the
Python 3 port, because now any regression on the unittests will be caught.
And, when the additional fixes are applied to address the functional tests
and plugin's tests, the whole set of tests will be enabled for Python 3.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The pystache library on Python 3 will generated an HTML file with entity
references for the single quote char, instead of the literal char.

Let's check for either one of them in the functional test
test_basic.RunnerSimpleTest.test_fs_unfriendly_run().
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The content on the file is being read as bytes, so let's use byte
literals in the comparison.

The error message received some adjustments, mainly because it could
produce weird stuff text such as "1th" or "2th".  To be honest, I'd
remove this error message completely, the assertion error is pretty
easy to spot IMO.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This fixes the functional test
test_loader.LoaderTestFunctional.test_yaml_loader_run.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Instead of as bytes in assertions, specially because of a part of it
is going to be used as a path.  This fixes the following issue:

    ======================================================================
    ERROR: test_default_enabled_plugins (test_output.OutputPluginTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/home/cleber/src/avocado/avocado/selftests/functional/test_output.py", line 509, in test_default_enabled_plugins
        self.check_output_files(debug_log)
      File "/home/cleber/src/avocado/avocado/selftests/functional/test_output.py", line 286, in check_output_files
        json_output_path = os.path.join(base_dir, 'results.json')
      File "/usr/lib64/python3.6/posixpath.py", line 92, in join
        genericpath._check_arg_types('join', a, *p)
      File "/usr/lib64/python3.6/genericpath.py", line 151, in _check_arg_types
        raise TypeError("Can't mix strings and bytes in path components") from None
    TypeError: Can't mix strings and bytes in path components
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This fixes the functional test "test_invalid_values" (from
test_job_timeout.JobTimeOutTest).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The config file generated and used on this test was missing a
separator between the section name and the first key/value line.

This adds a new line and makes the resulting file aesthetically
pleasing and able to be parsed.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The "url" and "test" fields are just legacy and duplicated fields
that carry the same data as the "id" field.  Let's remove those.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The avocado.utils.process module has been changed so that the content
of stdout and stderr are now bytes.  The content generated by the
iso9660 backend implementations should also operate on bytes, given
that it makes little sense for the library to assume a given content
type of a file (inside an iso9660 archive).

To make the data returned by the various backend `read()`
implementations always return bytes on both Python 2 and 3, an
explicit conversion is done from strings to bytes (which affects only
Python 2, since Python 3 already operates with bytes on files opened
in binary mode).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This brings a change in behaviour, in which the stdout/stderr of the
executed process will now be of bytes type, instead of a string type.

Two new attributes, which are implemented as properties, have been
added to the CmdResult class, `stdout_text` and `stderr_text`.  Those
are convenience methods that will return the same content that is in
`stdout` and `stderr`, reespectively, but decoded on the fly[1].

With regards to encoding, if one is not provided, the result of
`sys.getdefaultencoding()` will be used ("utf-8" for Python 3 and
"ascii" for Python 2).

Applications and/or tests using the APIs that return a CmdResult
should, to the best of my knowledge, set a default encoding themselves
so a stable behavior across Python versions.  But that if left to
users of this API.

A different tradeoff/design decision has to do with the tests modified
here.  One option is to have "text" (as in sequences of human readable
glyphs) as being of Python type "str".  On Python 2, "str" can be
compared to "bytes" because a conversion will happen on demand.  That
is, the following is fine on Python 2:

   >>> result = process.run("command")
   >>> "expected" in process.stdout

Where `expected` is of type "str" and `process.stdout` is of type
"bytes".  This is not true of Python 3, so either the types must match
or a conversion must be done explicitly.  The solutions to that are:

1) have these "text" as (of type) "bytes" in the source code itself,
   and avoid the conversion whenever possible
2) have "strings" in the source code itself, and use the conversion
   provided by `CmdResult.stdout_text` and `CmdResult.stderr_text`.

The approach chosen here is to avoid conversion if possible, that is,
use "byte" types, given the fact that the source code encoding is by
default 'ascii' and most of the "text" dealt with here can be
represented in 'ascii' too.  This is equivalent of doing:

   result = process.run("command")
   b"expected" in process.stdout
   "errors: %s" % 0 in process.stderr_text

[1] The obvious alternative, instead of decoding these on the fly
    would be to have multiple copies of the "same" data.  This assumes
    that binary data produced on the stdout/stderr will usually be
    larger than textual data.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Which makes data comparison predictable and stable across Python
versions and encodings.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is similar to the xunit unittest changes in 353fed04, but now
applied to the functional tests which happen to use the same features.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is similar to the change in 418bf92a, but now applied to the
functional tests that deal with xunit.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Automatically chosen on Python 3.  This mimmics the (proposed)
behavior in `avocado/core/runner.py`.

Reference: https://github.com/avocado-framework/avocado/pull/2451Signed-off-by: NCleber Rosa <crosa@redhat.com>

That's specific to Python 3, but works fine on Python 2.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Which includes the module name, and the different treatment of
exceptions that do not inherit from BaseExcpetion.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Avocado used to provide support for "test aliases", that is, one could
call "avocado run synctest" when the file containing the test was
named "synctest.py".  See e65ff614 for an example.

Since this "feature" has been long removed, let's remove the tests
also.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Remove variable assigments that are used just once and never actually
used.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And not a directory.  Tests were being always skipped becaused of that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The directory check is being performed, but without value to the test.
Let's turn it into an assertion.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently simple tests have a limited status API, being able to set only
the WARN status by generating an output string in a very specific and
hard-coded format.

This patch, while respects the current behaviour, creates configuration
keys, allowing users to provide regular expressions to search for in the
test outputs, aiming to set the final test status to either WARN or SKIP
when the test finishes with exit code 0.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Which is an optional plugin, and the test statuses feature is at the
innermost core of Avocado.  While the original test indeed saved a lot
of duplicated code, it's code that serves a specific testing purpose,
and it's better to be simpler and more verbose and with the right (no)
dependencies.

This issue came up during the Python 3 port work.  Now that all the
unittests (selftests/unit/*) were passing (locally, there's one other
PR pending to be accepted), the next logical step is getting the
"core" functional (selftests/functional/*) tests passing.  Because
the 'varianter_yaml_to_mux' optional plugins port is not done, the
"core" functional tests fail and the current work flow is broken.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Whiteboard was (probably accidentally) removed from the test attributes.
This patch recreates the whiteboard.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

In case on of the first cleanups fails the following ones are not
performed, leaving uncleaned files. Lets perform always all steps and
then check for occurred errors.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The prevailing coding style of Avocado (unittest.TestCase) tests is to
use the specialized assert methods, such as assertNotRegexpMatches.

Unfortunately, the regular expression related assertion methods have
been through a messy renaming.  In theory, assertNotRegexpMatches and
assertRegexpMatches should be available in Python 3.0 and 3.1, but not
in Python >= 3.2, where it got renamed to assertRegex and
assertNotRegex.

In practice, in my system with Python 3.6.3, I see:

>>> unittest.TestCase.assertNotRegexpMatches
<function TestCase._deprecate.<locals>.deprecated_func at 0x7f3b85b709d8>

But in Travis, with Python 3.4, it's not available:

ERROR: test_directives_regex (selftests.unit.test_safeloader.DocstringDirectives)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/avocado-framework/avocado/selftests/unit/test_safeloader.py", line 135, in test_directives_regex
    self.assertNotRegexpMatches(directive, safeloader.DOCSTRING_DIRECTIVE_RE)
AttributeError: 'DocstringDirectives' object has no attribute 'assertNotRegexpMatches'

Let's simplify things and just use a regex match and check for its
result.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The "realpath" is being deprecated in favor of "readlink -f", which
should mainly work the same way. This is required by Travis as it's
version of Ubuntu does not contain "realpath".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

In Travis more tests are failing when running in parallel, but still 20
failed tests in parallel re-ran in series are faster than all tests in
series.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

make spell is now covering more cases. This patch fixes the actual spell
errors and adds the required exceptions.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The docstring given on `_greedy_path()` has examples that don't match
the implementation.  To prevent the implementation from deviating
(again?) let's turn those into tests.

Also, to make the purpose of that function more clear, this
changes its name to `_greedy_path_to_re()`.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The order in which test classes and methods are found is relevant
to the loader result.  Currently, this ordering is not dependable
as the loader findings are stored on a regular dictionary.

This change makes the ordering dependable, and so happens to also
make the test pass on a Python 3 environment.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Which, different than standard Python xml.etree library, can do
schema validation.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

That's the only sane way to handle data, as bytes.  Let's not assume
any enconding on them.
Signed-off-by: NCleber Rosa <crosa@redhat.com>