Based on some reports of failures to behave properly on
unittest.TestCase's assertRaises methods, let's improve the test
coverage on these situations.

References: https://github.com/avocado-framework/avocado/issues/2791Signed-off-by: NCleber Rosa <crosa@redhat.com>

The newly released pylint version 2.2.0, changed its default behavior
(enabled) the following checks:

 * :unnecessary-pass (W0107): *Unnecessary pass statement*
   Used when a "pass" statement that can be avoided is encountered.

 * :unused-import (W0611): *Unused %s*
   Used when an imported module or variable is not used.

Let's apply fixes to those checks, and at the same time, pin the
pylint versions so that our CI environment is more stable (code that
test our code -- pylint -- doesn't change, while our code changes).

For Python 2, the latest release was 1.9.3, so let's use that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently, the only way to pass parameters to Avocado is by using the
variants layer, that is, one of the existing varianter plugin
implementations.  This was based on earlier design decisions, that
we're trying to fix without breaking expectation and compatibility.

While it'd makes sense to support parameters created by the varianter
layer itself, together with non-varianter parameters, it seems that we
can start by adding support for one or the other being used at a time.

This commit does exactly that: add support for a method of passing
parameters via the command line for all tests in a job, without
having to resort to the varianter.  When the varianter is used,
it takes precedence and the core test parameter passing support
is ignored.

This should solve a number of use cases that require one-time
executions of tests with different parameters, especially
executions during test development and debugging.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

This pylint checks for possible redefining outer names. Some occurrences
was preserved with disable flag because they would broke some public
api, like method/functions names or parameters.
Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

The W0613 check for unnused arguments. The most of the current cases
seems legitmate so the proper flag was added in each line so pylint can
ignore it.
Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

Commit 5cb976a9 attempted to remove avocado-server from release test
plan, but forgot this one.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The testIdentity changed here was using assertTrue with a constant
instead of a conditional. So, to make lint pass it was updated to a more
appropriate assert.
Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

Implementation of the CIT varianter optional plugin.
Signed-off-by: NAmador Pahim <amador@apahim.org>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The "avocado-server" is not being actively developed, still we spend 2
extra manual steps on trying whether it works. Let's stop and depend on
user-input.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

There are 5 extra steps every release on "avocado-virt" which is not
actively developed. Let's stop that and depend on user-inputs.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The "aexpect" package should be installed from distribution, there is no
need to build it nor install from additional repositories.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `python-avocado` package does not exists anymore, now we have py2
and py3 version. Let's check both and include the `src.rpm` as well.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This example test fetches two different files from different
locations, and uses a data file (shipped *with* the test) on
one of them.

The goal is to demonstrate a complete example of files you
may want to distribute with your test, and the assets you
may want to fetch from remote locations.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

AVOCADO_TEST_DATADIR is deprecated.
Removing them from the examples and selftests.
Signed-off-by: NNarasimhan V <sim@linux.vnet.ibm.com>

Moved original whiteboard example test to selftests and then
created a simpler example code.

Reference: https://trello.com/c/sG6J0VpwSigned-off-by: NWainer dos Santos Moschetta <wainersm@redhat.com>

After the deprecation, comes the removal.  Hopefully, "workdir" is now
being used in tests by now.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently Avocado crashes on py2 when the exception contains unicode.
Let's use the new "astring.to_text" method to safely get the exception
details.

As this exception is part of the results, this commit also fixes the
result plugins to cope with unicodes in them.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

When this test was initially written, the concept of a shared
directory available across tests did not exist.  Let's optimize the
execution by avoid compilation of the same binary on every single
test.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since the result of `genio.read_all_lines()` already is a list of all
lines (split by newlines).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

These warnings are described as:

 W0612: Used when a variable is defined but not used.
 W0622: Used when a variable or function override a built-in.

After a suggestion from Andrei Stepanov, we decided to immediately
enable these.

Rerefence: https://pagure.io/standard-test-roles/pull-request/144
Reference: https://trello.com/c/Ohh7Q6qj/1183-enable-w-in-travis-make-checkSigned-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NWilliam Tambe <william.tambe@amd.com>

The latest jobs have been failing because of the following issues:

    ************* Module raise
    E1130: 48,30: Raise.test: bad operand type for unary -: NoneType
    ************* Module selftests.functional.test_basic
    I1101:1191,24: PluginsXunitTest.run_and_check: Module 'lxml.etree'
    has not 'XMLSchema' member, but source is unavailable. Consider
    adding this module to extension-pkg-whitelist if you want to
    perform analysis based on run-time introspection of living
    objects.
    I1101:1191,40: PluginsXunitTest.run_and_check: Module 'lxml.etree'
    has not 'parse' member, but source is unavailable. Consider adding
    this module to extension-pkg-whitelist if you want to perform
    analysis based on run-time introspection of living objects.
    I1101:1193,43: PluginsXunitTest.run_and_check: Module 'lxml.etree'
    has not 'parse' member, but source is unavailable. Consider adding
    this module to extension-pkg-whitelist if you want to perform
    analysis based on run-time introspection of living objects.
    ************* Module selftests.unit.test_xunit
    I1101: 91,24: xUnitSucceedTest.test_add_success: Module
    'lxml.etree' has not 'XMLSchema' member, but source is
    unavailable. Consider adding this module to
    extension-pkg-whitelist if you want to perform analysis based on
    run-time introspection of living objects.
    I1101: 91,40: xUnitSucceedTest.test_add_success: Module
    'lxml.etree' has not 'parse' member, but source is
    unavailable. Consider adding this module to
    extension-pkg-whitelist if you want to perform analysis based on
    run-time introspection of living objects.
    I1101: 92,43: xUnitSucceedTest.test_add_success: Module
    'lxml.etree' has not 'parse' member, but source is
    unavailable. Consider adding this module to
    extension-pkg-whitelist if you want to perform analysis based on
    run-time introspection of living objects.

We can't whitelist the `lxml.etree` module at this time (inspekt doesn't
have such an option) and the raise failure is a false positive.  Let's
ignore those issues then.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

After much discussion, it was decided that `srcdir` must go, and
`workdir` should be used instead.  At this time, let's deprecate
it, and in the near future, remove it.

The deprecation approach chosen is to flag tests that use `srcdir`
with warning, so users will hopefully take notice.

Reference: https://github.com/avocado-framework/avocado/issues/1924
Reference: https://trello.com/c/U1aaJjPJ/1158-deprecate-testsrcdir
Reference: https://trello.com/c/dCcxhJc2/1267-remove-testsrcdirSigned-off-by: NCleber Rosa <crosa@redhat.com>

The skip() method has been deprecated for quite some time.  Let's
remove it for good and have users using the decorators and cancel()
instead.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The API used to get the file, by name, is `get_data()`, which looks
for that file in one of the data directories that the class provides
(file-level, class-level, variant-level, etc).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

These parameters are not class instantiation parameters, which is
the indentend usage of those docstring notations.  But, even worse,
they don't exist in these example tests at all.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Multiple tests executed by "avocado_loader_yaml" result in the same
name, which makes it hard to distinguish which one belongs to which
params. This commit allows one to specify "mux_suite_test_name_prefix"
which gets prefixed to the test name.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This brings a change in behaviour, in which the stdout/stderr of the
executed process will now be of bytes type, instead of a string type.

Two new attributes, which are implemented as properties, have been
added to the CmdResult class, `stdout_text` and `stderr_text`.  Those
are convenience methods that will return the same content that is in
`stdout` and `stderr`, reespectively, but decoded on the fly[1].

With regards to encoding, if one is not provided, the result of
`sys.getdefaultencoding()` will be used ("utf-8" for Python 3 and
"ascii" for Python 2).

Applications and/or tests using the APIs that return a CmdResult
should, to the best of my knowledge, set a default encoding themselves
so a stable behavior across Python versions.  But that if left to
users of this API.

A different tradeoff/design decision has to do with the tests modified
here.  One option is to have "text" (as in sequences of human readable
glyphs) as being of Python type "str".  On Python 2, "str" can be
compared to "bytes" because a conversion will happen on demand.  That
is, the following is fine on Python 2:

   >>> result = process.run("command")
   >>> "expected" in process.stdout

Where `expected` is of type "str" and `process.stdout` is of type
"bytes".  This is not true of Python 3, so either the types must match
or a conversion must be done explicitly.  The solutions to that are:

1) have these "text" as (of type) "bytes" in the source code itself,
   and avoid the conversion whenever possible
2) have "strings" in the source code itself, and use the conversion
   provided by `CmdResult.stdout_text` and `CmdResult.stderr_text`.

The approach chosen here is to avoid conversion if possible, that is,
use "byte" types, given the fact that the source code encoding is by
default 'ascii' and most of the "text" dealt with here can be
represented in 'ascii' too.  This is equivalent of doing:

   result = process.run("command")
   b"expected" in process.stdout
   "errors: %s" % 0 in process.stderr_text

[1] The obvious alternative, instead of decoding these on the fly
    would be to have multiple copies of the "same" data.  This assumes
    that binary data produced on the stdout/stderr will usually be
    larger than textual data.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

base64.encodestring() requires bytes on Python 3, and the whiteboard,
on the other hand, requires a string.

The default encodings will be used for encoding, and for encoding,
we want to be safe and use ascii which can be represented in the
various result formats.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There's a better maintained trinity test at:

   https://github.com/avocado-framework-tests/avocado-misc-tests/blob/master/fuzz/trinity.py

Let's drop this outdated example.

Reference: https://github.com/avocado-framework/avocado/issues/2285
Reference: https://trello.com/c/uCcng8dN/1168-drop-trinity-example-testSigned-off-by: NCleber Rosa <crosa@redhat.com>

examples/tests/gdbtest.py already imports the portable range() from
the six library, but was not using it.  Let's fix this.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

On Python 2, input() has a different behavior, including an evaluation
of the obtained string.  On Python 3, it just returns the stripped
string.

Let's hand off the difference handling to six.

PS: we won't use use six.moves.input() on the failtest_ugly.py file,
given that it's intended to be a bad example.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

pylint catches what the (on purpose) nasty example test does with:
"Raising a new style class which doesn't inherit from BaseException".

But, since that's on purpose, let's silent pylint instead.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is an attempt to catch most (if not all) usages of open
that do not follow the context manager pattern.

Sometimes, for better readability, our own `genio` library is
used.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The default parameters mechanism proved to be confusing, overlapping
and thus was scheduled to be removed.  This implements the removal
of the default_params API.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Double free errors are silent in Fedora 27, making the test
doublefree.py to fail.

Using `MALLOC_CHECK_=1` is guaranteed to expose the double free
errors, regardless the OS double free handling. This patch updates
the doublefree.py test to use the `MALLOC_CHECK_=1` approach.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The documentation and examples on the output check feature are
outdated or innaccurate.

For instance, the synctest.py example would produce different output
than recorded in the accompanying reference files because the API
parameters were not being respected.
Signed-off-by: NCleber Rosa <crosa@redhat.com>