The newly released pylint version 2.2.0, changed its default behavior
(enabled) the following checks:

 * :unnecessary-pass (W0107): *Unnecessary pass statement*
   Used when a "pass" statement that can be avoided is encountered.

 * :unused-import (W0611): *Unused %s*
   Used when an imported module or variable is not used.

Let's apply fixes to those checks, and at the same time, pin the
pylint versions so that our CI environment is more stable (code that
test our code -- pylint -- doesn't change, while our code changes).

For Python 2, the latest release was 1.9.3, so let's use that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently, the only way to pass parameters to Avocado is by using the
variants layer, that is, one of the existing varianter plugin
implementations.  This was based on earlier design decisions, that
we're trying to fix without breaking expectation and compatibility.

While it'd makes sense to support parameters created by the varianter
layer itself, together with non-varianter parameters, it seems that we
can start by adding support for one or the other being used at a time.

This commit does exactly that: add support for a method of passing
parameters via the command line for all tests in a job, without
having to resort to the varianter.  When the varianter is used,
it takes precedence and the core test parameter passing support
is ignored.

This should solve a number of use cases that require one-time
executions of tests with different parameters, especially
executions during test development and debugging.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

This pylint checks for possible redefining outer names. Some occurrences
was preserved with disable flag because they would broke some public
api, like method/functions names or parameters.
Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

The testIdentity changed here was using assertTrue with a constant
instead of a conditional. So, to make lint pass it was updated to a more
appropriate assert.
Signed-off-by: NCaio Carrara <ccarrara@redhat.com>

This example test fetches two different files from different
locations, and uses a data file (shipped *with* the test) on
one of them.

The goal is to demonstrate a complete example of files you
may want to distribute with your test, and the assets you
may want to fetch from remote locations.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

AVOCADO_TEST_DATADIR is deprecated.
Removing them from the examples and selftests.
Signed-off-by: NNarasimhan V <sim@linux.vnet.ibm.com>

Moved original whiteboard example test to selftests and then
created a simpler example code.

Reference: https://trello.com/c/sG6J0VpwSigned-off-by: NWainer dos Santos Moschetta <wainersm@redhat.com>

After the deprecation, comes the removal.  Hopefully, "workdir" is now
being used in tests by now.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently Avocado crashes on py2 when the exception contains unicode.
Let's use the new "astring.to_text" method to safely get the exception
details.

As this exception is part of the results, this commit also fixes the
result plugins to cope with unicodes in them.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

When this test was initially written, the concept of a shared
directory available across tests did not exist.  Let's optimize the
execution by avoid compilation of the same binary on every single
test.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since the result of `genio.read_all_lines()` already is a list of all
lines (split by newlines).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

These warnings are described as:

 W0612: Used when a variable is defined but not used.
 W0622: Used when a variable or function override a built-in.

After a suggestion from Andrei Stepanov, we decided to immediately
enable these.

Rerefence: https://pagure.io/standard-test-roles/pull-request/144
Reference: https://trello.com/c/Ohh7Q6qj/1183-enable-w-in-travis-make-checkSigned-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NWilliam Tambe <william.tambe@amd.com>

The latest jobs have been failing because of the following issues:

    ************* Module raise
    E1130: 48,30: Raise.test: bad operand type for unary -: NoneType
    ************* Module selftests.functional.test_basic
    I1101:1191,24: PluginsXunitTest.run_and_check: Module 'lxml.etree'
    has not 'XMLSchema' member, but source is unavailable. Consider
    adding this module to extension-pkg-whitelist if you want to
    perform analysis based on run-time introspection of living
    objects.
    I1101:1191,40: PluginsXunitTest.run_and_check: Module 'lxml.etree'
    has not 'parse' member, but source is unavailable. Consider adding
    this module to extension-pkg-whitelist if you want to perform
    analysis based on run-time introspection of living objects.
    I1101:1193,43: PluginsXunitTest.run_and_check: Module 'lxml.etree'
    has not 'parse' member, but source is unavailable. Consider adding
    this module to extension-pkg-whitelist if you want to perform
    analysis based on run-time introspection of living objects.
    ************* Module selftests.unit.test_xunit
    I1101: 91,24: xUnitSucceedTest.test_add_success: Module
    'lxml.etree' has not 'XMLSchema' member, but source is
    unavailable. Consider adding this module to
    extension-pkg-whitelist if you want to perform analysis based on
    run-time introspection of living objects.
    I1101: 91,40: xUnitSucceedTest.test_add_success: Module
    'lxml.etree' has not 'parse' member, but source is
    unavailable. Consider adding this module to
    extension-pkg-whitelist if you want to perform analysis based on
    run-time introspection of living objects.
    I1101: 92,43: xUnitSucceedTest.test_add_success: Module
    'lxml.etree' has not 'parse' member, but source is
    unavailable. Consider adding this module to
    extension-pkg-whitelist if you want to perform analysis based on
    run-time introspection of living objects.

We can't whitelist the `lxml.etree` module at this time (inspekt doesn't
have such an option) and the raise failure is a false positive.  Let's
ignore those issues then.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

After much discussion, it was decided that `srcdir` must go, and
`workdir` should be used instead.  At this time, let's deprecate
it, and in the near future, remove it.

The deprecation approach chosen is to flag tests that use `srcdir`
with warning, so users will hopefully take notice.

Reference: https://github.com/avocado-framework/avocado/issues/1924
Reference: https://trello.com/c/U1aaJjPJ/1158-deprecate-testsrcdir
Reference: https://trello.com/c/dCcxhJc2/1267-remove-testsrcdirSigned-off-by: NCleber Rosa <crosa@redhat.com>

The skip() method has been deprecated for quite some time.  Let's
remove it for good and have users using the decorators and cancel()
instead.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The API used to get the file, by name, is `get_data()`, which looks
for that file in one of the data directories that the class provides
(file-level, class-level, variant-level, etc).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

These parameters are not class instantiation parameters, which is
the indentend usage of those docstring notations.  But, even worse,
they don't exist in these example tests at all.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This brings a change in behaviour, in which the stdout/stderr of the
executed process will now be of bytes type, instead of a string type.

Two new attributes, which are implemented as properties, have been
added to the CmdResult class, `stdout_text` and `stderr_text`.  Those
are convenience methods that will return the same content that is in
`stdout` and `stderr`, reespectively, but decoded on the fly[1].

With regards to encoding, if one is not provided, the result of
`sys.getdefaultencoding()` will be used ("utf-8" for Python 3 and
"ascii" for Python 2).

Applications and/or tests using the APIs that return a CmdResult
should, to the best of my knowledge, set a default encoding themselves
so a stable behavior across Python versions.  But that if left to
users of this API.

A different tradeoff/design decision has to do with the tests modified
here.  One option is to have "text" (as in sequences of human readable
glyphs) as being of Python type "str".  On Python 2, "str" can be
compared to "bytes" because a conversion will happen on demand.  That
is, the following is fine on Python 2:

   >>> result = process.run("command")
   >>> "expected" in process.stdout

Where `expected` is of type "str" and `process.stdout` is of type
"bytes".  This is not true of Python 3, so either the types must match
or a conversion must be done explicitly.  The solutions to that are:

1) have these "text" as (of type) "bytes" in the source code itself,
   and avoid the conversion whenever possible
2) have "strings" in the source code itself, and use the conversion
   provided by `CmdResult.stdout_text` and `CmdResult.stderr_text`.

The approach chosen here is to avoid conversion if possible, that is,
use "byte" types, given the fact that the source code encoding is by
default 'ascii' and most of the "text" dealt with here can be
represented in 'ascii' too.  This is equivalent of doing:

   result = process.run("command")
   b"expected" in process.stdout
   "errors: %s" % 0 in process.stderr_text

[1] The obvious alternative, instead of decoding these on the fly
    would be to have multiple copies of the "same" data.  This assumes
    that binary data produced on the stdout/stderr will usually be
    larger than textual data.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

base64.encodestring() requires bytes on Python 3, and the whiteboard,
on the other hand, requires a string.

The default encodings will be used for encoding, and for encoding,
we want to be safe and use ascii which can be represented in the
various result formats.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There's a better maintained trinity test at:

   https://github.com/avocado-framework-tests/avocado-misc-tests/blob/master/fuzz/trinity.py

Let's drop this outdated example.

Reference: https://github.com/avocado-framework/avocado/issues/2285
Reference: https://trello.com/c/uCcng8dN/1168-drop-trinity-example-testSigned-off-by: NCleber Rosa <crosa@redhat.com>

examples/tests/gdbtest.py already imports the portable range() from
the six library, but was not using it.  Let's fix this.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

On Python 2, input() has a different behavior, including an evaluation
of the obtained string.  On Python 3, it just returns the stripped
string.

Let's hand off the difference handling to six.

PS: we won't use use six.moves.input() on the failtest_ugly.py file,
given that it's intended to be a bad example.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

pylint catches what the (on purpose) nasty example test does with:
"Raising a new style class which doesn't inherit from BaseException".

But, since that's on purpose, let's silent pylint instead.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is an attempt to catch most (if not all) usages of open
that do not follow the context manager pattern.

Sometimes, for better readability, our own `genio` library is
used.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The default parameters mechanism proved to be confusing, overlapping
and thus was scheduled to be removed.  This implements the removal
of the default_params API.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Double free errors are silent in Fedora 27, making the test
doublefree.py to fail.

Using `MALLOC_CHECK_=1` is guaranteed to expose the double free
errors, regardless the OS double free handling. This patch updates
the doublefree.py test to use the `MALLOC_CHECK_=1` approach.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The documentation and examples on the output check feature are
outdated or innaccurate.

For instance, the synctest.py example would produce different output
than recorded in the accompanying reference files because the API
parameters were not being respected.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This change deprecates the use of the of the `datadir` property,
and switches to `get_data()` instead.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The whiteboard test is an example test, and, since, we're about to
deprecate the datadir attribute, let's remote this call to chdir.

If we believe that's necessary to stress this part of the API, let's
write a (non-example) test for that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Basically switch to use the alias provided by six, with the Python
3 name.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Most of the LazyProperty variables in Test are actually always read on
"get_state" and other occasions so it does not make sense to set it
Lazily.

Also protect the remaining lazily initialized property from overriding
by using @property instead of our custom LazyProperty, which is not
really a property.

This commit requires some adjustments to nasty example tests which used
to override self.srcdir, which is not allowed (and restricted since this
commit).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Each test executed in Avocado gets its own process.  But, there are a
few peculiarities that test writers may have to deal with, so let's
document those.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

self.skip() is to be deprecated. This is the first step, adding a
message to the exception raised by the self.skip() and adjusting the
examples tests accordingly.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

We have received many requests to support skip tests from inside the
test method. As we can do that without breaking our own concepts, we
decided to introduce the CANCEL status and the corresponding
self.cancel() method to the Test class.

Reference: https://trello.com/c/viBJIEwISigned-off-by: NAmador Pahim <apahim@redhat.com>

The message obtained trying to execute doublefree will
be different when using other C libraries. Let's expand
the list of accepted outputs by adding the macOS C library
message.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>