The test can be asked to report its state, or it can do it itself.
Given that the test knows when it enters each phase, let's ask it to
report it.

Because the *state* is now reported, it's possible for a test to
finish (for instance by killing itself) without setting a valid (user
visible) status, such as "PASS", "FAIL", etc.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The avocado test runner mostly "flies blind" when it comes to the
individual test phases, that is, it doesn't know if a test is
currently being initialized, running its setup, running the test
method itself or its teardown.

With this, the runner gets access to that information, and can act
differently based on this knowledge.  One use case is to allow for
different timeouts on different test phases.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is a really bad example of a test, in the sense that it requires
a huge library just to draw an image.

Let's drop it, given that the remaining example works just as well.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

In my experience, a very small percentage of the changes actually
introduce Sphinx based documentation.  Say, a feature may be composed
of 10 patches, and at the end of the series, the feature is
documented.

Requiring the documentation to be build all the time takes a toll on
both local and remote systems CI systems.  Let's make the testing
slimmer, and rely on readthedocs.org to tell if any issues are
introduced.

Instead, let's deal with eventual issues before a release is cut.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And use the avocado module entry point, instead of any specific
script.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The former command line option is an old version, and the "--show"
form is preferred.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is actually an ancient option, and the preferred syntax has been
"--show=none" for a while.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

These utilities are useful for expected changes in any file on a
tested system. More info is provided within the module docstrings.
Signed-off-by: NPlamen Dimitrov <pdimitrov@pevogam.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And test that the interface (only) classes can not be used directly.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

This drainer (partial) implementation reads data from a file
descriptor.  Users should build on top of this class to come up with a
full implementation (as exemplified in the accompanying tests).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is a utility module that hopes to make it easier to drain and
process data that is generated concurrently with other code.

One example is data that is produced by the test, and needs to be
kept/processed/logged while the test logic is running.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Avocado's restclient tool, and also the avocado-server related tool
avocado-journal-replay, are features that have not received any
maintenance for a long time.  The reason is because avocado-server
itself hasn't been maintained.

One of the post 69.0 goals was to make Avocado slimmer, and try to make
its core features depend on nothing but Python, so let's drop this
feature for now.  If avocado-server resurrects, it can always be
brought back from version control.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The later is a pure Python library, designed for the specific purpose
of validating schema, while the former requires a compiler tool chain
to be installed in a number of scenarios.

Let's also pin the version of its direct depency, elementpath.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This shell script can not actually be used with non-bash sheels.  This
only manifests on some environments where /bin/sh is not bash, and the
exact outcome is similar to:

   * Checking for avocado.core imports from examples/tests: selftests/modules-boundaries.sh: 9: selftests/modules-boundaries.sh: RESULT: not found
   0
   * Checking for non-relative avocado imports from avocado/core: selftests/modules-boundaries.sh: 20: selftests/modules-boundaries.sh: RESULT: not found
   0
   * Checking for avocado imports from avocado/utils: selftests/modules-boundaries.sh: 31: selftests/modules-boundaries.sh: RESULT: not found
   0
   PASS: no module boundary infringement(s) found
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Rules such as "check" on our Makefile make sure that no temporary
directory created by Avocado is left behind, but, it's very hard to
tell who created a rogue directory.

This brings improvements to the naming of the temporary directory, so
that it's trivial to tell which test created in the first place.

Additionally, it improves the temporary directory creation, usually by
making use of a tearDown() test phase, instead of other more fragile
methods.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When this test used to be a function, it was OK to use "return",
but now that it's a script (and the code is not wrapped in a
function) that is invalid shell syntax.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When CI systems work on branches for testing, they usually create
merge commits.  And those won't be signed, specially not from the
author of the other commits.

Let's skip the merge commits to work around that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The current Python unittest discovery is too permissive, in the sense
that it will match any "test*" method on any class, wether it inherits
from the TestCase class of the unittest module or not.

This new implementation is more strict, but a lot more capable too.
This makes the Python unittest detection very similar (in code and
capabilities) to the Avocado instrumented test detection.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When a module with more than one level is imported, such as:

   import unittest.mock

All levels of the module will be available, that is, it's possible to
write code that makes use of whathever is in `unittest`, for instance,
`unittest.TestCase`.

Let's keep track of all intermediary levels imported to improve the
behavior of PythonModule.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The real goal of the test_imports attribute is to keep track of
classes that were imported.  The naming made sense given that original
this class (PythonModule) was called AvocadoModule and was hardcoded
to look for Test classes from the avocado module.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And those that don't.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

A base class exists in this module, and number of the test classes
inherit from it, from the single supposed benefit of sharing a
(static) method.

Let's flatten the inheritance, and make the static method a utility
function, because, if not for anything else, It improves code
readability.

Note: this was actually observed while working on the safeloader
Python unittest detection improvements, meaning that in theory the
safeloader improvements would make this unnecessary for the actual
loading support.  Still, I believe there are benefits (readability,
maintenance) with these changes.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

By renaming it to PythonModule, and making the module and klass names
configurable, it should be reusable for other types of Python modules
and classes, including unittest.TestCase.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The network utility used to (attempt to) get a free port defaults to
picking sequential ports.  But, that's not a requirement of this test.
When running tests in parallel, the same port may get allocated when a
sequential approach is used, and that is greatly minimized when random
ports are requested.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Instead of one large test method, let's have one for each.  Also,
let's define the base_logdir from the calling side, as with that
we can be sure that the test won't do any funky business, and
the tearDown() won't be trying to recursively remove what it
shouldn't.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

On all tested Python 3 versions, the SimpleQueue is available from
the multiprocessing module, so let's drop the conditional import.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

A "from <module> import <symbol>" statement will not always have
a module name set, one example is:

   from . import BASEDIR

In these situations, the statement's module attribute will be None.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Which are only necessary under Python 2, and thus, not needed
anymore.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The previous working dropping Python 2, removed the explicit
requirement of the six module, but because stevedore depends
on it, it would still be installed.

Now, every trace of explicit six usage has been removed.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There's no need to keep the special conditions for Python 2, now that
it's not supported anymore.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This test actually works fine under Python 3, the skip was not
properly lifted before.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When the recent_mock() utility was introduced, I may have missed the
fact that it would be possible to mock builtins.open().  Or, it may
be that was only possible in Python 3.

Anyway, now that we only support Python 3, let's just mock
builtins.open() and drop this custom decorator.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And the compatiblity helper six.  With Python 2 gone, a number of
package changes are also introduced, so that the packages are always
named python3-<suffix>.

Also, because it would be confusing not having an "avocado" script,
and to conform with the system wide change introduced by distros that
now offer Python 3 by default, the scripts are no longer called
avocado-3 (or avocado-3.x), but simply "avocado".
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And by removing the data, we won't have any test to be run, so
we're also removing the test itself.
Signed-off-by: NCleber Rosa <crosa@redhat.com>