AFAICT, most users of find_free_port() don't really care about the
port number being the first in the range, and to be honest, most users
I've seen don't even give a range.

So, the result when using the first port in the range is a high change
of collision, with no real benefit.  Let's change the default policy,
using a random port allocation.

PS: this is motivated by real world collisions I've experienced when
running tests in parallel, *and* when running other loads in the same
machine along tests.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NJan Richter <jarichte@redhat.com>

I clearly missed that on 71d915dd, as there's no point of using
a locally defined "basedir" and the one from the sefltests module.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NJan Richter <jarichte@redhat.com>

The data type expected to be given to each yield variant contains the
variant id, the "variant" itself, which should be a list of
TreeNode-like instances, and finally the applicable default search
paths.

The "variant" itself, has to be a list, given that on the very first
varianter implementation, it's supposed to contain one item for each
branch of the tree.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

This shell script can not actually be used with non-bash sheels.  This
only manifests on some environments where /bin/sh is not bash, and the
exact outcome is similar to:

   * Checking for avocado.core imports from examples/tests: selftests/modules-boundaries.sh: 9: selftests/modules-boundaries.sh: RESULT: not found
   0
   * Checking for non-relative avocado imports from avocado/core: selftests/modules-boundaries.sh: 20: selftests/modules-boundaries.sh: RESULT: not found
   0
   * Checking for avocado imports from avocado/utils: selftests/modules-boundaries.sh: 31: selftests/modules-boundaries.sh: RESULT: not found
   0
   PASS: no module boundary infringement(s) found
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Rules such as "check" on our Makefile make sure that no temporary
directory created by Avocado is left behind, but, it's very hard to
tell who created a rogue directory.

This brings improvements to the naming of the temporary directory, so
that it's trivial to tell which test created in the first place.

Additionally, it improves the temporary directory creation, usually by
making use of a tearDown() test phase, instead of other more fragile
methods.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Merge branch 'configure_net' of https://github.com/vaishnavibhat/avocado into vaishnavibhat-configure_net
Signed-off-by: NPraveen K Pandey <praveen@linux.vnet.ibm.com>

Resending the patch after exception handling and review comments.
This patch generates network config file when IP address, interface
name and netmask is provided.
Signed-off-by: NVaishnavi Bhat <vaishnavi@linux.vnet.ibm.com>

Merge branch 'fix-no-html-output' of https://github.com/pevogam/avocado into pevogam-fix-no-html-output
Signed-off-by: NPraveen K Pandey <praveen@linux.vnet.ibm.com>

Current 'None' evaluates as string where 'None' is not None.
Signed-off-by: NPlamen Dimitrov <pdimitrov@pevogam.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

When this test used to be a function, it was OK to use "return",
but now that it's a script (and the code is not wrapped in a
function) that is invalid shell syntax.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When CI systems work on branches for testing, they usually create
merge commits.  And those won't be signed, specially not from the
author of the other commits.

Let's skip the merge commits to work around that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

For a long time we have resorted to a hack, let's call it "all commits
hack", to test all commits in a branch on Travis-CI.  While it clear
had a positive side, it also had its annoyances.

Now, more and more often, we're faced with the obligation of breaking
PRs in smaller pieces just because of the limitations that the free
tier of Travis-CI imposes (for instance max job time is 50 minutes).

For instance, as this is written, the status of the master branch
is broken, because of the max job time.  And there's no way to
waive it or tweak it.

Also, the hack caused some really interesting problems, and caused
a lot of time wasted, for instance, because it masks the merge commit
that Travis-CI creates, and consequently, makes the signedoff-check.sh
fail on the real HEAD of the merged branch.  Basically, the "all
commits hack" is a requirement to currently have signedoff-check.sh
passing on Travis-CI.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NPraveen K Pandey <praveen@linux.vnet.ibm.com>

The current Python unittest discovery is too permissive, in the sense
that it will match any "test*" method on any class, wether it inherits
from the TestCase class of the unittest module or not.

This new implementation is more strict, but a lot more capable too.
This makes the Python unittest detection very similar (in code and
capabilities) to the Avocado instrumented test detection.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

So that the specific Avocado checks are kept in its own function, and
other types of checks for other types of tests can be implemented more
cleanly.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The disabled variable should contain a set of the disabled tests, and
a list was being defined as a fallback if `iter_classes()` doesn't
yield anything.

Let's also make the naming of the variable uniform.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

When looking for tests, the base class and the module where it
comes from, is of uttermost importance.  But, they have so far
been implicitly defined.

Let's make them explicit, which makes improves readability and
will help to make the utility functions reusable.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When a module with more than one level is imported, such as:

   import unittest.mock

All levels of the module will be available, that is, it's possible to
write code that makes use of whathever is in `unittest`, for instance,
`unittest.TestCase`.

Let's keep track of all intermediary levels imported to improve the
behavior of PythonModule.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The real goal of the test_imports attribute is to keep track of
classes that were imported.  The naming made sense given that original
this class (PythonModule) was called AvocadoModule and was hardcoded
to look for Test classes from the avocado module.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

And those that don't.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

A base class exists in this module, and number of the test classes
inherit from it, from the single supposed benefit of sharing a
(static) method.

Let's flatten the inheritance, and make the static method a utility
function, because, if not for anything else, It improves code
readability.

Note: this was actually observed while working on the safeloader
Python unittest detection improvements, meaning that in theory the
safeloader improvements would make this unnecessary for the actual
loading support.  Still, I believe there are benefits (readability,
maintenance) with these changes.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

On `_examine_class()`, the original assigment of the avocado tags to
the `cl_tags` variables is not necessary because when it's used in the
first occurrence, a `continue` statement follows the assigment, so
it's never reused.  On the other hand, it may be assigned without ever
being used.

On `find_avocado_tests()`, it's just an opmization because `cl_tags`
was only being used once in the function, and at the same time matches
the style used in the previous function.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

By renaming it to PythonModule, and making the module and klass names
configurable, it should be reusable for other types of Python modules
and classes, including unittest.TestCase.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

By adding internal methods to deal with import from and plain
import statements.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Just like the next block is doing.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The module path only get's updated on the AvocadoModule constructor,
so after that, it won't change and the `path` variable is thus
unnecessary.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Adding info about the missing `is_avocado` parameter, and remove
docstring for non existing parameter (most probably a leftover from
_examine_class()).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NJan Richter <jarichte@redhat.com>

Merge branch 'selftests_extract_from_checkall' of https://github.com/clebergnu/avocado into clebergnu-selftests_extract_from_checkall

Selftests: extract from checkall

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>