There are a few leftover references to the old Plugin classes in
the functional tests. Let's remove them.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

If a test has local imports (a fairly common use case),
we have to append the base test dir to sys.path. Otherwise,
we end up having a horrible traceback as a result:

Exception loading test
Traceback (most recent call last):
  File "/home/lmr/Code/avocado.lmr/avocado/core/runner.py", line 200, in _run_test
    instance = loader.load_test(test_factory)
  File "/home/lmr/Code/avocado.lmr/avocado/core/loader.py", line 262, in load_test
    test_module = imp.load_module(module_name, f, p, d)
  File "scylla_longevity.py", line 5, in <module>
    from sdcm.tester import ScyllaClusterTester
ImportError: No module named sdcm.tester

Which is clearly not wanted. We had this problem before
and now we have a regression. It's time for a functional
test to make sure we don't regress again.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It was noticed that the Python standard library provides os.ttyname(),
and, while it only returns the Pseudo Terminal file path when given
the slave file descriptor, it looks to be sufficient for most, if not
all, use cases.

This reverts commit 003bfaed.

CC: Lukáš Doktor <ldoktor@redhat.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Based on previous experiences (GDB terminal redirection attempts),
it's common for applications that need to allocate a pseudo to have
the name of the allocated file. This is, unfortunately, something that
the Python standard library doesn't give for free.

This simple utility module wraps and extends the Python standard
library `pty` module, providing a `openpty` function that, besides
allocating the pseudo terminal, also returns its name.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Including the old base plugin class definition.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Yes, too many plugin mentions, in this commit title, but that's what
it really is, so please bear with it.

Now that the plugin types have a finer granularity, let's show each
plugin type (based on the namespace) as a separate section.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@scylladb.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The Avocado command line application has "subcommands". So far,
to execute these commands, a special method, named "dispatch"
was set on the command line parser result object (aka Namespace)
pointing to the `run` method of the same class.

In truthfullness, the command line parser doesn't have too much
to do with these methods. Also, the `set_defaults` method of the
command line argument parser is not exactly intended for that use.
So, let's ask the subcommands subparser to formally save the chosen
subcommand, at well, `subcommand`, and nothing more.

Since the application knows about the command plugins (the ones that
inherit from CLICmdBase), and their interface now *require* a method
called `run`, let's use the extension entry-point name to locate the
underlying object and its `run` method.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

At this point, no plugin support exists in Avocado. This is in
preparation for the new plugin code to be cleanly introduced.

The plugins that play a role in the plugin architecture have
been removed, since they wouldn't be functional under the new
plugin management code. The ones that are add extra functionality
to Avocado have been kept, and will be ported to the new
architecture.

Also, most of the functional tests have been temporarily disabled.
The reason is that most of them run avocado, which depends on the
run command, which in turn, depends on the plugin archicture code.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This shows a more succint version of the help message (the so called
"usage" information). It's a certainly a matter of taste and change
of behavior, so this is marked as an RFC.

It also avoids a hackish injection of the print_help method as the
default dispatch method.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This fixes the recently introduced commit 5f7bafd0, running the skip
function instead of passing the function itself to `unittest.skipIf`.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `test_gendata` test of `OutputTest` runs the example test
`gendata.py`, but the `gendata.py` itself has a conditional in one of
the tests that make use of the PIL (aka Pillow package) library and
does nothing if it's missing.

PIL (Pillow) is highly dependent on C compilers and libraries to be
able to build itself. This makes systems it harder to run the full
Avocado self tests on more diverse (usually minimalist) systems.

So, let's make that functional test optional, if the libraries are not
present.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There are a couple of functional unittests that assume that a given
output was produced by the regular UI, that is, the Human output
format.

That is fine, since they're functional tests. But, some situations
lead to test errors, instead of pointing failures. Let's turn those
into explicit checks and fail properly instead of producing errors.

Note: These errors were visible when the human output was disabled, in
the plugin overhaul work, but the fixes still make sense independently
of that work.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

While going through the dependency list (requirements*.txt files) and
performing our self tests out of virtual environments, I noticed that
some tests are run outside the virtual environments.

The reason is that, even though the virtual environment is activated
for the test session (and say, `which python` gives `/venv/bin/python`),
we have hard coded `/usr/bin/python` in most places.

According to the some discussions on the virtualenv project itself[1],
a quick solution is to revert to the also common `/usr/bin/env python`
way of pointing to the Python interpreter.

[1] - https://github.com/pypa/virtualenv/issues/124Signed-off-by: NCleber Rosa <crosa@redhat.com>

We used to support subtests filter to load only some of the avocado
tests, but the functionality is broken. This patch brings this
functionality back. It uses "re.search" so people can use partial
matches. Additionally it includes selftests to prevent future breakages.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch adds unittest for loading simple tests with arguments.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

shlex.split() can fail on some strange commands (eg. on an unfinished
quotation). This patch fallbacks to simple string.split() in such cases,
which might return incorrect results, but is safer to use. Although it
logs warning about this notifying the user about possible issue.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

It has been reported that avocado crashes when a malformed command
is attempted to be executed, even when using `shell=True`.

The reason is that `shlex.split` will raise `ValueError` on those
commands. Since improving `shlex.split` may be too complex, and
possibly not the best place to "fix" this, let's just assume safely
that this command should be run on GDB.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Since we define the meaning of each exit code, we should be using that
ourselves.

Functional tests have been coded using literal values, and there may
be a good reason for that: proving that the API stands.  But there's a
big issue: we have not declared those codes as some kind of an
external API and guarantee stability. Also the subsequent changes in
this series actually change the meaning and values of those.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It's prone to mistakes maintain two separate ways to query for tests
(running them and displaying available ones).

This patch implements --dry-run, which follows the full "run" process,
only instead of running the test it skips them.

Additionally the --dry-run uses some tweaks to tell them apart from
normal executions, namely:

    1. No sysinfo gathering
    2. Unique id is 0000000000000000000000000000000000000000
    3. Unless user provides custom log location, fresh tmpdir is used
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Improve the `AvocadoParams.iteritems()` to return unique items only
in the same fashion `multiplexer` does it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

In preparation for the large plugin refactor, let's simplify the code ASAP
and drop related (custom) functionality that is going to be replaced.

Standard setuptools entrypoints is going to provide better capability then
our custom code that supports external 'avocado_*.py'module loading.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

From the Python `tempfile.mkstemp()` docs:

The default directory is chosen from a platform-dependent list, but
the user of the application can control the directory location by
setting the TMPDIR, TEMP or TMP environment variables.

So, let's let `tempfile.mkstemp()` itself choose the temp dir, based
on what the use may have set in environment variables.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is an automated mass import cleanup across all
avocado source files:

1) Imports follow the order:
 * Standard library imports
 * Non standard library external imports
 * Internal imports
 All separated by a single line
2) One line between imports and the rest of the code
3) Remove unused imports in the process
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Since the test loader now attempts to find Avocado instrumented tests
by using a parser and doesn't really load/execute test files, it can't
tell if a given class inherits from avocado.Test.

For those cases, a developer may signal, using a docstring tag, that
either:

 * A test class that doesn't look like an Avocado test class is indeed
   one ("🥑 enable")
 * A test class that looks like an Avocado test class should not be
   treated as such ("🥑 disable")

Also included is a a simple optimization by breaking earlier from the
parser loop.

Changes from v1:
   * Added two more tests and suggestions by Lukáš Doktor

Changes from v0:
   * Added fix reported by Lukáš Doktor
Signed-off-by: NCleber Rosa <crosa@redhat.com>
CC: Lukáš Doktor <ldoktor@redhat.com>

Inside test classes there may docstrings which may contain tags that
give special meaning. The tags are defined by "🥑 <value>" in a
single line. For now the only two "official" Avocado tags values
exist:

 * enable: the class is an Avocado INSTRUMENTED test, even if it
   doesn't "look like" one.
 * disable: the class is *NOT* an Avocado INSTRUMENTED test, even if
   looks like one.

Changes from v0:
 * Use unittest2 on Python 2.6
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Sphinx will attempt to fetch the Python objects inventory during the build
process. If for some reason, this test is being run on a machine that can
not access that address simply because of network restrictions (or the
developer may simply be on a plane) then it's better to SKIP the test than
to give a false positive.

Changes from v0:
 * Use unittest2 on Python 2.6
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This patch doesn't change any functionality, it only renames the
inner_runner to external_runner (the plugin is called simply external)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>