Autopep8 style cleanup + alphabetic import + import cleanup in
avocado.core.{loader,test} and selftests.functional.test_basic.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently InnerRunner lives within the FileLoader plugin and overrides
it's function in case inner_runner is supplied. This patch extracts
it and creates separate plugin for it. This plugin is still part of the
core and is registered automatically during the runtime.

On --inner-runner usage it overrides the --loader order and registers
only itself as by specifying --inner-runner people asks to use it.

When inner_runner is specified in --loader, it's processed accordingly
to the priority. When one uses only "inner_runner" without extra_params,
it never matches any tests (to keep the behavior).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The usage usage of incorrect test-types (types which can't be executed
like missing file, broken symlink ...) was not found useful. This
patch only allows "good" test types to be listed and possible to use.

Additionally the test type now requires to specify loader and type to
avoid possible clashes (and to emphasize the relation between loader
and test type).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The "avocado.core.output" spawns own instance of term_support on the
module level, let's avoid creating new one when not necessary.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The safe(r) loader does not load the classes, thus avocado tests are
not represented by class, but by string. This patch addjusts the loader
to support string-like expressions.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

InnerRunner tests are inherited from SimpleTest, thus "--loaders
SIMPLE:/bin/echo" detects inner_runner as subclass and proceeds.
This patch adds exclude to avoid InnerRunner to be detected as
SimpleTest.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

docs: Add a readthedocs specific requirements file

Add a file specifically for our automated documentation
builds (on readthedocs.org).
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

.travis.yml: Skip installing packages using apt

Those packages are not being installed anyway, and do not
interfere with general CI testing.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Update docs and testplan

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Now we don't need to explicitly specify --vm-hostname.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

avocado.core.data_dir: Make logdir creation safer [v3]

When one execute 2 jobs with first 7 chars of unique id at the same
time, they end-up in the same logdir.

This patch makes the creation safer. First it creates the main
directory, then it tries to create the directory with 7 chars of the
unique id. On failure it adds another char until it finds empty dir.

In case you spawn 2 jobs with the same id at the same time, it adds
a number starting with 0.

If you spawn more tham 1000 jobs at the same time with the exact same
id, it fails with IOError.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado.core.remote: Improbe extra_params usage

avocado.core.tree: Improve error message on missing !include file [v1]

extra_params compounds of multiple items. Let's use " ".join() instead
of manually adding spaces.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

When a multiplex file includes a missing file, avocado only reports
"there is a missing file" without any explanations whatsoever. This
patch changes it to a ValueError with the original file and the
missing one.

Additionally it adds missing Exception to try/except, which was
already possible on certain errors, to avoid tracebacks.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

docs: remove static modules.png file and its reference in the docs

The modules.png file is a static file that would need manual update
with almost every single commit. This is rather impractical and
doesn't add that much value to the regular user.

If a developer wants to take a look at the dependency graph, a
Makefile target still exists that will give a user a fresh graph.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Safe(r) test loader

And while at it, also include the source (SVG) version of the same
diagram.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

With the changes that now do not load the test at discovery time,
the test class is a string with the class name, and not a class
itself.

Therefore, this discovery test is only going to check for the test
class name. Other test may be added to do the real module load and
test class instantiation.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently, the test loader will load/execute the Python test files at
discovery time. This is not desirable because, among other things,
it's unsafe to execute code that was not explicitly requested to be
executed.

The new approach is to use the `ast` module and parse, but not
execute, the Python source code files, and attempt to use some
heuristics to detect an Avocado INSTRUMENTED test.

Since the current loader is too tighthly coupled to the idea of having
test classes and parameters (which implies having loaded the code) as
the building blocks of tests, this new approach gives the system class
names only. Also, since the code is not loaded, the path to the source
code file is given inside the test parameters (and later pulled
out). A more complete solution would be to redesign the Loader so that
it gives test information only at discovery time for *all* test types.

Since there's another major work going on in the loader at this time,
I decided to keep things simple(r) and just deal with the string based
"classes" when necessary.

Also included (transparently) is the support for multiple classes with
multiple tests inside a single Python source code file.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since loading/executing tests are not desirable before actually
running them, these tests must be dropped.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The BUGGY test concept relies on loading/executing the Python file
containing the tests and failing to do so. Since Avocado won't
load/execute Python test files anymore, it's not possible or desirable
to keep the BUGGY tests around.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There's no real reason why a single Python file can not hold multiple
test classes, each one with its own tests.

This test adds a simple Python test file with two classes, each one
with its own test, and expects the loader to find two instrumented
tests.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The test loader should only load (import, thus execute) Python test
modules at test execution time. During, for example, test discovery
and listing time, the Python modules should never be imported (thus
executed).

This introduces a test called "sleep eleven" (because "ten" is boring)
that "accidentally" has a `time.sleep(11)` at the global scope. This
test is listed by executing `avocado list -V sleepeleven.py` and is
given 3 seconds to finish. If the loader actually loads/executes the
Python code, it will fail because it will take at least a little bit
more then 11 seconds.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Currently we can only specify the loader plugins order via settings.
This patch adds "--loaders" argument and improves the granularity.
Now it's possible to set either @plugin_name, or TEST_TYPE, where
TEST_TYPE is the mapped name of the plugin (eg. SIMPLE, VT, ...)

There is one special name "DEFAULT", which injects all unspecified
loaders in the position of the "DEFAULT" keyword. When "DEFAULT" is
not specified, not-listed plugins/test_types are not evaluated.
(therefor it's possible to restrict some test loaders or only some
test types).

As the usecase changed slightly, the settings option was renamed
from plugins.loader_plugins_priority to plugins.loaders.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The mapping doesn't use `self` method and we might need to call it
before the plugin is initialized. Let's require it to be defined as
`staticmethod`.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The concept of inner runner is a modification of the file-loader. Lets
move the code there.

I also added --inner-runner to "avocado list". It doesn't handle
exceptions nicely as I did not want to spend too much time on it (as
we want to deprecate the list soon).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

With this patch loader can mark options as invalid (with LoaderError)
and avocado passes this message to the user. This should be cleaner way
than "sys.exit".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Remote/VM timeouts: introduce timeout command line options

avocado.core.data_dir: Suppress AttributeError when pickling _TmpDirT…

If the tracker is pickled/unpickled (common in avocado-vt,
for example), we might call garbage collection and the
temporary dir might be long gone, and the original os
and shutil imports as well. So let's handle this case
properly.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>