avocado.job: Refactor how multiplexation happens [v3]

This pull request changes the way multiplexation is used.

1. tests are discovered by test_loader.discover()
2. multiplexer is initialized
3. suite is executed with initialized multiplexer class
4. each test works as template and iterates over multiplexer variants

Additionaly support for multiplex_files overriding was removed as I
haven't found any users of this API.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch is initial support for people using custom bash scripts
with avocado. Tests should use:

    PATH=$(avocado "exec-path"):$PATH

and then they can utilize the helpers.

This version contain functions to write to Test.log the same way it's
possible from python including failing the test with TestWarn in case
avocado_warn was used.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

No actual change, only refactor code to use single function to get all
paths.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado.job: Print test results dir even on --show-job-log

We were printing the results dir only when standalone test
mode was used. Let's extend that to --show-job-log as well
by moving the message to the end of the job execution
method.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

When no tag is supplied get_tagged_name() generates unique name. On the
other hand when tag is supplied, it silently reuses existing path. This
patch adds check for these cases and use $name.$tag.$number in case
$name.$tag exists.

This should avoid clashes when executing multiple tests with the same
name with multiplexation.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

No actual change, only merge 2 same branches together.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Makefile: have a better target/recipe with deps for man pages

The `man` target is not currently using make's dependency system, let's
fix that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Print Job ID in job log

The job ID is an important information that should've
been printed to our job log. This patch fixes it and
also adds a functional verification test.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

It is more natural [1] that output of --show-job-log goes to
stdout than stderr. Let's change that behavior and update
unittests accordingly.

[1] Natural in this sense is defined by the user's expectation -
    when setting avocado to --show-job-log, you expect that the
    job log *is* in the application's stdout.

Changes from v1:
 * Python 2.6 constructor param for StreamHandler is strm,
   that was changed in 2.7 to be stream. Let's keep compatibility
   by using positional arguments.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Turns out that depending on guest config options, sometimes
the paginator kicks in and won't release the stdout until
a human comes in and presses 'q'. This is very, very bad for
automated procedures such as the one used by the remote plugin.

So add --disable-paginator in both plugins that use that
functionality, so they can be used in automated scripts.
Afterwards, we'll use that option in the remote plugin.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Under .el6, we depend on a series of EPEL available backports,
so we have to define the dependencies conditionally. The
changes here make avocado to finally build on RHEL6/CentOS6.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add a new python 2.6 target so that we can make sure
avocado works on older distros (our specific focus here
is RHEL 6, but of course, we expect the work on RHEL 6
to be useful to other distros sticking with older versions
of the python runtime).

In order to do that, we added a new 2.6 related requirements
file, and changed the requirements a bit so that it all
works under the particular Travis CI environment.

Changes from v1:
 * Per ldoktor's suggestion, use conditional install of backports
   using $TRAVIS_PYTHON_VERSION
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Python 2.6's os.path.relpath function returns some funny
results, such as:

In [3]: os.path.relpath('/tmp', '/')
Out[3]: '../tmp'

So let's skip relpath entirely.

Changes from v1:
 * Per ldoktor's suggestion use lstrip('/') instead of
   relpath workarounds.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

We can't install libvirt-python using pip since the
VM used by Travis runs Ubuntu 12.04 LTS (libvirt headers
are too old), so we have to settle for not requiring
all plugins (VM) to be available. It will still be tested
on 2.7, though.

Changes from v1:
 * Per ldoktor's suggestion, change check of exact python
   version being 2.7.x to python version >= 2.7.0.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

This one uses --vm, which is redundant and also uses
the vm functionality, one thing we still can't do in
Travis.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Instead of adding version at the class init time, add it
as one of the program's arguments. This will get rid of
a deprecation message while executing on python 2.6.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

EPEL ships with pystache 0.3.5, that has different API.
Let's make a resonable effort that the report works
there as well.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Use the logutils backport, available on pip and EPEL,
that allows us to keep using the dict logging config
method in python 2.6.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Dict comprehensions using curly braces is a
new feature, introduced in python 2.7. Let's
use equivalent 2.6 friendly constructs (list
comprehension and generating a list of tuples
with the dict() builtin).

Changes from v1:
 * Per ldoktor's suggestion, use list
   comprehensions and dict() instead of the
   more straight forward (and possibly slower)
   for loop.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

avocado.multiplexer: Remove unnecessary pylint ignore

It turns out that ignore not being there still makes
the pylint check to pass on both python 2.6 and 2.7.

The reason why this commit is necessary is that the
ignore type is unknown in pylint 1.4, causing the
2.7 related CI job to fail.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Release test plan: more tests, more precise descriptions and optimal test flow [v3]

This adds more tests to the release test plan, adds more precise descriptions
of the test and expected results.

Also, the test plan follows a sequence of steps that is more concise and eases
the manual testing process.

Changes from v2:
* Added Remote Machine HTML links test

Changes from v1:
* Added HTML sysinfo test
* Added HTML external links test
Signed-off-by: NCleber Rosa <crosa@redhat.com>

avocado/utils/io.py: rename to genio.py (meaning generic I/O)

The reason for this rename is that there's a name clash with Python's
standard library `io` module. This name clash manifest while performing
a simple Python validation of the avocado/utils/archive.py module:

$ python avocado/utils/archive.py
Traceback (most recent call last):
  File "archive.py", line 22, in <module>
    import zipfile
  File "/usr/lib64/python2.7/zipfile.py", line 501, in <module>
    class ZipExtFile(io.BufferedIOBase):
AttributeError: 'module' object has no attribute 'BufferedIOBase'

The explanation to that error is that Python itself adds the base
directory where the "executed" module file lives to the head of
the module search path list.

After some dicussion we realized that we should strive to not
name modules with the same names as Python's standard ones.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

loader: Ignore imported classes.

pyliblzma: make it optional since it is not essential [v2]