Add some functional tests that ensure that we don't
get caught by mistakes made when modifying the human
test reporting code again (or at least, not by the
obvious ones).
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The correct mapping for FAIL tests is term_support.fail_str,
not term_support.error_str. This will make FAIL tests to
report as FAIL again.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

[V2] Output tweaks

avocado.core.job: Print variants after the tree representation

Change docs to show an output that reflects the current
state of avocado.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Display the variants after the tree representation in the job log.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

[V2] avocado.core.job: Log avocado temporary dir

Display the temporary directory in the job log.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

[V2] Fix unhelpful message when calling process/GDB APIs

Makefile: Modify avocado-vt plugin links

The process and gdb related APIs trust on subprocess.Popen
to start a process. When shell=False and the command being
executed is missing, Subprocss.Popen will raise an OSError
with errno 2, with no additional message about what file
is actually missing.

OSError: [Errno 2] No such file or directory

As you might have guessed, my workaround is to capture
OSErrors with errno 2, and re-raise them with a more
helpful message.

'File /usr/bin/gdbserver not found'
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

This is a slight tweak on the appearance of the test
list in the human readable output - a single space
between the beginning of the line and the beginning
of the output (test numbering).

Before:

avocado run examples/tests/gdbtest.py
JOB ID     : 569340b20c85ed09a6164e8ce21e7b547e75802a
JOB LOG    : /home/lmr/avocado/job-results/job-2015-08-12T18.12-569340b/job.log
TESTS      : 21
(1/21) examples/tests/gdbtest.py:GdbTest.test_connect_multiple_clients: PASS (2.43 s)
(2/21) examples/tests/gdbtest.py:GdbTest.test_disconnect: PASS (2.45 s)
 ...
(21/21) examples/tests/gdbtest.py:GdbTest.test_stream_messages: PASS (0.15 s)
RESULTS    : PASS 21 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
JOB HTML   : /home/lmr/avocado/job-results/job-2015-08-12T18.12-569340b/html/results.html
TIME       : 19.34 s

After:

avocado run examples/tests/gdbtest.py
JOB ID     : 569340b20c85ed09a6164e8ce21e7b547e75802a
JOB LOG    : /home/lmr/avocado/job-results/job-2015-08-12T18.12-569340b/job.log
TESTS      : 21
 (1/21) examples/tests/gdbtest.py:GdbTest.test_connect_multiple_clients: PASS (2.43 s)
 (2/21) examples/tests/gdbtest.py:GdbTest.test_disconnect: PASS (2.45 s)
 ...
 (21/21) examples/tests/gdbtest.py:GdbTest.test_stream_messages: PASS (0.15 s)
RESULTS    : PASS 21 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
JOB HTML   : /home/lmr/avocado/job-results/job-2015-08-12T18.12-569340b/html/results.html
TIME       : 19.34 s
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Instead of showing it at the beginning, when it's not
ready yet.

Before:

JOB ID     : 751ceef63264d7749211875de16a0a719b80ccd3
JOB LOG    : /home/lmr/avocado/job-results/job-2015-08-12T18.17-751ceef/job.log
JOB HTML   : /home/lmr/avocado/job-results/job-2015-08-12T18.17-751ceef/html/results.html
TESTS      : 1
(1/1) passtest.py:PassTest.test: PASS (0.00 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
TIME       : 0.00 s

After:

JOB ID     : b92df2a8389bbc0c3569263968ca6509ec562825
JOB LOG    : /home/lmr/avocado/job-results/job-2015-08-12T18.17-b92df2a/job.log
TESTS      : 1
(1/1) passtest.py:PassTest.test: PASS (0.00 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0
JOB HTML   : /home/lmr/avocado/job-results/job-2015-08-12T18.17-b92df2a/html/results.html
TIME       : 0.00 s
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

With avocado-vt plugins renamed, 'make link' must
change.

ATTENTION: Please do a 'make clean' before applying
this patch (or clean your source tree before 'make link').
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

avocado.core.output: Use stdout to log test statuses

avocado.core.test: Use GDB-friendly srcdir

Throbber (test progress) supports two colors. Currently it turns green
only when new status arrives, which is currently only used when we
interrupt the test (GDB) or when the test finishes. Then the throbber
turns green for one iteration and back to yellow in next iteration.

This patch changes the behavior so the throbber is green when the test
process is alive and yellow when it's interrupted.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The human output prints the test name, progress and pass/skip test
statuses into stdout and fail/error/interrupt ones as stderr. This
results in unreadable output. This patch logs all of them in stdout.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Avocado allows the usage of GDB, but our default self.srcdir is
incompatible with it causing many tests to fail. This patch simply
replaces the ':' in basename to '_'. It's already created in a temporary
directory, so there shouldn't be any name clashes.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We can't set signal handlers inside a thread. In such
cases, let's not try to set the signal handler. This
fixes a problem where subprocesses executed with
`avocado.utils.process` APIs would fail if executed
inside threads.

One example of a thread that runs subprocesses can
be found on our avocado-vt plugin: The virt related
tests run a screendump thread, that will produce
screendumps on a specific directory. These screendumps
will be encoded later into a video. The libvirt VM
implementation takes screendumps by executing
process.run('virsh screendump'). So that's one of
the possible examples where process APIs can be used
inside of threads, and we should support such examples.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

VM plugin cleanup bug

Fix example test "linuxbuild.py"

GNU make (mis)understands paths that contain ":". The reason is that they look
like static patterns:

 https://www.gnu.org/software/make/manual/html_node/Static-Usage.html#Static-Usage

The exact error message one would see would be similar to:

 Makefile:XXX: *** multiple target patterns.  Stop.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `avocado.utils.kernel_build.KernelBuild` utility class can now
build a Linux kernel without an external configuration file, so let's
add this possibility to the test itself.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The Linux kernel compilation instructions suggest that a separate
directory is used for building the kernel and not the source tree
itself. Let's make this "exemplary" helper class follow those
guidelines.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And let's use a generated config by default, that is, let's drop the
custom config file we ship with the test.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

On most recent kernel make dep is deprecated and doesn't add any value
to the build routine itself.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

A Linux kernel can be built without an external configuration file
because it can generate a default one for the current target
platform. So, let's make that parameter optional.

Also, if a user does provide a config file, we use a completely silent
target for updating that config file, using defaults for missing
configuration keys.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It's reasonable to let the kernel build class to pick a work directory
itself, thus making that parameter optional.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It's more common to have variables with all caps defined at the class
level.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since it's not possible (yet?) to depend on existing virtual machines
to run tests for some of the Avocado features, including many of the
VM Plugin features, let's add a test as an example that can be run
manually instead. While they will not be part (at this time?) of the
functional test suite, it may serve as a reference/example in the
future.

The approach chosen here is to have a first run of Avocado making sure that
a flag file does not exist. A second run of Avocado, now executed with an
additional `--vm-cleanup` parameter runs a test that creates the flag file.
Finally, the third and last execution of re-runs the same test as the first
run, checking that the flag file does *not* exist, that is, the clean up of
the VM did work.

Because this test requires a libvirt domain and the hostname/address
of that virtual machine, these parameters assume no defaults for
safety and/or security causes. The test will be skipped if these are
not configured properly.  Please edit the vm-cleanup.yaml file with
the appropriate parameters.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Let's be practical and simply destroy (pull the plug on the VM) that
will have its state (snapshot reverted).

Not shutting down (or powering off/destroying) the VM has the side
effect of not reverting the state of the (still running) VM on a
couple of platforms, including EL7.

This fixed issue #396.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The given dest parameters actually reflect the default generated by
argparse. So, let's clean it up.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

avocado.core.runner: Make getting the timeout from test plugins more …

Some test loader plugins will return the 'timeout' parameter
as a string, so this will cause avocado to break, given that
it will add the timeout to another float value later in the
runner code. Let's ensure that value will be converted to
a float before the addition is made.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

They are necessary for the GDB functionality in avocado,
so let's bring them in.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

trinity example test: don't output ANSI characters to the logs