These files were captured using our new runner option

$ avocado run --output-check-record all examples/tests/synctest.py

And should now be checked every time that particular test
will run.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

It is a common use case in test automation that we want
to check the output of the test or applications started
by it and compare it with a reference output.

Introduce the option --output-check-record, that will
run the test you specify and generate the files
'stdout.expected' and 'stderr.expected' in the test
data dir (same directory as the source file, with the
name 'filename.data').

In case that option is not provided, if there is an
expected stdout or stderr file, those files will be
checked against the output the test just produced. If
no files are found, the check will be skipped. If
--output-check-record, no check will be made on that
particular execution, since we are recording what
is going to be expected in future test runs.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add the record_stream_file param to the execution of the
dropin tests. This ensures their outputs are properly logged.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add the record_stream_files argument to the synctest
command execution, to showcase how to use it. The idea
is that the full output of the synctest command goes to
the 'stdout.actual' and 'stderr.actual' files, excluding
all previous 'make' commands used to build the test suite.

With this, we pave the way for a feature that people can
use to record the outputs of commands, and later compare
with the reference output automatically.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add the new record_stream_files to the process APIs, which
basically means, 'log stdout and stderr of that subprocess
to the test stdout.actual and stderr.actual files'. We
thought about making that transparent, but frequently people
will want to execute programs inside tests that are part
of the setup of those tests, for example, make commands and
the such.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Create the file handler for both streams and connect them to
the loggers, so whatever goes on the stdout of those tests
will be logged to those files.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

So whatever goes to the stdout/stderr of those subprocesses
gets properly logged in the stdout.actual/stderr.actual files
that the test execution will record.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add a new object that can properly stub the stdout
and stderr streams for test subprocess executions,
delivering the messages to our stdout and stderr
loggers instead of directly to sys.stdout and/or
sys.stderr.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Both will be important to store stdout and stderr streams
of tests for our future output check mode feature.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Organize the log APIs into:

* APIs that log operation statuses (.log_ui_status)
* More generic ones (.log_ui)

The distinction helps and gives more insight on what is
actually accomplished by each method. This is a fairly
intrusive change, although it received a fair bit of
testing before submission.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

As this class is evolving to become the one stop shop
for avocado output needs. Also, move some methods around
to help organize better the class layout.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

One less than ideal aspect of our output module is that
plugins have to use a bunch of primitives of that module
out of necessity (such as get_paginator()).

Avocado needs a single entity that controls output for
the application. Let's start by covering the needs of
plugins, by introducing the list_mode argument to the
constructor of that class. When list_mode is set to
True, instead of the regular logging system we'll use
the output paginator class (that will use less if
available to display large amounts of output line with
pagination. Also, turn a useful method that was private
to public, ._log() -> .log().
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Multiplexer documentation fixes

/root was removed in a9ea39d6, but the documentation was not fixed.

Get rid of "/root" for the root node and use empty string as name,
in order to avoid to prefix filters and to display information
with the /root prefix in every output.

The output will be just like this:

./scripts/avocado multiplex mux.yaml --filter-only /hw/cpu/arm --filter-out /os
Variants generated:
Variant 1:    /hw/cpu/arm, /hw/disk/scsi, /env/debug
Variant 2:    /hw/cpu/arm, /hw/disk/scsi, /env/prod
Variant 3:    /hw/cpu/arm, /hw/disk/virtio, /env/debug
Variant 4:    /hw/cpu/arm, /hw/disk/virtio, /env/prod
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Update documentation regarding external plugins and the hello world plugin example.

Make it clear that we need to prefix every external plugin
with "avocado_" to load it. Also, using the new updated "Hello
World!" plugin example.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Update the "Hello World!" plugin example to match
the recent changes that we've made in the plugin structure.

avocado.plugins.vm: Add option group for libvirt domain options

As it helps organizing the help output of the application.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

GDB: deal with fatal signals and generate core dumps (v2)

This adds two related features:

1) dealing with fatal signals such as SIGSEGV
2) generating core dumps automatically (disabled by default) if requested

So now the behaviour on fatal signals (SISEGV and SIGABRT) for now cause
the test to be skipped. Other ways to deal with this proved to be complex
and not really effective.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

multiplexer: Reintroduce command line options for filters.

docs/ContributionGuide: Update with new ML address

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Reintroduce command line options --filter-only and --filter-out
to control the global filters for multiplexing, both by the test runner
and the multiplex plugin.

Group multiplex command line options together in the group
"multiplex" inside the test runner.

Examples:

avocado run --help
... other options ...
multiplex arguments:
  -m MULTIPLEX_FILE, --multiplex-file MULTIPLEX_FILE
                        Path to an avocado multiplex (.yaml) file
  --filter-only [FILTER_ONLY [FILTER_ONLY ...]]
                        Filter only path(s) from multiplexing
  --filter-out [FILTER_OUT [FILTER_OUT ...]]
                        Filter out path(s) from multiplexing

avocado run footest --multiplex foo.yaml \
	--filter-only /root/env /root/os/linux \
        --filter-out /root/hw/virtio

avocado multiplex mux.yaml \
	--filter-only /root/os/win /root/hw/cpu/intel \
        --filter-out /root/hw/disk/virtio
Variants generated:
Variant 1:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/winxp, /root/env/debug
Variant 2:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/winxp, /root/env/prod
Variant 3:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/win7, /root/env/debug
Variant 4:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/win7, /root/env/prod
Variant 5:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/win8, /root/env/debug
Variant 6:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/win8, /root/env/prod
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Ruda's multiplex updates

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Include documentation for path_parent and any_sibling and
use variable names more relevant.

In function any_sibling, if the number of arguments is less than 2,
then returns False, because it is obviously alone.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Include unit test for support functions used in the multiplexer.py
module: path_parent() and any_sibling().
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Provide a better output when the number of variants of a multiplex
is zero (for a giving set of tests and one multiplex file).

Also, when there is no tests passed, report early.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

For the filter_only and filter_out functions for filtering,
always accept a string or a list of string for for the parameters.

So this is possible to parse a YAML file like:

foo:
    filter-only: /root/env/prod

or

foo:
    filter-only: [/root/env/prod]

The previous behavior was treating the string as sequence,
so the filters were using the chars as input values.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

To make things easy for debugging, changing the default
behavior of repr() and str() to the class TreeNode.

* For repr() just output the node name.
  Example: TreeNode(name='prod')
* For str() output the node path and the variables from the environment.
  Example: /root/env/prod: foo=123, bar=456

So the output looks like that:

... after a break pointing into the multiplex() function...

(Pdb) print args[0]
[TreeNode(name='short'), TreeNode(name='medium'), TreeNode(name='long'), TreeNode(name='longest')]

(Pdb) print args[0][0]
/root/short: sleep_length=0.5
Signed-off-by: NRuda Moura <rmoura@redhat.com>

VM plugin improvements

It was something that we forgot to do back when the feature
was implemented. This basic guide should be enough to
understand and use the feature.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Reword and align messages that inform libvirt domain
and VM login.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>