Updates the docs with all behavior changes and comments
on text structure/phrasing.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

As pointed out during code review by Rudá.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

As pointed out during review, it is more straightforward
to just call the stream files 'stdout' and 'stderr'.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Make 'none' the default for --output-check-record, and
add --disable-output-check, for when you don't check the
test expected outputs for some reason (you are running on
a machine that makes your test output to behave differently, but
you still want to do the rest of the test.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

After review, we settled in 'allow_output_check' rather than
'record_stream_files', since we are allowing some of the output
streams of the program to be checked. Also, in order to make
the feature easier to use, turn it on by default (set
allow_output_check=True by default).
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

A number of programs will honor the value of $PAGER if that
is set as the program to use as a paginator, such as git.
Let's make our paginator code to honor it too.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add an example of script that will have its output
checked if executed through avocado.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add an example of how avocado exposes information to
shell scripts through environment variables.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

These files were captured using our new runner option

$ avocado run --output-check-record all examples/tests/synctest.py

And should now be checked every time that particular test
will run.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

It is a common use case in test automation that we want
to check the output of the test or applications started
by it and compare it with a reference output.

Introduce the option --output-check-record, that will
run the test you specify and generate the files
'stdout.expected' and 'stderr.expected' in the test
data dir (same directory as the source file, with the
name 'filename.data').

In case that option is not provided, if there is an
expected stdout or stderr file, those files will be
checked against the output the test just produced. If
no files are found, the check will be skipped. If
--output-check-record, no check will be made on that
particular execution, since we are recording what
is going to be expected in future test runs.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add the record_stream_file param to the execution of the
dropin tests. This ensures their outputs are properly logged.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add the record_stream_files argument to the synctest
command execution, to showcase how to use it. The idea
is that the full output of the synctest command goes to
the 'stdout.actual' and 'stderr.actual' files, excluding
all previous 'make' commands used to build the test suite.

With this, we pave the way for a feature that people can
use to record the outputs of commands, and later compare
with the reference output automatically.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add the new record_stream_files to the process APIs, which
basically means, 'log stdout and stderr of that subprocess
to the test stdout.actual and stderr.actual files'. We
thought about making that transparent, but frequently people
will want to execute programs inside tests that are part
of the setup of those tests, for example, make commands and
the such.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Create the file handler for both streams and connect them to
the loggers, so whatever goes on the stdout of those tests
will be logged to those files.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

So whatever goes to the stdout/stderr of those subprocesses
gets properly logged in the stdout.actual/stderr.actual files
that the test execution will record.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add a new object that can properly stub the stdout
and stderr streams for test subprocess executions,
delivering the messages to our stdout and stderr
loggers instead of directly to sys.stdout and/or
sys.stderr.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Both will be important to store stdout and stderr streams
of tests for our future output check mode feature.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Organize the log APIs into:

* APIs that log operation statuses (.log_ui_status)
* More generic ones (.log_ui)

The distinction helps and gives more insight on what is
actually accomplished by each method. This is a fairly
intrusive change, although it received a fair bit of
testing before submission.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

As this class is evolving to become the one stop shop
for avocado output needs. Also, move some methods around
to help organize better the class layout.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

One less than ideal aspect of our output module is that
plugins have to use a bunch of primitives of that module
out of necessity (such as get_paginator()).

Avocado needs a single entity that controls output for
the application. Let's start by covering the needs of
plugins, by introducing the list_mode argument to the
constructor of that class. When list_mode is set to
True, instead of the regular logging system we'll use
the output paginator class (that will use less if
available to display large amounts of output line with
pagination. Also, turn a useful method that was private
to public, ._log() -> .log().
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Multiplexer documentation fixes

/root was removed in a9ea39d6, but the documentation was not fixed.

Get rid of "/root" for the root node and use empty string as name,
in order to avoid to prefix filters and to display information
with the /root prefix in every output.

The output will be just like this:

./scripts/avocado multiplex mux.yaml --filter-only /hw/cpu/arm --filter-out /os
Variants generated:
Variant 1:    /hw/cpu/arm, /hw/disk/scsi, /env/debug
Variant 2:    /hw/cpu/arm, /hw/disk/scsi, /env/prod
Variant 3:    /hw/cpu/arm, /hw/disk/virtio, /env/debug
Variant 4:    /hw/cpu/arm, /hw/disk/virtio, /env/prod
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Update documentation regarding external plugins and the hello world plugin example.

Make it clear that we need to prefix every external plugin
with "avocado_" to load it. Also, using the new updated "Hello
World!" plugin example.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Update the "Hello World!" plugin example to match
the recent changes that we've made in the plugin structure.

avocado.plugins.vm: Add option group for libvirt domain options

As it helps organizing the help output of the application.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

GDB: deal with fatal signals and generate core dumps (v2)

This adds two related features:

1) dealing with fatal signals such as SIGSEGV
2) generating core dumps automatically (disabled by default) if requested

So now the behaviour on fatal signals (SISEGV and SIGABRT) for now cause
the test to be skipped. Other ways to deal with this proved to be complex
and not really effective.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

multiplexer: Reintroduce command line options for filters.

docs/ContributionGuide: Update with new ML address

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Reintroduce command line options --filter-only and --filter-out
to control the global filters for multiplexing, both by the test runner
and the multiplex plugin.

Group multiplex command line options together in the group
"multiplex" inside the test runner.

Examples:

avocado run --help
... other options ...
multiplex arguments:
  -m MULTIPLEX_FILE, --multiplex-file MULTIPLEX_FILE
                        Path to an avocado multiplex (.yaml) file
  --filter-only [FILTER_ONLY [FILTER_ONLY ...]]
                        Filter only path(s) from multiplexing
  --filter-out [FILTER_OUT [FILTER_OUT ...]]
                        Filter out path(s) from multiplexing

avocado run footest --multiplex foo.yaml \
	--filter-only /root/env /root/os/linux \
        --filter-out /root/hw/virtio

avocado multiplex mux.yaml \
	--filter-only /root/os/win /root/hw/cpu/intel \
        --filter-out /root/hw/disk/virtio
Variants generated:
Variant 1:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/winxp, /root/env/debug
Variant 2:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/winxp, /root/env/prod
Variant 3:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/win7, /root/env/debug
Variant 4:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/win7, /root/env/prod
Variant 5:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/win8, /root/env/debug
Variant 6:    /root/hw/cpu/intel, /root/hw/disk/scsi, /root/os/win/win8, /root/env/prod
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Ruda's multiplex updates