Enable a mode where instead of the UI, only the job log
shows up in the avocado output. Also, clarify the meaning
of --silent on the command line help.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add a helper method on TestResultProxy() to get information
of output plugins using stdout, making up for a cleaner
way of letting the user know about the conflict.

Also, rephrase the error messages and update functional
tests to reflect the new status quo.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The stdout fd could be inadvertedly closed when doing
that. Besides, it's good practice to not initialize
something that will not be used.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

That was a wrong behavior of our application. When providing
a command line such as:

scripts/avocado run sleeptest --json /tmp/json

This command should display human output. This patch fixes this
issue, and corrects the functional test for this behavior.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

When passing --silent to the avocado command line,
stop emiting messages to the logging system altogether.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Now, introducing .notify() methods that do not reference
the UI directly, and migrate all plugins to use it.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Updates the docs with all behavior changes and comments
on text structure/phrasing.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

As pointed out during code review by Rudá.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

As pointed out during review, it is more straightforward
to just call the stream files 'stdout' and 'stderr'.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Make 'none' the default for --output-check-record, and
add --disable-output-check, for when you don't check the
test expected outputs for some reason (you are running on
a machine that makes your test output to behave differently, but
you still want to do the rest of the test.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

After review, we settled in 'allow_output_check' rather than
'record_stream_files', since we are allowing some of the output
streams of the program to be checked. Also, in order to make
the feature easier to use, turn it on by default (set
allow_output_check=True by default).
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

A number of programs will honor the value of $PAGER if that
is set as the program to use as a paginator, such as git.
Let's make our paginator code to honor it too.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add an example of script that will have its output
checked if executed through avocado.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add an example of how avocado exposes information to
shell scripts through environment variables.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

These files were captured using our new runner option

$ avocado run --output-check-record all examples/tests/synctest.py

And should now be checked every time that particular test
will run.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

It is a common use case in test automation that we want
to check the output of the test or applications started
by it and compare it with a reference output.

Introduce the option --output-check-record, that will
run the test you specify and generate the files
'stdout.expected' and 'stderr.expected' in the test
data dir (same directory as the source file, with the
name 'filename.data').

In case that option is not provided, if there is an
expected stdout or stderr file, those files will be
checked against the output the test just produced. If
no files are found, the check will be skipped. If
--output-check-record, no check will be made on that
particular execution, since we are recording what
is going to be expected in future test runs.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add the record_stream_file param to the execution of the
dropin tests. This ensures their outputs are properly logged.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add the record_stream_files argument to the synctest
command execution, to showcase how to use it. The idea
is that the full output of the synctest command goes to
the 'stdout.actual' and 'stderr.actual' files, excluding
all previous 'make' commands used to build the test suite.

With this, we pave the way for a feature that people can
use to record the outputs of commands, and later compare
with the reference output automatically.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add the new record_stream_files to the process APIs, which
basically means, 'log stdout and stderr of that subprocess
to the test stdout.actual and stderr.actual files'. We
thought about making that transparent, but frequently people
will want to execute programs inside tests that are part
of the setup of those tests, for example, make commands and
the such.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Create the file handler for both streams and connect them to
the loggers, so whatever goes on the stdout of those tests
will be logged to those files.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

So whatever goes to the stdout/stderr of those subprocesses
gets properly logged in the stdout.actual/stderr.actual files
that the test execution will record.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add a new object that can properly stub the stdout
and stderr streams for test subprocess executions,
delivering the messages to our stdout and stderr
loggers instead of directly to sys.stdout and/or
sys.stderr.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Both will be important to store stdout and stderr streams
of tests for our future output check mode feature.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Organize the log APIs into:

* APIs that log operation statuses (.log_ui_status)
* More generic ones (.log_ui)

The distinction helps and gives more insight on what is
actually accomplished by each method. This is a fairly
intrusive change, although it received a fair bit of
testing before submission.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

As this class is evolving to become the one stop shop
for avocado output needs. Also, move some methods around
to help organize better the class layout.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

One less than ideal aspect of our output module is that
plugins have to use a bunch of primitives of that module
out of necessity (such as get_paginator()).

Avocado needs a single entity that controls output for
the application. Let's start by covering the needs of
plugins, by introducing the list_mode argument to the
constructor of that class. When list_mode is set to
True, instead of the regular logging system we'll use
the output paginator class (that will use less if
available to display large amounts of output line with
pagination. Also, turn a useful method that was private
to public, ._log() -> .log().
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Multiplexer documentation fixes

/root was removed in a9ea39d6, but the documentation was not fixed.

Get rid of "/root" for the root node and use empty string as name,
in order to avoid to prefix filters and to display information
with the /root prefix in every output.

The output will be just like this:

./scripts/avocado multiplex mux.yaml --filter-only /hw/cpu/arm --filter-out /os
Variants generated:
Variant 1:    /hw/cpu/arm, /hw/disk/scsi, /env/debug
Variant 2:    /hw/cpu/arm, /hw/disk/scsi, /env/prod
Variant 3:    /hw/cpu/arm, /hw/disk/virtio, /env/debug
Variant 4:    /hw/cpu/arm, /hw/disk/virtio, /env/prod
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Update documentation regarding external plugins and the hello world plugin example.

Make it clear that we need to prefix every external plugin
with "avocado_" to load it. Also, using the new updated "Hello
World!" plugin example.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Update the "Hello World!" plugin example to match
the recent changes that we've made in the plugin structure.

avocado.plugins.vm: Add option group for libvirt domain options

As it helps organizing the help output of the application.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>