The message is being printed even woth different result formats (json,
tap, ...). This patch makes it to be printed only when using default
TUI.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Double free errors are silent in Fedora 27, making the test
doublefree.py to fail.

Using `MALLOC_CHECK_=1` is guaranteed to expose the double free
errors, regardless the OS double free handling. This patch updates
the doublefree.py test to use the `MALLOC_CHECK_=1` approach.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The documentation and examples on the output check feature are
outdated or innaccurate.

For instance, the synctest.py example would produce different output
than recorded in the accompanying reference files because the API
parameters were not being respected.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The avocado.utils.process APIs offer the `allow_output_check`
parameter which exists at a function call level, and thus should be
applied only during the scope of that function.

While it's questionable if a test process should be able to skip
some of its generated output (by means of an `avocado.utils.process`
function parameter), the fact is that the utility libraries offer
that at a lower level.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

In commit 4514a8ad, the default value from the command line option
`--output-check-record` was being used to set the mode of the
`avocado.utils.process` APIs with regards to output record mode.

The problem with that is it makes it impossible for the API to
distinguish between "no explicit behavior requested" and "explicitly
being asked to disable recording".
Signed-off-by: NCleber Rosa <crosa@redhat.com>

First, the recording of the output should be governed by the
corresponding setting alone.  There's no need to check the test being
run in job standalone mode, because if that's being run like that,
there's no way to actually set the record mode.

Then, for the output check mode, it won't happen if the the test is
being run in standalone mode or if the output recording has been
explicitly disabled (then we have no material to check the reference
with) or if the checking itself has not been enabled.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The script module is frequently used in Avocado itself in its
functional tests to generate tests files that will be run by the
`avocado` command line test runner.

Being able to create files that are *not* executable is important
because it can determine if a file will be identified as a SIMPLE or
INSTRUMENTED test.  Let's make that read only file permission
available as a definition in the module level, just like
`DEFAULT_MODE` is there.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Avocado recently introduced the support to record the combined
output generated by a test.  Now, this introduces support for
checking against a previously record combined output.

If an 'output.expected' file exists, it'll take precedence
over the `stdout.expected` and `stderr.expected` files.

Also it's necessary to switch the default operation mode for
the `avocado.utils.process` functions, since to compare with
an `output.expected` file, the record mode for the currently
running test has to be the same.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `_reference_check()` utility method now provides more meaningful
results.  This allows one to check wether a check was actually
performed.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The code is basically the same for all stream and files.  Let's
have a single method instead.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The code is basically the same for all different files.  Let's
have a single method instead.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The same code is run on multiple other tests.  It's a waste to run
it yet another time by itself.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

There was once an idea that the "mux" would be an interface that would
be reused by other implementations (say, json_to_mux, foo_to_mux,
etc).  Now, it seems much more important to make the Avocado varianter
design and interfaces better, and advocate its use instead.

Additionally, there has always been a gray line between the
Multiplexer and the Avocado params.  Moving the mux outside of the
Avocado core will help to draw clearer lines.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Test to cover the fixes in subtests filter. The filter
should work in a directory discovery for both INSTRUMENTED
and SIMPLE tests.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Simple tests discovery is not considering the subtests filter.

This patch adds to the simple test discovery process the capability of
filtering the tests by subtests filter.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Currently the subtests_filter is not used under test discovery in
directories.

This patch fixes that by adding the `subtests_filter` argument to
`_make_tests()` also in directory dicovery mode.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The call of `__del__` is not guaranteed and we are responsible for
cleaning those files. Let's call this function late in our `app` code.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

And include jobdata results and a test for 52.0 (LTS).

The 52.0 release, an LTS release, was meant to allow transitions from
the previous LTS version (36.0lts) and it did so including the ability
to replay jobs created with the older version.

Now, we're 6+ months into the 52.0 release, and as promised, we'll
stop maintaining 36.0lts.  To keep to code clean, let's remove all
this compatibility with versions before 52.0.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

The FDDrainer reads from the PIPEs and writes to both the internal
StringIO instances and to stream logging handlers.  There are no
guarantees that the logging handlers will flush the content before the
FDDrainer finishes.

Let's explicitly close all the handlers (which should really be
FileHandlers) associated with the stream loggers, which should flush
and sync the content on the files themselves.

Additionally, the combined drainer was being missed from the flush
process, and was causing some output in that mode to be unavailable on
generated output.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Sometimes, we want to use function has arguments in wait_for().
This patch supported optional arguments for wait function.
It's optional, so nothing effect for original usage.

| Example:
| def is_ready(status1, status2=None):
|     return status1 == status2
|
| wait.wait_for(is_ready, timeout=10, args=[False], kwargs={'status2': True})
Signed-off-by: NAmos Kong <amos@scylladb.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

there is a typo error in command execution in multipath.py in
get_path_status() method. so corrected it and it is working fine now.
Signed-off-by: NNaresh Bannoth <nbannoth@linux.vnet.ibm.com>

Currently test's note contains full job.state['fail_reason'].
And it can be huge unreadable single line  backtrace which
looks ugly in resultsdb_frontend. In my case it populate whole
webpage. IMHO front end note should be small, if someone want to
investigate failure reason we should goto logs link.

This patch add config option which allow to limit note's size,
which is limited by default to preserve original behaviour.

Also map job.INTERRUPTED state to resultsdb.INFO because when human
manually interrupts test he probably knows the reason, so test should
not being treated as NEEDS_INSPECTION.
Signed-off-by: NDmitry Monakhov <dmonakhov@openvz.org>

Avocado is amazing tool for journal everyday routines. But it is very
script oriented, so if one want to quick run simple command under avocado
script should be created first, which is not too handy.

Let's add trivial wrapper which transparently do it for us
#Example: 
$ avocado-run-inplace.sh yum update -y
JOB ID     : 26b320e99a8413f1fe73d23afaae746d8436bf8e
JOB LOG    : /root/avocado/job-results/job-2017-11-22T12.06-26b320e/job.log
 (1/1) yum_update_-y.sh: PASS (47.48 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 51.52 s
Signed-off-by: NDmitry Monakhov <dmonakhov@openvz.org>

It would be nice to show result url in output.
ResultEvent class is not good for that because it is called too early,
and makes output looks ugly. Let's use Result.render hook which is
specially designed for this stage
Signed-off-by: NDmitry Monakhov <dmonakhov@openvz.org>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

On `make rpm`, docs build fails with:

    docs/source/optional_plugins/varianter_pict.rst:33: WARNING: Could
    not lex literal_block as "ini".

This patch changes the way to define a block of code for a
proven-to-be-good one.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The remote runner is currently assuming not only about the existence
of mux_to_yaml plugin, which is bad enough, but relying on it to be
installed and active.

Let's default to no "mux files" if the plugin is not available.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>