Test to cover the fixes in subtests filter. The filter
should work in a directory discovery for both INSTRUMENTED
and SIMPLE tests.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Simple tests discovery is not considering the subtests filter.

This patch adds to the simple test discovery process the capability of
filtering the tests by subtests filter.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Currently the subtests_filter is not used under test discovery in
directories.

This patch fixes that by adding the `subtests_filter` argument to
`_make_tests()` also in directory dicovery mode.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

The FDDrainer reads from the PIPEs and writes to both the internal
StringIO instances and to stream logging handlers.  There are no
guarantees that the logging handlers will flush the content before the
FDDrainer finishes.

Let's explicitly close all the handlers (which should really be
FileHandlers) associated with the stream loggers, which should flush
and sync the content on the files themselves.

Additionally, the combined drainer was being missed from the flush
process, and was causing some output in that mode to be unavailable on
generated output.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Sometimes, we want to use function has arguments in wait_for().
This patch supported optional arguments for wait function.
It's optional, so nothing effect for original usage.

| Example:
| def is_ready(status1, status2=None):
|     return status1 == status2
|
| wait.wait_for(is_ready, timeout=10, args=[False], kwargs={'status2': True})
Signed-off-by: NAmos Kong <amos@scylladb.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

there is a typo error in command execution in multipath.py in
get_path_status() method. so corrected it and it is working fine now.
Signed-off-by: NNaresh Bannoth <nbannoth@linux.vnet.ibm.com>

Currently test's note contains full job.state['fail_reason'].
And it can be huge unreadable single line  backtrace which
looks ugly in resultsdb_frontend. In my case it populate whole
webpage. IMHO front end note should be small, if someone want to
investigate failure reason we should goto logs link.

This patch add config option which allow to limit note's size,
which is limited by default to preserve original behaviour.

Also map job.INTERRUPTED state to resultsdb.INFO because when human
manually interrupts test he probably knows the reason, so test should
not being treated as NEEDS_INSPECTION.
Signed-off-by: NDmitry Monakhov <dmonakhov@openvz.org>

Avocado is amazing tool for journal everyday routines. But it is very
script oriented, so if one want to quick run simple command under avocado
script should be created first, which is not too handy.

Let's add trivial wrapper which transparently do it for us
#Example: 
$ avocado-run-inplace.sh yum update -y
JOB ID     : 26b320e99a8413f1fe73d23afaae746d8436bf8e
JOB LOG    : /root/avocado/job-results/job-2017-11-22T12.06-26b320e/job.log
 (1/1) yum_update_-y.sh: PASS (47.48 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 51.52 s
Signed-off-by: NDmitry Monakhov <dmonakhov@openvz.org>

It would be nice to show result url in output.
ResultEvent class is not good for that because it is called too early,
and makes output looks ugly. Let's use Result.render hook which is
specially designed for this stage
Signed-off-by: NDmitry Monakhov <dmonakhov@openvz.org>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

On `make rpm`, docs build fails with:

    docs/source/optional_plugins/varianter_pict.rst:33: WARNING: Could
    not lex literal_block as "ini".

This patch changes the way to define a block of code for a
proven-to-be-good one.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The remote runner is currently assuming not only about the existence
of mux_to_yaml plugin, which is bad enough, but relying on it to be
installed and active.

Let's default to no "mux files" if the plugin is not available.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

It looks like Travis does some different handling of process, and
we need to manually force the writes to the stdout.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

- Revert "travis: Use parallel check in travis"

    This reverts commit 4b296996.

    Parallel check was enabled in Travis, but it's overloading the system
    and making the time sensitive tests to FAIL.

- avocado.utils.process: fix default prefix

    Log prefix needs at least the %-style string formatter. Let's include
    that as the default so we don't break under 'no prefix'.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NAmador Pahim <apahim@redhat.com>

PICT is a free software (MIT licensed) tool that implements
combinatorial testing.  Information about it can be found here:

   https://github.com/Microsoft/pict/

This is a very simple, but functional varianter implementation.  It
performs the basic tasks of creating variants and making parameters
accessible to tests.  To test this version, it's recommended to
download PICT, build it (it only takes a C compiler such as GCC or
clang and make) and put it in a location included in your PATH (or use
`--pict-binary`).

To list the variants generated with the default order (2, that is,
do a pairwise idenpendent combinatorial testing):

 $ avocado variants --pict-parameter-file=examples/params.pict
 Pict Variants (11):
 Variant amd-scsi-rtl8139-windows-rhel6-acff:    /run
 ...
 Variant amd-ide-e1000-linux-rhel6-eb43:    /run

To list the variants generated with a 3-way combination:

 $ avocado variants --pict-parameter-file=examples/params.pict \
   --pict-order-of-combinations=3

 Pict Variants (28):
 Variant intel-ide-virtio-windows-rhel7-aea5:    /run
 ...
 Variant intel-scsi-e1000-linux-rhel7-9f61:    /run

It's also possible to run tests with variants generated by the same
command line switches.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The official fabric project doesn't support Python 3 yet.  Let's use
the unofficial Python 3 port for now.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

We have never used the subprocess32 backport module, and probably
wouldn't even support if users were doing that.  Let's remove the
the possibility of confusion and byte rot.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The snakefood package is not available on Python 3.  Let's skip
this check then.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

And the support for combined mode.  So far, there was no need to
determine how processes would be run, and how their output would be
collected: the stdout and stderr would always be collcted separately.

Now, with the need to support the "combined" mode, the
`--output-check-record` command line switch not only has to control
which files are populated (stdout.expected, stderr.expected) but also
it needs to control the execution of process, because it's what will
generate the "raw material" for a possible "output.expected" file.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is refactor of the current locks/file/threads created in the
SubProcess class to a more contained an isolated (and tested)
block of functionality.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The stream_prefix does nothing, let's remove it.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Which simplifies the code and makes it more Pythonic.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There are a few active players in the handling of the threads that
handle the reading from PIPEs and writing into files (and logging
streams), including files, locks and thread objects.

Let's define them in advance at instance initialization time, make
them protected instead of public.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Besides a small optmization, it seems more correct a single I/O
readiness decision in each loop, instead of possible different
results in the two calls.
Signed-off-by: NCleber Rosa <crosa@redhat.com>