Signed-off-by: NCleber Rosa <crosa@redhat.com>

IMHO, the generated sphinx config file is too noisy and thus hard to
manage. Let's clean it up.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

avocado.plugins.builtin: Builtin plugins are dynamic.

the builtin plugin list were once static coded, but now
they are search and loaded by run-time,
by inspecting the directory <avocado_home>/avocado/plugins.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Not found fixes

We have introduced the new NOT_FOUND status, but haven't
updated the output plugins with it. Let's fix this bug.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

avocado.cli.app: Fix to display all --help options.

Fix the way that we handle argument parsing, to display
all options for subcommands when --help is passed.

We fix the problem by using parents parse.

Plus: cosmetic fixes in avocado.plugins regarding
argument parse.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Looks like a travis worker can be a lot slower than I thought at first,
in the order of 50 times slower than a 2 year old development machine.

So now, the timeout is 250 times higher than the test run would take
under the mentioned development machine (0.12s), but still able to
catch bugs if the loop does not cycle immediatelly on failure for the
100 failtests.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Job main loop: functional tests timing

Looks like test bots, such as travis, can have very little resources
at times and run things really slowly, example:

======================================================================
FAIL: Sleeptest is supposed to take 1 second, let's make a sandwich of
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/travis/build/avocado-framework/avocado/selftests/all/
functional/avocado/basic_tests.py", line 238, in test_runner_sleep_fail_sleep_timing
    self.assertLess(actual_time, 4.0)
AssertionError: 4.036144018173218 not less than 4.0

Let's increase the maximum expected times.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Test execution times async

Signed-off-by: NCleber Rosa <crosa@redhat.com>

As spotted by Ruda. Thanks!

CC: Ruda Moura <rmoura@redhat.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There's a bug in the current job mainloop that causes significant delays.
The test state is received from the test via the queue in an async fashion,
but if a proper state is received, we failt to mark the loop as done.

This patch, in conjunction with the previous reverted commit, puts the
queue timeout in sync with the UI timeout and puts the cycle timeout
back to 1s, without the adverse consequences.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This reverts commit 10e6af662844a4c9df6994bfc7536f0413fbc82d.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

avocado: Add NOT_FOUND test status

To avoid confusion, when passing an invalid test ID
(typo, incorrect path, alias that could not be resolved),
introduce a new status, NOT_FOUND inside avocado, as
well as proper handling handling for such situations.

$ scripts/avocado run sbrubles
JOB ID    : 312b690173a8f5f261d9c5cefdb8699ac0798ded
JOB LOG   : /home/lmr/avocado/job-results/job-2014-08-14T10.02-312b690/job.log
TESTS     : 1
(1/1) sbrubles: NOT_FOUND (0.00 s)
PASS      : 0
ERROR     : 0
FAIL      : 0
SKIP      : 0
WARN      : 0
NOT FOUND : 1
TIME      : 0.00 s

Also, add a functional test for this new status.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

avocado.job: Give an error message on an empty test list

If an empty list of tests was passed, give an error validation
message and exit. Provide a functional test for it.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

New feature: silent (human) test result.

Implement new option --silent, to mute the human test result.
It has no effect with --json or --xunit options.

Example:

$ avocado --silent run failtest
$ echo $?
$ 1
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Interrupt fixes

Turns out some subprocesses might misbehave (see tests
like trinity - run on a VM), so it's not enough to send
the interrupt signal. For a clean stop output wise,
SIGKILL is necessary.

As a side effect, the previous Ctrl+C can be resurrected,
this time renamed to _interrupt, since we're not really
simulating Ctrl+C.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

This way we avoid the bug that was fixed on the previous commit.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

This way we can correctly catch tests that just timed out.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

So we don't trigger multiprocess logging inadvertedly.
Just fill out the instance with appropriate data.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

avocado: Fix install and execution under virtual environment

This ends up making the subprocesses to receive the
SIGINT multiple times, leading to the multiprocess
logging to misbehave.

Also, remove the ctrl+c test, given that we couldn't
figure out a way to properly reproduce a SIGINT passed
to the process group of the running avocado instance.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The use of virtual environments is a very useful feature
for developers and helps us to sanity check code instalability.
However, we were not able to use it because the data file
install has to be different for virt env and for regular
system wide install (RPMs).

This patch makes avocado aware of the execution mode,
and sets the install process so that it works differently
(and fine) for both normal install and virtenv install.

Additionally, this fixes a bug I introduced on installing
test files, that broke the avocado rpms.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Sprint 11

Since we are not running short anytime of error code definitions, I
believe we can give a more descriptive return code when jobs are
actually interrupted.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

That is, within a second, if the test process is running, the user
will see a "full cycle" of the throbber.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The command now is 'datadir'.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Inside the job result instance dir, introduce a 'test-results'
directory that holds the results of all the other test instances.
This organizes things better and is a standard dir name,
since now we mimic a the origin test directory structure.

Update the docs accordingly.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

synctest fails on the CI VM, let's use the simpler,
bogus datadir test to do that functional verification.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Create a test that uses files from the data dir, as
a functional test, so we notice when that functionality
is broken.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>