avocado.core.data_dir: Contemplate read only dirs

The avocado tests location can actually be a read
only location, so take that into account when looking
for test dirs.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add new avocado Functional Tests

They raise the following exceptions:

1) TestError (failure)
2) TestNAError (skip)
3) TestWarn (failure)

The corresponding functional tests for them go
in the Travis CI file.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Since someone might want to be funny and raise those
exceptions, it is safer to make them to return "FAIL"
instead of "NEVER_RAISE_THIS", so avocado does not
crash.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Since both can possibly fail and mask the actual test
results.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

It is easier to show when a test actually failed, or
we were not able to successfully set the test up.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Fix stand alone tests rc

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

When running stand alone tests, ie, tests without
the avocado runner, I surely do expect they return
exit codes according to wether it passed or failed.

This patch fixed that by making the TestModuleRunner
class to return appropriate return codes from the
Job.run() function.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Fix test execution when running in standalone mode.

Fix errors:

* AttributeError: 'NoneType' object has no attribute 'unique_id'
* AttributeError: 'NoneType' object has no attribute 'keep_tmp_files'

When running tests standalone, like:

python tests/sleeptest/sleeptest.py
Signed-off-by: NRuda Moura <rmoura@redhat.com>

avocado.plugins.manager: Skip loading bad builtin plugins.

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Introduce Avocado journal replay utility

Signed-off-by: NRuda Moura <rmoura@redhat.com>

A command line utility that can read a test journal (when the test
is run using '--journal' (and thus the journal plugin), and can
send all the test activities recorded to a central Avocado Server.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Auto generate api docs

Instead of manually updating the API rst files upon
new module additions, auto generate the .rst files
through a trick on conf.py.

With this we have reduced maintenance and always
up to date API docs.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

We'll introduce code in docs/source/conf.py to call
sphinx-apidoc to generate them on the fly, ensuring
API docs will always be complete on each new build
triggered on read the docs, due to the web commit
hooks configured on github.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

docs: API - Update new modules recently added to avocado

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Report link

Class names can be found on instance.__class__.__name__,
not instance.__class__.name. Very silly bug that is
fixed here.

Also, Print error headers to provide visual cues of
the error that happened, also, provide a bug report
link for avocado crashes. Example:

Avocado crashed: ValueError: BUG!
Traceback (most recent call last):

  File "/home/lmr/Code/avocado/avocado/job.py", line 117, in _run
    raise ValueError("BUG!")

ValueError: BUG!

Please include the traceback info and command line used on your bug report
Report bugs visiting https://github.com/avocado-framework/avocado/issues/newSigned-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

In order to color error messages more prominently,
introduce fail headers, colored red, to provide
visual cues of what's going on.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Fix assorted bugs, make avocado return codes and functional test more complete

Avocado should handle gracefully and not crash.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

We have to tell apart 'expected' avocado error from
plain crashes. Refactor the Job.run() method, separating
it to an unhandled exception auxiliary method, and a
handled one, that properly sets the following return
codes for avocado execution:

* AVOCADO_ALL_OK (0)
    Both job and tests PASSed
* AVOCADO_TESTS_FAIL (1)
    Job went fine, but some tests FAILed or ERRORed
* AVOCADO_JOB_FAIL (2)
    Something went wrong with the Job itself, by explicit
    :class:`avocado.core.exceptions.JobError` exception.
* AVOCADO_CRASH (3)
    Something else went wrong and avocado plain crashed.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add a mapping of possible return codes for avocado,
to the meaning of them. We need to differentiate
simple jobs from tests failed, jobs that failed, and
avocado crashes.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

For the (yet non existing) cases where we want to throw
a specific JobError, that will be managed by the runner
plugin.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

When we skip a test, that is not a failure. Only the
test was not applicable for the given test environment.
Therefore, we should not report those as failures.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

When tests don't exist, current avocado breaks on
a very spectacular fashion:

Traceback (most recent call last):
  File "scripts/avocado", line 29, in <module>
    sys.exit(app.run())
  File "/home/lmr/Code/avocado2/avocado/cli/app.py", line 72, in run
    return self.args.func(self.args)
  File "/home/lmr/Code/avocado2/avocado/plugins/runner.py", line 106, in
run_tests
    return job_instance.run()
  File "/home/lmr/Code/avocado2/avocado/job.py", line 111, in run
    test_instance = self.run_test(url)
  File "/home/lmr/Code/avocado2/avocado/job.py", line 76, in run_test
    test_instance = self._load_test_instance(url)
  File "/home/lmr/Code/avocado2/avocado/job.py", line 63, in
_load_test_instance
    f, p, d = imp.find_module(url, [test_module_dir])
ImportError: No module named bogustest

We should be of course more classy when such a problem happens.
Introduce a NonExistentTest class to handle such problems,
and use it when the ImportError happens. The end result is
a lot better:

DEBUG LOG: /home/lmr/avocado/logs/run-2014-04-29-13.12.34/debug.log
TOTAL TESTS: 1
(1/1) bogustest.1:  ERROR (0.10 s)
TOTAL PASSED: 0
TOTAL ERROR: 1
TOTAL FAILED: 0
TOTAL SKIPPED: 0
TOTAL WARNED: 0
ELAPSED TIME: 0.10 s
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

An oversight happened when writing the output plugins made
the ERROR status to be ignored and reported as PASS.
This patch fixes this problem.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add new rule 'build-rpm-all' to the Makefile.

Add rule to build both source and binary RPMs.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Display init docs

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>