The sleeptenmin example test contains `self.report_state` which should
not be called directly and it defines variable which is not used.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We have sysinfo job hooks and test hooks. Job hooks are only executed
when sysinfo is enabled, but test hooks are executed anyway. This patch
makes the test to respect sysinfo setting and only execute test hooks
when sysinfo is enabled.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Using asset fetcher for KernelBuild download() so we can take
advantage of the cache mechanism.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Signed-off-by: NWei Jiangang <weijg.fnst@cn.fujitsu.com>

There's no reason for having a separated simpletests directory, let's move
them to the tests directory.

Reference: https://trello.com/c/VFcxxwIpSigned-off-by: NAmador Pahim <apahim@redhat.com>

updated module name in examples/tests/linuxbuild.py

The get_data_path utility API of the Test class is a convenience,
it simply joins the name of a given file to the datadir location.

IMHO the value is low when compared to the pollution of the namespace
and yet another compatibility/stability promise.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

These tests were moved to
https://github.com/avocado-framework/avocado-misc-testsSigned-off-by: NAmador Pahim <apahim@redhat.com>

Add new 'aiostress' test based on autotest.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Add new 'compilebench' test based on autotest.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Add new 'bonnie' test based on autotest.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

With the same one that comes from the test documentation.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Add new 'stress' test based on autotest.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

While going through the dependency list (requirements*.txt files) and
performing our self tests out of virtual environments, I noticed that
some tests are run outside the virtual environments.

The reason is that, even though the virtual environment is activated
for the test session (and say, `which python` gives `/venv/bin/python`),
we have hard coded `/usr/bin/python` in most places.

According to the some discussions on the virtualenv project itself[1],
a quick solution is to revert to the also common `/usr/bin/env python`
way of pointing to the Python interpreter.

[1] - https://github.com/pypa/virtualenv/issues/124Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is an automated mass import cleanup across all
avocado source files:

1) Imports follow the order:
 * Standard library imports
 * Non standard library external imports
 * Internal imports
 All separated by a single line
2) One line between imports and the rest of the code
3) Remove unused imports in the process
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Make sure we have avocado_ + __name__ prefixes to the dirs
created with tempfile, so that we have an easy way to spot
when we are not cleaning up directories properly. This mostly
concerns unittests, although the kernel_build lib and the
iso9660 lib also uses that API.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The name was not correct as the test should error-out.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently when one passes exception, which is not instance of Exception,
avocado proceeds and returns "INTERRUPTED" test instead of "ERROR". This
for example happens when old-style-class is used as exception. This
patch uses pure "except" without any argument to catch such exceptions
and then it wraps them in "exceptions.TestError" instead.

The same applies to "setUp" and "tearDown", they only use different
class as a wrapper.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

With ldoktor's fix 136a78b6 this is unnecessary and we
risk it being confused for people looking at the
examples.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

GNU make (mis)understands paths that contain ":". The reason is that they look
like static patterns:

 https://www.gnu.org/software/make/manual/html_node/Static-Usage.html#Static-Usage

The exact error message one would see would be similar to:

 Makefile:XXX: *** multiple target patterns.  Stop.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `avocado.utils.kernel_build.KernelBuild` utility class can now
build a Linux kernel without an external configuration file, so let's
add this possibility to the test itself.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And let's use a generated config by default, that is, let's drop the
custom config file we ship with the test.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since it's not possible (yet?) to depend on existing virtual machines
to run tests for some of the Avocado features, including many of the
VM Plugin features, let's add a test as an example that can be run
manually instead. While they will not be part (at this time?) of the
functional test suite, it may serve as a reference/example in the
future.

The approach chosen here is to have a first run of Avocado making sure that
a flag file does not exist. A second run of Avocado, now executed with an
additional `--vm-cleanup` parameter runs a test that creates the flag file.
Finally, the third and last execution of re-runs the same test as the first
run, checking that the flag file does *not* exist, that is, the clean up of
the VM did work.

Because this test requires a libvirt domain and the hostname/address
of that virtual machine, these parameters assume no defaults for
safety and/or security causes. The test will be skipped if these are
not configured properly.  Please edit the vm-cleanup.yaml file with
the appropriate parameters.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The logs from the trinity test are full of ANSI characters that add
colors to messages, which makes it difficult to read/manipulate
them.

Fix that by passing -m to trinity. From trinity --help:

   '--monochrome,-m: don't output ANSI codes'

This patch renames the fail_on_error decorator to fail_on and adds
support to supply list of exceptions. This way the decorator can be used
not only to make tests fail on generic exceptions but also to specify
parts of code with expected failures of given type.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Give to test writers the fail_on_error decorator, which
should turn any generic exception into an avocado
TestFail.

This is important for some tests that can raise
any exception during a complex operation, and any of those
exceptions can be considered a test failure.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Make sure avocado complains loudly when people try to call
skip() from outside setUp() - this shouldn't be allowed in
avocado tests. Let the user know of the problem and ask them
to fix their tests. Also, update the docstring for skip(),
which will reduce the chances of test writers using it in
inappropriate places.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

In order to keep things simple and clear, enforce the
following test results rules:

 * `PASS`: the test passed, which means all conditions
   being tested have passed.
 * `FAIL`: the test failed, which means at least one
   condition being tested has failed. Ideally, it should
   mean a problem in the software being tested has been found.
 * `ERROR`: an error happened during the test execution.
   This can happen, for example, if there's a bug in the test
   runner, in its libraries or if a resource breaks unexpectedly.
   Uncaught exceptions in the test code will also result in this
   status.
 * `SKIP`: the test runner decided a requested test should not
   be executed. This can happen, for example, due to missing
   requirements in the test environment or when there's a job
   timeout.

In order to do that, let's not make the behavior on exceptions
other than TestFail to be configurable. Therefore, the config
value uncaught_exception_result must be removed and the status
of the test always be 'ERROR'.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Summarize the test params to help people understand and run and
adjust test variants via multiplexer easily.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The gendata is a nice example of multiple-subtests test. Add
a functional test that checks gendata creates the files
correctly and that two subtests are spawned.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The runTest method has been obsoleted and everyone should be using
"test*" method(s) instead. This patch removes the support and replaces
the default and example entry points to "test".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

On unhandeled unknown exceptions the test should finish with ERROR
rather than FAIL as the test developer didn't think it might happen.

This behavior is configurable in config when the user have a different
opinion.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Introduced with commit d6abdcd4,
avocado tests no longer report correctly to the unittest runner.
The reason is that there is a structure that must be correctly
updated by the run() method (the result instance), that was not
being updated in the new implementation.

Therefore, one possible fix, presented here, is to once again
*not* implement run() at all. this way we avoid messing around
with internal data structures, and when avocado is executed
under nosetests, or the default unittest runner, we get the
expected behavior of the method. The downside is that we have
less perceived integration among both classes.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

It was previously not possible for a test to skip itself (due to say,
missing requirements) during the setUp() phase. Any exception caught
at this point was reflected as a test failure.
Signed-off-by: NCleber Rosa <crosa@redhat.com>
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Instead of playing with core exceptions or downgrading those exceptions
to the base Test API, let's allow users to do so with convenience
methods.

The example tests now make use of those, and can now remain in the
examples directory instead of the former proposal of turning them
into "functional test assets" and moving them to the selftests
directory.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This means also moving its dependencies, namely spark and gdbmi_parser,
to a place under avocado/utils. I've chosen to add another level here
to clearly identify those deps as 3rd party external modules that are
reused in Avocado.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And while doing that, also remove avocado.linux for good, since no one else
lives there.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Only avocado.Test is of general interest to test writers. For that reason
the entire test module has been moved to the avocado.core namespace.

The recommended way to go about writing instrumented tests is something like:

   from avocado import Test
   class MyTest(Test):
      def test_foo(self):
         do_stuff()

Changes from v2:
* Fixed references in docstrings
* Fixed refrences of avocado.test in Writing Test Guide docs

Changes from v1:
* Fixed comment typo on commit message
* Fixed old reference of avocado.test (then test.Test) instead of
  avocado then Test in GDB docs.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Because of the parameter behaviour change this test was failing to get
the name of the tarball to use during the test.

Once that was fixed, there was a couple of build errors. Instead of
debugging the build of the 1.4 version, I tried version 1.5, which
built without any problems. Let's use this oppotunity to fix the
test (wrapper) and update the (original test) tarball.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This patch adds documentation of the "new params API" and removes the
compatility layer. From now on only the API with paths is valid:

    self.params.get(key, path='*', default=None)

where the default path '*' matches anything from --mux-entry path.
Default path is `/test`.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>