The gendata is a nice example of multiple-subtests test. Add
a functional test that checks gendata creates the files
correctly and that two subtests are spawned.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The runTest method has been obsoleted and everyone should be using
"test*" method(s) instead. This patch removes the support and replaces
the default and example entry points to "test".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

On unhandeled unknown exceptions the test should finish with ERROR
rather than FAIL as the test developer didn't think it might happen.

This behavior is configurable in config when the user have a different
opinion.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Introduced with commit d6abdcd4,
avocado tests no longer report correctly to the unittest runner.
The reason is that there is a structure that must be correctly
updated by the run() method (the result instance), that was not
being updated in the new implementation.

Therefore, one possible fix, presented here, is to once again
*not* implement run() at all. this way we avoid messing around
with internal data structures, and when avocado is executed
under nosetests, or the default unittest runner, we get the
expected behavior of the method. The downside is that we have
less perceived integration among both classes.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

It was previously not possible for a test to skip itself (due to say,
missing requirements) during the setUp() phase. Any exception caught
at this point was reflected as a test failure.
Signed-off-by: NCleber Rosa <crosa@redhat.com>
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Instead of playing with core exceptions or downgrading those exceptions
to the base Test API, let's allow users to do so with convenience
methods.

The example tests now make use of those, and can now remain in the
examples directory instead of the former proposal of turning them
into "functional test assets" and moving them to the selftests
directory.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This means also moving its dependencies, namely spark and gdbmi_parser,
to a place under avocado/utils. I've chosen to add another level here
to clearly identify those deps as 3rd party external modules that are
reused in Avocado.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The initial idea about the avocado.restclient namespace was that
eventually users could write scripts using that as an API to
interact with avocado-server using Python code.

While the idea is still valid, the avocado-server interface is
a simple and standards compliant RESTful interface. Thus, users
can still easily interact with it without us having to support
a formal Python API at this point.

In the future, if need arises, we can improve the Python REST
client API and promote it to a public namespace.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

And while doing that, also remove avocado.linux for good, since no one else
lives there.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Only avocado.Test is of general interest to test writers. For that reason
the entire test module has been moved to the avocado.core namespace.

The recommended way to go about writing instrumented tests is something like:

   from avocado import Test
   class MyTest(Test):
      def test_foo(self):
         do_stuff()

Changes from v2:
* Fixed references in docstrings
* Fixed refrences of avocado.test in Writing Test Guide docs

Changes from v1:
* Fixed comment typo on commit message
* Fixed old reference of avocado.test (then test.Test) instead of
  avocado then Test in GDB docs.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Because of the parameter behaviour change this test was failing to get
the name of the tarball to use during the test.

Once that was fixed, there was a couple of build errors. Instead of
debugging the build of the 1.4 version, I tried version 1.5, which
built without any problems. Let's use this oppotunity to fix the
test (wrapper) and update the (original test) tarball.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When running "Avocado Virt Boot Test Run and HTML report"
the user must provide '--sysinfo on' in order to acquire system
information.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

This patch adds documentation of the "new params API" and removes the
compatility layer. From now on only the API with paths is valid:

    self.params.get(key, path='*', default=None)

where the default path '*' matches anything from --mux-entry path.
Default path is `/test`.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch modifies all example tests to use the new API. It uses the
default path, which is `*`, which matches the mux-entry paths ONLY.
In order to get the values it's necessarily to inject the YAML files
into the mux-entry path, by default `/test`, eg:

    avocado run sleeptest -m /test:examples/tests/sleeptest.py.data/
    sleeptest.yaml

It might seen as a bit more complicated to execute, but it's definitely
easier for maintaining and combining existing YAML files. To execute
matrix of tests you simply inject them into 2 different `/test` children,
eg: `-m /test/by_length:lengths.yaml /test/by_method:methods.yaml`.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch modifies the multiplexer logic to support full recursive
multiplexation, unlike the old version which only flattened the
multiplexed nodes. This allows greater flexibility.

Additionally it removes the !join flag and inverts the logic to use !mux
instead. By default leaves are not multiplexed and all are part of the
current params. Users can specify exact nodes which get multiplexed by
using !mux keyword.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Do sanity checks in Avocado sources.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Move avocado.plugins to avocado.core.plugins, in conformance to our
purpose of keeping things not related to test developers,
inside avocado.core, isolated from users.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

This reverts commit 71d47611.

The idea in the mentioned commit were sent to the upstream mailing
list and a better solution was proposed, so it makes no sense to
maintain support in Avocado for something that will never make into
GDB itself.

Export avocado.core.job.main as avocado.main, so that make it look
like an API for test developers to use.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Move avocado.job to avocado.core.job, in conformance to our
purpose of keeping things not related to test developers,
inside avocado.core, isolated from users.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

the executed test is passtest, thus the name in results should be
passtest.py and not boot.py
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

There are python traps (they can't be called scripts) which on import
modify stdin/out/err. This even on discovery makes avocado behave very
badly and it's hard to discover what and when it happened. This patch
provides dummy stdin/out/err when discovering the tests.

This solution only takes care of stdin/out/err related issues, safer
method would be to spawn another process and gather the info there.
Still we can't avoid modifying the OS during the discovery, but that's
IMO acceptable for a test-framework.

The examples/tests/failtest_ugly.py is used in basic_tests unittest
using "avocado list" (without this patch it waits for input, than it
prints the test's outputs and exits without listing the tests).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

After all, this is not a unittest of avocado per se, but
an example test. Besides, when executing it as a unittest, we
create an undesirable job directory in ~/avocado/job-results.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Multiplextest contains in /guest/os contains Linux and Windows branches.
These branches are mutually exclusive thus they need to be !join-ed.

Additionally I reworked the /host/kernel_config and split the "numa" vs.
"page_size" variants and added "default". This IMO reflects the
real-world usage better.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This is a testing version of params with paths support. By default
it acts as the old Params, but you can use params.get(key, path=....)
to utilize new-params handling.

For now the PATH for relative paths is hardcoded to "/test/*" but it
will be specifiable on the command line. I just need to pass it to the
test which I'd like to do over "metadata" (see below) rather then adding
it temporarily to the params and risk colisions.

The code is not optimized and in order to be compatible it uses old
concepts. Some of them I'd like to get rid of in near future:

* default_params => currently used only to set default "metadata" like
  test timeout. I'd like to replace it for RW Test.metadata, which would
  be overwritten durint __init__ if params contain the keys in specific
  paths. IMO they shouldn't be shared with test params.
* test_factory's params are currently dict with some metadata like
  wether this test is executed from directory or by uri. Again, they are
  metadata and should be handled separately. Some of them might be reused
  to params if needed, but not generaly.
* create separated multiplexer plugin generate multiple variants
* reconsider the need for "objects", "object_params" and "object_counts"
* couple of others I forget to mention...

There is one real change I made, I got rid of ${key}_${type} to specify
type of the argument. Yaml supports any type we might like and by
our convention all tests should be written to work without any params.
Thus this is IMO more confusing, than beneficial. But if you insist
I can copy&paste the support for it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We decided to use params.get(key, default) rather than flat dictionary
in test header. This way should be easier to read to code without the
need to look up and also use different defaults for different namespaces
(or even code branches).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We're going one step ahead to unittest  compatibility.
Now we obsolete action() method and use runTest().
Then what is runTest() will be run().

Update the selftests and tests regarding this modification.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Use -e to exit on first failure.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch is initial support for people using custom bash scripts
with avocado. Tests should use:

    PATH=$(avocado "exec-path"):$PATH

and then they can utilize the helpers.

This version contain functions to write to Test.log the same way it's
possible from python including failing the test with TestWarn in case
avocado_warn was used.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This adds more tests to the release test plan, adds more precise descriptions
of the test and expected results.

Also, the test plan follows a sequence of steps that is more concise and eases
the manual testing process.

Changes from v2:
* Added Remote Machine HTML links test

Changes from v1:
* Added HTML sysinfo test
* Added HTML external links test
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The NastyException error is back:

class NastyException(Exception):
    def __init__(self, msg):
        self.msg = msg
    def __str__(self):
        return self.msg

crashes Avocado. The problem is we are trying to re-raise the exceptions
and then read the traceback again. Even reading of the details of this
very nasty written exception causes critical failure.

What this patch does instead of handling issues wherever we produce
them, it waits till the exception bubbles to the entry-point and it
tries to read it. If it fails, we log the original traceback
(information about the failure) and then user-friendly message saying
we're unable to retrieve the original failure.

Additionally I had to modify the xunit output as in case of very nasty
inputs it's unable to reproduce the values.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Change behavior to expect the test to pass, by using test asserts.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

avocado-server has been given the functionality to list the APIs
that it serves, and this code adds a simple connection method and
an example script to use that.

This should allow clients to dynamically adjust to available APIs.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

To actually perform any online task, one needs a valid connection
to an avocado-server. This introduces the connection module, along
with API examples on how to use them.

This functionality is configurable by default using the standard
avocado configuration file, so reasonable connection options are
set.

Changes from v1:
 * Be more specific with exceptions during connection initialization
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This patch simplifies the concept of WARN test. In case user uses the
self.log.warn or self.log.warning methods, test is marked as dirty and
in case everything else worked fine it finishes with TestWarn exception.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This example demonstrate simple usage of GDB API.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Add new test `raise.py` to raise signals to itself and check results.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Add new test `cabort.py` to call abort() from lib C.
Signed-off-by: NRudá Moura <rmoura@redhat.com>