Signed-off-by: NCleber Rosa <crosa@redhat.com>

 avocado.test: Add support for params with paths [v4]

Multiplextest contains in /guest/os contains Linux and Windows branches.
These branches are mutually exclusive thus they need to be !join-ed.

Additionally I reworked the /host/kernel_config and split the "numa" vs.
"page_size" variants and added "default". This IMO reflects the
real-world usage better.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This is a testing version of params with paths support. By default
it acts as the old Params, but you can use params.get(key, path=....)
to utilize new-params handling.

For now the PATH for relative paths is hardcoded to "/test/*" but it
will be specifiable on the command line. I just need to pass it to the
test which I'd like to do over "metadata" (see below) rather then adding
it temporarily to the params and risk colisions.

The code is not optimized and in order to be compatible it uses old
concepts. Some of them I'd like to get rid of in near future:

* default_params => currently used only to set default "metadata" like
  test timeout. I'd like to replace it for RW Test.metadata, which would
  be overwritten durint __init__ if params contain the keys in specific
  paths. IMO they shouldn't be shared with test params.
* test_factory's params are currently dict with some metadata like
  wether this test is executed from directory or by uri. Again, they are
  metadata and should be handled separately. Some of them might be reused
  to params if needed, but not generaly.
* create separated multiplexer plugin generate multiple variants
* reconsider the need for "objects", "object_params" and "object_counts"
* couple of others I forget to mention...

There is one real change I made, I got rid of ${key}_${type} to specify
type of the argument. Yaml supports any type we might like and by
our convention all tests should be written to work without any params.
Thus this is IMO more confusing, than beneficial. But if you insist
I can copy&paste the support for it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We decided to use params.get(key, default) rather than flat dictionary
in test header. This way should be easier to read to code without the
need to look up and also use different defaults for different namespaces
(or even code branches).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Support for multiple tests [V3]

The code seems to be dead (ineffective) at this part, since argparse
already checks previously if the url argument is passed.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

It's now possible to select test methods to execute, in
the multiple tests context, by using a special form.

When you select `filename.py:test_method_pattern` it will filter out
every test methods and use just the test_method_pattern passed.
The pattern is a shell like feature, so `test_connection*` will select
every test method that starts with `test_connection` to execute.

Example:

$ avocado run examples/tests/gdbtest.py:\*connect*\

will run:

(1/3) gdbtest.py:GdbTest.test_connect_multiple_clients: PASS (2.41 s)
(2/3) gdbtest.py:GdbTest.test_disconnect: PASS (2.52 s)
(3/3) gdbtest.py:GdbTest.test_disconnect_raw: PASS (4.52 s)

and

$ avocado run examples/tests/gdbtest.py:test_connect_multiple_clients

will test just:

(1/1) gdbtest.py:GdbTest.test_connect_multiple_clients: PASS (2.34 s)

Note that we've scape the test method pattern to avoid shell expansion!
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Remove words for setup, cleanup and action and update to the new slang:
setUp, tearDown and runTest.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

When a class has multiple tests, display each test.

Example:

INSTRUMENTED examples/tests/gdbtest.py:GdbTest.test_connect_multiple_clients
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Set the datadir for test based upon the filename of the class,
so that multiple test methods in a same test class will share
the same datadir directory.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Introduce multiple test methods defined in Avocado Test class.
The tests looks like the one of unittest module.

The test methods are defined when you create test<something> methods,
inside the test class. Each test methods becomes a test case unit
to execute.

Example:

class MyTest(avocado.test.Test):
    def test_eone(self)...
    def test_two(self)...
    def testAnotherone(self)...
Signed-off-by: NRudá Moura <rmoura@redhat.com>

We're going one step ahead to unittest  compatibility.
Now we obsolete action() method and use runTest().
Then what is runTest() will be run().

Update the selftests and tests regarding this modification.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

avocado.runner: Fortify early_state.get(timeout)

We shouldn't heavily rely on early_state. This patch tries to get
value from early_state and when it's 0/None uses DEFAULT_TIMEOUT.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

When running 'run --help' piped through a non existing command,
exit gracefully with a shell error and don't print the whole trace
back message.

Reference: https://trello.com/c/Nm4elZrb/363-bug-avocado-should-fail-gracefully-when-redirected-to-a-broken-pipeSigned-off-by: NRudá Moura <rmoura@redhat.com>

examples.tests.simplewarning: Use -e to exit on failure

In tree execution worked fine in virtual environment, but not without
it. This commit tries to iterate through PATH and detect the in-tree
libexec path.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We need to execute avocado in simpletests, thus we need both, libexec
and scripts directories inside PATH.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Use -e to exit on first failure.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Run unittests during the build of RPM packages [v2]

It seems a good idea, and another safety net, to run the selftests
during the build of packages. For now, it's not trivial to run the
functional tests, so let's at least run the unittests.

Another current limitation is that unittests are only run on Fedora
and not on EL6 or EL7, and the reason for that are missing
dependencies on both EPEL6 and EPEL7.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The basic idea here is to put all common requirements on a common line
and deal with the differences in conditional blocks.

Also, python-flexmock is not available on EPEL 7, but available on all
other supported platforms.

Finally, python-yaml and PyYAML are in their own lines because it
makes it easier to see that they're the same package with different
names.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This means that we include the Makefile in the source tarball and just
use the `man` target.

With regards to portability among Python version, this should not be
an issue as rst2man on the system itself will the correct version. In
a sense, it's just another utility, it doesn't really matter it's
written in Python.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There are two reasons for introducing this new target:

1) It adds symmetry to the Debian targets, which have both source only
   and complete versions.

2) It is useful when you only want to make the source package and
   build the "binary" elsewhere.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

avocado.remote: Add support for host-timeout [v2]

Sometimes avocado on guest hangs. This patch adds host timeout to
interrupt the execution.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

To help users to understand what each config option does.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Those are not necessary in the avocado options and can
safely stay as a config file only option.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The option --relative links instructed the htmlresult plugin
whether to use absolute log paths in anchor tags, or to use
relative paths, something that dates back to the old autotest
html job report. After thinking about it I realized that this
option doesn't need to be here, as using relative links is
basically better for all intents and purposes (the job results
can be safely redistributed and the report would still work).

So get rid of that option and always use relative links for
the job results dir links and logs. Tested with local execution
as well as remote/vm execution.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Include virt API on readthedocs.org

This patch will checkout the avocado-virt repo during the documentation
build, so that all of avocado's API (including -virt) is presented at a
single and comprehensive location.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The following things were changed and for the following reasons:

1) Remove the current hack to force the program name at the module
entry point, for obvious reasons.

2) Return the exit status code, for obvious reasons.

3) Add some safety with a `__main__` check, because Sphinx will
load the avocado module in such a way that the avocado app is run
and the build fails. This is true on my system, and may be the case
on others.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

avocado.aexpect: Fix bugs that broke avocado-virt

Let's try to avoid mistakes by explaining that we should not
use avocado imports in the stand alone part of the aexpect
module.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

We can't require avocado imports in the server part of
the aexpect process because that will be executed
stand alone, and will not have access to the rest
of the avocado libraries. The bugs were introduced
in commits:

6f380e0d
3fa9077f

This patch fixes the problem by moving the imports
to the client side of the module.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>