At this point, no plugin support exists in Avocado. This is in
preparation for the new plugin code to be cleanly introduced.

The plugins that play a role in the plugin architecture have
been removed, since they wouldn't be functional under the new
plugin management code. The ones that are add extra functionality
to Avocado have been kept, and will be ported to the new
architecture.

Also, most of the functional tests have been temporarily disabled.
The reason is that most of them run avocado, which depends on the
run command, which in turn, depends on the plugin archicture code.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This fixes the recently introduced commit 5f7bafd0, running the skip
function instead of passing the function itself to `unittest.skipIf`.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `test_gendata` test of `OutputTest` runs the example test
`gendata.py`, but the `gendata.py` itself has a conditional in one of
the tests that make use of the PIL (aka Pillow package) library and
does nothing if it's missing.

PIL (Pillow) is highly dependent on C compilers and libraries to be
able to build itself. This makes systems it harder to run the full
Avocado self tests on more diverse (usually minimalist) systems.

So, let's make that functional test optional, if the libraries are not
present.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There are a couple of functional unittests that assume that a given
output was produced by the regular UI, that is, the Human output
format.

That is fine, since they're functional tests. But, some situations
lead to test errors, instead of pointing failures. Let's turn those
into explicit checks and fail properly instead of producing errors.

Note: These errors were visible when the human output was disabled, in
the plugin overhaul work, but the fixes still make sense independently
of that work.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since we define the meaning of each exit code, we should be using that
ourselves.

Functional tests have been coded using literal values, and there may
be a good reason for that: proving that the API stands.  But there's a
big issue: we have not declared those codes as some kind of an
external API and guarantee stability. Also the subsequent changes in
this series actually change the meaning and values of those.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Make sure we have avocado_ + __name__ prefixes to the dirs
created with tempfile, so that we have an easy way to spot
when we are not cleaning up directories properly. This mostly
concerns unittests, although the kernel_build lib and the
iso9660 lib also uses that API.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Even though I may be the one to blame about coming up with this
"simple import magic", I believe it was a mistake and we should get
rid of them.

There are a couple of other ways to actually do development out of
of a source tree, including running unittests that do not require
this amount of boiler plate code. Examples include just setting
the PYTHONPATH environment variable to actually run (setuptools based)
`python setup.py develop`.

The little bits of what looks like the import magic that was left, is
not really import magic. It's just that functional tests need to locate
the Avocado source tree base directory to run the test runner from it.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

To be honest, our test code could still be kept in the same directories and
have the same names. But I think we can improve two things here:

1) Flatten a little bit the directory structure of selftests. Two path
components are being dropped here: "all" and "avocado". So that
"selftests/all/functional/avocado" becomes simply
"selftests/functional".

2) File names match what is, by default, recognized by unittest (the
Standard Library module) based discovery of tests. That means that
doc_build_test.py becomes test_doc_build.py. Not a big deal IMHO.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

On multiple places we modify the sys.path to make avocado work from
sources. Instead of `append` we should `insert` the path as when running
from avocado sources directory, we want to use the modules from here,
rather than from installed version.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The gendata is a nice example of multiple-subtests test. Add
a functional test that checks gendata creates the files
correctly and that two subtests are spawned.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The runTest method has been obsoleted and everyone should be using
"test*" method(s) instead. This patch removes the support and replaces
the default and example entry points to "test".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The job ID is an important information that should've
been printed to our job log. This patch fixes it and
also adds a functional verification test.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

This one uses --vm, which is redundant and also uses
the vm functionality, one thing we still can't do in
Travis.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NRudá Moura <rmoura@redhat.com>

Even though we have changed functional tests to use passtest instead of
sleeptest, there still has 3 more tests can go with passtest.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Change behavior to expect the test to pass, by using test asserts.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

vm plugin only extends the remote plugin and shares the test execution
part. This patch inherits vm plugin from remote one and adds only
the setup/cleanup part.

Additionally it removes the --vm argument and enables this plugin when
--vm-domain is set.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

* Use --sysinfo=off in the current functional tests. It will
drop the total execution time of the tests.

* Add a new functional test `sysinfo_tests.py` which will actually
test if the sysinfo directory is being created or not, depending on
the proper command line option.

Self test files used to be executable, but that changed on 143b574c.
So, it doesn't make much sense to keep the shebangs + python commands
on them.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Verify if we are not printing terminal codes inside
redirected outputs, when we in fact shouldn't.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Remove the license text from files inside docs, examples and selftest,
to keep the license text only in relevant parts of the project.

References:

* https://fedoraproject.org/wiki/Packaging:LicensingGuidelines
* https://www.gnu.org/licenses/gpl-howto.htmlSigned-off-by: NRudá Moura <rmoura@redhat.com>

This patch replaces most of the calls to sleeptest in our functional
test with passtest, in an attempt to reduce, without invalidating the
purporse, the time it takes to run them.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Functional verification of the HTML output plugin functionality.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add a helper method on TestResultProxy() to get information
of output plugins using stdout, making up for a cleaner
way of letting the user know about the conflict.

Also, rephrase the error messages and update functional
tests to reflect the new status quo.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

That was a wrong behavior of our application. When providing
a command line such as:

scripts/avocado run sleeptest --json /tmp/json

This command should display human output. This patch fixes this
issue, and corrects the functional test for this behavior.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Fix functional tests regarding the changes that moves
options from the main application to the test runner (avocado run).
Signed-off-by: NRuda Moura <rmoura@redhat.com>

Make some changes to bring the concept of job ID as a
user visible information:

* Make the human output to display the job ID
* Make the json output to have the job ID
* Record the job id in a file inside the results dir

Adapt functional tests to that new schema.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Simplify commands for xUnit and JSON output plugins:

* Remove options --xunit-output and --json-output.
* Use '-' to stdout or filename to options --xunit and --json.

Usage examples:

$ avocado --xunit - run sleeptest  # goes to stdout
$ avocado --xunit /tmp/foo.xml run sleeptest
$ avocado --xunit - --json /tmp/foo.json run sleeptest.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

As I realized we did not have the journal plugin out here.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add extra tests that verify that the always on output
plugins (JSON and Xunit inside the debug dir) are being
correctly generated and the outputs are valid.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

It's necessary if we're generating valid outputs in
the appropriate places, and that they respect the
rules established so far.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>