Incremental smoke-check

This target checks very basic function of avocado and is used
incrementally in travis to verify this functionality for each
commit up to the origin/master.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Makefile: move the removal of avocado directories to the clean target

The check target have a couple of (relevant) directory cleaning up
commands that are, IMHO, out of place. Let's move them to the clean
target.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Last time it did not work, let's try it again to see whether it works.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado/core/settings.py: simple optimization

Reuse the value of "_config_path_intree" to set "_config_path_intree_extra".
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Test result small improvements

Avocado defines avocado.core.result.TestResult, so that alternative
test result classes can inherit from it. When a user writes a new
test result class, he's left wondering how to activate that.

First, for consistency, Avocado should define proper interfaces
for "test result plugins". This is on sight, and will ultimately
allow users to register new test result implementations simply by
registering the setuptools entry points, just as it's currently
done (on versions >= 0.31.0) for command line related plugins.

Until then, let's at least improve the current state of things.
These changes believe that the following improvements are given here:

* avocado.core.job.Job._set_output_plugins now won't look at every
  possible key in the application arguments, and won't silence
  possible errors while trying to instantiate test result classes.
* test result writers can now use a documented method for registering
  their test result classes.

The same approach was not chosen to be done with the test runner
classes because there can only by one test runner per Avocado job.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This is really intended to be the same for all instances of a given
class. Every other (xunit, json, journal, html) result class already
does it like that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The code for executing binary is the same for booth, let's require
self._command to contain the binary and merge the code to avoid future
diversion.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado.core.runner: Force methodName on replay_map substitution

avocado.utils.software_manager: Bugfixes and cleanups

Test namespace cleanup [3rd round]

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

tempfile.mktemp is insecure. Let's use NamedTemporaryFile instead and
leave the cleanup on failure on python.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The cfgparser converts arguments internally. If we supply integer,
it raises TypeError when it tires to read it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

There is an issue when running SIMPLE tests with non absolute
paths, example:

 $ echo "#!/bin/bash" > /tmp/test.sh
 $ echo "exit 0" >> /tmp/test.sh
 $ chmod +x /tmp/test.sh
 $ cd /tmp
 $ avocado run test.sh

The previous work on the simple test runner, adding the filename
property, actually fixes that bug, so let's add a functional test to
avoid an regression.

Reference: https://trello.com/c/nbhyILO0Signed-off-by: NCleber Rosa <crosa@redhat.com>

Run commands that require admin privileges with 'sudo' [2]

Test and SimpleTest now have a filename property, so no need to
keep the same information in the path attribute.

Also, path is being abused by ExternalRunnerTest. Let's implement
a proper filename attribute (one that returns None, since it may not
be backed by a file), and use a custom attribute to hold the command
name that will be executed.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When there's no filename (test not backed by a file), there's no
basedir and then also no expected stdout/stderr files.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

On 639ae27c, basedir is implemented with an explicit return. Let's
do the same here for consistency.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The "replay_map" allows to substitute the test class with custom class.
The problem is, that the "methodName" defined in "test_params" might not
exist in this class.

We have two choices, either leave it up to the user, to substitute with
a compatible class (eg. by creating the "methodName" in "__init__"
before calling the super.__init__), or we can require them to always use
given "methodName". This patch uses the second approach as it seems less
magical to me.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

selftests/functional/test_replay.py: fix typo that caused sysinfo col…

I noticed during a run of the self test suite that system information
was being collected (`rpm -qa` and the like can usually be seen using
quite a few resources).

It looks like this little typo was causing that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This reverts commit 732a400a.

avocado.main(): avoid an infinite fork loop bomb [v4]

Because the current implementation of avocado.main() creates a job
and runs "sys.argv[0]" to implement standalone mode, it ends up
running itself over and over.

This simple proposed fix, prevents avocado.main() from running
itself again if called from itself. Since they are on different
processes, the mechanism chosen to do this is to set an environment
variable, that will be seen by the next process.

Also, by exiting from main() with an error code, the test first
level test will fail. This will let the user know that the chosen
approach (SIMPLE tests written in Python and calling main()) are
not worthy of a PASS.

The functional tests make use of Python's standard library utilities
(subprocess module) directly for running Avocado because of current
issues with Avocado's own process utility module.

This adresses issue #961.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Test namespace cleanup [2nd round] [v2]

IMHO it makes more sense to only publish these variables when they
actually contain meaningful value. Right now it *may* contain "None"
as this is maybe the value of the `basedir` and `datadir` attributes
when the test is not backed by a file.

Also, it's a lot more common, say, for a shell script to check if
a variable is set (if [ -n $AVOCADO_TEST_DATA_DIR ] ...) then to
check if it's not equal to "None".
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Avocado doesn't intent to support the use of these attributes by
test (writers), so let's at the very least hide them.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The implementation of basedir from Avocado Test is also sufficient for
SimpleTest now that the `filename` property is implemented.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

SIMPLE tests are backed by files, executable ones. So, it makes sense
to have the `filename` property accordingly. The base Avocado Test
class implementation of `basedir` applies to SimpleTest as well
with this implementation.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since basedir is closely related to `filename` and `datadir`, let's
also make it a property for coherency.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Instead of assuming we have administrative privileges to
run software manager related commands, use sudo in commands
that do require root. We now assume you're running the
test as a user that has sudo configuration such that no
password will be prompted to execute that command [1].

A place where we need such a provision is when running on
Amazon AWS instances (we can't run as root, but the base
user of the image has the appropriate sudo configuration).

[1] Running as root means that sudo won't be necessary,
but figuring that out is responsibility of the process
APIs, that were modified to take a sudo parameter and
figure things out based on that.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

Instead of assuming we have administrative privileges to
run service manager related commands, use sudo in commands
that do require root. We now assume you're running the
test as a user that has sudo configuration such that no
password will be prompted to execute that command [1].

A place where we need such a provision is when running on
Amazon AWS instances (we can't run as root, but the base
user of the image has the appropriate sudo configuration).

[1] Running as root means that sudo won't be necessary,
but figuring that out is responsibility of the process
APIs, that were modified to take a sudo parameter and
figure things out based on that.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>