At this point, no plugin support exists in Avocado. This is in
preparation for the new plugin code to be cleanly introduced.

The plugins that play a role in the plugin architecture have
been removed, since they wouldn't be functional under the new
plugin management code. The ones that are add extra functionality
to Avocado have been kept, and will be ported to the new
architecture.

Also, most of the functional tests have been temporarily disabled.
The reason is that most of them run avocado, which depends on the
run command, which in turn, depends on the plugin archicture code.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since we define the meaning of each exit code, we should be using that
ourselves.

Functional tests have been coded using literal values, and there may
be a good reason for that: proving that the API stands.  But there's a
big issue: we have not declared those codes as some kind of an
external API and guarantee stability. Also the subsequent changes in
this series actually change the meaning and values of those.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Make sure we have avocado_ + __name__ prefixes to the dirs
created with tempfile, so that we have an easy way to spot
when we are not cleaning up directories properly. This mostly
concerns unittests, although the kernel_build lib and the
iso9660 lib also uses that API.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Even though I may be the one to blame about coming up with this
"simple import magic", I believe it was a mistake and we should get
rid of them.

There are a couple of other ways to actually do development out of
of a source tree, including running unittests that do not require
this amount of boiler plate code. Examples include just setting
the PYTHONPATH environment variable to actually run (setuptools based)
`python setup.py develop`.

The little bits of what looks like the import magic that was left, is
not really import magic. It's just that functional tests need to locate
the Avocado source tree base directory to run the test runner from it.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

To be honest, our test code could still be kept in the same directories and
have the same names. But I think we can improve two things here:

1) Flatten a little bit the directory structure of selftests. Two path
components are being dropped here: "all" and "avocado". So that
"selftests/all/functional/avocado" becomes simply
"selftests/functional".

2) File names match what is, by default, recognized by unittest (the
Standard Library module) based discovery of tests. That means that
doc_build_test.py becomes test_doc_build.py. Not a big deal IMHO.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

On multiple places we modify the sys.path to make avocado work from
sources. Instead of `append` we should `insert` the path as when running
from avocado sources directory, we want to use the modules from here,
rather than from installed version.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

With the new information that is going to be added to
the job log, keeping up with the number of lines of that
log is a fool's errand at this point. Let's drop the checks.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The current implementation iterates through the avaliable key/values
and displays only the last value per unique key. This patch uses
the value origin and displays the "origin:key => value" for all
unique available records per variant.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch changes the default mux-entry to `/run` as it tends to be
more convenient.

The biggest change is that this location is used as default location
for added YAML files. It's also used as the base when using relative
path YAML file injection. The only way to include file into the exact
location is to use absolute path (`/your/location`).

The benefit is that single simple YAML file doesn't require any additional
location and is in the default mux-entry location. Check the documentation
for details.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This patch adds support to specify where to inject the yaml file,
meaning it extends the yaml file's location in the same fashion as
!using on top of the file does (combination of both at the same time
is possible). You can do that by specifying the path on the cmdline
"-m $location:$yaml_file".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

This is a testing version of params with paths support. By default
it acts as the old Params, but you can use params.get(key, path=....)
to utilize new-params handling.

For now the PATH for relative paths is hardcoded to "/test/*" but it
will be specifiable on the command line. I just need to pass it to the
test which I'd like to do over "metadata" (see below) rather then adding
it temporarily to the params and risk colisions.

The code is not optimized and in order to be compatible it uses old
concepts. Some of them I'd like to get rid of in near future:

* default_params => currently used only to set default "metadata" like
  test timeout. I'd like to replace it for RW Test.metadata, which would
  be overwritten durint __init__ if params contain the keys in specific
  paths. IMO they shouldn't be shared with test params.
* test_factory's params are currently dict with some metadata like
  wether this test is executed from directory or by uri. Again, they are
  metadata and should be handled separately. Some of them might be reused
  to params if needed, but not generaly.
* create separated multiplexer plugin generate multiple variants
* reconsider the need for "objects", "object_params" and "object_counts"
* couple of others I forget to mention...

There is one real change I made, I got rid of ${key}_${type} to specify
type of the argument. Yaml supports any type we might like and by
our convention all tests should be written to work without any params.
Thus this is IMO more confusing, than beneficial. But if you insist
I can copy&paste the support for it.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

It is more natural [1] that output of --show-job-log goes to
stdout than stderr. Let's change that behavior and update
unittests accordingly.

[1] Natural in this sense is defined by the user's expectation -
    when setting avocado to --show-job-log, you expect that the
    job log *is* in the application's stdout.

Changes from v1:
 * Python 2.6 constructor param for StreamHandler is strm,
   that was changed in 2.7 to be stream. Let's keep compatibility
   by using positional arguments.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Fix regression when the number of lines from the output was reduced,
so we're now assuming 13 lines, instead of 14.

Fix issue #472.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

Also, change the code a little to tell the difference
between stdout and stderr.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The new pep8 adds W503 line break before %s check. This is mass style
update of avocado framework without any actual changes.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Sysinfo module now allows to define profilers, which are programs
defined in command line, to start as a background processes and log
the output in a proper directory. Example: vmstat 1.

The profilers are started before a job execution and then, stopped at
the end of the job. The output is stored inside `sysinfo/profile/*`
subdirectory, in job result directory.

The profilers can be customized in `avocado.conf`, section
`sysinfo.collect`.  The default configuration define
profilers as the following: `journalctl -f` and `vmstat 1`.

This feature is turned off by default.
Signed-off-by: NRudá Moura <rmoura@redhat.com>

* Use --sysinfo=off in the current functional tests. It will
drop the total execution time of the tests.

* Add a new functional test `sysinfo_tests.py` which will actually
test if the sysinfo directory is being created or not, depending on
the proper command line option.

This complex patchset adds debug version of yaml parser. It stores
the origin of each value from environment. For list it stores
per-slice origin too.

Debug mode is NOT suitable for run-ner, only for multiplex-er. It
corrupts the values by adding the origin after the value.

The unittests were adjusted and enhanced. I added explanation into the
examples/mux-selftests.yaml describing what the weird names stands for
and what they test. I added couple of check which stress the old tests
and couple of them test the new features.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Instead of adjusting existing tests I rewrote them from scratch with
parallel execution in mind.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Replace test_run_mplex_timeout with test_run_mplex_params test which
executes 'env_variables.sh' and checks that output contain correct
output. This test checks the multiplexer capability to change variable
value.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

multiplex_test uses sleeptest to check it supplies the values correctly.
This patch uses passtest instead.

For now it uses the same multiplex file and in `test_run_mplex_timeout`
still uses sleeptest, but there is FIXME message to use envtest once
it supports params passing (another pull request)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Remove the license text from files inside docs, examples and selftest,
to keep the license text only in relevant parts of the project.

References:

* https://fedoraproject.org/wiki/Packaging:LicensingGuidelines
* https://www.gnu.org/licenses/gpl-howto.htmlSigned-off-by: NRudá Moura <rmoura@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

We have tests/ and selftests/ in avocado (the test runner).
The former includes examples of tests that make use of the
avocado API, and are also used to test avocado via CI.

This is confusing, as it doesn't make it clear to users that
tests are not part of the avocado framework. Since we have a
directory called examples with some other files (multiplexer,
plugins), move the tests to there (and keep using them in the CI).
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

The parser for the subcommand run now accepts one or more
tests ids (paths / aliases) and there isn't a need to quote the list
as a string.

Example:

* Before: avocado 'sleeptest failtest /usr/bin/true'
* Now: avocado sleeptest failtest /usr/bin/true

So it's now possible to run avocado like `avocado run *.py`.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

That change has to be made all at once for the sake of
bisectability. Update avocado.job and the multiplexer
plugin with the new multiplexing library code.

Also, update functional tests, the avocado sample tests
with the new YAML based config files, and the requirements.txt
file with the new prominent dependency of avocado, PyYAML.
Signed-off-by: NRuda Moura <rmoura@redhat.com>

To avoid confusion, when passing an invalid test ID
(typo, incorrect path, alias that could not be resolved),
introduce a new status, NOT_FOUND inside avocado, as
well as proper handling handling for such situations.

$ scripts/avocado run sbrubles
JOB ID    : 312b690173a8f5f261d9c5cefdb8699ac0798ded
JOB LOG   : /home/lmr/avocado/job-results/job-2014-08-14T10.02-312b690/job.log
TESTS     : 1
(1/1) sbrubles: NOT_FOUND (0.00 s)
PASS      : 0
ERROR     : 0
FAIL      : 0
SKIP      : 0
WARN      : 0
NOT FOUND : 1
TIME      : 0.00 s

Also, add a functional test for this new status.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Instead of using 'debug log' we can use 'job log', as it
is a human readable log of the job execution.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Make avocado tests to refer to their test ids (paths),
relative to the directory you are in. For example,
running:

avocado run tests/sleeptest.py

will give you

DEBUG LOG: /home/lmr/avocado/logs/run-2014-07-31-12.54.01/debug.log
TOTAL TESTS: 1
(1/1) tests/synctest.py.1: PASS (1.93 s)

Test log and data directories were changed accordingly. Now,
if your test is synctest.py, your data dir will be called
synctest.py.data.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Add tests to verify if passing the full test path
works, rather than the test ID, and if an empty
test ID/alias triggers the correct error code.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Make a reasonable effort in checking the amount of lines
avocado outputs to the job log. If it is less than a certain
amount, we might be having output problems.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>

In order for people to run all avocado tests at once,
move and reorganize tests under a directory called
selftests. For someone to run all self tests in
avocado, one would have to execute:

    selftests/run selftests/all

This is a shortcut for developers. .travis.yml
was also updated to run tests based on the new
directory structure.
Signed-off-by: NLucas Meneghel Rodrigues <lmr@redhat.com>