This script introduces a pair of directories that will be searched
for executable files, to be run before and after the actual execution
of a job.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

These new plugin interfaces, to be registered under the
'avocado.plugins.job.prepost' namespace, allow for any custom action
to be executed before and/or after the job execution.

The JobPre plugin interface requires the `pre` method to be
implemented, while the JobPost plugin interface requires the `post`
method to be implemented.  The separation of the interfaces allows for
mandatory implementation of the given interfaces, which was the whole
point of using Abstract base classes, instead of having empty default
implementations.

One single class may implement either one, or both of those
interfaces.  Both `pre` and `post` methods receive the current Job
instance as an argument, which allows the plugin to fetch any job
information or even to tweak as it may seem fit.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Since one of the main goals of the Job is to run tests, and given the
initial use of the Job class internals, now by plugins and in the near
future as a full blown API, let's make the test suite public.

This will help plugins to take a look at what is going to be
(attempted to be) run, and react to that.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

When running the per-commit checks on CI, the previous environment is
not as clean as it should be.

One example is when a given work introduces a new plugin. First the
HEAD of the branch is tested, which includes the installation of the
new plugin.  Then, the per-commit checks would return to the first
commit in the branch, which doesn't contain the plugin, but the
environment still knows about it.  This is a matter of forcing the
clean up of setuptools entrypoints (or any setuptools generated info
for that matter).

While adding the deeper cleaning, it seemed useful to split the
Makefile "link" target into "develop", which runs "python setup.py
develop" on the Avocado source tree alone.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This commit implements the serialized test ids described in the
Introduce proper test IDs RFC.

https://www.redhat.com/archives/avocado-devel/2016-March/msg00024.html

It implements `TestName` class, which contains the test uid, test name
and the variant uid and allows querying for the file-system-friendly
name.

The workflow is:

1. tests are discovered, name is translated to "Test Name" by loader
2. test_suite is passed to the runner (new) along with the number of
   tests+variants to be executed (used to get number of digits)
3. the Mux (params generator) yields the template + (new) variant id
4. the runner replaces template['name'] to TestName(uid, test_name,
   variant_id); where uid is currently no executed tests, test_name is
   the original name from Loader and variant_id is either None or the
   variant index.

This commit makes the `tag` argument unused. To avoid problems a warning
is issued on it's usage so we can remove it in the upcoming releases.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

When using a system V init service manager to check an unavailable
package, the error messages are sent via stderr, not stdout.
Signed-off-by: NHao Liu <hliu@redhat.com>

Fabric is not thread safe, but we can get away with executing
threads that don't have a lot of concurrency (that is, not a
lot of commands are happening at the same time). Let's try to
ensure the new command is being executed with proper
authentication parameters.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

The static methods can be turned into module level functions,
in particular the command running code, that can be reused
by any code that updates fabric environment info accordingly,
so let's do it.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

This parameter is not really necessary, as we can simply
decide at runtime if we want the operation to be quiet
or not, and have method default values to establish default
behavior.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

Since we have a fairly limited used of fabric in the vm
and remote plugins (one single host per test), we did not
notice an important limitation of the remoter code: It does
not support multiple instances of it, due to the fabric env
singleton nature. Having to connect to more than one system
with different authentication settings will cause failures
due to the fact that fabric was using old authentication
settings.

So let's use fabric execute function to make safer run
and send/receive file methods.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

We can just call fabric.api.env.update() instead.
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

Add support for identity files as valid SSH options
to our remote plugin. The use case for this is connecting
to Amazon EC2 instances using the provided keypair (.pem
files).

avocado run --remote-hostname my-machine.com --remote-username myuser --remote-key-file /path/to/myfile.pem passtest
Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

Signed-off-by: NLucas Meneghel Rodrigues <lookkas@gmail.com>

examples/testplans/release.json: fix vt_list plugin name

We use `False` in avocado.conf and remoter should have the same default.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The slides links are broken.  Let's remove them, and eventually re-add
them when they are put back in place.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It's `vt_list` and not `vt_lister`, so let's fix it.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The slides links are broken.  Let's remove them, and eventually re-add
them when they are put back in place.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

when the user provides --remote-password and the host is not found
in known_hosts, it raises fabric.network.NetworkError. We catch this
exception in remoter as usually it means the connection timed out
and if we have enough time we try it again.

This patch checks if 'not found in known_hosts' is present in the
exception message breaks the loop to retry, re-raising the exception.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

avocado.core.remoter option to reject remote unknown hosts

avocado: Rename the `args.default_multiplex_tree`

When Avocado runs tests on remote machines it always accepts the
host SSH key fingerprint. This could allow credentials to be stolen
if the remote host name or address is spoofed.

This patch adds to Avocado the configuration option to enable/disable
the reject_unknown_hosts option, so user can choose between safe or
flexible.

Notice there is an issue in Paramiko where it does not recognizes
ecdsa keys in known_hosts file. So for reject_unknown_hosts to work,
we currently have to accept the remote host key fingerprint in RSA format.
To do so, the SSH command shouled be like below:

  $ ssh -oHostKeyAlgorithms='ssh-rsa' <host>

Reference: https://trello.com/c/oqyAvZBqSigned-off-by: NAmador Pahim <apahim@redhat.com>

When the total number of tests is lower than number of executed tests,
avocado reports incorrect number of tests. Let's increase the number of
tests in such case.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This selftest makes sure test statuses are assessed correctly on the
test end inside jsonresult (which also verifies TestResult handling).

Additionally it makes sure jsonresult handles unknown statuses and
missing values in the status correctly.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The TestResult baseclass keeps the count of executed tests and
additionally on test end it calls function associated with the finish
status. This is not used by any plugin as they usually don't care. The
only one which could benefit from it is xunit, but it uses simplified
version. I think it's time to say good bye to this and simplify the
TestResult class (along with minor speedup)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Add test as "ERROR" when the status is unknown.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The time related variables cause troubles, when the test is interrupted.
Let's initialize them to 0 so they can be used in result postprocessing
if the test is interrupted between early_status and finish.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

TestResult depends on the test status containing all information. The
problem is that on interruption the status might not be complete. Let's
provide understandable defaults for such cases.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Currently all classes inherited from TestResult are storing the full
state per each executed test, although there is actually none using it.
Those who need additional information store them again in their
variables. Let's avoid wasting of memory and only store the counts
instead.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `args.default_multiplex_tree` stands for default avocado params,
which currently supports only `multiplexer` implementation. This might
change in the future, but this variable could be used by other backends
too. So let's rename it now to `default_avocado_params`.

Additionally it's only set when the system is multiplexer-enabled, but
it has nothing to do with the multiplexer (directly). Multiplexer is
only one of the sources of params. So let's initialize it always. This
avoids a bug in `avocado-virt`, which uses this default params to set
it's params and it does that via this generic API (not dependent on the
multiplexer at all).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>