Let's increase the GDB init timeout up to 5s and while on it improve a
bit the while condition.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

On heavy loaded machines (mainly IO) it might happen that test's
`__init__` is not executed in 10s causing the execution to fail. The
same applies to when we try to interrupt the job, only here the default
timeout was 0.5s. Let's increase both to 60s, they are both busy loops
so this timeout is only a corner case.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

We do produce `results.xml` and `results.json`, let's also include
`results.tap` by default and allow disabling this by `--tap-job-result`.

Note it's impossible to initialize the default result.tap in `__init__`
as we only get `args` and not the job itself, therefor this patch adds
it as first thing inside the `pre_tests` step.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

docs: Update the documentation test output

It looks like the test result is not copy-paste from real test. Since
there is two `(6/6)`, and actually no `ERROR` occurs but get 2 in
result sumary.

* https://github.com/avocado-framework/avocado:
  spell: fix spelling for v44 release

Only two occurrences. One is a typo and another is an exception.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

The package manager for both is zypper. Both have an /etc/os-release
file, but the content of it is quite different.

Some examples:

NAME="openSUSE Leap"
VERSION="42.2"
ID=opensuse
VERSION_ID="42.2"

NAME="SLES"
VERSION="12-SP2"
VERSION_ID="12.2"

NAME="openSUSE Tumbleweed"
ID=opensuse
VERSION_ID="20161125"

So I took the easy path - what they all have in common is the SUSE
string. And as no other probe looks at /etc/os-release, this is very
safe for now.

* https://github.com/avocado-framework/avocado:
  Example tests: add sample tags
  Test tags: support test tags in loader, list and run
  Loader: add support for test tags
  safeloader: add support for "🥑 tags=category1,category2"
  Docstring directives: use the same terminology in the code

This adds tags to most of the example tests.  Since tags are free
form, this adds the following (hopefully self documenting) tags:

 * fast
 * slow
 * failure_expected
 * requires_c_compiler

Then, they can be used in various ways, such as:

 $ avocado run examples/tests/{abort,passtest.py} --filter-by-tags='-failure_expected'

To filter out tests that are known to fail (and thus run only
passtest.py).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This adds support for test tags (to choose categorized tests) to the
loader, and also to the list and run commands.

The user experience is based on two options:

 * --filter-by-tags: this can be supplied any number of times.  Each
   time it must be given a comma separated list of tags.
 * --filter-by-tags-include-empty: this will always include tests that
   have no tags set.

These options are equally available to the list and run commands.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

* https://github.com/avocado-framework/avocado:
  don't add logger without handlers to output.LoggingFile instances

We currently override sys.stdout and sys.stderr with instances of
output.LoggingFile with some loggers. But bu adding test loggers in the
runner is causing output.LoggingFile to have loggers without handlers.

This patch creates a mechanism to include loggers later in a
output.LoggingFile instance so we can delay the test loggers inclusion
in sys.stderr/sys.stdout to the correct moment.

Also, create the proper cleanup in _stop_logging()
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Test tags are a way to categorize tests, at a class level.  This adds
that information to the test factory, along the other information
already present.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This adds utility functions to parse class-level docstring contents
with a "tags" directive.

The idea is that test writers should be able to categorize their tests
and test runners should be able to further filter the result of the
test reference resolution using that information.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Commit 2adc953e changed how we used to refer to "avocado tags", and now
we call them "docstring directives".  Let's do that in the code too,
so that our users and hackers (and ourselves) have a more consistent
experience.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

* https://github.com/avocado-framework/avocado:
  Makefile: fix help message

We renamed make check-long to make check-full. Let's reflect that change
in the help message.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

* https://github.com/avocado-framework/avocado:
  Docs: rename the "avocado tags" to "docstring directives"

* https://github.com/avocado-framework/avocado:
  docs: use ResultEvents interface

The term we've used so far, it's too specific and conflicts with
the idea of "tagging", that is, categorizing tests.

Independent of the upcoming work on "tags", it looks like a good
idea to change the terminology being used here.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Instead of the `Result` (old-style plugin) approach.

Also, the example `xunit` module was incorrect and it's fixed here.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

* https://github.com/avocado-framework/avocado:
  avocado/core/loader.py: remove "FILTERED" type
  avocado/plugins/list.py: replace keywords for (test) reference
  avocado/plugins/list.py: optmize type and name of INSTRUMENTED tests

We've been seen failures on resource constrained environments on some
tests that are either time sensitive and/or assume a some computing
resources will be available.

Environments such as TravisCI and RPM build servers may not have the
resources these tests expect, so let's move them to runs of `make
check-full`.  This will hopefully put an end to the false positives
we've been getting on TravisCI.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Running `make check` or `make check-full` during development should be
the choice of the developer, but before doing a release, it makes
sense to require a full check.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

We aim to mark more tests that are not suitable to run everywhere, so
it looks like a good idea to rename this to check-full.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This test type (discovery status, really), has never been used.  Even
though there's been work on filtering tests (by tags), this doesn't
really apply to that work at this point.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

It's been a while since the safeloader started to be used to load
INSTRUMENTED tests.  Since we don't load the code, and can't compare
the class types, we stick to the string representation of the test
class name.

Now, the code on `avocado list` that deals with INSTRUMENTED tests
still contains the old check for instances of `avocado.test.Test`.
This shouldn't be necessary anymore, so let's remove it.

With that removal, the overall type mapping code can be made simpler.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It has been some time now since replay feature accepts the job to be
identified by its results directory. Now it's time to remove the
--replay-data-dir option, since this feature became redundant.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

When user runs a replay job with the option --job-results-dir, we are
using the --job-results-dir only as the job output directory.

This patch makes the replay job to look for the source job on the
--job-results-dir, when it's provided.

Reference: https://trello.com/c/CUQxstnhSigned-off-by: NAmador Pahim <apahim@redhat.com>

Recent pylint is again a bit more pedantic. Let's fix places where
possible and locally disable checks with false-detections.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The `get_named_tree_cls` is a factory to produce TreeNode class with
embedded origin. Later commits introduce different types of TreeNodes
and we'll need this factory to produce class of given type so add it as
an optional argument here.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>