The current runner loader/discovery method is a custom build and finds
modules/tests recursively on the current folder. With the new runner,
part of the python unittest discover was delegated to
'loadTestsFromName()' on the standard library that uses a different
logic for this (assumes that we need to import the test). Because of
that we need 1) create a custom base loader here, or 2) prepend the path
with the current dir so loadTestsFromName will return a valid test
suite.

This might not be the perfect solution but was a hot-fix found for LTS.
Fixes #3851

Reference: https://docs.python.org/3/library/unittest.html#unittest.TestLoader.loadTestsFromNameSigned-off-by: NBeraldo Leal <bleal@redhat.com>

The only reason for the "avocado/core/nrunner.py" to be standalone is
to ease its deployment into an environment that contains nothing but
Python itself.  This will hopefully change in the future when we solve
the deployment problem.

For now, it's clear that at least the StatusServer does not need to
live in that module and be deployed when tests are executed in a
different environment.  The StatusServer is used solely by the
`avocado nrun` command, and thus, can live on another module.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

If the "avocado.core.nrunner" module is being used by the Avocado Job,
it can benefit from being on a system that has setuptools installed
and can pick runner for runnables from there.

But, if the "nrunner" module is being executed by itself, let's say,
on a remote system, it may not access to setuptools or entrypoints,
so it should still rely on its internal runner registry.

This change allows the Avocado Job to run all ranges of test types
(runnable kinds, really) currently supported.  This is true for
executions without spawners and always local, which currently, in a
job, it's the only way they're done.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The "avocado-instrumented" test and the Python unittest share a
good deal of similarity, including a common ancestry, and this
attempts to improve it even further.

In Avocado, the "reference resolution" produces a user visible
"test name", which, according to our own guidelines, should be
a valid reference.  This allows users to copy and past those test
names and run tests from them.  Example:

 $ avocado list examples/tests/passtest.py
 INSTRUMENTED examples/tests/passtest.py:PassTest.test

 $ avocado list examples/tests/passtest.py:PassTest.test
 INSTRUMENTED examples/tests/passtest.py:PassTest.test

This is what enables a user to do:

 $ avocado list multiple.py
 INSTRUMENTED multiple.py:Test.test_1
 INSTRUMENTED multiple.py:Test.test_2
 INSTRUMENTED multiple.py:Test.test_3

 $ avocado run multiple.py:Test.test_3

And have a valid test executed.

So, the idea here is to allow the same to happen for Python unittests.
Internally, at execution time, the valid "unittest name" is still
used, but to the user, a test name that is reusable and coherent with
the overall Avocado experience and guidelines is used.

The example given, though, is about the principle, using the current
runner (and loader), while these changes are clearly about the nrunner
and resolver.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

We need to query for the output_dir on a task object, this small change is
making this property public.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

Follow handle_task_started method pattern and print just when
`self.verbose = True`.
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

If the status server is not waiting for a pending task, aka
`wait_on_tasks_pending = True`, there is no need to remove it
from the list, or it will crash the status server:

```
Task exception was never retrieved
future: <Task finished name='Task-7' coro=<StatusServer.cb() done, defined at /home/wrampazz/src/avocado/avocado.dev/avocado/core/nrunner.py:689> exception=ValueError('list.remove(x): x not in list')>
Traceback (most recent call last):
  File "/home/wrampazz/src/avocado/avocado.dev/avocado/core/nrunner.py", line 717, in cb
    self.handle_task_finished(data)
  File "/home/wrampazz/src/avocado/avocado.dev/avocado/core/nrunner.py", line 739, in handle_task_finished
    self.tasks_pending.remove(task_id)
ValueError: list.remove(x): x not in list
```
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

Do not crash the StatusServer when an unknown command or a
malformed JSON is received.
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

While testing the status server using telnet, when a `bye` is
sent, it is translated to `b'bye\r\n'`, making the status server
crash and not exit.
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

This is a small refactor to make the list of commands available as
a class attribute avoiding multiple resolutions and string
manipulation when a command needs to be resolved.
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

Multiple warnings during `make html` for the documentation.
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

Python 3.8 started to show the following deprecation warning:

"DeprecationWarning: "@coroutine" decorator is deprecated since
Python 3.8, use "async def" instead"

This changes the code in the nrunner to comply with "async def".
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

Trying out the resilience of nrunner, I found that it would crash
when found runners were not executable.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Including the list of tasks ending with fail or error.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Exiting early if not result is present, and avoid constant access to
the dictionary values.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Not all messages coming from tasks will contain messages with
'result' keys, actually, only the final messages will.

This change avoids KeyError exceptions from being raised on these
coroutines.  There was no apparent error because they were simply
never collected by the coroutine callers.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Extend Avocado to recognize requirements in json format on tests
docstrings. This is part of BP002.

Ps. previous commit 6934a0ee
missed changes to the Python unittest resolver plugin.
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

When calling subprocess with env=None will cause to copy the current env
by default to the new process. So, currently there is no consistency
when calling with and without kwargs. This change will always send the
current env.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Signed-off-by: NBeraldo Leal <bleal@redhat.com>

If verbose is enabled then will print the output_dir when the task
starts.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

This is a small refact to add a helper method that will prepare a status
dict with a basic information like the time the event was generated.
Runners don't need to fill the time information every time. From now on,
all event will have a timestamp with it.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

This small change only split the callback method for a better
organization.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Tasks must inform to the runners where will be the task output_dir.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

Spawners are not being used by the current nrunner module. To improve
readability and future grow this small change will organize the spawners into
modules.
Signed-off-by: NBeraldo Leal <bleal@redhat.com>

This reverts commit 1549e362, reversing
changes made to fa5aab1f.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Extend Avocado to recognize requirements in json format on tests
docstrings. This is part of BP002.
Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

Analogous to the pick_runner_command() method, the pick_runner_class()
method selects the known runner class based on the runnable kind.

There are a few differences from the command selection, given that
for a Python class we don't attempt to find a Python module in the system
and load its code.  For Python classes we depend on a proper and active
registration.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Which is a registry containing the name of standalone executables.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The selection of a runner was initially introduced as a set of methods
to the Task class.  Most use cases will involve running a Task (which
always contain a Runnable, but not using a Runnable by itself), but
the runner really acts on the Runnable.  This can be evidenced by the
"task.runnable.kind" references in the methods being moved to the
Runnable class.

With this change, it's expected that we will be able to determine if a
runnable has its requirements satisfied.  Right now the only
requirement is really a suitable Runner implementation for the
Runnable kind.

Then, if found to be necessary, we can then implement and determine
extra requirements that are related to a Task that are not necessarily
related to a Runnable.  For instance, the a Task may be configured to
require a specific scheduling/spawing condition, such as to be
executed on a container, which to a Runnable should be transparent (a
Runnable should only care that it's being executed by a suitable
Runner).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

To make it more consistent in the same block of code.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NWillian Rampazzo <willianr@redhat.com>

And introduce the "result" keyword for `ExecTestRunner`,
`PythonUnittestRunner`, `NoOpRunner` and `RobotRunner`, while also
changing the status server to use that key.

Issue: https://github.com/avocado-framework/avocado/issues/3547Signed-off-by: NDenis Karpelevich <dkarpele@redhat.com>
Signed-off-by: NCleber Rosa <crosa@redhat.com>

There's a confusion when it comes to what is a the (most) current
state of an ongoing execution performed by runners and other similar
terms, such as output, status, etc.

This should make the code easier to understand and allow for a better
common terminology to be formed.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This brings an initial and limited check to the spawing of tasks.
The different spawn implementations show how other implementations
could do the same.

The goal after this is  to:

 1) expand the amount of information provided, such as telling if the
    spawning was successful

 2) use the information to identify if tasks that have not being given
    results back to the status server (or some other replacement
    implementation) are still alive but unable to communicate

 3) if a spawned task is not alive anymore, we can give up on it
    and give it a status such as "UNKNOWN" or something similar.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This introduces a simplistic implementation of spawner that, although
limited in features at this time, serves to prove the concept of the
spawner interface.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The reason being that this module is a complete and standalone
implementation ready to be used, as long as it's executable.

Also and by making it executable it can be deployed on demand, say to
containers, much more easily.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

By splitting some of its functionality into a task method that can be
used for checking each task requirement availability.
Signed-off-by: NCleber Rosa <crosa@redhat.com>