This is an attempt to catch most (if not all) usages of open
that do not follow the context manager pattern.

Sometimes, for better readability, our own `genio` library is
used.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Despite the reviews we have some W0102s that potentially corrupts the
arguments. Let's get rid of them.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

As in previous commits, job should not count on arguments being
present, such as "base_logdir".  This also needs to be handled
when using dry run mode.

Since the dry run implementation creates a temporary directory
itself, it needs to be cleaned up in addition to the tmpdir
created by setUp.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

One of the premises of the Job class is that it can be created without
any mandatory information set in the args parameter.  But, because of
the way we're testing it, we always pass a base_logdir.

This test is about making sure the job gets a base_logdir, and
consequently a logdir, even when a base_logdir is not set in the args.
The implementation of this test is based on the fact that there's
already a tmpdir that can be used for playing the role of a base_logdir.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

We already handle the lack of a unique_job_id in a job, but when
using dry run, the check is not being done in a way that waives
that requirement.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The Job class code uses the same name, logdir, for both the directory
that will hold a given job's results (job-<timestamp>-<shortid>) and
the base directory, where the first one will be created.

This changes the naming so that the base log dir, only present in
the job arguments (the argparse.Namespace instance) will be named
"base_logdir".  The job's result dir, which value is kept in the
job instance itself, is still named "logdir".
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It may be useful to users of the Job class to have information on how
much time it took for it to run.  Since the goal for the Job class is
to be flexible when it comes to its usage, it may not be possible to
track the execution time if users follow a different path.

For now, the time accouting is done automatically if users use the
`run()` method.

If advanced users of the Job class choose to set the start time, end
time and elapsed time themselves, they are free to do it and `run()`
will never overwrite it.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

In some tests we override the job workflow, let's make sure the
post_tests is executed whenever pre_test was.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The unittest standard library on Python 2.7 an later has everything
that the unittest2 backport is supposed to have.  Let's then drop all
the conditional imports of unittest2 and stick with unittest.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This PR adjusts our tests to avoid producing results in the default
results directory (usually caused by instantiating Job or Test without a
results dir).

One of the tests still produces test results as it is essential for the
test execution, but it goes through the produced tests and removes the
base directory.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Recent pylint update is more pedantic about module level spacing. There
are no changes to code, only couple of extra spaces to make it happy.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This is the right place to criticize the test suite.  Even though I
don't feel it's right to abort the application from within the job
class, this prevents the current user experience to break.

In the near future, I'd like to see removed, and avocado runs without
a test suite to run and give these exact results: no tests were
available to be run, so none were executed.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

It's been a while since the "Test ID RFC" came out:

https://www.redhat.com/archives/avocado-devel/2016-March/msg00024.html

Still, we've been referring to test URLs all around our code
and also on user visible strings.  This is an attempt to rename
the mentions of "URLs" that really should be "test references"
or simply "references" at this point.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The `run()` method is supposed to be the simplified interface for
jobs, that is, they run all phases of a job.  This should allow for
users of job instances to simply run:

   >>> job = avocado.core.job.Job()
   >>> result = job.run()

Of course the other more granular methods can still be used.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The post_tests phase is intended for the job to run any actions after
the tests are run.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The run_tests phase is responsible for actually running the test suite
that has been created in the create_test_suite phase.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

The pre_tests phase is intended for the job to execute any actions it
deems necessary before any test is executed.  Right now, the Job
Pre/Post plugins have their "pre" methods executed here.

Following this, there is going to be a rename for the plugin interface
names, to better reflect that those actions do not happen before or
after a job, but happen *within* a job, before and after *tests* are
executed.

While at it, let's also make the Job Pre/Post plugin dispatcher
(`_job_pre_post_dispatcher`) private.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This introduces the create_test_suite job phase.  Its goal is to
fire up the test resolver and populate the instance attribute
`test_suite`.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

This introduces a skeleton for testing the use of a job
programmatically.  The most basic assumptions (such as the job id) are
tested here initially.
Signed-off-by: NCleber Rosa <crosa@redhat.com>