When an invalid python code is present in a test class, avocado crashes
badly. This patch adds a TestLoaderError class to be used in those cases,
so we use it to fake the invalid test class, fail the test with ERROR estatus
and report the exception in job log.

Reference: https://trello.com/c/3zDIjTuYSigned-off-by: NAmador Pahim <apahim@redhat.com>

Handle long filenames in avocado and fix unknown test status [v4]

The leading "." makes files invisible on unix systems, which might be
confusing to users. Let's replace with "_".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

This adds selftests regarding the long filenames fixes.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

When avocado fails early in the process (for example in
_setup_environment_variables) it crashes, because the test status is
None. Other example could be user-defined status, which is not inside
user-facing-statuses.

This patch modifies the "status" to "ERROR" when test reports
unsupported status and sets related values to provide details regarding
this failure.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Maximal allowed file name is 255 on ext4/python2.7. Note that badly
written code depending on this function can produce infinite hangs (when
the code appends different char on position greater than 255 as it get's
ignored)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The logdir depends on the safe-tagged-name, which is initialized in
`_get_tagged_name` function, which is checking if the safe-tagged-name
exists and reports the unsafe tagged name only to construct safe-tagged-
name few lines after and again check if the dir exists.

Instead of this this patch looks for unused safe-tagged-name and
immediately creates it and reports booth, unsafe name + the already
created logdir.

Additionally this adds adds limit on filename length and limits the
number of tries to 9999, to avoid infinite hangs.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Avoid crash when we try to record reference stdout when filename is not
associated.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The maximal lentght of a file name on ext4/python2.7 is 255 chars. Let's
avoid using datadir with longer name (to avoid crash when we record
stdio)
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The "self.name" can contain dangerous characters. Let's use
"self.logdir" as base for "self.workdir".
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado.core.loader: Use make_broken in _make_avocado_tests

avocado.utils.astring: Be more lenient regarding utf-8 chars

Fix job status on timeout [v6]

When a job is timed out during a test execution, we fail the test
and put status ERROR in the test. The job then exits with the rc
AVOCADO_TESTS_FAIL. This patch fixes this, making the test status
INTERRUPTED and the job to exit with AVOCADO_JOB_INTERRUPTED.

Also, when a job is timed out before a test, the test is skipped
and the job exits with rc AVOCADO_ALL_OK. For that case, this patch
makes the job to exit with AVOCADO_JOB_INTERRUPTED instead, keeping
the test status as SKIP.

Given this change, now we have the following combinations of test status
and job return code:

Case1:
    - Test1: PASS
    - Test2: SKIP (TestTimeoutSkip)

    Job RC: AVOCADO_JOB_INTERRUPTED

Case2:
    - Test1: PASS
    - Test2: INTERRUPTED (TestTimeoutInterrupted)
    - Test3: SKIP (TestTimeoutSkip)

    Job RC: AVOCADO_JOB_INTERRUPTED

Case3:
    - Test1: PASS
    - Test2: FAIL
    - Test3: INTERRUPTED (TestTimeoutInterrupted)
    - Test4: SKIP (TestTimeoutSkip)

    Job RC: AVOCADO_JOB_INTERRUPTED
Signed-off-by: NAmador Pahim <apahim@redhat.com>

Interrupted tests will have improved relevance with the coming
fix in job status report. This patch includes interrupted tests
report in xml xunit output, adding the interrupted tests to the
'error' counter.
Signed-off-by: NAmador Pahim <apahim@redhat.com>

*.svg, .min.js and .min.css are text files, but they're not
intended to be edited by hand.

Worst of all, they keep poluting the output of git grep.

The _make_avocado_tests accepts "make_broken" function and not
"list_non_tests" flag. We forgot to update one occurrence of this.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

add nargs='?' to toplevel options so we accept empty argument [v2]

avocado.core.xunit: Escape non-printable characters

avocado.core.job: Provide a better message when no tests found

avocado.core.output: Two bugfixes

This is intended fo fix the odd message for '--show' and '--config'
inside 'run' without argument, assuming the default on empty argument,
making the user to face the right message on first wrong usage.

Reference: https://trello.com/c/yfrhQOtqSigned-off-by: NAmador Pahim <apahim@redhat.com>

People tend to mix unicode, strings and encoded strings. This patch adds
"avocado.utils.astring.string_safe_encode" function, which tries to
allow all of the listed inputs and it assumes "utf-8" is in use for
encoded strings.

This function is then used in the tabular_output to avoid
"UnicodeDecodeErrors" for all listed input types.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

In case no tests are found we advice users to run `avocado list -V`
followed by list of urls, which does not really work with no-urls given.
This patch adds second message in case the user does not provide urls
(eg. is using the `--vt-config` option).
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

avocado.core.remoter: Correct the timeout condition

The "remote" log does not contain any valuable informations in >INFO log
levels and we currently don't support built-in-streams log level
specification. Let's use "DEBUG" for "remote" built-in stream as default
as that correlates with the "remote.log" in results.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

The --show stream:level is actually broken as the "name" variable is not
tuple, but only the stream name. The variable was renamed to
"stream_level" instead.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

We need to finish when the time.time() is greater than the timeout, not
lower.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Xunit/XML CDATA/attributes must not contain non-printable characters.
This patch escapes such characters using hex value.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

Signed-off-by: NCleber Rosa <crosa@redhat.com>

docs: add logging capabilities docs to test writers [v2]

Let's give a real example on how to use logging streams when writing
tests, and use the runner options that allow for logging streams to
be captured into log files.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

External Runner: deal with empty test URL

A test base on the external runner feature depends on both the
external runner and the test URL. These two are always combined to
generate the complete command that will be executed by Avocado.

But, the Avocado command line application does not require URLs to be
given on the command line, because these may come from other sources
on loaders other than the basic FileLoader.

So, let's also explicitly require a URL to be given to to the External
Runner loader to return a test factory (class and parameters).
Signed-off-by: NCleber Rosa <crosa@redhat.com>

Command line argument parser: disable abbreviations [v2]

To avoid ambiguity, and allow for precise argument names and this
command line examples, let's disable abbreviations. One example where
this led to confusion:

 $ avocado run --show test passtest

Instead of

 $ avocado --show test run passtest

With this change, the first command line would result in:

 avocado run: error: unrecognized arguments: --show

Which is much more precise than:

 Unable to discover url(s) 'test' with loader plugins(s) 'file',
 'external', try running 'avocado list -V test' to see the details.

Disabling the arguments abbreviation without depending on the fix
applied to Python itself is suggested here:

 http://bugs.python.org/msg204678

Reference: https://trello.com/c/Z3lWiJp0Signed-off-by: NCleber Rosa <crosa@redhat.com>

And deprecate the --multiplex-files option, which was actually
the complete name of the option, but was not being used formally
by our documentation.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

examples.testplans: Add tty cleanup logging selftest