avocado.rst

:title: avocado
:subtitle: test runner command line tool
:title_upper: AVOCADO
:manual_section: 1

SYNOPSIS
========

 avocado [-h] [-v] [-V] [--logdir LOGDIR] [--loglevel LOG_LEVEL] [--plugins PLUGINS_DIR]
 {run,list,sysinfo,multiplex,plugins,datadir} ...

DESCRIPTION
===========

Avocado is an experimental test framework that is built on the experience
accumulated with `autotest` (`http://autotest.github.io`).

`avocado` is also the name of its test runner command line tool.

OPTIONS
=======

The following list of options are builtin, application level `avocado`
options. Most other options are implemented via plugins and will depend
on them being loaded::

 -h, --help             show this help message and exit
 -v, --version          show program's version number and exit
 -V, --verbose          print extra debug messages
 --logdir LOGDIR        Alternate logs directory
 --loglevel LOG_LEVEL   Debug Level
 --plugins PLUGINS_DIR  Load extra plugins from directory

Real use of avocado depends on running avocado subcommands. This a typical list
of avocado subcommands::

 run         Run one or more tests (test module in .py, test alias or dropin)
 list        List available test modules
 sysinfo     Collect system information
 multiplex   Generate a list of dictionaries with params from a multiplex file
 plugins     List all plugins loaded
 datadir     List all relevant directories used by avocado

To get usage instructions for a given subcommand run it with `--help`. Example::

 $ avocado multiplex --help

 usage: avocado multiplex [-h] [-t] [-c] [multiplex_file]

 positional arguments:
   multiplex_file  Path to a multiplex file

 optional arguments:
   -h, --help      show this help message and exit
   -t, --tree      Shows the multiplex tree structure
   -c, --contents  Keep temporary files generated by tests


RUNNING A TEST
==============

The most common use of the `avocado` command line tool is to run a test::

 $ avocado run sleeptest

This command will run the `sleeptest` test, as found on the standard test
directories. The output should be similar to::

 JOB ID    : <id>
 JOB LOG   : /home/<user>/avocado/job-results/job-<date>-<shortid>/job.log
 TESTS     : 1
 (1/1) sleeptest.py: PASS (1.00 s)
 PASS      : 1
 ERROR     : 0
 FAIL      : 0
 SKIP      : 0
 WARN      : 0
 NOT FOUND : 0
 TIME      : 1.00 s

The test directories will vary depending on you system and
installation method used. Still, it's pretty easy to find that out as shown
in the next section.

LISTING TESTS
=============

The `avocado` command line tool also has a `list` command, that lists the
known tests in the standard test directory::

 $ avocado list

The output should be similar to::

 Tests dir: /home/<user>/local/avocado/tests
     Alias         Path
     sleeptest     /home/<user>/local/avocado/tests/sleeptest.py
     ...
     warntest      /home/<user>/local/avocado/tests/warntest.py
     sleeptenmin   /home/<user>/local/avocado/tests/sleeptenmin.py

EXPLORING RESULTS
=================

When `avocado` runs tests, it saves all its results on your system::

 JOB ID    : <id>
 JOB LOG   : /home/<user>/avocado/job-results/job-<date>-<shortid>/job.log

For your convenience, `avocado` maintains a link to the latest job run
(an `avocado run` command in this context), so you can always use `"latest"`
to browse your test results::

 $ ls /home/<user>/avocado/job-results/latest
 id
 job.log
 results.json
 results.xml
 sysinfo
 test-results

The main log file is `job.log`, but every test has its own results directory::

 $ ls -1 ~/avocado/job-results/latest/test-results/
 sleeptest.py

Since this is a directory, it should have content similar to::

 $ ls -1 ~/avocado/job-results/latest/test-results/sleeptest.py/
 data
 debug.log
 sysinfo

MULTIPLEX
=========

Avocado has a powerful tool that enables multiple test scenarios to be run
using a single, unmodified test. This mechanism uses a multiplex file, that
multiplies all possible variations automatically.

A command by the same name, `multiplex`, is available on the `avocado`
command line tool, and enables you to see all the test scenarios that can
be run::

 $ avocado multiplex tests/sleeptest.py.data/sleeptest.yaml
 Variants generated:
 Variant 1:    ["short={'sleep_length': 0.5}"]
 Variant 2:    ["medium={'sleep_length': 1}"]
 Variant 3:    ["long={'sleep_length': 5}"]
 Variant 4:    ["longest={'sleep_length': 10}"]

 $ avocado run --multiplex tests/sleeptest.py.data/sleeptest.yaml sleeptest

And the output should look like::

 ...
 (1/4) sleeptest.py: PASS (0.50 s)
 (2/4) sleeptest.py.1: PASS (1.00 s)
 (3/4) sleeptest.py.2: PASS (5.01 s)
 (4/4) sleeptest.py.3: PASS (10.01 s)
 ...

DEBUGGING BINARIES RUN AS PART OF A TEST
========================================

One interesting avocado feature is the ability to automatically and
transparently run binaries that are used on a given test inside the
GNU debugger.

Suppose you are running a test that uses an external, compiled, image
converter. Now suppose you're feeding it with different types of images,
including broken image files, and it fails at a given point. You wish
you could connect to the debugger at that given source location while
your test is running. This is how to do just that with avocado::

 $ avocado run --gdb-run-bin=convert:convert_ppm_to_raw converttest

The job starts running just as usual, and so does your test::

 JOB ID    : <id>
 JOB LOG   : /home/<user>/avocado/job-results/job-<date>-<shortid>/job.log
 TESTS     : 1
 (1/1) converttest.py: /

The `convert` binary though, automatically runs inside GDB. Avocado will
stop when the given breakpoint is reached::

 TEST PAUSED because of debugger breakpoint. To DEBUG your application run:
 /home/<user>/avocado/job-results/job-<date>-<shortid>/test-results/converttest.py/data/convert.gdb.sh

 NOTE: please use *disconnect* command in gdb before exiting, or else the debugged process will be KILLED

From this point, you can run the generated script (`convert.gdb.sh`) to
debug you application.

As noted, it is strongly recommended that you *disconnect* from gdb while
your binary is still running. That is, if the binary finished running
while you are debugging it, avocado has no way to know about its status.

Avocado will automatically send a `continue` command to the debugger
when you disconnect from and exit gdb.

FILES
=====

::

 /etc/avocado/settings.ini
    system wide configuration file

BUGS
====

If you find a bug, please report it over our github page as an issue.

MORE INFORMATION
================

For more information check Avocado's online documentation at: `http://avocado-framework.readthedocs.org/`

Or the project github page at: `http://github.com/avocado-framework`


AUTHOR
======

Cleber Rosa <cleber@redhat.com>