Skip to content

A
avocado

openeuler / avocado

通知 0
Star 0
Fork 0
A
avocado

体验新版 GitCode，发现更多精彩内容 >>

提交 d727bfe0 编写于 作者: L Lukáš Doktor
浏览文件
操作

avocado.multiplexer: Switch to the new API 

This patch adds documentation of the "new params API" and removes the
compatility layer. From now on only the API with paths is valid:

    self.params.get(key, path='*', default=None)

where the default path '*' matches anything from --mux-entry path.
Default path is `/test`.
Signed-off-by: NLukáš Doktor <ldoktor@redhat.com>
上级 8e2ddf58
显示空白变更内容
内联 并排
Showing with 389 addition and 381 deletion
avocado/multiplexer.py
浏览文件 @ d727bfe0
......@@ -253,63 +253,14 @@ class AvocadoParams(object):
logging.getLogger("avocado.test").warn(msg)
return self.get(attr)
def get(self, *args, **kwargs):
"""
Retrieve params
Old API: ``params.get(key, failobj=None)`` (any matching param)
New API: ``params.get(key, path=$MUX_ENTRY/*, default=None)``
As old and new API overlaps, you must use all 3 arguments or
explicitely use key argument "path" or "default".
"""
def compatibility(args, kwargs):
"""
Be 100% compatible with old API while allow _SOME OF_ the new APIs
calls:
OLD: get(key), get(key, default), get(key, failobj=default)
NEW: get(key, path, default), get(key, path=path),
get(key, default=default)
:warning: We are unable to distinguish old get(key, default) vs.
new get(key, path), therefor if you want to use the new
API you must specify path/default using named arguments
or supply all 3 arguments:
get(key, path, default), get(key, path=path),
get(key, default=default).
This will be removed in final version.
"""
if len(args) < 1:
raise TypeError("Incorrect arguments: params.get(%s, %s)"
% (args, kwargs))
elif 'failobj' in kwargs:
return [args[0], '/*', kwargs['failobj']] # Old API
elif len(args) > 2 or 'default' in kwargs or 'path' in kwargs:
try:
if 'default' in kwargs:
default = kwargs['default']
elif len(args) > 2:
default = args[2]
else:
default = None
if 'path' in kwargs:
path = kwargs['path']
elif len(args) > 1:
path = args[1]
else:
path = None
key = args[0]
return [key, path, default]
except IndexError:
raise TypeError("Incorrect arguments: params.get(%s, %s)"
% (args, kwargs))
else: # Old API
if len(args) == 1:
return [args[0], '/*', None]
else:
return [args[0], '/*', args[1]]
key, path, default = compatibility(args, kwargs)
def get(self, key, path=None, default=None):
"""
Retrieve value associated with key from params
:param key: Key you're looking for
:param path: namespace ['*']
:param default: default value when not found
:raise KeyError: In case of multiple different values (params clash)
"""
if path is None: # default path is any relative path
path = '*'
try:
......@@ -368,10 +319,6 @@ class AvocadoParam(object):
"""
This is a single slice params. It can contain multiple leaves and tries to
find matching results.
Currently it doesn't care about params origin, it requires single result
or failure. In future it'll get the origin from LeafParam and if it's the
same it'll proceed, otherwise raise exception (as it can't decide which
variable is desired)
"""
def __init__(self, leaves, name):
......
docs/source/MultiplexConfig.rst
浏览文件 @ d727bfe0
......@@ -4,197 +4,91 @@
Multiplex Configuration
=======================
Multiplex Configuration is a specialized way of providing lists
of key/value pairs within combination's of various categories,
that will be passed to avocado test as parameters in a dictionary
called ``params``. The format simplifies and condenses complex
multidimensional arrays of test parameters into a flat list. The
combinatorial result can be filtered and adjusted prior to testing.
The parser relies on `YAML <http://www.yaml.org/>`_, a human friendly
markup language. The YAML format allows one to create, manually or
with code automation, multiple configurations for the tests. You can use any
text editor to write YAML files, but choose one that supports syntax
enhancements and basic validation, it saves time!
Here is how a simple and valid multiplex configuration looks like::
# Multiplex config example, file sleep.yaml
short:
sleep_length: 1
medium:
sleep_length: 60
long:
sleep_length: 600
The key concepts here are ``nodes`` (provides context and scope), ``keys`` (think of variables) and ``values`` (scalar or lists).
In order to get a good coverage one always needs to execute the same test
with different parameters or in various environments. Avocado uses the
term ``Multiplexation`` to generate multiple variants of the same test with
different values. To define these variants and values
`YAML <http://www.yaml.org/>`_ files are used. The benefit of using YAML
file is the visible separation of different scopes. Even very advanced setups
are still human readable, unlike traditional sparse, multi-dimensional-matrices
of parameters.
Let's start with an example (line numbers added at the beginning)::
1 hw:
2 cpu: !mux
3 intel:
4 cpu_CFLAGS: '-march=core2'
5 amd:
6 cpu_CFLAGS: '-march=athlon64'
7 arm:
8 cpu_CFLAGS: '-mabi=apcs-gnu -march=armv8-a -mtune=arm8'
9 disk: !mux
10 scsi:
11 disk_type: 'scsi'
12 virtio:
13 disk_type: 'virtio'
14 distro: !mux
15 fedora:
16 init: 'systemd'
17 mint:
18 init: 'systemv'
19 env: !mux
20 debug:
21 opt_CFLAGS: '-O0 -g'
22 prod:
23 opt_CFLAGS: '-O2'
There are couple of key=>value pairs (4,6,8,11,13,...) and there are
named nodes which define scope (1,2,3,5,7,9,...). There are also additional
flags (2, 9, 14, 19) which modifies the behavior.
In the next section, we will describe these concepts in more details.
.. _nodes:
Nodes
=====
Nodes servers for two purposes, to name or describe a discrete point of information
and to store in a set of key/values (possibly empty). Basically nodes can contains
other nodes, so will have the parent and child relationship in a tree structure.
The tree node structure can be obtained by using the command line
``avocado multiplex --tree <file>`` and for previous example,
it looks just like this::
They define context of the key=>value pairs allowing us to easily identify
for what this values might be used for and also it makes possible to define
multiple values of the same keys with different scope.
avocado multiplex --tree sleep.yaml
Config file tree structure:
Nodes are organized in parent-child relationship and together they create
a tree. To view this structure use ``avocado multiplex --tree <file>``::
/-short
/-intel
|
----|-medium
/cpu-<>--amd
| |
/hw \-arm
| |
| | /-scsi
| \disk-<>
| \-virtio
-|
| /-fedora
|-distro-<>
| \-mint
|
\-long
It helps if you see the tree structure as a set of paths
separated by ``/``, much like paths in the file system.
In the example we have being working on, there are only three paths:
- ``//short``
- ``//medium``
- ``//long``
The ending nodes (the leafs on the tree) will become part of all lower-level
(i.e. further indented) variant stanzas (see section variants_).
However, the precedence is evaluated in top-down or ``last defined`` order.
In other words, the last parsed has precedence over earlier definitions.
| /-debug
\env-<>
\-prod
When you provide multiple files they are processed and merged together using
the common root (`/`). When certain paths overlap (`$file1:/my/path`,
`$file2:/my/path`), we first create the tree of `$file1` and then process
`$file2`. This means all children of `/my/path` of the first file are in
correct order and `$file2` either updates values or appends new children
as next ones. This of course happens recursively so you update valures and add
children of all the nodes beneath.
During this merge it's also possible to remove nodes using python regular
expressions, which can be useful when extending upstream file using downstream
yaml files. This is done by `!remove_node : $value_name` directive::
os:
fedora:
windows:
3.11:
95:
os:
!remove_node : windows
windows:
win3.11:
win95:
Removes the `windows` node from structure. It's different from `filter-out`
as it really removes the node (and all children) from the tree and
it can be replaced by you new structure as shown in the example. It removes
`windows` with all children and then replaces this structure with slightly
modified version.
As `!remove_node` is processed during merge, when you reverse the order,
windows is not removed and you end-up with `/windows/{win3.11,win95,3.11,95}`
nodes.
Due to yaml nature, it's __mandatory__ to put space between `!remove_node`
and `:`!
Additionally you can prepend multiple nodes to the given node by using
`!using : $prepended/path`. This is useful when extending complex structure,
for example imagine having distro variants in separate ymal files. In the
end you want to merge them into the `/os` node. The main file can be simply::
# main.yaml
os:
!include : os/fedora/21.yaml
....
You can see that ``hw`` has 2 children ``cpu`` and ``disk``. All parameters
defined in parent node are inherited to children and extended/overwritten by
their values up to the leaf nodes. The leaf nodes (``intel``, ``amd``, ``arm``,
``scsi``, ...) are the most important as after multiplexation they form the
parameters available in tests.
And each file can look either like this::
# fedora/21.yaml
fedora:
21:
some: value
or you can use `!using` which prepends the `fedora/21`::
# fedora/21.yaml
!using : /fedora/21
some: value
To be precise there is a way to define the structure in the main yaml file::
# main.yaml
os:
fedora:
21:
!include : fedora_21.yaml
Or use recursive `!include` (slower)::
# main.yaml
os:
fedora:
!include : os/fedora.yaml
# os/fedora.yaml
21:
!include : fedora/21.yaml
# os/fedora/21.yaml
some: value
Due to yaml nature, it's __mandatory__ to put space between `!using` and `:`!
.. _keys_and_values:
Keys and Values
===============
Keys and values are the most basic useful facility provided by the
format. A statement in the form ``<key>: <value>`` sets ``<key>`` to
``<value>``.
Values are numbers, strings and lists. Some examples of literal values:
- Booleans: ``true`` and ``false``.
- Numbers: 123 (integer), 3.1415 (float point)
- Strings: 'This is a string'
And lists::
Every value other than dict (4,6,8,11) is used as value of the antecedent
node.
cflags:
- '-O2'
- '-g'
- '-Wall'
The list above will become ``['-O2', '-g', '-Wall']`` to Python. In fact,
YAML is compatible to JSON.
It's also possible to remove key using python's regexp, which can be useful
when extending upstream file using downstream yaml files. This is done by
`!remove_value : $value_name` directive::
debug:
CFLAGS: '-O0 -g'
debug:
!remove_value: CFLAGS
removes the CFLAGS value completely from the debug node. This happens during
the merge and only once. So if you switch the two, CFLAGS would be defined.
Due to yaml nature, it's __mandatory__ to put space between `!remove_value`
and `:`!
.. _environment:
Environment
===========
The environment is a set of key/values constructed by the moment
we walk the path (beginning from the root) until we reach a specific node.
Each node can define key/value pairs (lines 4,6,8,11,...). Additionally
each children node inherits values of it's parent and the result is called
node ``environment``.
Given the node structure bellow::
......@@ -223,14 +117,111 @@ The environment created for the nodes ``fedora`` and ``osx`` are:
- Node ``//devtools/fedora`` environment ``compiler: 'gcc'``, ``flags: ['-O2', '-Wall']``
- None ``//devtools/osx`` environment ``compiler: 'clang'``, ``flags: ['-O2', '-arch i386', '-arch x86_64']``
.. _multiple_files:
Variants
========
In the end all leafs are gathered and turned into parameters, more specifically into
``AvocadoParams``::
setup:
graphic:
user: "guest"
password: "pass"
text:
user: "root"
password: "123456"
produces ``[graphic, text]``. In the test code you'll be able to query only
those leaves. Intermediary or root nodes are available.
The example above generates a single test execution with parameters separated
by path. But the most powerful multiplexer feature is that it can generate
multiple variants. To do that you need to tag a node whose children are
ment to be multiplexed. Effectively it returns only leaves of one child at the
time.In order to generate all possible variants multiplexer creates cartesian
product of all of these variants::
cpu: !mux
intel:
amd:
arm:
fmt: !mux
qcow2:
raw:
Produces 6 variants::
/cpu/intel, /fmt/qcow2
/cpu/intel, /fmt/raw
...
/cpu/arm, /fmt/raw
The !mux evaluation is recursive so one variant can expand to multiple
ones::
fmt: !mux
qcow: !mux
2:
2v3:
raw:
Results in::
/fmt/qcow2/2
/fmt/qcow2/2v3
/raw
Resolution order
================
You can see that only leaves are part of the test parameters. It might happen
that some of these leaves contain different values of the same key. Then
you need to make sure your queries separate them by different paths. When
the path matches multiple results with different origin, an exception is raised
as it's impossible to guess which key was originally intended.
To avoid these problems it's recommended to use unique names in test parameters if
possible, to avoid the mentioned clashes. It also makes it easier to extend or mix
multiple YAML files for a test.
For multiplex YAML files that are part of a framework, contain default
configurations, or serve as plugin configurations and other advanced setups it is
possible and commonly desirable to use non-unique names. But always keep those points
in mind and provide sensible paths.
Multiplexer also supports something called "multiplex entry points" or
"resolution order". By default it's ``/tests/*`` but it can be overridden by
``--mux-entry``, which accepts multiple arguments. What it does it splits
leaves by the provided paths. Each query goes one by one through those
sub-trees and first one to hit the match returns the result. It might not solve
all problems, but it can help to combine existing YAML files with your ones::
qa: # large and complex read-only file, content injected into /qa
tests:
timeout: 10
...
my_variants: !mux # your YAML file injected into /my_variants
short:
timeout: 1
long:
timeout: 1000
You want to use an existing test which uses ``params.get('timeout', '*')``. Then you
can use ``--mux-entry '/my_variants/*' '/qa/*'`` and it'll first look in your
variants. If no matches are found, then it would proceed to ``/qa/*``
Keep in mind that only slices defined in mux-entry are taken into account for
relative paths (the ones starting with ``*``)
Multiple files
==============
You can provide multiple files. In such scenario final tree is a combination
of the provided files where later nodes with the same name override values of
the precending corresponding node. New nodes are appended as new children::
the preceding corresponding node. New nodes are appended as new children::
file-1.yaml:
debug:
......@@ -253,8 +244,8 @@ results in::
fast:
CFLAGS: '-Ofast' # appended
It's also possilbe to include existing file into other file's node. This
is done by `!include : $path` directive::
It's also possible to include existing file into another a given node in another
file. This is done by the `!include : $path` directive::
os:
fedora:
......@@ -262,117 +253,161 @@ is done by `!include : $path` directive::
gentoo:
!include : gentoo.yaml
Due to yaml nature, it's __mandatory__ to put space between `!include` and `:`!
Due to YAML nature, it's __mandatory__ to put space between `!include` and `:`!
The file location can be either absolute path or relative path to the yaml
The file location can be either absolute path or relative path to the YAML
file where the `!include` is called (even when it's nested).
Whole file is __merged__ into the node where it's defined.
.. _variants:
Variants
========
Advanced YAML tags
==================
When tree parsing and filtering is finished, we create set of variants.
Each variant uses one leaf of each sibling group. For example::
There are additional features related to YAML files. Most of them require values
separated by ``:``. Again, in all such cases it's mandatory to add a white space (``
``) between the tag and the ``:``, otherwise ``:`` is part of the tag name and the
parsing fails.
cpu:
intel:
amd:
arm:
fmt:
qcow2:
raw:
!include
--------
Produces 2 groups `[intel, amd, arm]` and `[qcow2, raw]`, which results in
6 variants (all combinations; product of the groups)
Includes other file and injects it into the node it's specified in::
It's also possible to join current node and its children by `!join` tag::
my_other_file:
!include : other.yaml
fmt: !join
qcow:
2:
2v3:
raw:
The content of ``/my_other_file`` would be parsed from the ``other.yaml``. It's
the hardcoded equivalent of the ``-m $using:$path``.
Without the join this would produce 2 groups `[2, 2v3]` and `[raw]` resulting
in 2 variants `[2, raw]` and `[2v3, raw]`, which is really not useful.
But we said that `fmt` children should join this sibling group
so it results in one group `[qcow/2, qcow/2v3, raw]` resulting in 3 variants
each of different fmt. This is useful when some
of the variants share some common key. These keys are set inside the
parent, for example here `qcow2.0` and `qcow2.2v3` share the same key
`type: qcow2` and `qcow2.2v3` adds `extra_params` into his params::
Relative paths start from the original file's directory.
fmt:
qcow2:
type: qcow2
0:
v3:
extra_params: "compat=1.1"
raw:
type: raw
!using
------
Complete example::
Prepends path to the node it's defined in::
hw:
cpu:
intel:
amd:
arm:
fmt: !join
qcow:
qcow2:
qcow2v3:
raw:
os: !join
linux: !join
Fedora:
19:
Gentoo:
!using : /foo
bar:
!using : baz
``bar`` is put into ``baz`` becoming ``/baz/bar`` and everything is put into
``/foo``. So the final path of ``bar`` is ``/foo/baz/bar``.
!remove_node
------------
Removes node if it existed during the merge. It can be used to extend
incompatible YAML files::
os:
fedora:
windows:
3.11:
95:
os:
!remove_node : windows
windows:
win3.11:
win95:
Removes the `windows` node from structure. It's different from `filter-out`
as it really removes the node (and all children) from the tree and
it can be replaced by you new structure as shown in the example. It removes
`windows` with all children and then replaces this structure with slightly
modified version.
As `!remove_node` is processed during merge, when you reverse the order,
windows is not removed and you end-up with `/windows/{win3.11,win95,3.11,95}`
nodes.
While preserving names and environment values. Then all combinations are
created resulting into 27 unique variants covering all possible combinations
of given tree::
Variant 1: /hw/cpu/intel, /hw/fmt/qcow/qcow2, /os/linux/Fedora/19
Variant 2: /hw/cpu/intel, /hw/fmt/qcow/qcow2, /os/linux/Gentoo
Variant 3: /hw/cpu/intel, /hw/fmt/qcow/qcow2, /os/windows/3.11
Variant 4: /hw/cpu/intel, /hw/fmt/qcow/qcow2v3, /os/linux/Fedora/19
Variant 5: /hw/cpu/intel, /hw/fmt/qcow/qcow2v3, /os/linux/Gentoo
Variant 6: /hw/cpu/intel, /hw/fmt/qcow/qcow2v3, /os/windows/3.11
Variant 7: /hw/cpu/intel, /hw/fmt/raw, /os/linux/Fedora/19
Variant 8: /hw/cpu/intel, /hw/fmt/raw, /os/linux/Gentoo
Variant 9: /hw/cpu/intel, /hw/fmt/raw, /os/windows/3.11
Variant 10: /hw/cpu/amd, /hw/fmt/qcow/qcow2, /os/linux/Fedora/19
Variant 11: /hw/cpu/amd, /hw/fmt/qcow/qcow2, /os/linux/Gentoo
Variant 12: /hw/cpu/amd, /hw/fmt/qcow/qcow2, /os/windows/3.11
Variant 13: /hw/cpu/amd, /hw/fmt/qcow/qcow2v3, /os/linux/Fedora/19
Variant 14: /hw/cpu/amd, /hw/fmt/qcow/qcow2v3, /os/linux/Gentoo
Variant 15: /hw/cpu/amd, /hw/fmt/qcow/qcow2v3, /os/windows/3.11
Variant 16: /hw/cpu/amd, /hw/fmt/raw, /os/linux/Fedora/19
Variant 17: /hw/cpu/amd, /hw/fmt/raw, /os/linux/Gentoo
Variant 18: /hw/cpu/amd, /hw/fmt/raw, /os/windows/3.11
Variant 19: /hw/cpu/arm, /hw/fmt/qcow/qcow2, /os/linux/Fedora/19
Variant 20: /hw/cpu/arm, /hw/fmt/qcow/qcow2, /os/linux/Gentoo
Variant 21: /hw/cpu/arm, /hw/fmt/qcow/qcow2, /os/windows/3.11
Variant 22: /hw/cpu/arm, /hw/fmt/qcow/qcow2v3, /os/linux/Fedora/19
Variant 23: /hw/cpu/arm, /hw/fmt/qcow/qcow2v3, /os/linux/Gentoo
Variant 24: /hw/cpu/arm, /hw/fmt/qcow/qcow2v3, /os/windows/3.11
Variant 25: /hw/cpu/arm, /hw/fmt/raw, /os/linux/Fedora/19
Variant 26: /hw/cpu/arm, /hw/fmt/raw, /os/linux/Gentoo
Variant 27: /hw/cpu/arm, /hw/fmt/raw, /os/windows/3.11
You can generate this list yourself by executing::
avocado multiplex /path/to/multiplex.yaml [-c]
Note that there's no need to put extensions to a multiplex file, although
doing so helps with organization. The optional -c param is used to provide
the contents of the dictionaries generated, not only their shortnames.
With Nodes, Keys, Values & Filters, we have most of what you
actually need to construct most multiplex files.
!remove_value
-------------
It's similar to `!remove_node`_ only with values.
!mux
----
Children of this node will be multiplexed. This means that in first variant
it'll return leaves of the first child, in second the leaves of the second
child, etc. Example is in section `Variants`_
Complete example
================
Let's take a second look at the first example::
1 hw:
2 cpu: !mux
3 intel:
4 cpu_CFLAGS: '-march=core2'
5 amd:
6 cpu_CFLAGS: '-march=athlon64'
7 arm:
8 cpu_CFLAGS: '-mabi=apcs-gnu -march=armv8-a -mtune=arm8'
9 disk: !mux
10 scsi:
11 disk_type: 'scsi'
12 virtio:
13 disk_type: 'virtio'
14 distro: !mux
15 fedora:
16 init: 'systemd'
17 mint:
18 init: 'systemv'
19 env: !mux
20 debug:
21 opt_CFLAGS: '-O0 -g'
22 prod:
23 opt_CFLAGS: '-O2'
After filters are applied (simply removes non-matching variants), leaves
are gathered and all variants are generated::
./scripts/avocado multiplex examples/mux-environment.yaml
Variants generated:
Variant 1: /hw/cpu/intel, /hw/disk/scsi, /distro/fedora, /env/debug
Variant 2: /hw/cpu/intel, /hw/disk/scsi, /distro/fedora, /env/prod
Variant 3: /hw/cpu/intel, /hw/disk/scsi, /distro/mint, /env/debug
Variant 4: /hw/cpu/intel, /hw/disk/scsi, /distro/mint, /env/prod
Variant 5: /hw/cpu/intel, /hw/disk/virtio, /distro/fedora, /env/debug
Variant 6: /hw/cpu/intel, /hw/disk/virtio, /distro/fedora, /env/prod
Variant 7: /hw/cpu/intel, /hw/disk/virtio, /distro/mint, /env/debug
Variant 8: /hw/cpu/intel, /hw/disk/virtio, /distro/mint, /env/prod
Variant 9: /hw/cpu/amd, /hw/disk/scsi, /distro/fedora, /env/debug
Variant 10: /hw/cpu/amd, /hw/disk/scsi, /distro/fedora, /env/prod
Variant 11: /hw/cpu/amd, /hw/disk/scsi, /distro/mint, /env/debug
Variant 12: /hw/cpu/amd, /hw/disk/scsi, /distro/mint, /env/prod
Variant 13: /hw/cpu/amd, /hw/disk/virtio, /distro/fedora, /env/debug
Variant 14: /hw/cpu/amd, /hw/disk/virtio, /distro/fedora, /env/prod
Variant 15: /hw/cpu/amd, /hw/disk/virtio, /distro/mint, /env/debug
Variant 16: /hw/cpu/amd, /hw/disk/virtio, /distro/mint, /env/prod
Variant 17: /hw/cpu/arm, /hw/disk/scsi, /distro/fedora, /env/debug
Variant 18: /hw/cpu/arm, /hw/disk/scsi, /distro/fedora, /env/prod
Variant 19: /hw/cpu/arm, /hw/disk/scsi, /distro/mint, /env/debug
Variant 20: /hw/cpu/arm, /hw/disk/scsi, /distro/mint, /env/prod
Variant 21: /hw/cpu/arm, /hw/disk/virtio, /distro/fedora, /env/debug
Variant 22: /hw/cpu/arm, /hw/disk/virtio, /distro/fedora, /env/prod
Variant 23: /hw/cpu/arm, /hw/disk/virtio, /distro/mint, /env/debug
Variant 24: /hw/cpu/arm, /hw/disk/virtio, /distro/mint, /env/prod
Where the first variant contains::
/hw/cpu/intel/ => cpu_CFLAGS: -march=core2
/hw/disk/ => disk_type: scsi
/distro/fedora/ => init: systemd
/env/debug/ => opt_CFLAGS: -O0 -g
The second one::
/hw/cpu/intel/ => cpu_CFLAGS: -march=core2
/hw/disk/ => disk_type: scsi
/distro/fedora/ => init: systemd
/env/prod/ => opt_CFLAGS: -O2
From this example you can see that querying for ``/env/debug`` works only in
the first variant, but returns nothing in the second variant. Keep this in mind
and when you use the ``!mux`` flag always query for the pre-mux path,
``/env/*`` in this example.
docs/source/WritingTests.rst
浏览文件 @ d727bfe0
......@@ -110,34 +110,66 @@ to analyze that particular benchmark result).
Accessing test parameters
=========================
Each test has a set of parameters that can be accessed through ``self.params.[param-name]``.
Avocado finds and populates ``self.params`` with all parameters you define on a Multiplex
Config file (see :doc:`MultiplexConfig`), in a way that they are available as attributes,
not just dict keys. This has the advantage of reducing the boilerplate code necessary to
access those parameters. As an example, consider the following multiplex file for sleeptest::
variants:
- sleeptest:
sleep_length_type = float
variants:
- short:
sleep_length = 0.5
- medium:
sleep_length = 1
- long:
sleep_length = 5
You may notice some things here: there is one test param to sleeptest, called ``sleep_length``. We could have named it
``length`` really, but I prefer to create a param namespace of sorts here. Then, I defined
``sleep_length_type``, that is used by the config system to convert a value (by default a
:class:`basestring`) to an appropriate value type (in this case, we need to pass a :class:`float`
to :func:`time.sleep` anyway). Note that this is an optional feature, and you can always use
:func:`float` to convert the string value coming from the configuration anyway.
Another important design detail is that sometimes we might not want to use the config system
at all (for example, when we run an avocado test as a stand alone test). To account for this
case, we have to specify a ``default_params`` dictionary that contains the default values
for when we are not providing config from a multiplex file.
Each test has a set of parameters that can be accessed through
``self.params.get($name, $path=None, $default=None)``.
Avocado finds and populates ``self.params`` with all parameters you define on
a Multiplex Config file (see :doc:`MultiplexConfig`). As an example, consider
the following multiplex file for sleeptest::
test:
sleeptest:
type: "builtin"
short:
sleep_length: 0.5
medium:
sleep_length: 1
long:
sleep_length: 5
In this example 3 variants are executed (see :doc:`MultiplexConfig` for
details). All of them contain variable "type" and "sleep_length". To obtain
current value, you need the name ("sleep_length") and its path. The path
differs for each variant so it's needed to use the most suitable portion
of the path, in this example: "/test/sleeptest/*" or perhaps "sleeptest/*"
might be enough. It depends on how your setups looks like.
The default value is optional, but always keep in mind to handle them nicely.
Someone might be executing your test with different params or without any
params at all. It should work fine.
So the complete example on how to access the "sleep_length" would be::
self.params.get("sleep_length", "/*/sleeptest/*", 1)
There is one way to make this even simpler. It's possible to define resolution
order, then for simple queries you can simply omit the path::
self.params.get("sleep_length", None, 1)
self.params.get("sleep_length", '*', 1)
self.params.get("sleep_length", default=1)
One should always try to avoid param clashes (multiple matching keys for given
path with different origin). If it's not possible (eg. when
you use multiple yaml files) you can modify the resolution order by modifying
``--mux-entry``. What it does is it slices the params and iterates through the
paths one by one. When there is a match in the first slice it returns
it without trying the other slices. Although relative queries only match
from ``--mux-entry`` slices.
There are many ways to use paths to separate clashing params or just to make
more clear what your query for. Usually in tests the usage of '*' is sufficient
and the namespacing is not necessarily, but it helps make advanced usage
clearer and easier to follow.
When thinking of the path always think about users. It's common to extend
default config with additional variants or combine them with different
ones to generate just the right scenarios they need. People might
simply inject the values elsewhere (eg. `/test/sleeptest` =>
`/upstream/test/sleeptest`) or they can merge other clashing file into the
default path, which won't generate clash, but would return their values
instead. Then you need to clarify the path (eg. `'*'` => `sleeptest/*`)
More details on that are in :doc:`MultiplexConfig`
Using a multiplex file
======================
......@@ -562,7 +594,7 @@ impact your test grid. You can account for that possibility and set up a
::
$ avocado run sleeptest --multiplex /test:/tmp/sleeptest-example.mplx
$ avocado run sleeptest --multiplex /test:/tmp/sleeptest-example.yaml
JOB ID : 6d5a2ff16bb92395100fbc3945b8d253308728c9
JOB LOG : $HOME/avocado/job-results/job-2014-08-12T15.52-6d5a2ff1/job.log
JOB HTML : $HOME/avocado/job-results/job-2014-08-12T15.52-6d5a2ff1/html/results.html
......@@ -583,8 +615,8 @@ impact your test grid. You can account for that possibility and set up a
15:52:51 test L0144 DEBUG|
15:52:51 test L0145 DEBUG| Test log: $HOME/avocado/job-results/job-2014-08-12T15.52-6d5a2ff1/sleeptest.1/test.log
15:52:51 test L0146 DEBUG| Test instance parameters:
15:52:51 test L0153 DEBUG| _name_map_file = {'sleeptest-example.mplx': 'sleeptest'}
15:52:51 test L0153 DEBUG| _short_name_map_file = {'sleeptest-example.mplx': 'sleeptest'}
15:52:51 test L0153 DEBUG| _name_map_file = {'sleeptest-example.yaml': 'sleeptest'}
15:52:51 test L0153 DEBUG| _short_name_map_file = {'sleeptest-example.yaml': 'sleeptest'}
15:52:51 test L0153 DEBUG| dep = []
15:52:51 test L0153 DEBUG| id = sleeptest
15:52:51 test L0153 DEBUG| name = sleeptest
......
examples/tests/synctest.py
浏览文件 @ d727bfe0
......@@ -21,7 +21,7 @@ class SyncTest(test.Test):
"""
self.cwd = os.getcwd()
tarball_path = self.get_data_path(self.params.get('sync_tarball', '*',
default='synctest.tar.bz2'))
'synctest.tar.bz2'))
archive.extract(tarball_path, self.srcdir)
self.srcdir = os.path.join(self.srcdir, 'synctest')
if self.params.get('debug_symbols', default=True):
......
selftests/all/unit/avocado/multiplexer_unittest.py
浏览文件 @ d727bfe0
......@@ -92,12 +92,6 @@ class TestAvocadoParams(unittest.TestCase):
str(multiplexer.AvocadoParams([], 'Unittest', None, [], {}))
self.assertEqual(26, sum([1 for _ in self.params1.iteritems()]))
def test_get_old_api(self):
self.assertEqual(self.params1.get('unique1'), 'unique1')
self.assertEqual(self.params1.get('missing'), None)
self.assertEqual(self.params1.get('missing', 'aaa'), 'aaa')
self.assertEqual(self.params1.root, 'root')
def test_get_abs_path(self):
# /ch0/ is not leaf thus it's not queryable
self.assertEqual(self.params1.get('root', '/ch0/', 'bbb'), 'bbb')
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或