未验证 提交 12ea1d71 编写于 作者: C Cleber Rosa

Merge remote-tracking branch 'apahim/release_notes_v2'

===================
0.32.0: Road Runner
===================
Hi everyone! A new year brings a new Avocado release as the result of
Sprint #32: Avocado 0.32.0, aka, "Road Runner".
Release Notes
=============
The major changes introduced in the previous releases were put to
trial on this release cycle, and as a result, we have responded with
documentation updates and also many fixes. This release also marks the
introduction of a great feature by a new member of our team: Amador
Pahim brought us the Job Replay feature! Kudos!!!
So, for Avocado the main changes are:
* Job Replay: users can now easily re-run previous jobs by using the
--replay command line option. This will re-run the job with the same
tests, configuration and multiplexer variants that were used on the
origin one. By using --replay-test-status, users can, for example,
only rerun the failed tests of the previous job. For more check
our docs[1].
* Documentation changes in response to our users feedback, specially
regarding the setup.py install/develop requirement.
* Fixed the static detection of test methods when using repeated
names.
* Ported some Autotest tests to Avocado, now available on their own
repository[2]. More contributions here are very welcome!
For a complete list of changes please check the Avocado changelog[3].
For Avocado-VT, there were also many changes, including:
* Major documentation updates, making them simpler and more in sync
with the Avocado documentation style.
* Refactor of the code under the avocado_vt namespace. Previously
most of the code lived under the plugin file itself, now it
better resembles the structure in Avocado and the plugin files
are hopefully easier to grasp.
Again, for a complete list of changes please check the Avocado-VT
changelog[4].
Install avocado
---------------
Instructions are available in our documentation on how to install
either with packages or from source[5].
Updated RPM packages are be available in the project repos for
Fedora 22, Fedora 23, EPEL 6 and EPEL 7.
Happy hacking and testing!
---
| [1] http://avocado-framework.readthedocs.org/en/0.32.0/Replay.html
| [2] http://github.com/avocado-framework/avocado-misc-tests
| [3] https://github.com/avocado-framework/avocado/compare/0.31.0...0.32.0
| [4] https://github.com/avocado-framework/avocado-vt/compare/0.31.0...0.32.0
| [5] http://avocado-framework.readthedocs.org/en/0.32.0/GetStartedGuide.html
===================================
0.33.0: Lemonade Joe or Horse Opera
===================================
Hello big farmers, backyard gardeners and supermarket reapers! Here is
a new announcement to all the appreciators of the most delicious green
fruit out here. Avocado release 0.33.0, aka, Lemonade Joe or Horse
Opera, is now out!
Release Notes
=============
The main changes in Avocado are:
* Minor refinements to the Job Replay feature introduced in the last
release.
* More consistency naming for the status of tests that were not
executed. Namely, the TEST_NA has been renamed to SKIP all across
the internal code and user visible places.
* The avocado Test class has received some cleanups and
improvements. Some attributes that back the class implementation but
are not intended for users to rely upon are now hidden or removed.
Additionally some the internal attributes have been turned into
proper documented properties that users should feel confident to
rely upon. Expect more work on this area, resulting in a cleaner
and leaner base Test class on the upcoming releases.
* The avocado command line application used to show the main app help
message even when help for a specific command was asked for. This
has now been fixed.
* It's now possible to use the avocado process utility API to run
privileged commands transparently via SUDO. Just add the "sudo=True"
parameter to the API calls and have your system configured to allow
that command without asking interactively for a password.
* The software manager and service utility API now knows about
commands that require elevated privileges to be run, such as
installing new packages and starting and stopping services (as
opposed to querying packages and services status). Those utility
APIs have been integrated with the new SUDO features allowing
unprivileged users to install packages, start and stop services more
easily, given that the system is properly configured to allow that.
* A nasty "fork bomb" situation was fixed. It was caused when a SIMPLE
test written in Python used the Avocado's "main()" function to run
itself.
* A bug that prevented SIMPLE tests from being run if Avocado was not
given the absolute path of the executable has been fixed.
* A cleaner internal API for registering test result classes has been
put into place. If you have written your own test result class,
please take a look at avocado.core.result.register_test_result_class.
* Our CI jobs now also do quick "smoke" checks on every new commit
(not only the PR's branch HEAD) that are proposed on github.
* A new utility function, binary_from_shell_cmd, has been added to
process API allows to extract the executable to be run from complex
command lines, including ones that set shell variable names.
* There have been internal changes to how parameters, including the
internally used timeout parameter, are handled by the test loader.
* Test execution can now be PAUSED and RESUMED interactively! By
hitting CTRL+Z on the Avocado command line application, all processes
of the currently running test are PAUSED. By hitting CTRL+Z again,
they are RESUMED.
* The Remote/VM runners have received some refactors, and most of the
code that used to live on the result test classes have been moved
to the test runner classes. The original goal was to fix a bug, but
turns out test runners were more suitable to house some parts of the
needed functionality.
For a complete list of changes please check the Avocado changelog[1].
For Avocado-VT, there were also many changes, including:
* A new utility function, get_guest_service_status, to get service
status in a VM.
* A fix for ssh login timeout error on remote servers.
* Fixes for usb ehci on PowerPC.
* Fixes for the screenshot path, when on a remote host
* Added libvirt function to create volumes with by XML files
* Added utility function to get QEMU threads (get_qemu_threads)
And many other changes. Again, for a complete list of changes please
check the Avocado-VT changelog[2].
Install avocado
---------------
Instructions are available in our documentation on how to install
either with packages or from source[3].
Updated RPM packages are be available in the project repos for
Fedora 22, Fedora 23, EPEL 6 and EPEL 7.
Happy hacking and testing!
---
| [1] https://github.com/avocado-framework/avocado/compare/0.32.0...0.33.0
| [2] https://github.com/avocado-framework/avocado-vt/compare/0.32.0...0.33.0
| [3] http://avocado-framework.readthedocs.org/en/latest/GetStartedGuide.html#installing-avocado
| Sprint Theme: https://www.youtube.com/watch?v=H5Lg_14m-sM
============================
0.34.0: The Hour of the Star
============================
Hello to all test enthusiasts out there, specially to those that
cherish, care or are just keeping an eye on the greenest test
framework there is: Avocado release 0.34.0, aka The Hour of the Star,
is now out!
Release Notes
=============
The main changes in Avocado for this release are:
* A complete overhaul of the logging and output implementation. This
means that all Avocado output uses the standard Python logging library
making it very consistent and easy to understand [1].
* Based on the logging and output overhaul, the command line test
runner is now very flexible with its output. A user can choose
exactly what should be output. Examples include application output
only, test output only, both application and test output or any
other combination of the builtin streams. The user visible command
line option that controls this behavior is `--show`, which is an
application level option, that is, it's available to all avocado
commands. [2]
* Besides the builtin streams, test writers can use the standard
Python logging API to create new streams. These streams can be shown
on the command line as mentioned before, or persisted automatically
in the job results by means of the `--store-logging-stream` command
line option. [3][4]
* The new `avocado.core.safeloader` module, intends to make it easier
to to write new test loaders for various types of Python
code. [5][6]
* Based on the new `avocado.core.safeloader` module, a contrib script
called `avocado-find-unittests`, returns the name of
unittest.TestCase based tests found on a given number of Python
source code files. [7]
* Avocado is now able to run its own selftest suite. By leveraging the
`avocado-find-unittests` contrib script and the External Runner [8]
feature. A Makefile target is available, allowing developers to run
`make selfcheck` to have the selftest suite run by Avocado. [9]
* Partial Python 3 support. A number of changes were introduced that
allow concurrent Python 2 and 3 support on the same code base. Even
though the support for Python 3 is still *incomplete*, the `avocado`
command line application can already run some limited commands at
this point.
* Asset fetcher utility library. This new utility library, and
INSTRUMENTED test feature, allows users to transparently request
external assets to be used in tests, having them cached for later
use. [10]
* Further cleanups in the public namespace of the avocado Test class.
* [BUG FIX] Input from the local system was being passed to remote
systems when running tests with either in remote systems or VMs.
* [BUG FIX] HTML report stability improvements, including better
Unicode handling and support for other versions of the Pystache
library.
* [BUG FIX] Atomic updates of the "latest" job symlink, allows for
more reliable user experiences when running multiple parallel jobs.
* [BUG FIX] The avocado.core.data_dir module now dynamically checks
the configuration system when deciding where the data directory
should be located. This allows for later updates, such as when
giving one extra `--config` parameter in the command line, to be
applied consistently throughout the framework and test code.
* [MAINTENANCE] The CI jobs now run full checks on each commit on
any proposed PR, not only on its topmost commit. This gives higher
confidence that a commit in a series is not causing breakage that
a later commit then inadvertently fixes.
For a complete list of changes please check the Avocado changelog[11].
For Avocado-VT, please check the full Avocado-VT changelog[12].
Avocado Videos
--------------
As yet another way to let users know about what's available in
Avocado, we're introducing short videos with very targeted content on
our very own YouTube channel:
https://www.youtube.com/channel/UCP4xob52XwRad0bU_8V28rQ
The first video available demonstrates a couple of new features
related to the advanced logging mechanisms, introduced on this
release:
https://www.youtube.com/watch?v=8Ur_p5p6YiQ
Install avocado
---------------
Instructions are available in our documentation on how to install
either with packages or from source[13].
Updated RPM packages are be available in the project repos for
Fedora 22, Fedora 23, EPEL 6 and EPEL 7.
Happy hacking and testing!
---
| [1] http://avocado-framework.readthedocs.org/en/0.34.0/LoggingSystem.html
| [2] http://avocado-framework.readthedocs.org/en/0.34.0/LoggingSystem.html#tweaking-the-ui
| [3] http://avocado-framework.readthedocs.org/en/0.34.0/LoggingSystem.html#storing-custom-logs
| [4] http://avocado-framework.readthedocs.org/en/0.34.0/WritingTests.html#advanced-logging-capabilities
| [5] https://github.com/avocado-framework/avocado/blob/0.34.0/avocado/core/safeloader.py
| [6] http://avocado-framework.readthedocs.org/en/0.34.0/api/core/avocado.core.html#module-avocado.core.safeloader
| [7] https://github.com/avocado-framework/avocado/blob/0.34.0/contrib/avocado-find-unittests
| [8] http://avocado-framework.readthedocs.org/en/0.34.0/GetStartedGuide.html#running-tests-with-an-external-runner
| [9] https://github.com/avocado-framework/avocado/blob/0.34.0/Makefile#L33
| [10] http://avocado-framework.readthedocs.org/en/0.34.0/WritingTests.html#fetching-asset-files
| [11] https://github.com/avocado-framework/avocado/compare/0.33.0...0.34.0
| [12] https://github.com/avocado-framework/avocado-vt/compare/0.33.0...0.34.0
| [13] http://avocado-framework.readthedocs.org/en/latest/GetStartedGuide.html#installing-avocado
| Sprint Theme: https://trello.com/c/QIbM3NvY/590-sprint-theme
===============
35.0: Mr. Robot
===============
This is another proud announcement: Avocado release 35.0, aka "Mr. Robot",
is now out!
Release Notes
=============
This release, while a "regular" release, will also serve as a beta for
our first "long term stability" (aka "lts") release. That means that
the next release, will be version "36.0lts" and will receive only bug
fixes and minor improvements. So, expect release 35.0 to be pretty
much like "36.0lts" feature-wise. New features will make into the
"37.0" release, to be released after "36.0lts". Read more about the
details on the specific RFC[9].
The main changes in Avocado for this release are:
* A big round of fixes and on machine readable output formats, such
as xunit (aka JUnit) and JSON. The xunit output, for instance,
now includes tests with schema checking. This should make sure
interoperability is even better on this release.
* Much more robust handling of test references, aka test URLs.
Avocado now properly handles very long test references, and also
test references with non-ascii characters.
* The avocado command line application now provides richer exit
status[1]. If your application or custom script depends on the
avocado exit status code, you should be fine as avocado still
returns zero for success and non-zero for errors. On error
conditions, though, the exit status code are richer and made of
combinable (ORable) codes. This way it's possible to detect that,
say, both a test failure and a job timeout occurred in a single
execution.
* [SECURITY RELATED] The remote execution of tests (including in
Virtual Machines) now allows for proper checks of host keys[2].
Without these checks, avocado is susceptible to a man-in-the-middle
attack, by connecting and sending credentials to the wrong machine.
This check is *disabled* by default, because users depend on this
behavior when using machines without any prior knowledge such as
cloud based virtual machines. Also, a bug in the underlying SSH
library may prevent existing keys to be used if these are in ECDSA
format[3]. There's an automated check in place to check for the
resolution of the third party library bug. Expect this feature to
be *enabled* by default in the upcoming releases.
* Pre/Post Job hooks. Avocado now defines a proper interface for
extension/plugin writers to execute actions while a Job is runnning.
Both Pre and Post hooks have access to the Job state (actually, the
complete Job instance). Pre job hooks are called before tests are
run, and post job hooks are called at the very end of the job (after
tests would have usually finished executing).
* Pre/Post job scripts[4]. As a feature built on top of the Pre/Post job
hooks described earlier, it's now possible to put executable scripts
in a configurable location, such as `/etc/avocado/scripts/job/pre.d`
and have them called by Avocado before the execution of tests. The
executed scripts will receive some information about the job via
environment variables[5].
* The implementation of proper Test-IDs[6] in the test result
directory.
Also, while not everything is (yet) translated into code, this release
saw various and major RFCs, which are definitely shaping the future of
Avocado. Among those:
* Introduce proper test IDs[6]
* Pre/Post *test* hooks[7]
* Multi-stream tests[8]
* Avocado maintainability and integration with avocado-vt[9]
* Improvements to job status (completely implemented)[10]
For a complete list of changes please check the Avocado changelog[11].
For Avocado-VT, please check the full Avocado-VT changelog[12].
Install avocado
---------------
Instructions are available in our documentation on how to install
either with packages or from source[13].
Updated RPM packages are be available in the project repos for
Fedora 22, Fedora 23, EPEL 6 and EPEL 7.
Packages
--------
As a heads up, we still package the latest version of the various
Avocado sub projects, such as the very popular Avocado-VT and the
pretty much experimental Avocado-Virt and Avocado-Server projects.
For the upcoming releases, there will be changes in our package
offers, with a greater focus on long term stability packages for
Avocado. Other packages may still be offered as a convenience, or
may see a change of ownership. All in the best interest of our users.
If you have any concerns or questions, please let us know.
Happy hacking and testing!
---
| [1] http://avocado-framework.readthedocs.org/en/35.0/ResultFormats.html#exit-codes
| [2] https://github.com/avocado-framework/avocado/blob/35.0/etc/avocado/avocado.conf#L41
| [3] https://github.com/avocado-framework/avocado/blob/35.0/selftests/functional/test_thirdparty_bugs.py#L17
| [4] http://avocado-framework.readthedocs.org/en/35.0/ReferenceGuide.html#job-pre-and-post-scripts
| [5] http://avocado-framework.readthedocs.org/en/35.0/ReferenceGuide.html#script-execution-environment
| [6] https://www.redhat.com/archives/avocado-devel/2016-March/msg00024.html
| [7] https://www.redhat.com/archives/avocado-devel/2016-April/msg00000.html
| [8] https://www.redhat.com/archives/avocado-devel/2016-April/msg00042.html
| [9] https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html
| [10] https://www.redhat.com/archives/avocado-devel/2016-April/msg00010.html
| [11] https://github.com/avocado-framework/avocado/compare/0.34.0...35.0
| [13] https://github.com/avocado-framework/avocado-vt/compare/0.34.0...35.0
| [12] http://avocado-framework.readthedocs.org/en/35.0/GetStartedGuide.html#installing-avocado
| Sprint Theme: https://trello.com/c/7dWknPDJ/637-sprint-theme
=======
36.0lts
=======
This is a very proud announcement: Avocado release 36.0lts, our very
first "Long Term Stability" release, is now out!
LTS in a nutshell
=================
This release marks the beginning of a special cycle that will last for
18 months. Avocado usage in production environments should favor the
use of this LTS release, instead of non-LTS releases.
Bug fixes will be provided on the "36lts"[1] branch until, at least,
September 2017. Minor releases, such as "36.1lts", "36.2lts" an so
on, will be announced from time to time, incorporating those stability
related improvements.
Keep in mind that no new feature will be added. For more information,
please read the "Avocado Long Term Stability" RFC[2].
Changes from 35.0
=================
As mentioned in the release notes for the previous release (35.0),
only bug fixes and other stability related changes would be added to
what is now 36.0lts. For the complete list of changes, please check
the GIT repo change log[3].
Install avocado
===============
The Avocado LTS packages are available on a separate repository, named
"avocado-lts". These repositories are available for Fedora 22, Fedora
23, EPEL 6 and EPEL 7.
Updated ".repo" files are available on the usual locations:
* https://repos-avocadoproject.rhcloud.com/static/avocado-fedora.repo
* https://repos-avocadoproject.rhcloud.com/static/avocado-el.repo
Those repo files now contain definitions for both the "LTS" and
regular repositories. Users interested in the LTS packages, should
disable the regular repositories and enable the "avocado-lts" repo.
Instructions are available in our documentation on how to install
either with packages or from source[4].
Happy hacking and testing!
---
| [1] https://github.com/avocado-framework/avocado/tree/36lts
| [2] https://www.redhat.com/archives/avocado-devel/2016-April/msg00038.html
| [3] https://github.com/avocado-framework/avocado/compare/35.0...36.0lts
| [4] http://avocado-framework.readthedocs.io/en/36lts/GetStartedGuide.html#installing-avocado
===============================
37.0: Trabant vs. South America
===============================
This is another proud announcement: Avocado release 37.0, aka "Trabant
vs. South America", is now out!
Release Notes
=============
This release is yet another collection of bug fixes and some new
features. Along with the same changes that made the 36.0lts
release[1], this brings the following additional changes:
* TAP[2] version 12 support, bringing better integration with other
test tools that accept this streaming format as input.
* Added niceties on Avocado's utility libraries "build" and "kernel",
such as automatic parallelism and resource caching. It makes tests
such as "linuxbuild.py" (and your similar tests) run up to 10 times
faster.
* Fixed an issue where Avocado could leave processes behind after the
test was finished.
* Fixed a bug where the configuration for tests data directory would
be ignored.
* Fixed a bug where SIMPLE tests would not properly exit with WARN
status.
For a complete list of changes please check the Avocado changelog[3].
For Avocado-VT, please check the full Avocado-VT changelog[4].
Happy hacking and testing!
---
| [1] https://www.redhat.com/archives/avocado-devel/2016-May/msg00025.html
| [2] https://en.wikipedia.org/wiki/Test_Anything_Protocol
| [3] https://github.com/avocado-framework/avocado/compare/35.0...37.0
| [4] https://github.com/avocado-framework/avocado-vt/compare/35.0...37.0
| [5] http://avocado-framework.readthedocs.io/en/37.0/GetStartedGuide.html#installing-avocado
| Sprint Theme: https://trello.com/c/XbIUqU1Y/673-sprint-theme
===============
38.0: Love, Ken
===============
You guessed it right: this is another Avocado release announcement:
release 38.0, aka "Love, Ken", is now out!
Release Notes
=============
Another development cycle has just finished, and our community will
receive this new release containing a nice assortment of bug fixes and
new features.
* The download of assets in tests now allow for an expiration time.
This means that tests that need to download any kind of external
asset, say a tarball, can now automatically benefit from the
download cache, but can also keep receiving new versions
automatically.
Suppose your asset uses an asset named `myproject-daily.tar.bz2`,
and that your test runs 50 times a day. By setting the expire time
to `1d` (1 day), your test will benefit from cache on most runs, but
will still fetch the new version when the 24 hours from the
first download have passed.
For more information, please check out the
`documentation <http://avocado-framework.readthedocs.io/en/38.0/WritingTests.html>`_
on the `expire` parameter to the `fetch_asset()` method.
* Environment variables can be propagated into tests running on remote
systems. It's a known fact that one way to influence application behavior,
including test, is to set environment variables. A command line such as::
$ MYAPP_DEBUG=1 avocado run myapp_test.py
Will work as expected on a local system. But Avocado also allows
running tests on remote machines, and up until now, it has been
lacking a way to propagate environment variables to the remote
system.
Now, you can use::
$ MYAPP_DEBUG=1 avocado run --env-keep MYAPP_DEBUG \
--remote-host test-machine myapp_test.py
* The plugin interfaces have been moved into the
`avocado.core.plugin_interfaces` module. This means that plugin
writers now have to import the interface definitions this namespace,
example::
...
from avocado.core.plugin_interfaces import CLICmd
class MyCommand(CLICmd):
...
This is a way to keep ourselves honest, and say that there's no
difference from plugin interfaces to Avocado's core implementation,
that is, they may change at will. For greater stability, one should
be tracking the LTS releases.
Also, it effectively makes all plugins the same, whether they're
implemented and shipped as part of Avocado, or as part of external
projects.
* A contrib script for running kvm-unit-tests. As some people are
aware, Avocado has indeed a close relation to virtualization
testing. Avocado-VT is one obvious example, but there are other
virtualization related test suites can Avocado can run.
This release adds a contrib script that will fetch, download,
compile and run kvm-unit-tests using Avocado's external runner
feature. This gives results in a better granularity than the
support that exists in Avocado-VT, which gives only a single
PASS/FAIL for the entire test suite execution.
For more information, please check out the `Avocado changelog
<https://github.com/avocado-framework/avocado/compare/37.0...38.0>`_.
Avocado-VT
----------
Also, while we focused on Avocado, let's also not forget that
Avocado-VT maintains it's own fast pace of incoming niceties.
* s390 support: Avocado-VT is breaking into new grounds, and now has
support for the s390 architecture. Fedora 23 for s390 has been added
as a valid guest OS, and s390-virtio has been added as a new machine
type.
* Avocado-VT is now more resilient against failures to persist its
environment file, and will only give warnings instead of errors when
it fails to save it.
* An improved implementation of the "job lock" plugin, which prevents
multiple Avocado jobs with VT tests to run simultaneously. Since
there's no finer grained resource locking in Avocado-VT, this is a
global lock that will prevent issues such as image corruption when
two jobs are run at the same time.
This new implementation will now check if existing lock files are
stale, that is, they are leftovers from previous run. If the
processes associated with these files are not present, the stale
lock files are deleted, removing the need to clean them up manually.
It also outputs better debugging information when failures to
acquire lock.
The complete list of changes to Avocado-VT are available on
`Avocado-VT changelog <https://github.com/avocado-framework/avocado-vt/compare/37.0...38.0>`_.
Miscellaneous
-------------
While not officially part of this release, this development cycle saw
the introduction of new tests on our
`avocado-misc-tests <https://github.com/avocado-framework/avocado-misc-tests>`_.
Go check it out!
Finally, since Avocado and Avocado-VT are not newly born anymore, we
decided to update information mentioning KVM-Autotest, virt-test on so
on around the web. This will hopefully redirect new users to the Avocado
community and avoid confusion.
Happy hacking and testing!
---
Sprint Theme: https://trello.com/c/Y6IIFXBS/732-sprint-theme
......@@ -47,6 +47,20 @@ API Reference
api/core/avocado.core.rst
api/plugins/avocado.plugins.rst
=====================
Avocado Release Notes
=====================
.. toctree::
:maxdepth: 1
38_0
37_0
36_0
35_0
0_34_0
0_33_0
0_32_0
Indices and tables
==================
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册