{ "name": "Release Test Plan", "description": "Test Plan to be performed before a release is cut", "tests": [ {"name": "Avocado source is sound", "description": "On your development machine, on a freshen Avocado source to be released, run `$ make check`. Expected result: Make command should say OK."}, {"name": "Avocado RPM build", "description": "On your development machine, build the Avocado RPM packages using: `$ make build-rpm-all`. Expected result: SRPM file `SRPMS/avocado-x.y.z-r.distro.src.rpm` and RPM files at `RPMS/noarch`"}, {"name": "Avocado RPM install", "description": "On a fresh virtual machine, perform the installation of Avocado using the packages built on test 'Avocado RPM build'."}, {"name": "Avocado Test Run on RPM based installation", "description": "On the same machine you just installed Avocado used during RPM packages ('Avocado RPM install'), run the simplest possible test with `$ avocado run passtest`. Expected results: `(1/1) passtest.py: PASS (0.00 s)`. After the test, shutdown the virtual machine."}, {"name": "Avocado Test Run on Virtual Machine", "description": "On your development machine, execute Avocado with the Virtual Machine plugin, so that a test will be run on the machine you just installed Avocado using RPM packages. Run: `$ avocado run passtest --vm-domain --vm-username --vm-password --vm-hostname `. Expected results: `(1/1) /home//(...)/passtest.py: PASS (0.00 s)`"}, {"name": "Avocado Test Run on Remote Machine", "description": "On your development machine, execute Avocado with the Remote plugin, so that a test will be run on the machine you just installed Avocado using RPM packages. Run: `$ avocado run passtest --remote-username --remote-password --remote-hostname --html /tmp/report.html`. Expected results: `(1/1) /home//(...)/passtest.py: PASS (0.00 s)`"}, {"name": "Avocado Remote Machine HTML report", "description": "On a web browser, open the previously generated HTML report at `/tmp/report.html`. Verify that all the links such as `job-YYYY-MM-DD...` (under `Results Dir`), `passtest.py` (under `Test ID`) and `debug.log` point to valid locations."}, {"name": "Avocado Server Source Checkout and Unittests", "description": "On the same virtual machine used during 'Avocado RPM install', checkout a copy of the avocado-server repository with `$ git clone git://github.com/avocado-framework/avocado-server.git`. Then install dependencies with `$ pip install -r requirements.txt`. Then finally run the unittests with `$ make check`. Expected result: `OK`"}, {"name": "Avocado Server Run", "description": "On the same machine used during 'Avocado Server Source Checkout and Unittests', run the quick and dirty script to get it running `$ ./reset-and-run.sh`. Set admin password to `123`. Expected result: `Starting development server at http://0.0.0.0:9405`"}, {"name": "Avocado Server Functional Test", "description": "On the same machine used during 'Avocado Server Run', on another console, run `$ avocado run selftests/all/functional/avocadoserver/api.py`. Expected result: `PASS`"}, {"name": "Avocado Virt and VT Source Checkout", "description": "On a virtualization capable machine, checkout a copy of the avocado-vt, avocado-virt, avocado-virt-tests repositories with `$ git clone git://github.com/avocado-framework/avocado-vt.git`, `$ git clone git://github.com/avocado-framework/avocado-virt.git` and `$ git clone git://github.com/avocado-framework/avocado-virt-tests.git`. On the Avocado (non-virt) repo, run `make link`. Now run `$ avocado plugins`. Expected result: `virt_bootstrap`, `virt`, `virt_test_compat_runner` and `virt_test_compat_lister` plugins enabled."}, {"name": "Avocado Virt Bootstrap", "description": "Now that the `virt_bootstrap` plugin and action `virt-bootstrap` is available run `$ avocado virt-bootstrap`. Expected result: `Your system appears to be all set to execute tests`"}, {"name": "Avocado Virt Boot Test Run and HTML report", "description": "Now that your system is setup to run virtualization tests, run the boot test with `$ avocado run avocado-virt-tests/qemu/boot.py --sysinfo on --open-browser`. Expected result: `PASS`, followed by a web browser showing an HTML report"}, {"name": "Avocado Virt - Assignment of values from the cmdline", "description": "Test that we can assign values from the cmdline. Create a new qemu bin path, for example sudo ln -s `/usr/bin/qemu-kvm /usr/bin/qemu-kvm-avocado-functional`, then run the boot test with `$ avocado run avocado-virt-tests/qemu/boot.py --qemu-bin /usr/bin/qemu-kvm-avocado-functional --sysinfo on --open-browser`. Expected result: `PASS`, followed by the web report. Look at the job log, it should list the path `/usr/bin/qemu-kvm-avocado-functional` in the QEMU command line."}, {"name": "Avocado Virt - Migration test", "description": "Run the boot test with `$ avocado run avocado-virt-tests/qemu/migration/migration.py --sysinfo on --open-browser`. Expected result: `PASS`, followed by a web browser showing an HTML report"}, {"name": "Avocado VT - Bootstrap", "description": "Configure avocado vt as described in http://avocado-vt.readthedocs.org/en/latest/GetStartedGuide.html. Expected result: No errors."}, {"name": "Avocado VT - List tests", "description": "Run `avocado list`. Expected result: The output should list avocado VT tests, such as `io-github-autotest-qemu.unattended_install.cdrom.extra_cdrom_ks.default_install.aio_native`"}, {"name": "Avocado VT - Run test", "description": "Run `.avocado run type_specific.io-github-autotest-qemu.migrate.default.tcp --sysinfo on --open-browser`. Expected result: `PASS`, followed by a web browser showing an HTML report"}, {"name": "Avocado HTML report sysinfo", "description": "On the HTML report, click on `Sysinfo (pre job, click to expand)` and verify that system information such as `hostname` and `cpuinfo` are present and accurate"}, {"name": "Avocado HTML report links", "description": "On the HTML report, verify that all the links such as `job-YYYY-MM-DD...` (under `Results Dir`), `boot.py` (under `Test ID`) and `debug.log` point to valid locations."} ] }