YAML Loader: move it into its own plugin and package

The newly introduced YAML Loader plugin is currently hosted on the
same plugin (directory and module) as the varianter one.  While they
share a lot of code, they have different purposes.

Let's move the YAML loader into its own plugin, so that its
installation is also optional and the separation is clearer.
Signed-off-by: NCleber Rosa <crosa@redhat.com>

docs/source/optional_plugins/index.rst

@@ -13,3 +13,4 @@ optional plugins:
    results
    robot
    varianter_yaml_to_mux
+   yaml_loader

docs/source/optional_plugins/varianter_yaml_to_mux.rst

@@ -543,29 +543,3 @@ From this example you can see that querying for ``/env/debug`` works only in
 the first variant, but returns nothing in the second variant. Keep this in mind
 and when you use the ``!mux`` flag always query for the pre-mux path,
 ``/env/*`` in this example.
-
-
-Yaml_to_mux_loader plugin
-=========================
-
-This plugin is part of the `Yaml_to_mux` plugin and it understands the same
-content, only it works on loader-level, rather than on test variants level.
-The result is that this plugin tries to open the test reference as if it was
-a file specifying variants and if it succeeds it iterates through variants
-and looks for `test_reference` entries. On success it attempts to discover
-the reference using either loader defined by `test_reference_resolver_class`
-or it fall-backs to `FileLoader` when not specified. Then it assigns the
-current variant's params to all of the discovered tests. This way one can
-freely assign various variants to different tests.
-
-Keep in mind YAML files (in Avocado) are ordered, therefor variant name won't
-re-arrange the test order. The only exception is when you use the same variant
-name twice, then the second one will get merged into the first one.
-
-Also note that in case of no `test_reference` or just when no tests are
-discovered in the current variant, there is no error, no warning and
-the loader reports the discovered tests (if any) without the variant
-which did not produced any tests.
-
-The simplest way to learn about this plugin is to look at examples in
-``examples/yaml_to_mux_loader/``.

docs/source/optional_plugins/yaml_loader.rst

+YAML Loader (yaml_loader)
+=========================
+
+This plugin is related to `Yaml_to_mux` plugin and it understands the same
+content, only it works on loader-level, rather than on test variants level.
+The result is that this plugin tries to open the test reference as if it was
+a file specifying variants and if it succeeds it iterates through variants
+and looks for `test_reference` entries. On success it attempts to discover
+the reference using either loader defined by `test_reference_resolver_class`
+or it fall-backs to `FileLoader` when not specified. Then it assigns the
+current variant's params to all of the discovered tests. This way one can
+freely assign various variants to different tests.
+
+Keep in mind YAML files (in Avocado) are ordered, therefor variant name won't
+re-arrange the test order. The only exception is when you use the same variant
+name twice, then the second one will get merged into the first one.
+
+Also note that in case of no `test_reference` or just when no tests are
+discovered in the current variant, there is no error, no warning and
+the loader reports the discovered tests (if any) without the variant
+which did not produced any tests.
+
+The simplest way to learn about this plugin is to look at examples in
+``examples/yaml_to_mux_loader/``.

optional_plugins/loader_yaml/MANIFEST.in

+include VERSION

optional_plugins/loader_yaml/VERSION

+52.0

optional_plugins/loader_yaml/avocado_loader_yaml/__init__.py

+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See LICENSE for more details.
+#
+# Copyright: Red Hat Inc. 2017
+# Author: Lukas Doktor <ldoktor@redhat.com>
+"""Avocado Plugin that loads tests from YAML files"""
+
+import copy
+
+from avocado.core import loader
+from avocado.core import mux
+from avocado.core import varianter
+from avocado.core.plugin_interfaces import CLI
+from avocado_varianter_yaml_to_mux import create_from_yaml
+
+
+class YamlTestsuiteLoader(loader.TestLoader):
+
+    """
+    Gets variants from a YAML file and uses `test_reference` entries
+    to create a test suite.
+    """
+
+    name = "yaml_testsuite"
+    _extra_type_label_mapping = {}
+    _extra_decorator_mapping = {}
+
+    @staticmethod
+    def get_type_label_mapping():
+        """
+        No type is discovered by default, uses "full_*_mappings" to report
+        the actual types after "discover()" is called.
+        """
+        return {}
+
+    def get_full_type_label_mapping(self):
+        return self._extra_type_label_mapping
+
+    @staticmethod
+    def get_decorator_mapping():
+        return {}
+
+    def get_full_decorator_mapping(self):
+        return self._extra_decorator_mapping
+
+    def _get_loader(self, params):
+        """
+        Initializes test loader according to params.
+
+        Uses params.get():
+          test_reference_resolver_class - loadable location of the loader class
+          test_reference_resolver_args - args to override current Avocado args
+                                         before being passed to the loader
+                                         class. (dict)
+          test_reference_resolver_extra - extra_params to be passed to resolver
+                                          (dict)
+        """
+        resolver_class = params.get("test_reference_resolver_class")
+        if not resolver_class:
+            if params.get("test_reference"):
+                resolver_class = "avocado.core.loader.FileLoader"
+            else:
+                # Don't supply the default when no `test_reference` is given
+                # to avoid listing default FileLoader tests
+                return None
+        mod, klass = resolver_class.rsplit(".", 1)
+        try:
+            loader_class = getattr(__import__(mod, fromlist=[klass]), klass)
+        except ImportError:
+            raise RuntimeError("Unable to import class defined by test_"
+                               "reference_resolver_class '%s.%s'"
+                               % (mod, klass))
+        _args = params.get("test_reference_resolver_args")
+        if not _args:
+            args = self.args
+        else:
+            args = copy.deepcopy(self.args)
+            for key, value in _args.iteritems():
+                setattr(args, key, value)
+        extra_params = params.get("test_reference_resolver_extra", default={})
+        return loader_class(args, extra_params)
+
+    def discover(self, reference, which_tests=loader.DEFAULT):
+        tests = []
+        try:
+            root = mux.apply_filters(create_from_yaml([reference], False),
+                                     getattr(self.args, "mux_suite_only", []),
+                                     getattr(self.args, "mux_suite_out", []))
+        except Exception:
+            return []
+        mux_tree = mux.MuxTree(root)
+        for variant in mux_tree:
+            params = varianter.AvocadoParams(variant, "YamlTestsuiteLoader",
+                                             ["/run/*"], {})
+            reference = params.get("test_reference")
+            test_loader = self._get_loader(params)
+            if not test_loader:
+                continue
+            _tests = test_loader.discover(reference, which_tests)
+            self._extra_type_label_mapping.update(
+                test_loader.get_full_type_label_mapping())
+            self._extra_decorator_mapping.update(
+                test_loader.get_full_decorator_mapping())
+            if _tests:
+                for tst in _tests:
+                    tst[1]["params"] = (variant, ["/run/*"])
+                tests.extend(_tests)
+        return tests
+
+
+class LoaderYAML(CLI):
+
+    name = 'loader_yaml'
+    description = "YAML test loader options for the 'run' subcommand"
+
+    def configure(self, parser):
+        for name in ("list", "run"):
+            subparser = parser.subcommands.choices.get(name, None)
+            if subparser is None:
+                continue
+
+        mux = subparser.add_argument_group("yaml to mux testsuite options")
+        mux.add_argument("--mux-suite-only", nargs="+",
+                         help="Filter only part of the YAML suite file")
+        mux.add_argument("--mux-suite-out", nargs="+",
+                         help="Filter out part of the YAML suite file")
+
+    def run(self, args):
+        loader.loader.register_plugin(YamlTestsuiteLoader)

optional_plugins/loader_yaml/setup.py

+#!/bin/env python
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+#
+# See LICENSE for more details.
+#
+# Copyright: Red Hat Inc. 2017
+# Author: Cleber Rosa <crosa@redhat.com>
+
+from setuptools import setup, find_packages
+
+
+setup(name='avocado-framework-plugin-loader-yaml',
+      description='Avocado Plugin that loads tests from YAML files',
+      version=open("VERSION", "r").read().strip(),
+      author='Avocado Developers',
+      author_email='avocado-devel@redhat.com',
+      url='http://avocado-framework.github.io/',
+      packages=find_packages(),
+      include_package_data=True,
+      install_requires=['avocado-framework-plugin-varianter-yaml-to-mux'],
+      entry_points={
+          "avocado.plugins.cli": [
+              "loader_yaml = avocado_loader_yaml:LoaderYAML"]})

optional_plugins/varianter_yaml_to_mux/avocado_varianter_yaml_to_mux/__init__.py

@@ -314,111 +314,10 @@ class YamlToMuxCLI(CLI):
                              help="DEPRECATED: Filter out path(s) from "
                              "multiplexing (use --mux-filter-out instead)")
 
-            mux = subparser.add_argument_group("yaml to mux testsuite options")
-            mux.add_argument("--mux-suite-only", nargs="+",
-                             help="Filter only part of the YAML suite file")
-            mux.add_argument("--mux-suite-out", nargs="+",
-                             help="Filter out part of the YAML suite file")
-
     def run(self, args):
         """
         The YamlToMux varianter plugin handles these
         """
-        loader.loader.register_plugin(YamlTestsuiteLoader)
-
-
-class YamlTestsuiteLoader(loader.TestLoader):
-
-    """
-    Gets variants from a YAML file and uses `test_reference` entries
-    to create a test suite.
-    """
-
-    name = "yaml_testsuite"
-    _extra_type_label_mapping = {}
-    _extra_decorator_mapping = {}
-
-    @staticmethod
-    def get_type_label_mapping():
-        """
-        No type is discovered by default, uses "full_*_mappings" to report
-        the actual types after "discover()" is called.
-        """
-        return {}
-
-    def get_full_type_label_mapping(self):
-        return self._extra_type_label_mapping
-
-    @staticmethod
-    def get_decorator_mapping():
-        return {}
-
-    def get_full_decorator_mapping(self):
-        return self._extra_decorator_mapping
-
-    def _get_loader(self, params):
-        """
-        Initializes test loader according to params.
-
-        Uses params.get():
-          test_reference_resolver_class - loadable location of the loader class
-          test_reference_resolver_args - args to override current Avocado args
-                                         before being passed to the loader
-                                         class. (dict)
-          test_reference_resolver_extra - extra_params to be passed to resolver
-                                          (dict)
-        """
-        resolver_class = params.get("test_reference_resolver_class")
-        if not resolver_class:
-            if params.get("test_reference"):
-                resolver_class = "avocado.core.loader.FileLoader"
-            else:
-                # Don't supply the default when no `test_reference` is given
-                # to avoid listing default FileLoader tests
-                return None
-        mod, klass = resolver_class.rsplit(".", 1)
-        try:
-            loader_class = getattr(__import__(mod, fromlist=[klass]), klass)
-        except ImportError:
-            raise RuntimeError("Unable to import class defined by test_"
-                               "reference_resolver_class '%s.%s'"
-                               % (mod, klass))
-        _args = params.get("test_reference_resolver_args")
-        if not _args:
-            args = self.args
-        else:
-            args = copy.deepcopy(self.args)
-            for key, value in _args.iteritems():
-                setattr(args, key, value)
-        extra_params = params.get("test_reference_resolver_extra", default={})
-        return loader_class(args, extra_params)
-
-    def discover(self, reference, which_tests=loader.DEFAULT):
-        tests = []
-        try:
-            root = mux.apply_filters(create_from_yaml([reference], False),
-                                     getattr(self.args, "mux_suite_only", []),
-                                     getattr(self.args, "mux_suite_out", []))
-        except Exception:
-            return []
-        mux_tree = mux.MuxTree(root)
-        for variant in mux_tree:
-            params = varianter.AvocadoParams(variant, "YamlTestsuiteLoader",
-                                             ["/run/*"], {})
-            reference = params.get("test_reference")
-            test_loader = self._get_loader(params)
-            if not test_loader:
-                continue
-            _tests = test_loader.discover(reference, which_tests)
-            self._extra_type_label_mapping.update(
-                test_loader.get_full_type_label_mapping())
-            self._extra_decorator_mapping.update(
-                test_loader.get_full_decorator_mapping())
-            if _tests:
-                for tst in _tests:
-                    tst[1]["params"] = (variant, ["/run/*"])
-                tests.extend(_tests)
-        return tests
 
 
 class YamlToMux(mux.MuxPlugin, Varianter):

python-avocado.spec

@@ -135,6 +135,9 @@ popd
 pushd optional_plugins/varianter_yaml_to_mux
 %{__python} setup.py build
 popd
+pushd optional_plugins/loader_yaml
+%{__python} setup.py build
+popd
 %{__make} man
 
 %install
@@ -157,6 +160,9 @@ popd
 pushd optional_plugins/varianter_yaml_to_mux
 %{__python} setup.py install --root %{buildroot} --skip-build
 popd
+pushd optional_plugins/loader_yaml
+%{__python} setup.py install --root %{buildroot} --skip-build
+popd
 %{__mkdir} -p %{buildroot}%{_mandir}/man1
 %{__install} -m 0644 man/avocado.1 %{buildroot}%{_mandir}/man1/avocado.1
 %{__install} -m 0644 man/avocado-rest-client.1 %{buildroot}%{_mandir}/man1/avocado-rest-client.1
@@ -183,6 +189,9 @@ popd
 pushd optional_plugins/varianter_yaml_to_mux
 %{__python} setup.py develop --user
 popd
+pushd optional_plugins/loader_yaml
+%{__python} setup.py develop --user
+popd
 # Package build environments have the least amount of resources
 # we have observed so far.  Let's avoid tests that require too
 # much resources or are time sensitive
@@ -225,6 +234,7 @@ AVOCADO_CHECK_LEVEL=0 selftests/run
 %exclude %{python_sitelib}/avocado_framework_plugin_runner_docker*
 %exclude %{python_sitelib}/avocado_framework_plugin_resultsdb*
 %exclude %{python_sitelib}/avocado_framework_plugin_varianter_yaml_to_mux*
+%exclude %{python_sitelib}/avocado_framework_plugin_loader_yaml*
 %{_libexecdir}/avocado/avocado-bash-utils
 %{_libexecdir}/avocado/avocado_debug
 %{_libexecdir}/avocado/avocado_error
@@ -329,6 +339,18 @@ defined in a yaml file(s).
 %{python_sitelib}/avocado_varianter_yaml_to_mux*
 %{python_sitelib}/avocado_framework_plugin_varianter_yaml_to_mux*
 
+%package plugins-loader-yaml
+Summary: Avocado Plugin that loads tests from YAML files
+Requires: %{name}-plugins-varianter-yaml-to-mux == %{version}
+
+%description plugins-loader-yaml
+Can be used to produce a test suite from definitions in a YAML file,
+similar to the one used in the yaml_to_mux varianter plugin.
+
+%files plugins-loader-yaml
+%{python_sitelib}/avocado_loader_yaml*
+%{python_sitelib}/avocado_framework_plugin_loader_yaml*
+
 %package examples
 Summary: Avocado Test Framework Example Tests
 Requires: %{name} == %{version}