Merge lp:~billy-olsen/charms/trusty/rabbitmq-server/ch-sync-cli-fix into lp:~openstack-charmers-archive/charms/trusty/rabbitmq-server/next

[Billy Olsen]

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #7375 rabbitmq-server-next for billy-olsen mp266619
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7375/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #6843 rabbitmq-server-next for billy-olsen mp266619
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/6843/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #5546 rabbitmq-server-next for billy-olsen mp266619
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/11975719/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5546/

[lp:~billy-olsen/charms/trusty/rabbitmq-server/ch-sync-cli-fix]

104. By Billy Olsen on 2015-07-31

Include cli in the rmq amulet

[uosci-testing-bot (uosci-testing-bot)]

charm_lint_check #7376 rabbitmq-server-next for billy-olsen mp266619
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/7376/

[uosci-testing-bot (uosci-testing-bot)]

charm_unit_test #6844 rabbitmq-server-next for billy-olsen mp266619
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/6844/

[uosci-testing-bot (uosci-testing-bot)]

charm_amulet_test #5548 rabbitmq-server-next for billy-olsen mp266619
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/11976402/
Build: http://10.245.162.77:8080/job/charm_amulet_test/5548/

[Ryan Beisner (1chb1n)]

I would suggest merging this, even though the amulet test fails. It was previously failing that functional test.

This merge proposal resolves an import error, introduced by a charm-helpers CLI.

review: Approve

[Ryan Beisner (1chb1n)]

PS FYI, this is the only os-charm which had an amulet test affected by the CLI helper addition.

[Ryan Beisner (1chb1n)]

Regarding the pre-existing test failure, we will still need to address that separately.

[Edward Hope-Morley (hopem)]

Lets get this landed. It does not touch any code outside of amulet tests (which are broken anyway).

review: Approve

=== modified file 'charm-helpers-tests.yaml'
--- charm-helpers-tests.yaml	2015-04-13 22:11:34 +0000
+++ charm-helpers-tests.yaml	2015-07-31 22:01:37 +0000
@@ -2,4 +2,5 @@
 branch: lp:charm-helpers
 include:
     - core
+    - cli
     - contrib.ssl

=== modified file 'hooks/charmhelpers/core/hookenv.py'
--- hooks/charmhelpers/core/hookenv.py	2015-07-31 13:11:07 +0000
+++ hooks/charmhelpers/core/hookenv.py	2015-07-31 22:01:37 +0000
@@ -34,7 +34,22 @@
 import tempfile
 from subprocess import CalledProcessError
 
-from charmhelpers.cli import cmdline
+try:
+    from charmhelpers.cli import cmdline
+except ImportError as e:
+    # due to the anti-pattern of partially synching charmhelpers directly
+    # into charms, it's possible that charmhelpers.cli is not available;
+    # if that's the case, they don't really care about using the cli anyway,
+    # so mock it out
+    if str(e) == 'No module named cli':
+        class cmdline(object):
+            @classmethod
+            def subcommand(cls, *args, **kwargs):
+                def _wrap(func):
+                    return func
+                return _wrap
+    else:
+        raise
 
 import six
 if not six.PY3:

=== added directory 'tests/charmhelpers/cli'
=== added file 'tests/charmhelpers/cli/__init__.py'
--- tests/charmhelpers/cli/__init__.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/cli/__init__.py	2015-07-31 22:01:37 +0000
@@ -0,0 +1,195 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+import inspect
+import argparse
+import sys
+
+from six.moves import zip
+
+from charmhelpers.core import unitdata
+
+
+class OutputFormatter(object):
+    def __init__(self, outfile=sys.stdout):
+        self.formats = (
+            "raw",
+            "json",
+            "py",
+            "yaml",
+            "csv",
+            "tab",
+        )
+        self.outfile = outfile
+
+    def add_arguments(self, argument_parser):
+        formatgroup = argument_parser.add_mutually_exclusive_group()
+        choices = self.supported_formats
+        formatgroup.add_argument("--format", metavar='FMT',
+                                 help="Select output format for returned data, "
+                                      "where FMT is one of: {}".format(choices),
+                                 choices=choices, default='raw')
+        for fmt in self.formats:
+            fmtfunc = getattr(self, fmt)
+            formatgroup.add_argument("-{}".format(fmt[0]),
+                                     "--{}".format(fmt), action='store_const',
+                                     const=fmt, dest='format',
+                                     help=fmtfunc.__doc__)
+
+    @property
+    def supported_formats(self):
+        return self.formats
+
+    def raw(self, output):
+        """Output data as raw string (default)"""
+        if isinstance(output, (list, tuple)):
+            output = '\n'.join(map(str, output))
+        self.outfile.write(str(output))
+
+    def py(self, output):
+        """Output data as a nicely-formatted python data structure"""
+        import pprint
+        pprint.pprint(output, stream=self.outfile)
+
+    def json(self, output):
+        """Output data in JSON format"""
+        import json
+        json.dump(output, self.outfile)
+
+    def yaml(self, output):
+        """Output data in YAML format"""
+        import yaml
+        yaml.safe_dump(output, self.outfile)
+
+    def csv(self, output):
+        """Output data as excel-compatible CSV"""
+        import csv
+        csvwriter = csv.writer(self.outfile)
+        csvwriter.writerows(output)
+
+    def tab(self, output):
+        """Output data in excel-compatible tab-delimited format"""
+        import csv
+        csvwriter = csv.writer(self.outfile, dialect=csv.excel_tab)
+        csvwriter.writerows(output)
+
+    def format_output(self, output, fmt='raw'):
+        fmtfunc = getattr(self, fmt)
+        fmtfunc(output)
+
+
+class CommandLine(object):
+    argument_parser = None
+    subparsers = None
+    formatter = None
+    exit_code = 0
+
+    def __init__(self):
+        if not self.argument_parser:
+            self.argument_parser = argparse.ArgumentParser(description='Perform common charm tasks')
+        if not self.formatter:
+            self.formatter = OutputFormatter()
+            self.formatter.add_arguments(self.argument_parser)
+        if not self.subparsers:
+            self.subparsers = self.argument_parser.add_subparsers(help='Commands')
+
+    def subcommand(self, command_name=None):
+        """
+        Decorate a function as a subcommand. Use its arguments as the
+        command-line arguments"""
+        def wrapper(decorated):
+            cmd_name = command_name or decorated.__name__
+            subparser = self.subparsers.add_parser(cmd_name,
+                                                   description=decorated.__doc__)
+            for args, kwargs in describe_arguments(decorated):
+                subparser.add_argument(*args, **kwargs)
+            subparser.set_defaults(func=decorated)
+            return decorated
+        return wrapper
+
+    def test_command(self, decorated):
+        """
+        Subcommand is a boolean test function, so bool return values should be
+        converted to a 0/1 exit code.
+        """
+        decorated._cli_test_command = True
+        return decorated
+
+    def no_output(self, decorated):
+        """
+        Subcommand is not expected to return a value, so don't print a spurious None.
+        """
+        decorated._cli_no_output = True
+        return decorated
+
+    def subcommand_builder(self, command_name, description=None):
+        """
+        Decorate a function that builds a subcommand. Builders should accept a
+        single argument (the subparser instance) and return the function to be
+        run as the command."""
+        def wrapper(decorated):
+            subparser = self.subparsers.add_parser(command_name)
+            func = decorated(subparser)
+            subparser.set_defaults(func=func)
+            subparser.description = description or func.__doc__
+        return wrapper
+
+    def run(self):
+        "Run cli, processing arguments and executing subcommands."
+        arguments = self.argument_parser.parse_args()
+        argspec = inspect.getargspec(arguments.func)
+        vargs = []
+        kwargs = {}
+        for arg in argspec.args:
+            vargs.append(getattr(arguments, arg))
+        if argspec.varargs:
+            vargs.extend(getattr(arguments, argspec.varargs))
+        if argspec.keywords:
+            for kwarg in argspec.keywords.items():
+                kwargs[kwarg] = getattr(arguments, kwarg)
+        output = arguments.func(*vargs, **kwargs)
+        if getattr(arguments.func, '_cli_test_command', False):
+            self.exit_code = 0 if output else 1
+            output = ''
+        if getattr(arguments.func, '_cli_no_output', False):
+            output = ''
+        self.formatter.format_output(output, arguments.format)
+        if unitdata._KV:
+            unitdata._KV.flush()
+
+
+cmdline = CommandLine()
+
+
+def describe_arguments(func):
+    """
+    Analyze a function's signature and return a data structure suitable for
+    passing in as arguments to an argparse parser's add_argument() method."""
+
+    argspec = inspect.getargspec(func)
+    # we should probably raise an exception somewhere if func includes **kwargs
+    if argspec.defaults:
+        positional_args = argspec.args[:-len(argspec.defaults)]
+        keyword_names = argspec.args[-len(argspec.defaults):]
+        for arg, default in zip(keyword_names, argspec.defaults):
+            yield ('--{}'.format(arg),), {'default': default}
+    else:
+        positional_args = argspec.args
+
+    for arg in positional_args:
+        yield (arg,), {}
+    if argspec.varargs:
+        yield (argspec.varargs,), {'nargs': '*'}

=== added file 'tests/charmhelpers/cli/benchmark.py'
--- tests/charmhelpers/cli/benchmark.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/cli/benchmark.py	2015-07-31 22:01:37 +0000
@@ -0,0 +1,36 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+from . import cmdline
+from charmhelpers.contrib.benchmark import Benchmark
+
+
[e-mail](command_name='benchmark-start')
+def start():
+    Benchmark.start()
+
+
[e-mail](command_name='benchmark-finish')
+def finish():
+    Benchmark.finish()
+
+
[e-mail]_builder('benchmark-composite', description="Set the benchmark composite score")
+def service(subparser):
+    subparser.add_argument("value", help="The composite score.")
+    subparser.add_argument("units", help="The units the composite score represents, i.e., 'reads/sec'.")
+    subparser.add_argument("direction", help="'asc' if a lower score is better, 'desc' if a higher score is better.")
+    return Benchmark.set_composite_score

=== added file 'tests/charmhelpers/cli/commands.py'
--- tests/charmhelpers/cli/commands.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/cli/commands.py	2015-07-31 22:01:37 +0000
@@ -0,0 +1,32 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+"""
+This module loads sub-modules into the python runtime so they can be
+discovered via the inspect module. In order to prevent flake8 from (rightfully)
+telling us these are unused modules, throw a ' # noqa' at the end of each import
+so that the warning is suppressed.
+"""
+
+from . import CommandLine  # noqa
+
+"""
+Import the sub-modules which have decorated subcommands to register with chlp.
+"""
+import host  # noqa
+import benchmark  # noqa
+import unitdata  # noqa
+from charmhelpers.core import hookenv  # noqa

=== added file 'tests/charmhelpers/cli/host.py'
--- tests/charmhelpers/cli/host.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/cli/host.py	2015-07-31 22:01:37 +0000
@@ -0,0 +1,31 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+from . import cmdline
+from charmhelpers.core import host
+
+
[e-mail]()
+def mounts():
+    "List mounts"
+    return host.mounts()
+
+
[e-mail]_builder('service', description="Control system services")
+def service(subparser):
+    subparser.add_argument("action", help="The action to perform (start, stop, etc...)")
+    subparser.add_argument("service_name", help="Name of the service to control")
+    return host.service

=== added file 'tests/charmhelpers/cli/unitdata.py'
--- tests/charmhelpers/cli/unitdata.py	1970-01-01 00:00:00 +0000
+++ tests/charmhelpers/cli/unitdata.py	2015-07-31 22:01:37 +0000
@@ -0,0 +1,39 @@
+# Copyright 2014-2015 Canonical Limited.
+#
+# This file is part of charm-helpers.
+#
+# charm-helpers is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# charm-helpers is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with charm-helpers.  If not, see <http://www.gnu.org/licenses/>.
+
+from . import cmdline
+from charmhelpers.core import unitdata
+
+
[e-mail]_builder('unitdata', description="Store and retrieve data")
+def unitdata_cmd(subparser):
+    nested = subparser.add_subparsers()
+    get_cmd = nested.add_parser('get', help='Retrieve data')
+    get_cmd.add_argument('key', help='Key to retrieve the value of')
+    get_cmd.set_defaults(action='get', value=None)
+    set_cmd = nested.add_parser('set', help='Store data')
+    set_cmd.add_argument('key', help='Key to set')
+    set_cmd.add_argument('value', help='Value to store')
+    set_cmd.set_defaults(action='set')
+
+    def _unitdata_cmd(action, key, value):
+        if action == 'get':
+            return unitdata.kv().get(key)
+        elif action == 'set':
+            unitdata.kv().set(key, value)
+            unitdata.kv().flush()
+            return ''
+    return _unitdata_cmd

=== modified file 'tests/charmhelpers/core/hookenv.py'
--- tests/charmhelpers/core/hookenv.py	2015-07-31 13:11:07 +0000
+++ tests/charmhelpers/core/hookenv.py	2015-07-31 22:01:37 +0000
@@ -34,7 +34,22 @@
 import tempfile
 from subprocess import CalledProcessError
 
-from charmhelpers.cli import cmdline
+try:
+    from charmhelpers.cli import cmdline
+except ImportError as e:
+    # due to the anti-pattern of partially synching charmhelpers directly
+    # into charms, it's possible that charmhelpers.cli is not available;
+    # if that's the case, they don't really care about using the cli anyway,
+    # so mock it out
+    if str(e) == 'No module named cli':
+        class cmdline(object):
+            @classmethod
+            def subcommand(cls, *args, **kwargs):
+                def _wrap(func):
+                    return func
+                return _wrap
+    else:
+        raise
 
 import six
 if not six.PY3: