Merge lp:~bigdata-dev/charms/trusty/apache-hadoop-client/minimal into lp:~bigdata-dev/charms/trusty/apache-hadoop-client/trunk

[Cory Johns]

Refactored to use apache-hadoop-plugin instead of duplicating all the code.

[lp:~bigdata-dev/charms/trusty/apache-hadoop-client/minimal]

99. By Cory Johns on 2015-05-12

Added missing scope: container to subordinate relation

100. By Cory Johns on 2015-05-12

Merged upstream

[amir sanjar (asanjar)]

looks good

review: Approve

[amir sanjar (asanjar)]

review: Approve

=== modified file 'README.md'
--- README.md	2015-05-06 17:04:07 +0000
+++ README.md	2015-05-12 23:32:24 +0000
@@ -10,11 +10,12 @@
 
 ## Usage
 
-This charm is intended to be deployed via one of the
-[bundles](https://jujucharms.com/q/bigdata-dev/apache?type=bundle).
-For example:
+This charm is intended to be connected to the
+[core bundle](https://jujucharms.com/u/bigdata-dev/apache-core-batch-processing/):
 
-    juju quickstart u/bigdata-dev/apache-core-batch-processing
+    juju quickstart apache-core-batch-processing
+    juju deploy apache-hadoop-client client
+    juju add-relation client plugin
 
 This will deploy the Apache Hadoop platform with a single client unit.
 From there, you can manually load and run map-reduce jobs:
@@ -24,46 +25,9 @@
     hadoop jar my-job.jar
 
 
-## Deploying in Network-Restricted Environments
-
-The Apache Hadoop charms can be deployed in environments with limited network
-access. To deploy in this environment, you will need a local mirror to serve
-the packages and resources required by these charms.
-
-
-### Mirroring Packages
-
-You can setup a local mirror for apt packages using squid-deb-proxy.
-For instructions on configuring juju to use this, see the
-[Juju Proxy Documentation](https://juju.ubuntu.com/docs/howto-proxies.html).
-
-
-### Mirroring Resources
-
-In addition to apt packages, the Apache Hadoop charms require a few binary
-resources, which are normally hosted on Launchpad. If access to Launchpad
-is not available, the `jujuresources` library makes it easy to create a mirror
-of these resources:
-
-    sudo pip install jujuresources
-    juju resources fetch --all apache-hadoop-client/resources.yaml -d /tmp/resources
-    juju resources serve -d /tmp/resources
-
-This will fetch all of the resources needed by this charm and serve them via a
-simple HTTP server. You can then set the `resources_mirror` config option to
-have the charm use this server for retrieving resources.
-
-You can fetch the resources for all of the Apache Hadoop charms
-(`apache-hadoop-hdfs-master`, `apache-hadoop-yarn-master`,
-`apache-hadoop-compute-slave`, `apache-hadoop-client`, etc) into a single
-directory and serve them all with a single `juju resources serve` instance.
-
-
 ## Contact Information
 
-* Amir Sanjar <[e-mail]>
-* Cory Johns <[e-mail]>
-* Kevin Monroe <[e-mail]>
+[[e-mail]](mailto:[e-mail])
 
 
 ## Hadoop

=== modified file 'config.yaml'
--- config.yaml	2015-04-03 16:49:17 +0000
+++ config.yaml	2015-05-12 23:32:24 +0000
@@ -1,6 +1,1 @@
-options:
-    resources_mirror:
-        type: string
-        default: ''
-        description: |
-            URL from which to fetch resources (e.g., Hadoop binaries) instead of Launchpad.
+options: {}

=== removed file 'dist.yaml'
--- dist.yaml	2015-04-16 15:45:57 +0000
+++ dist.yaml	1970-01-01 00:00:00 +0000
@@ -1,116 +0,0 @@
-# This file contains values that are likely to change per distribution.
-# The aim is to make it easier to update / extend the charms with
-# minimal changes to the shared code in charmhelpers.
-vendor: 'apache'
-hadoop_version: '2.4.1'
-packages:
-    - 'libsnappy1'
-    - 'libsnappy-dev'
-    - 'openssl'
-    - 'liblzo2-2'
-groups:
-    - 'hadoop'
-    - 'mapred'
-    - 'supergroup'
-users:
-    ubuntu:
-        groups: ['hadoop', 'mapred', 'supergroup']
-    hdfs:
-        groups: ['hadoop']
-    mapred:
-        groups: ['hadoop', 'mapred']
-    yarn:
-        groups: ['hadoop']
-dirs:
-    hadoop:
-        path: '/usr/lib/hadoop'
-        perms: 0777
-    hadoop_conf:
-        path: '/etc/hadoop/conf'
-    hadoop_tmp:
-        path: '/tmp/hadoop'
-        perms: 0777
-    mapred_log:
-        path: '/var/log/hadoop/mapred'
-        owner: 'mapred'
-        group: 'hadoop'
-        perms: 0755
-    mapred_run:
-        path: '/var/run/hadoop/mapred'
-        owner: 'mapred'
-        group: 'hadoop'
-        perms: 0755
-    yarn_tmp:
-        path: '/tmp/hadoop-yarn'
-        perms: 0777
-    yarn_log_dir:
-        path: '/var/log/hadoop/yarn'
-        owner: 'yarn'
-        group: 'hadoop'
-        perms: 0755
-    hdfs_log_dir:
-        path: '/var/log/hadoop/hdfs'
-        owner: 'hdfs'
-        group: 'hadoop'
-        perms: 0755
-    hdfs_dir_base:
-        path: '/usr/local/hadoop/data'
-        owner: 'hdfs'
-        group: 'hadoop'
-        perms: 0755
-    cache_base:
-        path: '{dirs[hdfs_dir_base]}/cache'
-        owner: 'hdfs'
-        group: 'hadoop'
-        perms: 01775
-    cache_dir:
-        path: '{dirs[hdfs_dir_base]}/cache/hadoop'
-        owner: 'hdfs'
-        group: 'hadoop'
-        perms: 0775
-ports:
-    # Ports that need to be exposed, overridden, or manually specified.
-    # Only expose ports serving a UI or external API (i.e., namenode and
-    # resourcemanager).  Communication among units within the cluster does
-    # not need ports to be explicitly opened.
-    # If adding a port here, you will need to update
-    # charmhelpers.contrib.bigdata.handlers.apache or hooks/callbacks.py
-    # to ensure that it is supported.
-    namenode:
-        port: 8020
-        exposed_on: 'hdfs-master'
-    nn_webapp_http:
-        port: 50070
-        exposed_on: 'hdfs-master'
-    dn_webapp_http:
-        port: 50075
-        exposed_on: 'compute-slave'
-    resourcemanager:
-        port: 8032
-        exposed_on: 'yarn-master'
-    rm_webapp_http:
-        port: 8088
-        exposed_on: 'yarn-master'
-    rm_log:
-        port: 19888
-    nm_webapp_http:
-        port: 8042
-        exposed_on: 'compute-slave'
-    jobhistory:
-        port: 10020
-    jh_webapp_http:
-        port: 19888
-        exposed_on: 'yarn-master'
-    # TODO: support SSL
-    #nn_webapp_https:
-    #    port: 50470
-    #    exposed_on: 'hdfs-master'
-    #dn_webapp_https:
-    #    port: 50475
-    #    exposed_on: 'compute-slave'
-    #rm_webapp_https:
-    #    port: 8090
-    #    exposed_on: 'yarn-master'
-    #nm_webapp_https:
-    #    port: 8044
-    #    exposed_on: 'compute-slave'

=== removed file 'hooks/callbacks.py'
--- hooks/callbacks.py	2015-05-08 17:26:18 +0000
+++ hooks/callbacks.py	1970-01-01 00:00:00 +0000
@@ -1,36 +0,0 @@
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-"""
-Callbacks for additional setup tasks.
-
-Add any additional tasks / setup here.  If a callback is used by mutliple
-charms, consider refactoring it up to the charmhelpers library.
-"""
-from charmhelpers.contrib.bigdata import utils
-from charmhelpers.core import hookenv
-
-
-def update_etc_hosts():
-    if hookenv.in_relation_hook():
-        # send our hostname on the relation
-        local_host = hookenv.local_unit().replace('/', '-')
-        hookenv.relation_set(hostname=local_host)
-
-        # get /etc/hosts entries from the master
-        master_hosts = hookenv.relation_get('etc_hosts')
-
-        # update /etc/hosts on the local unit if we have master hostname data
-        if master_hosts:
-            hookenv.log('Updating /etc/hosts from %s' % hookenv.remote_unit())
-            utils.update_etc_hosts(master_hosts)
-        else:
-            hookenv.log('No /etc/hosts updates from %s' % hookenv.remote_unit())

=== removed file 'hooks/common.py'
--- hooks/common.py	2015-05-12 23:29:29 +0000
+++ hooks/common.py	1970-01-01 00:00:00 +0000
@@ -1,95 +0,0 @@
-#!/usr/bin/env python
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-"""
-Common implementation for all hooks.
-"""
-
-import jujuresources
-
-
-def bootstrap_resources():
-    """
-    Install required resources defined in resources.yaml
-    """
-    mirror_url = jujuresources.config_get('resources_mirror')
-    if not jujuresources.fetch(mirror_url=mirror_url):
-        jujuresources.juju_log('Resources unavailable; manual intervention required', 'ERROR')
-        return False
-    jujuresources.install(['pathlib', 'pyaml', 'six', 'charmhelpers', 'jujubigdata'])
-    return True
-
-
-def manage():
-    if not bootstrap_resources():
-        # defer until resources are available, since charmhelpers, and thus
-        # the framework, are required (will require manual intervention)
-        return
-
-    from charmhelpers.core import charmframework
-    import jujubigdata
-    import callbacks  # noqa  (ignore when linting)
-
-    # list of keys required to be in the dist.yaml
-    client_reqs = ['vendor', 'hadoop_version', 'packages', 'groups', 'users',
-                   'dirs', 'ports']
-    dist_config = jujubigdata.utils.DistConfig(filename='dist.yaml',
-                                               required_keys=client_reqs)
-    hadoop = jujubigdata.handlers.HadoopBase(dist_config)
-    hdfs = jujubigdata.handlers.HDFS(hadoop)
-    yarn = jujubigdata.handlers.YARN(hadoop)
-    namenode = jujubigdata.relations.NameNode(spec=hadoop.client_spec)
-    resourcemanager = jujubigdata.relations.ResourceManager(spec=hadoop.client_spec)
-    manager = charmframework.Manager([
-        {
-            'name': 'hadoop-base',
-            'requires': [
-                hadoop.verify_conditional_resources,
-            ],
-            'callbacks': [
-                hadoop.install,
-            ],
-        },
-        {
-            'name': 'client-hdfs',
-            'provides': [
-            ],
-            'requires': [
-                hadoop.is_installed,
-                namenode,
-            ],
-            'callbacks': [
-                namenode.update_etc_hosts,
-                hdfs.configure_client,
-            ],
-        },
-        {
-            'name': 'client-yarn',
-            'provides': [
-            ],
-            'requires': [
-                hadoop.is_installed,
-                resourcemanager,
-            ],
-            'callbacks': [
-                resourcemanager.update_etc_hosts,
-                yarn.install_demo,
-                yarn.configure_client,
-            ],
-            'cleanup': [],
-        },
-    ])
-    manager.manage()
-
-
-if __name__ == '__main__':
-    manage()

=== removed file 'hooks/config-changed'
--- hooks/config-changed	2015-02-09 18:13:28 +0000
+++ hooks/config-changed	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-#!/usr/bin/env python
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import common
-common.manage()

=== added file 'hooks/hadoop-plugin-relation-changed'
--- hooks/hadoop-plugin-relation-changed	1970-01-01 00:00:00 +0000
+++ hooks/hadoop-plugin-relation-changed	2015-05-12 23:32:24 +0000
@@ -0,0 +1,4 @@
+#!/bin/bash
+if [[ "$(relation-get hdfs-ready)" == "True" ]]; then
+    hooks/status-set active "Ready to run mapreduce jobs"
+fi

=== added file 'hooks/hadoop-plugin-relation-joined'
--- hooks/hadoop-plugin-relation-joined	1970-01-01 00:00:00 +0000
+++ hooks/hadoop-plugin-relation-joined	2015-05-12 23:32:24 +0000
@@ -0,0 +1,2 @@
+#!/bin/bash
+hooks/status-set waiting "Waiting for Hadoop to be ready"

=== modified file 'hooks/install'
--- hooks/install	2015-02-09 18:13:28 +0000
+++ hooks/install	2015-05-12 23:32:24 +0000
@@ -1,17 +1,2 @@
-#!/usr/bin/python
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-import setup
-setup.pre_install()
-
-import common
-common.manage()
+#!/bin/bash
+hooks/status-set blocked "Please add relation to apache-hadoop-plugin"

=== removed file 'hooks/namenode-relation-changed'
--- hooks/namenode-relation-changed	2015-05-07 15:08:59 +0000
+++ hooks/namenode-relation-changed	1970-01-01 00:00:00 +0000
@@ -1,16 +0,0 @@
-#!/usr/bin/env python
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import common
-
-common.manage()

=== removed file 'hooks/resourcemanager-relation-changed'
--- hooks/resourcemanager-relation-changed	2015-05-07 15:08:59 +0000
+++ hooks/resourcemanager-relation-changed	1970-01-01 00:00:00 +0000
@@ -1,16 +0,0 @@
-#!/usr/bin/env python
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import common
-
-common.manage()

=== removed file 'hooks/setup.py'
--- hooks/setup.py	2015-03-03 21:29:00 +0000
+++ hooks/setup.py	1970-01-01 00:00:00 +0000
@@ -1,35 +0,0 @@
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-import subprocess
-from glob import glob
-
-def pre_install():
-    """
-    Do any setup required before the install hook.
-    """
-    install_pip()
-    install_jujuresources()
-
-
-def install_pip():
-    subprocess.check_call(['apt-get', 'install', '-yq', 'python-pip', 'bzr'])
-
-
-def install_jujuresources():
-    """
-    Install the bundled jujuresources library, if not present.
-    """
-    try:
-        import jujuresources  # noqa
-    except ImportError:
-        jr_archive = glob('resources/jujuresources-*.tar.gz')[0]
-        subprocess.check_call(['pip', 'install', jr_archive])

=== removed file 'hooks/start'
--- hooks/start	2015-02-09 18:13:28 +0000
+++ hooks/start	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-#!/usr/bin/env python
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import common
-common.manage()

=== added file 'hooks/status-set'
--- hooks/status-set	1970-01-01 00:00:00 +0000
+++ hooks/status-set	2015-05-12 23:32:24 +0000
@@ -0,0 +1,5 @@
+#!/bin/bash
+# Wrapper around status-set for use with older versions of Juju
+if which status-set > /dev/null; then
+    status-set "$@"
+fi

=== removed file 'hooks/stop'
--- hooks/stop	2015-02-09 18:13:28 +0000
+++ hooks/stop	1970-01-01 00:00:00 +0000
@@ -1,15 +0,0 @@
-#!/usr/bin/env python
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import common
-common.manage()

=== modified file 'metadata.yaml'
--- metadata.yaml	2015-05-06 17:04:07 +0000
+++ metadata.yaml	2015-05-12 23:32:24 +0000
@@ -8,8 +8,7 @@
   This charm manages a dedicated client node as a place to
   run mapreduce jobs.
 tags: ["applications", "bigdata", "hadoop", "apache"]
-requires:
-  namenode:
-    interface: dfs
-  resourcemanager:
-    interface: mapred
+provides:
+  hadoop-plugin:
+    interface: hadoop-plugin
+    scope: container

=== removed directory 'resources'
=== removed file 'resources.yaml'
--- resources.yaml	2015-05-12 23:29:29 +0000
+++ resources.yaml	1970-01-01 00:00:00 +0000
@@ -1,36 +0,0 @@
-options:
-  output_dir: /home/ubuntu/resources
-resources:
-  pathlib:
-    pypi: path.py>=7.0
-  pyaml:
-    pypi: pyaml
-  six:
-    pypi: six
-  jujubigdata:
-    pypi: jujubigdata>=1.2.5,<2.0.0
-  charmhelpers:
-    pypi: http://bazaar.launchpad.net/~bigdata-dev/bigdata-data/trunk/download/kevin.monroe%40canonical.com-20150511173636-5rblzf5r2o1zcv2p/charmhelpers0.2.3.ta-20150417221203-zg62z8c220egc3ch-1/charmhelpers-0.2.3.tar.gz
-    hash: 44340a6fd6f192bcc9d390c0d9c3901d4fc190166485b107047bc1c6ba102a2f
-    hash_type: sha256
-  java-installer:
-    # This points to a script which manages installing Java.
-    # If replaced with an alternate implementation, it must output *only* two
-    # lines containing the JAVA_HOME path, and the Java version, respectively,
-    # on stdout.  Upon error, it must exit with a non-zero exit code.
-    url: http://bazaar.launchpad.net/~bigdata-dev/bigdata-data/trunk/download/cory.johns%40canonical.com-20150312205309-2ji1etk44gep01w1/javainstaller.sh-20150311213053-4vq7369jhlvc6qy8-1/java-installer.sh
-    hash: 130984f1dc3bc624d4245234d0fca22f529d234d0eaa1241c5e9f701319bdea9
-    hash_type: sha256
-optional_resources:
-  hadoop-aarch64:
-    url: http://bazaar.launchpad.net/~bigdata-dev/bigdata-data/trunk/download/kevin.monroe%40canonical.com-20150303192631-swhrf8f7q82si75t/hadoop2.4.1.tar.gz-20150303192554-7gqslr4m8ahkwiax-2/hadoop-2.4.1.tar.gz
-    hash: 03ad135835bfe413f85fe176259237a8
-    hash_type: md5
-  hadoop-ppc64le:
-    url: http://bazaar.launchpad.net/~bigdata-dev/bigdata-data/trunk/download/kevin.monroe%40canonical.com-20150130165209-nuz1myezjpdx7eus/hadoop2.4.1ppc64le.t-20150130165148-s8i19s002ht88gio-2/hadoop-2.4.1-ppc64le.tar.gz
-    hash: 09942b168a3db0d183b281477d3dae9deb7b7bc4b5783ba5cda3965b62e71bd5
-    hash_type: sha256
-  hadoop-x86_64:
-    url: http://bazaar.launchpad.net/~bigdata-dev/bigdata-data/trunk/download/cory.johns%40canonical.com-20150116154822-x5osw3zfhw6e03b1/hadoop2.4.1.tar.gz-20150116154748-yfa2j12rr5m53xd3-1/hadoop-2.4.1.tar.gz
-    hash: a790d39baba3a597bd226042496764e0520c2336eedb28a1a3d5c48572d3b672
-    hash_type: sha256

=== removed file 'resources/jujuresources-0.2.5.tar.gz'
Binary files resources/jujuresources-0.2.5.tar.gz	2015-03-03 19:56:49 +0000 and resources/jujuresources-0.2.5.tar.gz	1970-01-01 00:00:00 +0000 differ
=== modified file 'tests/01-basic-deployment.py'
--- tests/01-basic-deployment.py	2015-03-04 00:07:45 +0000
+++ tests/01-basic-deployment.py	2015-05-12 23:32:24 +0000
@@ -29,12 +29,6 @@
         assert 'SecondaryNameNode' not in output, "SecondaryNameNode should not be started"
         assert 'DataNode' not in output, "DataServer should not be started"
 
-    def test_dist_config(self):
-        # test_dist_config.py is run on the deployed unit because it
-        # requires the Juju context to properly validate dist.yaml
-        output, retcode = self.unit.run("tests/remote/test_dist_config.py")
-        self.assertEqual(retcode, 0, 'Remote dist config test failed:\n{}'.format(output))
-
 
 if __name__ == '__main__':
     unittest.main()

=== removed directory 'tests/remote'
=== removed file 'tests/remote/test_dist_config.py'
--- tests/remote/test_dist_config.py	2015-03-19 20:09:17 +0000
+++ tests/remote/test_dist_config.py	1970-01-01 00:00:00 +0000
@@ -1,72 +0,0 @@
-#!/usr/bin/env python
-
-import grp
-import os
-import pwd
-import unittest
-
-from charmhelpers.contrib import bigdata
-
-
-class TestDistConfig(unittest.TestCase):
-    """
-    Test that the ``dist.yaml`` settings were applied properly, such as users, groups, and dirs.
-
-    This is done as a remote test on the deployed unit rather than a regular
-    test under ``tests/`` because filling in the ``dist.yaml`` requires Juju
-    context (e.g., config).
-    """
-    @classmethod
-    def setUpClass(cls):
-        config = None
-        config_dir = os.environ['JUJU_CHARM_DIR']
-        config_file = 'dist.yaml'
-        if os.path.isfile(os.path.join(config_dir, config_file)):
-            config = os.path.join(config_dir, config_file)
-        if not config:
-            raise IOError('Could not find {} in {}'.format(config_file, config_dir))
-        reqs = ['vendor', 'hadoop_version', 'packages', 'groups', 'users',
-                'dirs']
-        cls.dist_config = bigdata.utils.DistConfig(config, reqs)
-
-    def test_groups(self):
-        for name in self.dist_config.groups:
-            try:
-                grp.getgrnam(name)
-            except KeyError:
-                self.fail('Group {} is missing'.format(name))
-
-    def test_users(self):
-        for username, details in self.dist_config.users.items():
-            try:
-                user = pwd.getpwnam(username)
-            except KeyError:
-                self.fail('User {} is missing'.format(username))
-            for groupname in details['groups']:
-                try:
-                    group = grp.getgrnam(groupname)
-                except KeyError:
-                    self.fail('Group {} referenced by user {} does not exist'.format(
-                        groupname, username))
-                if group.gr_gid != user.pw_gid:
-                    self.assertIn(username, group.gr_mem, 'User {} not in group {}'.format(
-                        username, groupname))
-
-    def test_dirs(self):
-        for name, details in self.dist_config.dirs.items():
-            dirpath = self.dist_config.path(name)
-            self.assertTrue(dirpath.isdir(), 'Dir {} is missing'.format(name))
-            stat = dirpath.stat()
-            owner = pwd.getpwuid(stat.st_uid).pw_name
-            group = grp.getgrgid(stat.st_gid).gr_name
-            perms = stat.st_mode & ~0o40000
-            self.assertEqual(owner, details.get('owner', 'root'),
-                             'Dir {} ({}) has wrong owner: {}'.format(name, dirpath, owner))
-            self.assertEqual(group, details.get('group', 'root'),
-                             'Dir {} ({}) has wrong group: {}'.format(name, dirpath, group))
-            self.assertEqual(perms, details.get('perms', 0o755),
-                             'Dir {} ({}) has wrong perms: 0o{:o}'.format(name, dirpath, perms))
-
-
-if __name__ == '__main__':
-    unittest.main()