Juju Charms Collection
apache-hadoop-hdfs-master package

Merge lp:~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/status into lp:~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/trunk

  1. Trusty Tahr (14.04)
  2. status
  3. Merge into trunk
Proposed by Cory Johns
Status: Merged
Merge reported by: Cory Johns
Merged at revision: not available
Proposed branch: lp:~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/status
Merge into: lp:~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/trunk
Diff against target: 327 lines (+130/-33)
11 files modified
hooks/callbacks.py (+22/-0)
hooks/common.py (+22/-14)
hooks/config-changed (+11/-0)
hooks/datanode-relation-changed (+11/-0)
hooks/install (+11/-0)
hooks/namenode-relation-changed (+11/-0)
hooks/secondary-relation-changed (+11/-0)
hooks/setup.py (+8/-10)
hooks/start (+11/-0)
hooks/stop (+11/-0)
resources.yaml (+1/-9)
To merge this branch: bzr merge lp:~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/status
Reviewer Review Type Date Requested Status
Kevin W Monroe Approve
Review via email: mp+262926@code.launchpad.net

Description of the change

Added extended status support.

Example status (tabular) outputs:

http://pastebin.ubuntu.com/11769979/ (in progress)
http://pastebin.ubuntu.com/11770196/ (complete)

Example status-history outputs:

http://pastebin.ubuntu.com/11770192/ (hdfs-master)
http://pastebin.ubuntu.com/11770195/ (compute-slave)

To post a comment you must log in.
Revision history for this message
Cory Johns (johnsca) wrote :

I also bundled charmhelpers and its dependencies, since there is almost no churn on it now and it made the status handling code cleaner as I could depend on hookenv.status_set() always being there.

lp:~bigdata-dev/charms/trusty/apache-hadoop-hdfs-master/status updated
87. By Cory Johns

Nested python resources to avoid attempting to pip install future non-python resources

Revision history for this message
Kevin W Monroe (kwmonroe) wrote :

status + bundled resources LGTM, +1.

Merged into to -dev/trunk in r87.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'hooks/callbacks.py'
--- hooks/callbacks.py 2015-05-12 21:56:03 +0000
+++ hooks/callbacks.py 2015-06-25 13:44:10 +0000
@@ -15,3 +15,25 @@
15Add any additional tasks / setup here. If a callback is used by mutliple15Add any additional tasks / setup here. If a callback is used by mutliple
16charms, consider refactoring it up to the jujubigdata library.16charms, consider refactoring it up to the jujubigdata library.
17"""17"""
18
19from charmhelpers.core import hookenv
20from charmhelpers.core import unitdata
21from jujubigdata.relations import DataNode
22
23
24def update_working_status():
25 if unitdata.kv().get('charm.active', False):
26 hookenv.status_set('maintenance', 'Updating configuration')
27 return
28 hookenv.status_set('maintenance', 'Setting up HDFS master')
29
30
31def update_active_status():
32 datanode = DataNode()
33 if datanode.is_ready():
34 hookenv.status_set('active', 'Ready (%s DataNodes)' % len(datanode.filtered_data()))
35 unitdata.kv().set('charm.active', True)
36 elif datanode.connected_units():
37 hookenv.status_set('waiting', 'Waiting for compute slaves to provide DataNodes')
38 else:
39 hookenv.status_set('blocked', 'Waiting for relation to compute slaves')
1840
=== modified file 'hooks/common.py'
--- hooks/common.py 2015-06-19 17:25:28 +0000
+++ hooks/common.py 2015-06-25 13:44:10 +0000
@@ -12,37 +12,39 @@
12# limitations under the License.12# limitations under the License.
1313
14import jujuresources14import jujuresources
1515from charmhelpers.core import hookenv
1616from charmhelpers.core import unitdata
17def status_set(status, message):17from charmhelpers.core import charmframework
18 import subprocess
19 try:
20 subprocess.check_call(['status-set', status, message])
21 except (subprocess.CalledProcessError, OSError):
22 jujuresources.juju_log('status: {}: {}'.format(status, message), 'ERROR')
2318
2419
25def bootstrap_resources():20def bootstrap_resources():
26 """21 """
27 Install required resources defined in resources.yaml22 Install required resources defined in resources.yaml
28 """23 """
24 if unitdata.kv().get('charm.bootstrapped', False):
25 return True
26 hookenv.status_set('maintenance', 'Installing base resources')
29 mirror_url = jujuresources.config_get('resources_mirror')27 mirror_url = jujuresources.config_get('resources_mirror')
30 if not jujuresources.fetch(mirror_url=mirror_url):28 if not jujuresources.fetch(mirror_url=mirror_url):
31 status_set('blocked', 'Resources unavailable; manual intervention required')29 missing = jujuresources.invalid()
30 hookenv.status_set('blocked', 'Unable to fetch required resource%s: %s' % (
31 's' if len(missing) > 1 else '',
32 ', '.join(missing),
33 ))
32 return False34 return False
33 jujuresources.install(['pathlib', 'pyaml', 'six', 'charmhelpers', 'jujubigdata'])35 jujuresources.install(['pathlib', 'jujubigdata'])
36 unitdata.kv().set('charm.bootstrapped', True)
34 return True37 return True
3538
3639
37def manage():40def manage():
38 if not bootstrap_resources():41 if not bootstrap_resources():
39 # defer until resources are available, since charmhelpers, and thus42 # defer until resources are available, since jujubigdata, and thus the
40 # the framework, (will be) managed by jujuresources43 # classes needed for the requires blocks, (will be) managed by jujuresources
41 return44 return
4245
43 from charmhelpers.core import charmframework
44 import jujubigdata46 import jujubigdata
45 import callbacks # noqa (ignore when linting)47 import callbacks
4648
47 # list of keys required to be in the dist.yaml49 # list of keys required to be in the dist.yaml
48 master_reqs = ['vendor', 'hadoop_version', 'packages', 'groups', 'users',50 master_reqs = ['vendor', 'hadoop_version', 'packages', 'groups', 'users',
@@ -80,6 +82,11 @@
80 jujubigdata.relations.SecondaryNameNode(optional=True),82 jujubigdata.relations.SecondaryNameNode(optional=True),
81 ],83 ],
82 'callbacks': [84 'callbacks': [
85 # These callbacks will be executed once the Hadoop base packages
86 # are installed. New items can be added to the end of this list
87 # and to hooks/callbacks.py to extend the functionality of this
88 # charm.
89 callbacks.update_working_status,
83 datanodes.register_connected_hosts,90 datanodes.register_connected_hosts,
84 secondary.register_connected_hosts,91 secondary.register_connected_hosts,
85 jujubigdata.utils.manage_etc_hosts,92 jujubigdata.utils.manage_etc_hosts,
@@ -89,6 +96,7 @@
89 hdfs.start_namenode,96 hdfs.start_namenode,
90 hdfs.create_hdfs_dirs,97 hdfs.create_hdfs_dirs,
91 charmframework.helpers.open_ports(dist_config.exposed_ports('hdfs-master')),98 charmframework.helpers.open_ports(dist_config.exposed_ports('hdfs-master')),
99 callbacks.update_active_status,
92 ],100 ],
93 'cleanup': [101 'cleanup': [
94 charmframework.helpers.close_ports(dist_config.exposed_ports('hdfs-master')),102 charmframework.helpers.close_ports(dist_config.exposed_ports('hdfs-master')),
95103
=== modified file 'hooks/config-changed'
--- hooks/config-changed 2015-02-09 18:19:59 +0000
+++ hooks/config-changed 2015-06-25 13:44:10 +0000
@@ -11,5 +11,16 @@
11# See the License for the specific language governing permissions and11# See the License for the specific language governing permissions and
12# limitations under the License.12# limitations under the License.
1313
14"""
15All hooks in this charm are managed by the Charm Framework.
16The framework helps manage dependencies and preconditions to ensure that
17steps are only executed when they can be successful. As such, no additional
18code should be added to this hook; instead, please integrate new functionality
19into the 'callbacks' list in hooks/common.py. New callbacks can be placed
20in hooks/callbacks.py, if necessary.
21
22See http://big-data-charm-helpers.readthedocs.org/en/latest/examples/framework.html
23for more information.
24"""
14import common25import common
15common.manage()26common.manage()
1627
=== modified file 'hooks/datanode-relation-changed'
--- hooks/datanode-relation-changed 2015-05-12 21:56:03 +0000
+++ hooks/datanode-relation-changed 2015-06-25 13:44:10 +0000
@@ -11,5 +11,16 @@
11# See the License for the specific language governing permissions and11# See the License for the specific language governing permissions and
12# limitations under the License.12# limitations under the License.
1313
14"""
15All hooks in this charm are managed by the Charm Framework.
16The framework helps manage dependencies and preconditions to ensure that
17steps are only executed when they can be successful. As such, no additional
18code should be added to this hook; instead, please integrate new functionality
19into the 'callbacks' list in hooks/common.py. New callbacks can be placed
20in hooks/callbacks.py, if necessary.
21
22See http://big-data-charm-helpers.readthedocs.org/en/latest/examples/framework.html
23for more information.
24"""
14import common25import common
15common.manage()26common.manage()
1627
=== modified file 'hooks/install'
--- hooks/install 2015-02-09 18:19:59 +0000
+++ hooks/install 2015-06-25 13:44:10 +0000
@@ -13,5 +13,16 @@
13import setup13import setup
14setup.pre_install()14setup.pre_install()
1515
16"""
17All hooks in this charm are managed by the Charm Framework.
18The framework helps manage dependencies and preconditions to ensure that
19steps are only executed when they can be successful. As such, no additional
20code should be added to this hook; instead, please integrate new functionality
21into the 'callbacks' list in hooks/common.py. New callbacks can be placed
22in hooks/callbacks.py, if necessary.
23
24See http://big-data-charm-helpers.readthedocs.org/en/latest/examples/framework.html
25for more information.
26"""
16import common27import common
17common.manage()28common.manage()
1829
=== modified file 'hooks/namenode-relation-changed'
--- hooks/namenode-relation-changed 2015-05-12 21:56:03 +0000
+++ hooks/namenode-relation-changed 2015-06-25 13:44:10 +0000
@@ -11,5 +11,16 @@
11# See the License for the specific language governing permissions and11# See the License for the specific language governing permissions and
12# limitations under the License.12# limitations under the License.
1313
14"""
15All hooks in this charm are managed by the Charm Framework.
16The framework helps manage dependencies and preconditions to ensure that
17steps are only executed when they can be successful. As such, no additional
18code should be added to this hook; instead, please integrate new functionality
19into the 'callbacks' list in hooks/common.py. New callbacks can be placed
20in hooks/callbacks.py, if necessary.
21
22See http://big-data-charm-helpers.readthedocs.org/en/latest/examples/framework.html
23for more information.
24"""
14import common25import common
15common.manage()26common.manage()
1627
=== modified file 'hooks/secondary-relation-changed'
--- hooks/secondary-relation-changed 2015-05-12 21:56:03 +0000
+++ hooks/secondary-relation-changed 2015-06-25 13:44:10 +0000
@@ -11,5 +11,16 @@
11# See the License for the specific language governing permissions and11# See the License for the specific language governing permissions and
12# limitations under the License.12# limitations under the License.
1313
14"""
15All hooks in this charm are managed by the Charm Framework.
16The framework helps manage dependencies and preconditions to ensure that
17steps are only executed when they can be successful. As such, no additional
18code should be added to this hook; instead, please integrate new functionality
19into the 'callbacks' list in hooks/common.py. New callbacks can be placed
20in hooks/callbacks.py, if necessary.
21
22See http://big-data-charm-helpers.readthedocs.org/en/latest/examples/framework.html
23for more information.
24"""
14import common25import common
15common.manage()26common.manage()
1627
=== modified file 'hooks/setup.py'
--- hooks/setup.py 2015-03-03 21:30:47 +0000
+++ hooks/setup.py 2015-06-25 13:44:10 +0000
@@ -12,24 +12,22 @@
12import subprocess12import subprocess
13from glob import glob13from glob import glob
1414
15
15def pre_install():16def pre_install():
16 """17 """
17 Do any setup required before the install hook.18 Do any setup required before the install hook.
18 """19 """
19 install_pip()20 install_pip()
20 install_jujuresources()21 install_bundled_resources()
2122
2223
23def install_pip():24def install_pip():
24 subprocess.check_call(['apt-get', 'install', '-yq', 'python-pip', 'bzr'])25 subprocess.check_call(['apt-get', 'install', '-yq', 'python-pip', 'bzr'])
2526
2627
27def install_jujuresources():28def install_bundled_resources():
28 """29 """
29 Install the bundled jujuresources library, if not present.30 Install the bundled resources libraries.
30 """31 """
31 try:32 archives = glob('resources/python/*')
32 import jujuresources # noqa33 subprocess.check_call(['pip', 'install'] + archives)
33 except ImportError:
34 jr_archive = glob('resources/jujuresources-*.tar.gz')[0]
35 subprocess.check_call(['pip', 'install', jr_archive])
3634
=== modified file 'hooks/start'
--- hooks/start 2015-02-09 18:19:59 +0000
+++ hooks/start 2015-06-25 13:44:10 +0000
@@ -11,5 +11,16 @@
11# See the License for the specific language governing permissions and11# See the License for the specific language governing permissions and
12# limitations under the License.12# limitations under the License.
1313
14"""
15All hooks in this charm are managed by the Charm Framework.
16The framework helps manage dependencies and preconditions to ensure that
17steps are only executed when they can be successful. As such, no additional
18code should be added to this hook; instead, please integrate new functionality
19into the 'callbacks' list in hooks/common.py. New callbacks can be placed
20in hooks/callbacks.py, if necessary.
21
22See http://big-data-charm-helpers.readthedocs.org/en/latest/examples/framework.html
23for more information.
24"""
14import common25import common
15common.manage()26common.manage()
1627
=== modified file 'hooks/stop'
--- hooks/stop 2015-02-09 18:19:59 +0000
+++ hooks/stop 2015-06-25 13:44:10 +0000
@@ -11,5 +11,16 @@
11# See the License for the specific language governing permissions and11# See the License for the specific language governing permissions and
12# limitations under the License.12# limitations under the License.
1313
14"""
15All hooks in this charm are managed by the Charm Framework.
16The framework helps manage dependencies and preconditions to ensure that
17steps are only executed when they can be successful. As such, no additional
18code should be added to this hook; instead, please integrate new functionality
19into the 'callbacks' list in hooks/common.py. New callbacks can be placed
20in hooks/callbacks.py, if necessary.
21
22See http://big-data-charm-helpers.readthedocs.org/en/latest/examples/framework.html
23for more information.
24"""
14import common25import common
15common.manage()26common.manage()
1627
=== modified file 'resources.yaml'
--- resources.yaml 2015-06-19 17:25:28 +0000
+++ resources.yaml 2015-06-25 13:44:10 +0000
@@ -3,16 +3,8 @@
3resources:3resources:
4 pathlib:4 pathlib:
5 pypi: path.py>=7.05 pypi: path.py>=7.0
6 pyaml:
7 pypi: pyaml
8 six:
9 pypi: six
10 jujubigdata:6 jujubigdata:
11 pypi: jujubigdata>=2.0.0,<3.0.07 pypi: jujubigdata>=2.0.2,<3.0.0
12 charmhelpers:
13 pypi: http://bazaar.launchpad.net/~bigdata-dev/bigdata-data/trunk/download/kevin.monroe%40canonical.com-20150610215108-gq4ud2rriy1sp1h6/charmhelpers0.3.1.ta-20150610215050-vxrqhizdsrqwnmcm-1/charmhelpers-0.3.1.tar.gz
14 hash: 061fe7204289b96fab5d3ca02883040ea3026526ebf0ad38e21457527a18d139
15 hash_type: sha256
16 java-installer:8 java-installer:
17 # This points to a script which manages installing Java.9 # This points to a script which manages installing Java.
18 # If replaced with an alternate implementation, it must output *only* two10 # If replaced with an alternate implementation, it must output *only* two
1911
=== removed file 'resources/jujuresources-0.2.5.tar.gz'
20Binary files resources/jujuresources-0.2.5.tar.gz 2015-03-03 19:57:43 +0000 and resources/jujuresources-0.2.5.tar.gz 1970-01-01 00:00:00 +0000 differ12Binary files resources/jujuresources-0.2.5.tar.gz 2015-03-03 19:57:43 +0000 and resources/jujuresources-0.2.5.tar.gz 1970-01-01 00:00:00 +0000 differ
=== added directory 'resources/python'
=== added file 'resources/python/PyYAML-3.11.tar.gz'
21Binary files resources/python/PyYAML-3.11.tar.gz 1970-01-01 00:00:00 +0000 and resources/python/PyYAML-3.11.tar.gz 2015-06-25 13:44:10 +0000 differ13Binary files resources/python/PyYAML-3.11.tar.gz 1970-01-01 00:00:00 +0000 and resources/python/PyYAML-3.11.tar.gz 2015-06-25 13:44:10 +0000 differ
=== added file 'resources/python/charmhelpers-0.3.1.tar.gz'
22Binary files resources/python/charmhelpers-0.3.1.tar.gz 1970-01-01 00:00:00 +0000 and resources/python/charmhelpers-0.3.1.tar.gz 2015-06-25 13:44:10 +0000 differ14Binary files resources/python/charmhelpers-0.3.1.tar.gz 1970-01-01 00:00:00 +0000 and resources/python/charmhelpers-0.3.1.tar.gz 2015-06-25 13:44:10 +0000 differ
=== added file 'resources/python/jujuresources-0.2.8.tar.gz'
23Binary files resources/python/jujuresources-0.2.8.tar.gz 1970-01-01 00:00:00 +0000 and resources/python/jujuresources-0.2.8.tar.gz 2015-06-25 13:44:10 +0000 differ15Binary files resources/python/jujuresources-0.2.8.tar.gz 1970-01-01 00:00:00 +0000 and resources/python/jujuresources-0.2.8.tar.gz 2015-06-25 13:44:10 +0000 differ
=== added file 'resources/python/pyaml-15.5.7.tar.gz'
24Binary files resources/python/pyaml-15.5.7.tar.gz 1970-01-01 00:00:00 +0000 and resources/python/pyaml-15.5.7.tar.gz 2015-06-25 13:44:10 +0000 differ16Binary files resources/python/pyaml-15.5.7.tar.gz 1970-01-01 00:00:00 +0000 and resources/python/pyaml-15.5.7.tar.gz 2015-06-25 13:44:10 +0000 differ
=== added file 'resources/python/six-1.9.0-py2.py3-none-any.whl'
25Binary files resources/python/six-1.9.0-py2.py3-none-any.whl 1970-01-01 00:00:00 +0000 and resources/python/six-1.9.0-py2.py3-none-any.whl 2015-06-25 13:44:10 +0000 differ17Binary files resources/python/six-1.9.0-py2.py3-none-any.whl 1970-01-01 00:00:00 +0000 and resources/python/six-1.9.0-py2.py3-none-any.whl 2015-06-25 13:44:10 +0000 differ

Subscribers

People subscribed via source and target branches

to all changes: