Merge lp:~james-page/charms/trusty/swift-storage/xenial into lp:~openstack-charmers-archive/charms/trusty/swift-storage/next

Proposed by James Page
Status: Merged
Merged at revision: 104
Proposed branch: lp:~james-page/charms/trusty/swift-storage/xenial
Merge into: lp:~openstack-charmers-archive/charms/trusty/swift-storage/next
Diff against target: 663 lines (+177/-86)
12 files modified
charmhelpers/contrib/openstack/amulet/deployment.py (+3/-2)
charmhelpers/contrib/openstack/context.py (+11/-7)
charmhelpers/contrib/openstack/neutron.py (+18/-6)
charmhelpers/contrib/openstack/utils.py (+65/-23)
charmhelpers/contrib/python/packages.py (+22/-7)
charmhelpers/core/host.py (+41/-26)
charmhelpers/fetch/giturl.py (+3/-1)
lib/swift_storage_context.py (+0/-3)
lib/swift_storage_utils.py (+2/-1)
tests/charmhelpers/contrib/openstack/amulet/deployment.py (+3/-2)
unit_tests/test_swift_storage_context.py (+0/-4)
unit_tests/test_swift_storage_utils.py (+9/-4)
To merge this branch: bzr merge lp:~james-page/charms/trusty/swift-storage/xenial
Reviewer Review Type Date Requested Status
OpenStack Charmers Pending
Review via email: mp+284515@code.launchpad.net

This proposal supersedes a proposal from 2016-01-30.

Description of the change

Resync helpers, fixup xenial support.

To post a comment you must log in.
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #18391 swift-storage-next for james-page mp284515
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/14731093/
Build: http://10.245.162.77:8080/job/charm_lint_check/18391/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #17133 swift-storage-next for james-page mp284515
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/17133/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #9100 swift-storage-next for james-page mp284515
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/9100/

104. By James Page on 2016-02-09

Tidy lint

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #119 swift-storage-next for james-page mp284515
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/119/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #125 swift-storage-next for james-page mp284515
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/125/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #118 swift-storage-next for james-page mp284515
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/118/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #23 swift-storage-next for james-page mp284515
    AMULET OK: passed

Build: http://10.245.162.36:8080/job/charm_amulet_test/23/

105. By James Page on 2016-02-11

Baseline

106. By James Page on 2016-02-11

Enable xenial mitaka tests

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #237 swift-storage-next for james-page mp284515
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/237/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #223 swift-storage-next for james-page mp284515
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/223/

Revision history for this message
David Ames (thedac) wrote :

Looks good. Waiting on amulet. With the possible exception of the xenial-mitaka amulet test which may fail due to dependencies.

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #118 swift-storage-next for james-page mp284515
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15019707/
Build: http://10.245.162.36:8080/job/charm_amulet_test/118/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #124 swift-storage-next for james-page mp284515
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15022712/
Build: http://10.245.162.36:8080/job/charm_amulet_test/124/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #130 swift-storage-next for james-page mp284515
    AMULET OK: passed

Build: http://10.245.162.36:8080/job/charm_amulet_test/130/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'charmhelpers/contrib/openstack/amulet/deployment.py'
--- charmhelpers/contrib/openstack/amulet/deployment.py 2016-01-04 21:31:57 +0000
+++ charmhelpers/contrib/openstack/amulet/deployment.py 2016-02-11 16:47:55 +0000
@@ -121,11 +121,12 @@
121121
122 # Charms which should use the source config option122 # Charms which should use the source config option
123 use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',123 use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',
124 'ceph-osd', 'ceph-radosgw']124 'ceph-osd', 'ceph-radosgw', 'ceph-mon']
125125
126 # Charms which can not use openstack-origin, ie. many subordinates126 # Charms which can not use openstack-origin, ie. many subordinates
127 no_origin = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe',127 no_origin = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe',
128 'openvswitch-odl', 'neutron-api-odl', 'odl-controller']128 'openvswitch-odl', 'neutron-api-odl', 'odl-controller',
129 'cinder-backup']
129130
130 if self.openstack:131 if self.openstack:
131 for svc in services:132 for svc in services:
132133
=== modified file 'charmhelpers/contrib/openstack/context.py'
--- charmhelpers/contrib/openstack/context.py 2016-01-08 02:38:26 +0000
+++ charmhelpers/contrib/openstack/context.py 2016-02-11 16:47:55 +0000
@@ -90,6 +90,12 @@
90from charmhelpers.contrib.openstack.utils import get_host_ip90from charmhelpers.contrib.openstack.utils import get_host_ip
91from charmhelpers.core.unitdata import kv91from charmhelpers.core.unitdata import kv
9292
93try:
94 import psutil
95except ImportError:
96 apt_install('python-psutil', fatal=True)
97 import psutil
98
93CA_CERT_PATH = '/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt'99CA_CERT_PATH = '/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt'
94ADDRESS_TYPES = ['admin', 'internal', 'public']100ADDRESS_TYPES = ['admin', 'internal', 'public']
95101
@@ -1258,13 +1264,11 @@
12581264
1259 @property1265 @property
1260 def num_cpus(self):1266 def num_cpus(self):
1261 try:1267 # NOTE: use cpu_count if present (16.04 support)
1262 from psutil import NUM_CPUS1268 if hasattr(psutil, 'cpu_count'):
1263 except ImportError:1269 return psutil.cpu_count()
1264 apt_install('python-psutil', fatal=True)1270 else:
1265 from psutil import NUM_CPUS1271 return psutil.NUM_CPUS
1266
1267 return NUM_CPUS
12681272
1269 def __call__(self):1273 def __call__(self):
1270 multiplier = config('worker-multiplier') or 01274 multiplier = config('worker-multiplier') or 0
12711275
=== modified file 'charmhelpers/contrib/openstack/neutron.py'
--- charmhelpers/contrib/openstack/neutron.py 2016-01-04 21:31:57 +0000
+++ charmhelpers/contrib/openstack/neutron.py 2016-02-11 16:47:55 +0000
@@ -50,7 +50,7 @@
50 if kernel_version() >= (3, 13):50 if kernel_version() >= (3, 13):
51 return []51 return []
52 else:52 else:
53 return ['openvswitch-datapath-dkms']53 return [headers_package(), 'openvswitch-datapath-dkms']
5454
5555
56# legacy56# legacy
@@ -70,7 +70,7 @@
70 relation_prefix='neutron',70 relation_prefix='neutron',
71 ssl_dir=QUANTUM_CONF_DIR)],71 ssl_dir=QUANTUM_CONF_DIR)],
72 'services': ['quantum-plugin-openvswitch-agent'],72 'services': ['quantum-plugin-openvswitch-agent'],
73 'packages': [[headers_package()] + determine_dkms_package(),73 'packages': [determine_dkms_package(),
74 ['quantum-plugin-openvswitch-agent']],74 ['quantum-plugin-openvswitch-agent']],
75 'server_packages': ['quantum-server',75 'server_packages': ['quantum-server',
76 'quantum-plugin-openvswitch'],76 'quantum-plugin-openvswitch'],
@@ -111,7 +111,7 @@
111 relation_prefix='neutron',111 relation_prefix='neutron',
112 ssl_dir=NEUTRON_CONF_DIR)],112 ssl_dir=NEUTRON_CONF_DIR)],
113 'services': ['neutron-plugin-openvswitch-agent'],113 'services': ['neutron-plugin-openvswitch-agent'],
114 'packages': [[headers_package()] + determine_dkms_package(),114 'packages': [determine_dkms_package(),
115 ['neutron-plugin-openvswitch-agent']],115 ['neutron-plugin-openvswitch-agent']],
116 'server_packages': ['neutron-server',116 'server_packages': ['neutron-server',
117 'neutron-plugin-openvswitch'],117 'neutron-plugin-openvswitch'],
@@ -155,7 +155,7 @@
155 relation_prefix='neutron',155 relation_prefix='neutron',
156 ssl_dir=NEUTRON_CONF_DIR)],156 ssl_dir=NEUTRON_CONF_DIR)],
157 'services': [],157 'services': [],
158 'packages': [[headers_package()] + determine_dkms_package(),158 'packages': [determine_dkms_package(),
159 ['neutron-plugin-cisco']],159 ['neutron-plugin-cisco']],
160 'server_packages': ['neutron-server',160 'server_packages': ['neutron-server',
161 'neutron-plugin-cisco'],161 'neutron-plugin-cisco'],
@@ -174,7 +174,7 @@
174 'neutron-dhcp-agent',174 'neutron-dhcp-agent',
175 'nova-api-metadata',175 'nova-api-metadata',
176 'etcd'],176 'etcd'],
177 'packages': [[headers_package()] + determine_dkms_package(),177 'packages': [determine_dkms_package(),
178 ['calico-compute',178 ['calico-compute',
179 'bird',179 'bird',
180 'neutron-dhcp-agent',180 'neutron-dhcp-agent',
@@ -219,7 +219,7 @@
219 relation_prefix='neutron',219 relation_prefix='neutron',
220 ssl_dir=NEUTRON_CONF_DIR)],220 ssl_dir=NEUTRON_CONF_DIR)],
221 'services': [],221 'services': [],
222 'packages': [[headers_package()] + determine_dkms_package()],222 'packages': [determine_dkms_package()],
223 'server_packages': ['neutron-server',223 'server_packages': ['neutron-server',
224 'python-neutron-plugin-midonet'],224 'python-neutron-plugin-midonet'],
225 'server_services': ['neutron-server']225 'server_services': ['neutron-server']
@@ -233,6 +233,18 @@
233 'neutron-plugin-ml2']233 'neutron-plugin-ml2']
234 # NOTE: patch in vmware renames nvp->nsx for icehouse onwards234 # NOTE: patch in vmware renames nvp->nsx for icehouse onwards
235 plugins['nvp'] = plugins['nsx']235 plugins['nvp'] = plugins['nsx']
236 if release >= 'kilo':
237 plugins['midonet']['driver'] = (
238 'neutron.plugins.midonet.plugin.MidonetPluginV2')
239 if release >= 'liberty':
240 midonet_origin = config('midonet-origin')
241 if midonet_origin is not None and midonet_origin[4:5] == '1':
242 plugins['midonet']['driver'] = (
243 'midonet.neutron.plugin_v1.MidonetPluginV2')
244 plugins['midonet']['server_packages'].remove(
245 'python-neutron-plugin-midonet')
246 plugins['midonet']['server_packages'].append(
247 'python-networking-midonet')
236 return plugins248 return plugins
237249
238250
239251
=== modified file 'charmhelpers/contrib/openstack/utils.py'
--- charmhelpers/contrib/openstack/utils.py 2016-01-13 18:36:55 +0000
+++ charmhelpers/contrib/openstack/utils.py 2016-02-11 16:47:55 +0000
@@ -25,6 +25,7 @@
25import re25import re
2626
27import six27import six
28import tempfile
28import traceback29import traceback
29import uuid30import uuid
30import yaml31import yaml
@@ -41,6 +42,7 @@
41 config,42 config,
42 log as juju_log,43 log as juju_log,
43 charm_dir,44 charm_dir,
45 DEBUG,
44 INFO,46 INFO,
45 related_units,47 related_units,
46 relation_ids,48 relation_ids,
@@ -105,16 +107,26 @@
105107
106# The ugly duckling - must list releases oldest to newest108# The ugly duckling - must list releases oldest to newest
107SWIFT_CODENAMES = OrderedDict([109SWIFT_CODENAMES = OrderedDict([
108 ('diablo', ['1.4.3']),110 ('diablo',
109 ('essex', ['1.4.8']),111 ['1.4.3']),
110 ('folsom', ['1.7.4']),112 ('essex',
111 ('grizzly', ['1.7.6', '1.7.7', '1.8.0']),113 ['1.4.8']),
112 ('havana', ['1.9.0', '1.9.1', '1.10.0']),114 ('folsom',
113 ('icehouse', ['1.11.0', '1.12.0', '1.13.0', '1.13.1']),115 ['1.7.4']),
114 ('juno', ['2.0.0', '2.1.0', '2.2.0']),116 ('grizzly',
115 ('kilo', ['2.2.1', '2.2.2']),117 ['1.7.6', '1.7.7', '1.8.0']),
116 ('liberty', ['2.3.0', '2.4.0', '2.5.0']),118 ('havana',
117 ('mitaka', ['2.5.0']),119 ['1.9.0', '1.9.1', '1.10.0']),
120 ('icehouse',
121 ['1.11.0', '1.12.0', '1.13.0', '1.13.1']),
122 ('juno',
123 ['2.0.0', '2.1.0', '2.2.0']),
124 ('kilo',
125 ['2.2.1', '2.2.2']),
126 ('liberty',
127 ['2.3.0', '2.4.0', '2.5.0']),
128 ('mitaka',
129 ['2.5.0']),
118])130])
119131
120# >= Liberty version->codename mapping132# >= Liberty version->codename mapping
@@ -337,12 +349,42 @@
337349
338350
339def import_key(keyid):351def import_key(keyid):
340 cmd = "apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 " \352 key = keyid.strip()
341 "--recv-keys %s" % keyid353 if (key.startswith('-----BEGIN PGP PUBLIC KEY BLOCK-----') and
342 try:354 key.endswith('-----END PGP PUBLIC KEY BLOCK-----')):
343 subprocess.check_call(cmd.split(' '))355 juju_log("PGP key found (looks like ASCII Armor format)", level=DEBUG)
344 except subprocess.CalledProcessError:356 juju_log("Importing ASCII Armor PGP key", level=DEBUG)
345 error_out("Error importing repo key %s" % keyid)357 with tempfile.NamedTemporaryFile() as keyfile:
358 with open(keyfile.name, 'w') as fd:
359 fd.write(key)
360 fd.write("\n")
361
362 cmd = ['apt-key', 'add', keyfile.name]
363 try:
364 subprocess.check_call(cmd)
365 except subprocess.CalledProcessError:
366 error_out("Error importing PGP key '%s'" % key)
367 else:
368 juju_log("PGP key found (looks like Radix64 format)", level=DEBUG)
369 juju_log("Importing PGP key from keyserver", level=DEBUG)
370 cmd = ['apt-key', 'adv', '--keyserver',
371 'hkp://keyserver.ubuntu.com:80', '--recv-keys', key]
372 try:
373 subprocess.check_call(cmd)
374 except subprocess.CalledProcessError:
375 error_out("Error importing PGP key '%s'" % key)
376
377
378def get_source_and_pgp_key(input):
379 """Look for a pgp key ID or ascii-armor key in the given input."""
380 index = input.strip()
381 index = input.rfind('|')
382 if index < 0:
383 return input, None
384
385 key = input[index + 1:].strip('|')
386 source = input[:index]
387 return source, key
346388
347389
348def configure_installation_source(rel):390def configure_installation_source(rel):
@@ -354,16 +396,16 @@
354 with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:396 with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:
355 f.write(DISTRO_PROPOSED % ubuntu_rel)397 f.write(DISTRO_PROPOSED % ubuntu_rel)
356 elif rel[:4] == "ppa:":398 elif rel[:4] == "ppa:":
357 src = rel399 src, key = get_source_and_pgp_key(rel)
400 if key:
401 import_key(key)
402
358 subprocess.check_call(["add-apt-repository", "-y", src])403 subprocess.check_call(["add-apt-repository", "-y", src])
359 elif rel[:3] == "deb":404 elif rel[:3] == "deb":
360 l = len(rel.split('|'))405 src, key = get_source_and_pgp_key(rel)
361 if l == 2:406 if key:
362 src, key = rel.split('|')
363 juju_log("Importing PPA key from keyserver for %s" % src)
364 import_key(key)407 import_key(key)
365 elif l == 1:408
366 src = rel
367 with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:409 with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:
368 f.write(src)410 f.write(src)
369 elif rel[:6] == 'cloud:':411 elif rel[:6] == 'cloud:':
370412
=== modified file 'charmhelpers/contrib/python/packages.py'
--- charmhelpers/contrib/python/packages.py 2016-01-04 21:31:57 +0000
+++ charmhelpers/contrib/python/packages.py 2016-02-11 16:47:55 +0000
@@ -19,20 +19,35 @@
1919
20import os20import os
21import subprocess21import subprocess
22import sys
2223
23from charmhelpers.fetch import apt_install, apt_update24from charmhelpers.fetch import apt_install, apt_update
24from charmhelpers.core.hookenv import charm_dir, log25from charmhelpers.core.hookenv import charm_dir, log
2526
26try:
27 from pip import main as pip_execute
28except ImportError:
29 apt_update()
30 apt_install('python-pip')
31 from pip import main as pip_execute
32
33__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"27__author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
3428
3529
30def pip_execute(*args, **kwargs):
31 """Overriden pip_execute() to stop sys.path being changed.
32
33 The act of importing main from the pip module seems to cause add wheels
34 from the /usr/share/python-wheels which are installed by various tools.
35 This function ensures that sys.path remains the same after the call is
36 executed.
37 """
38 try:
39 _path = sys.path
40 try:
41 from pip import main as _pip_execute
42 except ImportError:
43 apt_update()
44 apt_install('python-pip')
45 from pip import main as _pip_execute
46 _pip_execute(*args, **kwargs)
47 finally:
48 sys.path = _path
49
50
36def parse_options(given, available):51def parse_options(given, available):
37 """Given a set of options, check if available"""52 """Given a set of options, check if available"""
38 for key, value in sorted(given.items()):53 for key, value in sorted(given.items()):
3954
=== modified file 'charmhelpers/core/host.py'
--- charmhelpers/core/host.py 2016-01-07 09:19:41 +0000
+++ charmhelpers/core/host.py 2016-02-11 16:47:55 +0000
@@ -138,7 +138,8 @@
138 except subprocess.CalledProcessError:138 except subprocess.CalledProcessError:
139 return False139 return False
140 else:140 else:
141 if ("start/running" in output or "is running" in output):141 if ("start/running" in output or "is running" in output or
142 "up and running" in output):
142 return True143 return True
143 else:144 else:
144 return False145 return False
@@ -160,13 +161,13 @@
160161
161162
162def init_is_systemd():163def init_is_systemd():
164 """Return True if the host system uses systemd, False otherwise."""
163 return os.path.isdir(SYSTEMD_SYSTEM)165 return os.path.isdir(SYSTEMD_SYSTEM)
164166
165167
166def adduser(username, password=None, shell='/bin/bash', system_user=False,168def adduser(username, password=None, shell='/bin/bash', system_user=False,
167 primary_group=None, secondary_groups=None):169 primary_group=None, secondary_groups=None):
168 """170 """Add a user to the system.
169 Add a user to the system.
170171
171 Will log but otherwise succeed if the user already exists.172 Will log but otherwise succeed if the user already exists.
172173
@@ -174,7 +175,7 @@
174 :param str password: Password for user; if ``None``, create a system user175 :param str password: Password for user; if ``None``, create a system user
175 :param str shell: The default shell for the user176 :param str shell: The default shell for the user
176 :param bool system_user: Whether to create a login or system user177 :param bool system_user: Whether to create a login or system user
177 :param str primary_group: Primary group for user; defaults to their username178 :param str primary_group: Primary group for user; defaults to username
178 :param list secondary_groups: Optional list of additional groups179 :param list secondary_groups: Optional list of additional groups
179180
180 :returns: The password database entry struct, as returned by `pwd.getpwnam`181 :returns: The password database entry struct, as returned by `pwd.getpwnam`
@@ -300,14 +301,12 @@
300301
301302
302def fstab_remove(mp):303def fstab_remove(mp):
303 """Remove the given mountpoint entry from /etc/fstab304 """Remove the given mountpoint entry from /etc/fstab"""
304 """
305 return Fstab.remove_by_mountpoint(mp)305 return Fstab.remove_by_mountpoint(mp)
306306
307307
308def fstab_add(dev, mp, fs, options=None):308def fstab_add(dev, mp, fs, options=None):
309 """Adds the given device entry to the /etc/fstab file309 """Adds the given device entry to the /etc/fstab file"""
310 """
311 return Fstab.add(dev, mp, fs, options=options)310 return Fstab.add(dev, mp, fs, options=options)
312311
313312
@@ -363,8 +362,7 @@
363362
364363
365def file_hash(path, hash_type='md5'):364def file_hash(path, hash_type='md5'):
366 """365 """Generate a hash checksum of the contents of 'path' or None if not found.
367 Generate a hash checksum of the contents of 'path' or None if not found.
368366
369 :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,367 :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
370 such as md5, sha1, sha256, sha512, etc.368 such as md5, sha1, sha256, sha512, etc.
@@ -379,10 +377,9 @@
379377
380378
381def path_hash(path):379def path_hash(path):
382 """380 """Generate a hash checksum of all files matching 'path'. Standard
383 Generate a hash checksum of all files matching 'path'. Standard wildcards381 wildcards like '*' and '?' are supported, see documentation for the 'glob'
384 like '*' and '?' are supported, see documentation for the 'glob' module for382 module for more information.
385 more information.
386383
387 :return: dict: A { filename: hash } dictionary for all matched files.384 :return: dict: A { filename: hash } dictionary for all matched files.
388 Empty if none found.385 Empty if none found.
@@ -394,8 +391,7 @@
394391
395392
396def check_hash(path, checksum, hash_type='md5'):393def check_hash(path, checksum, hash_type='md5'):
397 """394 """Validate a file using a cryptographic checksum.
398 Validate a file using a cryptographic checksum.
399395
400 :param str checksum: Value of the checksum used to validate the file.396 :param str checksum: Value of the checksum used to validate the file.
401 :param str hash_type: Hash algorithm used to generate `checksum`.397 :param str hash_type: Hash algorithm used to generate `checksum`.
@@ -410,6 +406,7 @@
410406
411407
412class ChecksumError(ValueError):408class ChecksumError(ValueError):
409 """A class derived from Value error to indicate the checksum failed."""
413 pass410 pass
414411
415412
@@ -515,7 +512,7 @@
515512
516513
517def list_nics(nic_type=None):514def list_nics(nic_type=None):
518 '''Return a list of nics of given type(s)'''515 """Return a list of nics of given type(s)"""
519 if isinstance(nic_type, six.string_types):516 if isinstance(nic_type, six.string_types):
520 int_types = [nic_type]517 int_types = [nic_type]
521 else:518 else:
@@ -557,12 +554,13 @@
557554
558555
559def set_nic_mtu(nic, mtu):556def set_nic_mtu(nic, mtu):
560 '''Set MTU on a network interface'''557 """Set the Maximum Transmission Unit (MTU) on a network interface."""
561 cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]558 cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
562 subprocess.check_call(cmd)559 subprocess.check_call(cmd)
563560
564561
565def get_nic_mtu(nic):562def get_nic_mtu(nic):
563 """Return the Maximum Transmission Unit (MTU) for a network interface."""
566 cmd = ['ip', 'addr', 'show', nic]564 cmd = ['ip', 'addr', 'show', nic]
567 ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')565 ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
568 mtu = ""566 mtu = ""
@@ -574,6 +572,7 @@
574572
575573
576def get_nic_hwaddr(nic):574def get_nic_hwaddr(nic):
575 """Return the Media Access Control (MAC) for a network interface."""
577 cmd = ['ip', '-o', '-0', 'addr', 'show', nic]576 cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
578 ip_output = subprocess.check_output(cmd).decode('UTF-8')577 ip_output = subprocess.check_output(cmd).decode('UTF-8')
579 hwaddr = ""578 hwaddr = ""
@@ -584,7 +583,7 @@
584583
585584
586def cmp_pkgrevno(package, revno, pkgcache=None):585def cmp_pkgrevno(package, revno, pkgcache=None):
587 '''Compare supplied revno with the revno of the installed package586 """Compare supplied revno with the revno of the installed package
588587
589 * 1 => Installed revno is greater than supplied arg588 * 1 => Installed revno is greater than supplied arg
590 * 0 => Installed revno is the same as supplied arg589 * 0 => Installed revno is the same as supplied arg
@@ -593,7 +592,7 @@
593 This function imports apt_cache function from charmhelpers.fetch if592 This function imports apt_cache function from charmhelpers.fetch if
594 the pkgcache argument is None. Be sure to add charmhelpers.fetch if593 the pkgcache argument is None. Be sure to add charmhelpers.fetch if
595 you call this function, or pass an apt_pkg.Cache() instance.594 you call this function, or pass an apt_pkg.Cache() instance.
596 '''595 """
597 import apt_pkg596 import apt_pkg
598 if not pkgcache:597 if not pkgcache:
599 from charmhelpers.fetch import apt_cache598 from charmhelpers.fetch import apt_cache
@@ -603,19 +602,27 @@
603602
604603
605@contextmanager604@contextmanager
606def chdir(d):605def chdir(directory):
606 """Change the current working directory to a different directory for a code
607 block and return the previous directory after the block exits. Useful to
608 run commands from a specificed directory.
609
610 :param str directory: The directory path to change to for this context.
611 """
607 cur = os.getcwd()612 cur = os.getcwd()
608 try:613 try:
609 yield os.chdir(d)614 yield os.chdir(directory)
610 finally:615 finally:
611 os.chdir(cur)616 os.chdir(cur)
612617
613618
614def chownr(path, owner, group, follow_links=True, chowntopdir=False):619def chownr(path, owner, group, follow_links=True, chowntopdir=False):
615 """620 """Recursively change user and group ownership of files and directories
616 Recursively change user and group ownership of files and directories
617 in given path. Doesn't chown path itself by default, only its children.621 in given path. Doesn't chown path itself by default, only its children.
618622
623 :param str path: The string path to start changing ownership.
624 :param str owner: The owner string to use when looking up the uid.
625 :param str group: The group string to use when looking up the gid.
619 :param bool follow_links: Also Chown links if True626 :param bool follow_links: Also Chown links if True
620 :param bool chowntopdir: Also chown path itself if True627 :param bool chowntopdir: Also chown path itself if True
621 """628 """
@@ -639,15 +646,23 @@
639646
640647
641def lchownr(path, owner, group):648def lchownr(path, owner, group):
649 """Recursively change user and group ownership of files and directories
650 in a given path, not following symbolic links. See the documentation for
651 'os.lchown' for more information.
652
653 :param str path: The string path to start changing ownership.
654 :param str owner: The owner string to use when looking up the uid.
655 :param str group: The group string to use when looking up the gid.
656 """
642 chownr(path, owner, group, follow_links=False)657 chownr(path, owner, group, follow_links=False)
643658
644659
645def get_total_ram():660def get_total_ram():
646 '''The total amount of system RAM in bytes.661 """The total amount of system RAM in bytes.
647662
648 This is what is reported by the OS, and may be overcommitted when663 This is what is reported by the OS, and may be overcommitted when
649 there are multiple containers hosted on the same machine.664 there are multiple containers hosted on the same machine.
650 '''665 """
651 with open('/proc/meminfo', 'r') as f:666 with open('/proc/meminfo', 'r') as f:
652 for line in f.readlines():667 for line in f.readlines():
653 if line:668 if line:
654669
=== modified file 'charmhelpers/fetch/giturl.py'
--- charmhelpers/fetch/giturl.py 2016-01-13 18:36:55 +0000
+++ charmhelpers/fetch/giturl.py 2016-02-11 16:47:55 +0000
@@ -15,7 +15,7 @@
15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.15# along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
1616
17import os17import os
18from subprocess import check_call18from subprocess import check_call, CalledProcessError
19from charmhelpers.fetch import (19from charmhelpers.fetch import (
20 BaseFetchHandler,20 BaseFetchHandler,
21 UnhandledSource,21 UnhandledSource,
@@ -63,6 +63,8 @@
63 branch_name)63 branch_name)
64 try:64 try:
65 self.clone(source, dest_dir, branch, depth)65 self.clone(source, dest_dir, branch, depth)
66 except CalledProcessError as e:
67 raise UnhandledSource(e)
66 except OSError as e:68 except OSError as e:
67 raise UnhandledSource(e.strerror)69 raise UnhandledSource(e.strerror)
68 return dest_dir70 return dest_dir
6971
=== modified file 'lib/swift_storage_context.py'
--- lib/swift_storage_context.py 2015-11-24 13:51:05 +0000
+++ lib/swift_storage_context.py 2016-02-11 16:47:55 +0000
@@ -80,14 +80,11 @@
80 interfaces = []80 interfaces = []
8181
82 def __call__(self):82 def __call__(self):
83 import psutil
84 multiplier = int(config('worker-multiplier')) or 1
85 ctxt = {83 ctxt = {
86 'local_ip': unit_private_ip(),84 'local_ip': unit_private_ip(),
87 'account_server_port': config('account-server-port'),85 'account_server_port': config('account-server-port'),
88 'container_server_port': config('container-server-port'),86 'container_server_port': config('container-server-port'),
89 'object_server_port': config('object-server-port'),87 'object_server_port': config('object-server-port'),
90 'workers': str(psutil.NUM_CPUS * multiplier),
91 'object_server_threads_per_disk': config(88 'object_server_threads_per_disk': config(
92 'object-server-threads-per-disk'),89 'object-server-threads-per-disk'),
93 'account_max_connections': config('account-max-connections'),90 'account_max_connections': config('account-max-connections'),
9491
=== modified file 'lib/swift_storage_utils.py'
--- lib/swift_storage_utils.py 2015-11-02 21:40:22 +0000
+++ lib/swift_storage_utils.py 2016-02-11 16:47:55 +0000
@@ -132,7 +132,8 @@
132 for server in ['account', 'object', 'container']:132 for server in ['account', 'object', 'container']:
133 configs.register('/etc/swift/%s-server.conf' % server,133 configs.register('/etc/swift/%s-server.conf' % server,
134 [SwiftStorageServerContext(),134 [SwiftStorageServerContext(),
135 context.BindHostContext()]),135 context.BindHostContext(),
136 context.WorkerConfigContext()]),
136 return configs137 return configs
137138
138139
139140
=== modified file 'tests/021-basic-xenial-mitaka' (properties changed: -x to +x)
=== modified file 'tests/charmhelpers/contrib/openstack/amulet/deployment.py'
--- tests/charmhelpers/contrib/openstack/amulet/deployment.py 2016-01-04 21:31:57 +0000
+++ tests/charmhelpers/contrib/openstack/amulet/deployment.py 2016-02-11 16:47:55 +0000
@@ -121,11 +121,12 @@
121121
122 # Charms which should use the source config option122 # Charms which should use the source config option
123 use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',123 use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',
124 'ceph-osd', 'ceph-radosgw']124 'ceph-osd', 'ceph-radosgw', 'ceph-mon']
125125
126 # Charms which can not use openstack-origin, ie. many subordinates126 # Charms which can not use openstack-origin, ie. many subordinates
127 no_origin = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe',127 no_origin = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe',
128 'openvswitch-odl', 'neutron-api-odl', 'odl-controller']128 'openvswitch-odl', 'neutron-api-odl', 'odl-controller',
129 'cinder-backup']
129130
130 if self.openstack:131 if self.openstack:
131 for svc in services:132 for svc in services:
132133
=== modified file 'unit_tests/test_swift_storage_context.py'
--- unit_tests/test_swift_storage_context.py 2015-07-17 15:52:38 +0000
+++ unit_tests/test_swift_storage_context.py 2016-02-11 16:47:55 +0000
@@ -67,18 +67,15 @@
67 _file.write.assert_called_with('RSYNC_ENABLE=true\n')67 _file.write.assert_called_with('RSYNC_ENABLE=true\n')
6868
69 def test_swift_storage_server_context(self):69 def test_swift_storage_server_context(self):
70 import psutil
71 self.unit_private_ip.return_value = '10.0.0.5'70 self.unit_private_ip.return_value = '10.0.0.5'
72 self.test_config.set('account-server-port', '500')71 self.test_config.set('account-server-port', '500')
73 self.test_config.set('object-server-port', '501')72 self.test_config.set('object-server-port', '501')
74 self.test_config.set('container-server-port', '502')73 self.test_config.set('container-server-port', '502')
75 self.test_config.set('object-server-threads-per-disk', '3')74 self.test_config.set('object-server-threads-per-disk', '3')
76 self.test_config.set('worker-multiplier', '3')
77 self.test_config.set('object-replicator-concurrency', '3')75 self.test_config.set('object-replicator-concurrency', '3')
78 self.test_config.set('account-max-connections', '10')76 self.test_config.set('account-max-connections', '10')
79 self.test_config.set('container-max-connections', '10')77 self.test_config.set('container-max-connections', '10')
80 self.test_config.set('object-max-connections', '10')78 self.test_config.set('object-max-connections', '10')
81 num_workers = psutil.NUM_CPUS * 3
82 ctxt = swift_context.SwiftStorageServerContext()79 ctxt = swift_context.SwiftStorageServerContext()
83 result = ctxt()80 result = ctxt()
84 ex = {81 ex = {
@@ -87,7 +84,6 @@
87 'account_server_port': '500',84 'account_server_port': '500',
88 'local_ip': '10.0.0.5',85 'local_ip': '10.0.0.5',
89 'object_server_threads_per_disk': '3',86 'object_server_threads_per_disk': '3',
90 'workers': str(num_workers),
91 'object_replicator_concurrency': '3',87 'object_replicator_concurrency': '3',
92 'account_max_connections': '10',88 'account_max_connections': '10',
93 'container_max_connections': '10',89 'container_max_connections': '10',
9490
=== modified file 'unit_tests/test_swift_storage_utils.py'
--- unit_tests/test_swift_storage_utils.py 2015-11-16 20:56:57 +0000
+++ unit_tests/test_swift_storage_utils.py 2016-02-11 16:47:55 +0000
@@ -292,6 +292,7 @@
292 renderer.assert_called_with(templates_dir=swift_utils.TEMPLATES,292 renderer.assert_called_with(templates_dir=swift_utils.TEMPLATES,
293 openstack_release='essex')293 openstack_release='essex')
294294
295 @patch('charmhelpers.contrib.openstack.context.WorkerConfigContext')
295 @patch('charmhelpers.contrib.openstack.context.BindHostContext')296 @patch('charmhelpers.contrib.openstack.context.BindHostContext')
296 @patch.object(swift_utils, 'SwiftStorageContext')297 @patch.object(swift_utils, 'SwiftStorageContext')
297 @patch.object(swift_utils, 'RsyncContext')298 @patch.object(swift_utils, 'RsyncContext')
@@ -299,11 +300,12 @@
299 @patch('charmhelpers.contrib.openstack.templating.OSConfigRenderer')300 @patch('charmhelpers.contrib.openstack.templating.OSConfigRenderer')
300 def test_register_configs_post_install(self, renderer,301 def test_register_configs_post_install(self, renderer,
301 swift, rsync, server,302 swift, rsync, server,
302 bind_context):303 bind_context, worker_context):
303 swift.return_value = 'swift_context'304 swift.return_value = 'swift_context'
304 rsync.return_value = 'rsync_context'305 rsync.return_value = 'rsync_context'
305 server.return_value = 'swift_server_context'306 server.return_value = 'swift_server_context'
306 bind_context.return_value = 'bind_host_context'307 bind_context.return_value = 'bind_host_context'
308 worker_context.return_value = 'worker_context'
307 self.get_os_codename_package.return_value = 'grizzly'309 self.get_os_codename_package.return_value = 'grizzly'
308 configs = MagicMock()310 configs = MagicMock()
309 configs.register = MagicMock()311 configs.register = MagicMock()
@@ -316,11 +318,14 @@
316 call('/etc/rsync-juju.d/050-swift-storage.conf',318 call('/etc/rsync-juju.d/050-swift-storage.conf',
317 ['rsync_context', 'swift_context']),319 ['rsync_context', 'swift_context']),
318 call('/etc/swift/account-server.conf', ['swift_context',320 call('/etc/swift/account-server.conf', ['swift_context',
319 'bind_host_context']),321 'bind_host_context',
322 'worker_context']),
320 call('/etc/swift/object-server.conf', ['swift_context',323 call('/etc/swift/object-server.conf', ['swift_context',
321 'bind_host_context']),324 'bind_host_context',
325 'worker_context']),
322 call('/etc/swift/container-server.conf', ['swift_context',326 call('/etc/swift/container-server.conf', ['swift_context',
323 'bind_host_context'])327 'bind_host_context',
328 'worker_context'])
324 ]329 ]
325 self.assertEquals(ex, configs.register.call_args_list)330 self.assertEquals(ex, configs.register.call_args_list)
326331

Subscribers

People subscribed via source and target branches