Juju Charms Collection
heat package

Merge lp:~hopem/charms/trusty/heat/lp1518975 into lp:~openstack-charmers-archive/charms/trusty/heat/next

  1. Trusty Tahr (14.04)
  2. lp1518975
  3. Merge into next
Proposed by Edward Hope-Morley
Status: Merged
Merged at revision: 70
Proposed branch: lp:~hopem/charms/trusty/heat/lp1518975
Merge into: lp:~openstack-charmers-archive/charms/trusty/heat/next
Diff against target: 761 lines (+244/-102)
10 files modified
hooks/charmhelpers/contrib/openstack/amulet/deployment.py (+3/-2)
hooks/charmhelpers/contrib/openstack/context.py (+16/-8)
hooks/charmhelpers/contrib/openstack/neutron.py (+18/-6)
hooks/charmhelpers/contrib/openstack/templates/section-keystone-authtoken (+11/-0)
hooks/charmhelpers/contrib/openstack/utils.py (+108/-43)
hooks/charmhelpers/contrib/python/packages.py (+22/-7)
hooks/charmhelpers/core/host.py (+41/-26)
hooks/charmhelpers/fetch/giturl.py (+5/-3)
tests/basic_deployment.py (+17/-5)
tests/charmhelpers/contrib/openstack/amulet/deployment.py (+3/-2)
To merge this branch: bzr merge lp:~hopem/charms/trusty/heat/lp1518975
Reviewer Review Type Date Requested Status
OpenStack Charmers Pending
Review via email: mp+285744@code.launchpad.net
To post a comment you must log in.
Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #214 heat-next for hopem mp285744
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/214/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #207 heat-next for hopem mp285744
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/207/

lp:~hopem/charms/trusty/heat/lp1518975 updated
70. By Edward Hope-Morley

charm-helpers sync

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #225 heat-next for hopem mp285744
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/225/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #215 heat-next for hopem mp285744
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/215/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #108 heat-next for hopem mp285744
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15018637/
Build: http://10.245.162.36:8080/job/charm_amulet_test/108/

lp:~hopem/charms/trusty/heat/lp1518975 updated
71. By Edward Hope-Morley

fix amulet

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #245 heat-next for hopem mp285744
    LINT FAIL: lint-test failed

LINT Results (max last 2 lines):
make: *** [lint] Error 1
ERROR:root:Make target returned non-zero.

Full lint test output: http://paste.ubuntu.com/15020043/
Build: http://10.245.162.36:8080/job/charm_lint_check/245/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #226 heat-next for hopem mp285744
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/226/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #121 heat-next for hopem mp285744
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15020059/
Build: http://10.245.162.36:8080/job/charm_amulet_test/121/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #352 heat-next for hopem mp285744
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/352/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #127 heat-next for hopem mp285744
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
heat config error: section [DEFAULT] deferred_auth_method:trusts != expected deferred_auth_method:password
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15022882/
Build: http://10.245.162.36:8080/job/charm_amulet_test/127/

lp:~hopem/charms/trusty/heat/lp1518975 updated
72. By Edward Hope-Morley

another amulet fix

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #278 heat-next for hopem mp285744
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/278/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #355 heat-next for hopem mp285744
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/355/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #128 heat-next for hopem mp285744
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15023432/
Build: http://10.245.162.36:8080/job/charm_amulet_test/128/

lp:~hopem/charms/trusty/heat/lp1518975 updated
73. By Edward Hope-Morley

another amulet fix

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #370 heat-next for hopem mp285744
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/370/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #295 heat-next for hopem mp285744
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/295/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #145 heat-next for hopem mp285744
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15025125/
Build: http://10.245.162.36:8080/job/charm_amulet_test/145/

lp:~hopem/charms/trusty/heat/lp1518975 updated
74. By Edward Hope-Morley

amulet

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #302 heat-next for hopem mp285744
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/302/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #379 heat-next for hopem mp285744
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/379/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #152 heat-next for hopem mp285744
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
2016-02-12 18:41:29,482 _stack_create DEBUG: template url: file:///var/lib/jenkins/checkout/heat/tests/files/hot_hello_world.yaml
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15027729/
Build: http://10.245.162.36:8080/job/charm_amulet_test/152/

lp:~hopem/charms/trusty/heat/lp1518975 updated
75. By Edward Hope-Morley

ch sync

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #573 heat-next for hopem mp285744
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/573/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #487 heat-next for hopem mp285744
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/487/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #234 heat-next for hopem mp285744
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
2016-02-15 11:07:55,082 resource_reaches_status DEBUG: Stack status wait status check: 4 [FAILED:COMPLETE] 4ce7d76c-82dc-4a37-9145-e9ead2d91745
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15073397/
Build: http://10.245.162.36:8080/job/charm_amulet_test/234/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #264 heat-next for hopem mp285744
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
2016-02-16 16:17:08,585 resource_reaches_status DEBUG: Stack status wait status check: 6 [FAILED:COMPLETE] 591e552e-cd4b-4702-a3b9-97199da24abd
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/15093232/
Build: http://10.245.162.36:8080/job/charm_amulet_test/264/

Revision history for this message
Ryan Beisner (1chb1n) wrote :

@hopem

The heat stack is going into a FAILED state. I re-triggered to re-run and saw the same again:

2016-02-16 16:17:08,585 resource_reaches_status DEBUG: Stack status wait status check: 6 [FAILED:COMPLETE] 591e552e-cd4b-4702-a3b9-97199da24abd

Revision history for this message
Ryan Beisner (1chb1n) wrote :

@hopem @gnuoy

In case this detail got lost in the bits, I want to point out that Kilo heat amulet tests were passing prior to the following being committed to the heat charm:

https://code.launchpad.net/~james-page/charms/trusty/heat/liberty-fixes/+merge/285606

lp:~hopem/charms/trusty/heat/lp1518975 updated
76. By Edward Hope-Morley

sync //next

77. By Edward Hope-Morley

call domain-setup action for >= Kilo

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_lint_check #826 heat-next for hopem mp285744
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/826/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_unit_test #729 heat-next for hopem mp285744
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/729/

Revision history for this message
uosci-testing-bot (uosci-testing-bot) wrote :

charm_amulet_test #327 heat-next for hopem mp285744
    AMULET OK: passed

Build: http://10.245.162.36:8080/job/charm_amulet_test/327/

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'hooks/charmhelpers/contrib/openstack/amulet/deployment.py'
2--- hooks/charmhelpers/contrib/openstack/amulet/deployment.py 2016-01-04 21:27:26 +0000
3+++ hooks/charmhelpers/contrib/openstack/amulet/deployment.py 2016-02-18 11:19:19 +0000
4@@ -121,11 +121,12 @@
5
6 # Charms which should use the source config option
7 use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',
8- 'ceph-osd', 'ceph-radosgw']
9+ 'ceph-osd', 'ceph-radosgw', 'ceph-mon']
10
11 # Charms which can not use openstack-origin, ie. many subordinates
12 no_origin = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe',
13- 'openvswitch-odl', 'neutron-api-odl', 'odl-controller']
14+ 'openvswitch-odl', 'neutron-api-odl', 'odl-controller',
15+ 'cinder-backup']
16
17 if self.openstack:
18 for svc in services:
19
20=== modified file 'hooks/charmhelpers/contrib/openstack/context.py'
21--- hooks/charmhelpers/contrib/openstack/context.py 2016-01-08 02:37:29 +0000
22+++ hooks/charmhelpers/contrib/openstack/context.py 2016-02-18 11:19:19 +0000
23@@ -90,6 +90,12 @@
24 from charmhelpers.contrib.openstack.utils import get_host_ip
25 from charmhelpers.core.unitdata import kv
26
27+try:
28+ import psutil
29+except ImportError:
30+ apt_install('python-psutil', fatal=True)
31+ import psutil
32+
33 CA_CERT_PATH = '/usr/local/share/ca-certificates/keystone_juju_ca_cert.crt'
34 ADDRESS_TYPES = ['admin', 'internal', 'public']
35
36@@ -404,6 +410,7 @@
37 auth_host = format_ipv6_addr(auth_host) or auth_host
38 svc_protocol = rdata.get('service_protocol') or 'http'
39 auth_protocol = rdata.get('auth_protocol') or 'http'
40+ api_version = rdata.get('api_version') or '2.0'
41 ctxt.update({'service_port': rdata.get('service_port'),
42 'service_host': serv_host,
43 'auth_host': auth_host,
44@@ -412,7 +419,8 @@
45 'admin_user': rdata.get('service_username'),
46 'admin_password': rdata.get('service_password'),
47 'service_protocol': svc_protocol,
48- 'auth_protocol': auth_protocol})
49+ 'auth_protocol': auth_protocol,
50+ 'api_version': api_version})
51
52 if self.context_complete(ctxt):
53 # NOTE(jamespage) this is required for >= icehouse
54@@ -1258,13 +1266,11 @@
55
56 @property
57 def num_cpus(self):
58- try:
59- from psutil import NUM_CPUS
60- except ImportError:
61- apt_install('python-psutil', fatal=True)
62- from psutil import NUM_CPUS
63-
64- return NUM_CPUS
65+ # NOTE: use cpu_count if present (16.04 support)
66+ if hasattr(psutil, 'cpu_count'):
67+ return psutil.cpu_count()
68+ else:
69+ return psutil.NUM_CPUS
70
71 def __call__(self):
72 multiplier = config('worker-multiplier') or 0
73@@ -1467,6 +1473,8 @@
74 rdata.get('service_protocol') or 'http',
75 'auth_protocol':
76 rdata.get('auth_protocol') or 'http',
77+ 'api_version':
78+ rdata.get('api_version') or '2.0',
79 }
80 if self.context_complete(ctxt):
81 return ctxt
82
83=== modified file 'hooks/charmhelpers/contrib/openstack/neutron.py'
84--- hooks/charmhelpers/contrib/openstack/neutron.py 2016-01-04 21:27:26 +0000
85+++ hooks/charmhelpers/contrib/openstack/neutron.py 2016-02-18 11:19:19 +0000
86@@ -50,7 +50,7 @@
87 if kernel_version() >= (3, 13):
88 return []
89 else:
90- return ['openvswitch-datapath-dkms']
91+ return [headers_package(), 'openvswitch-datapath-dkms']
92
93
94 # legacy
95@@ -70,7 +70,7 @@
96 relation_prefix='neutron',
97 ssl_dir=QUANTUM_CONF_DIR)],
98 'services': ['quantum-plugin-openvswitch-agent'],
99- 'packages': [[headers_package()] + determine_dkms_package(),
100+ 'packages': [determine_dkms_package(),
101 ['quantum-plugin-openvswitch-agent']],
102 'server_packages': ['quantum-server',
103 'quantum-plugin-openvswitch'],
104@@ -111,7 +111,7 @@
105 relation_prefix='neutron',
106 ssl_dir=NEUTRON_CONF_DIR)],
107 'services': ['neutron-plugin-openvswitch-agent'],
108- 'packages': [[headers_package()] + determine_dkms_package(),
109+ 'packages': [determine_dkms_package(),
110 ['neutron-plugin-openvswitch-agent']],
111 'server_packages': ['neutron-server',
112 'neutron-plugin-openvswitch'],
113@@ -155,7 +155,7 @@
114 relation_prefix='neutron',
115 ssl_dir=NEUTRON_CONF_DIR)],
116 'services': [],
117- 'packages': [[headers_package()] + determine_dkms_package(),
118+ 'packages': [determine_dkms_package(),
119 ['neutron-plugin-cisco']],
120 'server_packages': ['neutron-server',
121 'neutron-plugin-cisco'],
122@@ -174,7 +174,7 @@
123 'neutron-dhcp-agent',
124 'nova-api-metadata',
125 'etcd'],
126- 'packages': [[headers_package()] + determine_dkms_package(),
127+ 'packages': [determine_dkms_package(),
128 ['calico-compute',
129 'bird',
130 'neutron-dhcp-agent',
131@@ -219,7 +219,7 @@
132 relation_prefix='neutron',
133 ssl_dir=NEUTRON_CONF_DIR)],
134 'services': [],
135- 'packages': [[headers_package()] + determine_dkms_package()],
136+ 'packages': [determine_dkms_package()],
137 'server_packages': ['neutron-server',
138 'python-neutron-plugin-midonet'],
139 'server_services': ['neutron-server']
140@@ -233,6 +233,18 @@
141 'neutron-plugin-ml2']
142 # NOTE: patch in vmware renames nvp->nsx for icehouse onwards
143 plugins['nvp'] = plugins['nsx']
144+ if release >= 'kilo':
145+ plugins['midonet']['driver'] = (
146+ 'neutron.plugins.midonet.plugin.MidonetPluginV2')
147+ if release >= 'liberty':
148+ midonet_origin = config('midonet-origin')
149+ if midonet_origin is not None and midonet_origin[4:5] == '1':
150+ plugins['midonet']['driver'] = (
151+ 'midonet.neutron.plugin_v1.MidonetPluginV2')
152+ plugins['midonet']['server_packages'].remove(
153+ 'python-neutron-plugin-midonet')
154+ plugins['midonet']['server_packages'].append(
155+ 'python-networking-midonet')
156 return plugins
157
158
159
160=== modified file 'hooks/charmhelpers/contrib/openstack/templates/section-keystone-authtoken'
161--- hooks/charmhelpers/contrib/openstack/templates/section-keystone-authtoken 2015-04-01 12:18:48 +0000
162+++ hooks/charmhelpers/contrib/openstack/templates/section-keystone-authtoken 2016-02-18 11:19:19 +0000
163@@ -1,4 +1,14 @@
164 {% if auth_host -%}
165+{% if api_version == '3' -%}
166+[keystone_authtoken]
167+auth_url = {{ service_protocol }}://{{ service_host }}:{{ service_port }}
168+project_name = {{ admin_tenant_name }}
169+username = {{ admin_user }}
170+password = {{ admin_password }}
171+project_domain_name = default
172+user_domain_name = default
173+auth_plugin = password
174+{% else -%}
175 [keystone_authtoken]
176 identity_uri = {{ auth_protocol }}://{{ auth_host }}:{{ auth_port }}/{{ auth_admin_prefix }}
177 auth_uri = {{ service_protocol }}://{{ service_host }}:{{ service_port }}/{{ service_admin_prefix }}
178@@ -7,3 +17,4 @@
179 admin_password = {{ admin_password }}
180 signing_dir = {{ signing_dir }}
181 {% endif -%}
182+{% endif -%}
183
184=== modified file 'hooks/charmhelpers/contrib/openstack/utils.py'
185--- hooks/charmhelpers/contrib/openstack/utils.py 2016-01-08 02:37:29 +0000
186+++ hooks/charmhelpers/contrib/openstack/utils.py 2016-02-18 11:19:19 +0000
187@@ -25,6 +25,7 @@
188 import re
189
190 import six
191+import tempfile
192 import traceback
193 import uuid
194 import yaml
195@@ -41,6 +42,7 @@
196 config,
197 log as juju_log,
198 charm_dir,
199+ DEBUG,
200 INFO,
201 related_units,
202 relation_ids,
203@@ -103,29 +105,28 @@
204 ('2016.1', 'mitaka'),
205 ])
206
207-# The ugly duckling
208+# The ugly duckling - must list releases oldest to newest
209 SWIFT_CODENAMES = OrderedDict([
210- ('1.4.3', 'diablo'),
211- ('1.4.8', 'essex'),
212- ('1.7.4', 'folsom'),
213- ('1.8.0', 'grizzly'),
214- ('1.7.7', 'grizzly'),
215- ('1.7.6', 'grizzly'),
216- ('1.10.0', 'havana'),
217- ('1.9.1', 'havana'),
218- ('1.9.0', 'havana'),
219- ('1.13.1', 'icehouse'),
220- ('1.13.0', 'icehouse'),
221- ('1.12.0', 'icehouse'),
222- ('1.11.0', 'icehouse'),
223- ('2.0.0', 'juno'),
224- ('2.1.0', 'juno'),
225- ('2.2.0', 'juno'),
226- ('2.2.1', 'kilo'),
227- ('2.2.2', 'kilo'),
228- ('2.3.0', 'liberty'),
229- ('2.4.0', 'liberty'),
230- ('2.5.0', 'liberty'),
231+ ('diablo',
232+ ['1.4.3']),
233+ ('essex',
234+ ['1.4.8']),
235+ ('folsom',
236+ ['1.7.4']),
237+ ('grizzly',
238+ ['1.7.6', '1.7.7', '1.8.0']),
239+ ('havana',
240+ ['1.9.0', '1.9.1', '1.10.0']),
241+ ('icehouse',
242+ ['1.11.0', '1.12.0', '1.13.0', '1.13.1']),
243+ ('juno',
244+ ['2.0.0', '2.1.0', '2.2.0']),
245+ ('kilo',
246+ ['2.2.1', '2.2.2']),
247+ ('liberty',
248+ ['2.3.0', '2.4.0', '2.5.0']),
249+ ('mitaka',
250+ ['2.5.0']),
251 ])
252
253 # >= Liberty version->codename mapping
254@@ -227,6 +228,33 @@
255 error_out(e)
256
257
258+def get_os_version_codename_swift(codename):
259+ '''Determine OpenStack version number of swift from codename.'''
260+ for k, v in six.iteritems(SWIFT_CODENAMES):
261+ if k == codename:
262+ return v[-1]
263+ e = 'Could not derive swift version for '\
264+ 'codename: %s' % codename
265+ error_out(e)
266+
267+
268+def get_swift_codename(version):
269+ '''Determine OpenStack codename that corresponds to swift version.'''
270+ codenames = [k for k, v in six.iteritems(SWIFT_CODENAMES) if version in v]
271+ if len(codenames) > 1:
272+ # If more than one release codename contains this version we determine
273+ # the actual codename based on the highest available install source.
274+ for codename in reversed(codenames):
275+ releases = UBUNTU_OPENSTACK_RELEASE
276+ release = [k for k, v in six.iteritems(releases) if codename in v]
277+ ret = subprocess.check_output(['apt-cache', 'policy', 'swift'])
278+ if codename in ret or release[0] in ret:
279+ return codename
280+ elif len(codenames) == 1:
281+ return codenames[0]
282+ return None
283+
284+
285 def get_os_codename_package(package, fatal=True):
286 '''Derive OpenStack release codename from an installed package.'''
287 import apt_pkg as apt
288@@ -270,7 +298,7 @@
289 # < Liberty co-ordinated project versions
290 try:
291 if 'swift' in pkg.name:
292- return SWIFT_CODENAMES[vers]
293+ return get_swift_codename(vers)
294 else:
295 return OPENSTACK_CODENAMES[vers]
296 except KeyError:
297@@ -289,12 +317,14 @@
298
299 if 'swift' in pkg:
300 vers_map = SWIFT_CODENAMES
301+ for cname, version in six.iteritems(vers_map):
302+ if cname == codename:
303+ return version[-1]
304 else:
305 vers_map = OPENSTACK_CODENAMES
306-
307- for version, cname in six.iteritems(vers_map):
308- if cname == codename:
309- return version
310+ for version, cname in six.iteritems(vers_map):
311+ if cname == codename:
312+ return version
313 # e = "Could not determine OpenStack version for package: %s" % pkg
314 # error_out(e)
315
316@@ -319,12 +349,42 @@
317
318
319 def import_key(keyid):
320- cmd = "apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 " \
321- "--recv-keys %s" % keyid
322- try:
323- subprocess.check_call(cmd.split(' '))
324- except subprocess.CalledProcessError:
325- error_out("Error importing repo key %s" % keyid)
326+ key = keyid.strip()
327+ if (key.startswith('-----BEGIN PGP PUBLIC KEY BLOCK-----') and
328+ key.endswith('-----END PGP PUBLIC KEY BLOCK-----')):
329+ juju_log("PGP key found (looks like ASCII Armor format)", level=DEBUG)
330+ juju_log("Importing ASCII Armor PGP key", level=DEBUG)
331+ with tempfile.NamedTemporaryFile() as keyfile:
332+ with open(keyfile.name, 'w') as fd:
333+ fd.write(key)
334+ fd.write("\n")
335+
336+ cmd = ['apt-key', 'add', keyfile.name]
337+ try:
338+ subprocess.check_call(cmd)
339+ except subprocess.CalledProcessError:
340+ error_out("Error importing PGP key '%s'" % key)
341+ else:
342+ juju_log("PGP key found (looks like Radix64 format)", level=DEBUG)
343+ juju_log("Importing PGP key from keyserver", level=DEBUG)
344+ cmd = ['apt-key', 'adv', '--keyserver',
345+ 'hkp://keyserver.ubuntu.com:80', '--recv-keys', key]
346+ try:
347+ subprocess.check_call(cmd)
348+ except subprocess.CalledProcessError:
349+ error_out("Error importing PGP key '%s'" % key)
350+
351+
352+def get_source_and_pgp_key(input):
353+ """Look for a pgp key ID or ascii-armor key in the given input."""
354+ index = input.strip()
355+ index = input.rfind('|')
356+ if index < 0:
357+ return input, None
358+
359+ key = input[index + 1:].strip('|')
360+ source = input[:index]
361+ return source, key
362
363
364 def configure_installation_source(rel):
365@@ -336,16 +396,16 @@
366 with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:
367 f.write(DISTRO_PROPOSED % ubuntu_rel)
368 elif rel[:4] == "ppa:":
369- src = rel
370+ src, key = get_source_and_pgp_key(rel)
371+ if key:
372+ import_key(key)
373+
374 subprocess.check_call(["add-apt-repository", "-y", src])
375 elif rel[:3] == "deb":
376- l = len(rel.split('|'))
377- if l == 2:
378- src, key = rel.split('|')
379- juju_log("Importing PPA key from keyserver for %s" % src)
380+ src, key = get_source_and_pgp_key(rel)
381+ if key:
382 import_key(key)
383- elif l == 1:
384- src = rel
385+
386 with open('/etc/apt/sources.list.d/juju_deb.list', 'w') as f:
387 f.write(src)
388 elif rel[:6] == 'cloud:':
389@@ -460,11 +520,16 @@
390 cur_vers = get_os_version_package(package)
391 if "swift" in package:
392 codename = get_os_codename_install_source(src)
393- available_vers = get_os_version_codename(codename, SWIFT_CODENAMES)
394+ avail_vers = get_os_version_codename_swift(codename)
395 else:
396- available_vers = get_os_version_install_source(src)
397+ avail_vers = get_os_version_install_source(src)
398 apt.init()
399- return apt.version_compare(available_vers, cur_vers) == 1
400+ if "swift" in package:
401+ major_cur_vers = cur_vers.split('.', 1)[0]
402+ major_avail_vers = avail_vers.split('.', 1)[0]
403+ major_diff = apt.version_compare(major_avail_vers, major_cur_vers)
404+ return avail_vers > cur_vers and (major_diff == 1 or major_diff == 0)
405+ return apt.version_compare(avail_vers, cur_vers) == 1
406
407
408 def ensure_block_device(block_device):
409
410=== modified file 'hooks/charmhelpers/contrib/python/packages.py'
411--- hooks/charmhelpers/contrib/python/packages.py 2016-01-04 21:27:26 +0000
412+++ hooks/charmhelpers/contrib/python/packages.py 2016-02-18 11:19:19 +0000
413@@ -19,20 +19,35 @@
414
415 import os
416 import subprocess
417+import sys
418
419 from charmhelpers.fetch import apt_install, apt_update
420 from charmhelpers.core.hookenv import charm_dir, log
421
422-try:
423- from pip import main as pip_execute
424-except ImportError:
425- apt_update()
426- apt_install('python-pip')
427- from pip import main as pip_execute
428-
429 __author__ = "Jorge Niedbalski <jorge.niedbalski@canonical.com>"
430
431
432+def pip_execute(*args, **kwargs):
433+ """Overriden pip_execute() to stop sys.path being changed.
434+
435+ The act of importing main from the pip module seems to cause add wheels
436+ from the /usr/share/python-wheels which are installed by various tools.
437+ This function ensures that sys.path remains the same after the call is
438+ executed.
439+ """
440+ try:
441+ _path = sys.path
442+ try:
443+ from pip import main as _pip_execute
444+ except ImportError:
445+ apt_update()
446+ apt_install('python-pip')
447+ from pip import main as _pip_execute
448+ _pip_execute(*args, **kwargs)
449+ finally:
450+ sys.path = _path
451+
452+
453 def parse_options(given, available):
454 """Given a set of options, check if available"""
455 for key, value in sorted(given.items()):
456
457=== modified file 'hooks/charmhelpers/core/host.py'
458--- hooks/charmhelpers/core/host.py 2016-01-08 02:37:29 +0000
459+++ hooks/charmhelpers/core/host.py 2016-02-18 11:19:19 +0000
460@@ -138,7 +138,8 @@
461 except subprocess.CalledProcessError:
462 return False
463 else:
464- if ("start/running" in output or "is running" in output):
465+ if ("start/running" in output or "is running" in output or
466+ "up and running" in output):
467 return True
468 else:
469 return False
470@@ -160,13 +161,13 @@
471
472
473 def init_is_systemd():
474+ """Return True if the host system uses systemd, False otherwise."""
475 return os.path.isdir(SYSTEMD_SYSTEM)
476
477
478 def adduser(username, password=None, shell='/bin/bash', system_user=False,
479 primary_group=None, secondary_groups=None):
480- """
481- Add a user to the system.
482+ """Add a user to the system.
483
484 Will log but otherwise succeed if the user already exists.
485
486@@ -174,7 +175,7 @@
487 :param str password: Password for user; if ``None``, create a system user
488 :param str shell: The default shell for the user
489 :param bool system_user: Whether to create a login or system user
490- :param str primary_group: Primary group for user; defaults to their username
491+ :param str primary_group: Primary group for user; defaults to username
492 :param list secondary_groups: Optional list of additional groups
493
494 :returns: The password database entry struct, as returned by `pwd.getpwnam`
495@@ -300,14 +301,12 @@
496
497
498 def fstab_remove(mp):
499- """Remove the given mountpoint entry from /etc/fstab
500- """
501+ """Remove the given mountpoint entry from /etc/fstab"""
502 return Fstab.remove_by_mountpoint(mp)
503
504
505 def fstab_add(dev, mp, fs, options=None):
506- """Adds the given device entry to the /etc/fstab file
507- """
508+ """Adds the given device entry to the /etc/fstab file"""
509 return Fstab.add(dev, mp, fs, options=options)
510
511
512@@ -363,8 +362,7 @@
513
514
515 def file_hash(path, hash_type='md5'):
516- """
517- Generate a hash checksum of the contents of 'path' or None if not found.
518+ """Generate a hash checksum of the contents of 'path' or None if not found.
519
520 :param str hash_type: Any hash alrgorithm supported by :mod:`hashlib`,
521 such as md5, sha1, sha256, sha512, etc.
522@@ -379,10 +377,9 @@
523
524
525 def path_hash(path):
526- """
527- Generate a hash checksum of all files matching 'path'. Standard wildcards
528- like '*' and '?' are supported, see documentation for the 'glob' module for
529- more information.
530+ """Generate a hash checksum of all files matching 'path'. Standard
531+ wildcards like '*' and '?' are supported, see documentation for the 'glob'
532+ module for more information.
533
534 :return: dict: A { filename: hash } dictionary for all matched files.
535 Empty if none found.
536@@ -394,8 +391,7 @@
537
538
539 def check_hash(path, checksum, hash_type='md5'):
540- """
541- Validate a file using a cryptographic checksum.
542+ """Validate a file using a cryptographic checksum.
543
544 :param str checksum: Value of the checksum used to validate the file.
545 :param str hash_type: Hash algorithm used to generate `checksum`.
546@@ -410,6 +406,7 @@
547
548
549 class ChecksumError(ValueError):
550+ """A class derived from Value error to indicate the checksum failed."""
551 pass
552
553
554@@ -515,7 +512,7 @@
555
556
557 def list_nics(nic_type=None):
558- '''Return a list of nics of given type(s)'''
559+ """Return a list of nics of given type(s)"""
560 if isinstance(nic_type, six.string_types):
561 int_types = [nic_type]
562 else:
563@@ -557,12 +554,13 @@
564
565
566 def set_nic_mtu(nic, mtu):
567- '''Set MTU on a network interface'''
568+ """Set the Maximum Transmission Unit (MTU) on a network interface."""
569 cmd = ['ip', 'link', 'set', nic, 'mtu', mtu]
570 subprocess.check_call(cmd)
571
572
573 def get_nic_mtu(nic):
574+ """Return the Maximum Transmission Unit (MTU) for a network interface."""
575 cmd = ['ip', 'addr', 'show', nic]
576 ip_output = subprocess.check_output(cmd).decode('UTF-8').split('\n')
577 mtu = ""
578@@ -574,6 +572,7 @@
579
580
581 def get_nic_hwaddr(nic):
582+ """Return the Media Access Control (MAC) for a network interface."""
583 cmd = ['ip', '-o', '-0', 'addr', 'show', nic]
584 ip_output = subprocess.check_output(cmd).decode('UTF-8')
585 hwaddr = ""
586@@ -584,7 +583,7 @@
587
588
589 def cmp_pkgrevno(package, revno, pkgcache=None):
590- '''Compare supplied revno with the revno of the installed package
591+ """Compare supplied revno with the revno of the installed package
592
593 * 1 => Installed revno is greater than supplied arg
594 * 0 => Installed revno is the same as supplied arg
595@@ -593,7 +592,7 @@
596 This function imports apt_cache function from charmhelpers.fetch if
597 the pkgcache argument is None. Be sure to add charmhelpers.fetch if
598 you call this function, or pass an apt_pkg.Cache() instance.
599- '''
600+ """
601 import apt_pkg
602 if not pkgcache:
603 from charmhelpers.fetch import apt_cache
604@@ -603,19 +602,27 @@
605
606
607 @contextmanager
608-def chdir(d):
609+def chdir(directory):
610+ """Change the current working directory to a different directory for a code
611+ block and return the previous directory after the block exits. Useful to
612+ run commands from a specificed directory.
613+
614+ :param str directory: The directory path to change to for this context.
615+ """
616 cur = os.getcwd()
617 try:
618- yield os.chdir(d)
619+ yield os.chdir(directory)
620 finally:
621 os.chdir(cur)
622
623
624 def chownr(path, owner, group, follow_links=True, chowntopdir=False):
625- """
626- Recursively change user and group ownership of files and directories
627+ """Recursively change user and group ownership of files and directories
628 in given path. Doesn't chown path itself by default, only its children.
629
630+ :param str path: The string path to start changing ownership.
631+ :param str owner: The owner string to use when looking up the uid.
632+ :param str group: The group string to use when looking up the gid.
633 :param bool follow_links: Also Chown links if True
634 :param bool chowntopdir: Also chown path itself if True
635 """
636@@ -639,15 +646,23 @@
637
638
639 def lchownr(path, owner, group):
640+ """Recursively change user and group ownership of files and directories
641+ in a given path, not following symbolic links. See the documentation for
642+ 'os.lchown' for more information.
643+
644+ :param str path: The string path to start changing ownership.
645+ :param str owner: The owner string to use when looking up the uid.
646+ :param str group: The group string to use when looking up the gid.
647+ """
648 chownr(path, owner, group, follow_links=False)
649
650
651 def get_total_ram():
652- '''The total amount of system RAM in bytes.
653+ """The total amount of system RAM in bytes.
654
655 This is what is reported by the OS, and may be overcommitted when
656 there are multiple containers hosted on the same machine.
657- '''
658+ """
659 with open('/proc/meminfo', 'r') as f:
660 for line in f.readlines():
661 if line:
662
663=== modified file 'hooks/charmhelpers/fetch/giturl.py'
664--- hooks/charmhelpers/fetch/giturl.py 2016-01-08 02:37:29 +0000
665+++ hooks/charmhelpers/fetch/giturl.py 2016-02-18 11:19:19 +0000
666@@ -15,7 +15,7 @@
667 # along with charm-helpers. If not, see <http://www.gnu.org/licenses/>.
668
669 import os
670-from subprocess import check_call
671+from subprocess import check_call, CalledProcessError
672 from charmhelpers.fetch import (
673 BaseFetchHandler,
674 UnhandledSource,
675@@ -49,8 +49,8 @@
676 cmd = ['git', '-C', dest, 'pull', source, branch]
677 else:
678 cmd = ['git', 'clone', source, dest, '--branch', branch]
679- if depth:
680- cmd.extend(['--depth', depth])
681+ if depth:
682+ cmd.extend(['--depth', depth])
683 check_call(cmd)
684
685 def install(self, source, branch="master", dest=None, depth=None):
686@@ -63,6 +63,8 @@
687 branch_name)
688 try:
689 self.clone(source, dest_dir, branch, depth)
690+ except CalledProcessError as e:
691+ raise UnhandledSource(e)
692 except OSError as e:
693 raise UnhandledSource(e.strerror)
694 return dest_dir
695
696=== modified file 'tests/basic_deployment.py'
697--- tests/basic_deployment.py 2015-06-12 13:50:03 +0000
698+++ tests/basic_deployment.py 2016-02-18 11:19:19 +0000
699@@ -128,6 +128,9 @@
700 # Authenticate admin with heat endpoint
701 self.heat = u.authenticate_heat_admin(self.keystone)
702
703+ if self._get_openstack_release() >= self.trusty_kilo:
704+ u.wait_on_action(u.run_action(self.heat_sentry, 'domain-setup'))
705+
706 def _image_create(self):
707 """Create an image to be used by the heat template, verify it exists"""
708 u.log.debug('Creating glance image ({})...'.format(IMAGE_NAME))
709@@ -482,12 +485,7 @@
710 'instance_driver': 'heat.engine.nova',
711 'plugin_dirs': '/usr/lib64/heat,/usr/lib/heat',
712 'environment_dir': '/etc/heat/environment.d',
713- 'deferred_auth_method': 'password',
714 'host': 'heat',
715- 'rabbit_userid': 'heat',
716- 'rabbit_virtual_host': 'openstack',
717- 'rabbit_password': rmq_rel['password'],
718- 'rabbit_host': rmq_rel['hostname']
719 },
720 'keystone_authtoken': {
721 'auth_uri': auth_uri,
722@@ -513,6 +511,20 @@
723 },
724 }
725
726+ rabbit_entries = {'rabbit_userid': 'heat',
727+ 'rabbit_virtual_host': 'openstack',
728+ 'rabbit_password': rmq_rel['password'],
729+ 'rabbit_host': rmq_rel['hostname']}
730+ if self._get_openstack_release() <= self.utopic_juno:
731+ expected['DEFAULT']['deferred_auth_method'] = 'password'
732+ expected['DEFAULT'].update(rabbit_entries)
733+ else:
734+ expected['DEFAULT']['deferred_auth_method'] = 'trusts'
735+ expected['oslo_messaging_rabbit'] = rabbit_entries
736+ del expected['keystone_authtoken']['auth_host']
737+ del expected['keystone_authtoken']['auth_port']
738+ del expected['keystone_authtoken']['auth_protocol']
739+
740 for section, pairs in expected.iteritems():
741 ret = u.validate_config_data(unit, conf, section, pairs)
742 if ret:
743
744=== modified file 'tests/charmhelpers/contrib/openstack/amulet/deployment.py'
745--- tests/charmhelpers/contrib/openstack/amulet/deployment.py 2016-01-04 21:27:26 +0000
746+++ tests/charmhelpers/contrib/openstack/amulet/deployment.py 2016-02-18 11:19:19 +0000
747@@ -121,11 +121,12 @@
748
749 # Charms which should use the source config option
750 use_source = ['mysql', 'mongodb', 'rabbitmq-server', 'ceph',
751- 'ceph-osd', 'ceph-radosgw']
752+ 'ceph-osd', 'ceph-radosgw', 'ceph-mon']
753
754 # Charms which can not use openstack-origin, ie. many subordinates
755 no_origin = ['cinder-ceph', 'hacluster', 'neutron-openvswitch', 'nrpe',
756- 'openvswitch-odl', 'neutron-api-odl', 'odl-controller']
757+ 'openvswitch-odl', 'neutron-api-odl', 'odl-controller',
758+ 'cinder-backup']
759
760 if self.openstack:
761 for svc in services:

Subscribers

People subscribed via source and target branches