Merge lp:~chris.macnaughton/charms/trusty/ceph-osd/storage-hooks into lp:~openstack-charmers-archive/charms/trusty/ceph-osd/next

Proposed by Chris MacNaughton on 2016-01-29
Status: Needs review
Proposed branch: lp:~chris.macnaughton/charms/trusty/ceph-osd/storage-hooks
Merge into: lp:~openstack-charmers-archive/charms/trusty/ceph-osd/next
Diff against target: 131 lines (+53/-23)
3 files modified
hooks/add-storage (+7/-0)
hooks/ceph_hooks.py (+37/-23)
metadata.yaml (+9/-0)
To merge this branch: bzr merge lp:~chris.macnaughton/charms/trusty/ceph-osd/storage-hooks
Reviewer Review Type Date Requested Status
James Page 2016-01-29 Needs Fixing on 2016-02-18
Review via email: mp+284445@code.launchpad.net

Description of the change

Add ability to manage ceph OSD and Journal devices with storage hooks

To post a comment you must log in.

charm_unit_test #17081 ceph-osd-next for chris.macnaughton mp284445
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/17081/

charm_lint_check #18334 ceph-osd-next for chris.macnaughton mp284445
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/18334/

charm_amulet_test #9098 ceph-osd-next for chris.macnaughton mp284445
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 124
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/14695879/
Build: http://10.245.162.77:8080/job/charm_amulet_test/9098/

62. By Chris MacNaughton on 2016-01-29

finish hooks

charm_unit_test #17082 ceph-osd-next for chris.macnaughton mp284445
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/17082/

charm_lint_check #18335 ceph-osd-next for chris.macnaughton mp284445
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/18335/

charm_amulet_test #9099 ceph-osd-next for chris.macnaughton mp284445
    AMULET FAIL: amulet-test failed

AMULET Results (max last 2 lines):
make: *** [functional_test] Error 1
ERROR:root:Make target returned non-zero.

Full amulet test output: http://paste.ubuntu.com/14697247/
Build: http://10.245.162.77:8080/job/charm_amulet_test/9099/

63. By Chris MacNaughton on 2016-02-01

update to get around storage behaviour

charm_lint_check #18448 ceph-osd-next for chris.macnaughton mp284445
    LINT OK: passed

Build: http://10.245.162.77:8080/job/charm_lint_check/18448/

charm_unit_test #17187 ceph-osd-next for chris.macnaughton mp284445
    UNIT OK: passed

Build: http://10.245.162.77:8080/job/charm_unit_test/17187/

charm_amulet_test #9145 ceph-osd-next for chris.macnaughton mp284445
    AMULET OK: passed

Build: http://10.245.162.77:8080/job/charm_amulet_test/9145/

charm_unit_test #122 ceph-osd-next for chris.macnaughton mp284445
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/122/

charm_lint_check #136 ceph-osd-next for chris.macnaughton mp284445
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/136/

charm_amulet_test #21 ceph-osd-next for chris.macnaughton mp284445
    AMULET OK: passed

Build: http://10.245.162.36:8080/job/charm_amulet_test/21/

James Page (james-page) wrote :

Chris

I have reservations about the actions being implemented here; they overlap heavily with what juju storage should be doing and I'm not sure they are the right direction.

That said, the juju storage stuff looks great - does the ceph charm suffer from the 'new versions' problem for early unit lifecycle as well?

review: Needs Fixing
64. By Chris MacNaughton on 2016-02-18

remove actions stuff

Didn't mean for the actions (incomplete) to be on here; you'll notice the mini hook, add-storage, that shims in support to make the hooks ok later since they will fire before the install hook.

charm_unit_test #740 ceph-osd-next for chris.macnaughton mp284445
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/740/

charm_lint_check #837 ceph-osd-next for chris.macnaughton mp284445
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/837/

65. By Chris MacNaughton on 2016-02-18

Resolve conflict with get_devices

charm_unit_test #743 ceph-osd-next for chris.macnaughton mp284445
    UNIT OK: passed

Build: http://10.245.162.36:8080/job/charm_unit_test/743/

charm_lint_check #841 ceph-osd-next for chris.macnaughton mp284445
    LINT OK: passed

Build: http://10.245.162.36:8080/job/charm_lint_check/841/

charm_amulet_test #336 ceph-osd-next for chris.macnaughton mp284445
    AMULET OK: passed

Build: http://10.245.162.36:8080/job/charm_amulet_test/336/

Unmerged revisions

65. By Chris MacNaughton on 2016-02-18

Resolve conflict with get_devices

64. By Chris MacNaughton on 2016-02-18

remove actions stuff

63. By Chris MacNaughton on 2016-02-01

update to get around storage behaviour

62. By Chris MacNaughton on 2016-01-29

finish hooks

61. By Chris MacNaughton on 2016-01-29

add storage hooks for OSD and Journal devices

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'hooks/add-storage'
2--- hooks/add-storage 1970-01-01 00:00:00 +0000
3+++ hooks/add-storage 2016-02-18 14:13:10 +0000
4@@ -0,0 +1,7 @@
5+#!/bin/bash
6+# Wrapper to deal with newer Ubuntu versions that don't have py2 installed
7+# by default.
8+
9+dpkg -l|grep python-apt || exit
10+
11+exec ./hooks/storage.real
12
13=== modified file 'hooks/ceph_hooks.py'
14--- hooks/ceph_hooks.py 2016-02-02 14:54:17 +0000
15+++ hooks/ceph_hooks.py 2016-02-18 14:13:10 +0000
16@@ -26,6 +26,8 @@
17 UnregisteredHookError,
18 service_name,
19 status_set,
20+ storage_get,
21+ storage_list
22 )
23 from charmhelpers.core.host import (
24 umount,
25@@ -157,24 +159,26 @@
26 prepare_disks_and_activate()
27
28
29+@hooks.hook('storage.real')
30 def prepare_disks_and_activate():
31- osd_journal = get_journal_devices()
32- check_overlap(osd_journal, set(get_devices()))
33- log("got journal devs: {}".format(osd_journal), level=DEBUG)
34- already_zapped = read_zapped_journals()
35- non_zapped = osd_journal - already_zapped
36- for journ in non_zapped:
37- ceph.maybe_zap_journal(journ)
38- write_zapped_journals(osd_journal)
39+ if os.path.exists('/var/lib/ceph'):
40+ osd_journal = get_journal_devices()
41+ check_overlap(osd_journal, set(get_devices()))
42+ log("got journal devs: {}".format(osd_journal), level=DEBUG)
43+ already_zapped = read_zapped_journals()
44+ non_zapped = osd_journal - already_zapped
45+ for journ in non_zapped:
46+ ceph.maybe_zap_journal(journ)
47+ write_zapped_journals(osd_journal)
48
49- if ceph.is_bootstrapped():
50- log('ceph bootstrapped, rescanning disks')
51- emit_cephconf()
52- for dev in get_devices():
53- ceph.osdize(dev, config('osd-format'),
54- osd_journal, config('osd-reformat'),
55- config('ignore-device-errors'))
56- ceph.start_osds(get_devices())
57+ if ceph.is_bootstrapped():
58+ log('ceph bootstrapped, rescanning disks')
59+ emit_cephconf()
60+ for dev in get_devices():
61+ ceph.osdize(dev, config('osd-format'),
62+ osd_journal, config('osd-reformat'),
63+ config('ignore-device-errors'))
64+ ceph.start_osds(get_devices())
65
66
67 def get_mon_hosts():
68@@ -218,19 +222,29 @@
69
70 def get_devices():
71 if config('osd-devices'):
72- return [
73+ devices = [
74 os.path.realpath(path)
75 for path in config('osd-devices').split(' ')]
76 else:
77- return []
78+ devices = []
79+ # List storage instances for the 'osd-devices'
80+ # store declared for this charm too, and add
81+ # their block device paths to the list.
82+ storage_ids = storage_list('osd-devices')
83+ devices.extend((storage_get('location', s) for s in storage_ids))
84+ return devices
85
86
87 def get_journal_devices():
88- osd_journal = config('osd-journal')
89- if not osd_journal:
90- return set()
91- osd_journal = [l.strip() for l in config('osd-journal').split(' ')]
92- osd_journal = set(filter(os.path.exists, osd_journal))
93+ if config('osd-journal'):
94+ osd_journals = config('osd-journal').split(' ')
95+ else:
96+ osd_journals = []
97+ storage_ids = storage_list('osd-journals')
98+ osd_journals.extend((storage_get('location', s) for s in storage_ids))
99+
100+ osd_journals = [l.strip() for l in osd_journals]
101+ osd_journal = set(filter(os.path.exists, osd_journals))
102 return osd_journal
103
104
105
106=== added symlink 'hooks/osd-devices-storage-attached'
107=== target is u'add-storage'
108=== added symlink 'hooks/osd-devices-storage-detached'
109=== target is u'add-storage'
110=== added symlink 'hooks/osd-journals-storage-attached'
111=== target is u'add-storage'
112=== added symlink 'hooks/osd-journals-storage-detached'
113=== target is u'add-storage'
114=== added symlink 'hooks/storage.real'
115=== target is u'ceph_hooks.py'
116=== modified file 'metadata.yaml'
117--- metadata.yaml 2015-11-18 10:30:34 +0000
118+++ metadata.yaml 2016-02-18 14:13:10 +0000
119@@ -19,3 +19,12 @@
120 requires:
121 mon:
122 interface: ceph-osd
123+storage:
124+ osd-devices:
125+ type: block
126+ multiple:
127+ range: 0-
128+ osd-journals:
129+ type: block
130+ multiple:
131+ range: 0-

Subscribers

People subscribed via source and target branches