Merge lp:~ltrager/maas-images/centos_storage_curthooks into lp:maas-images

Proposed by Lee Trager
Status: Merged
Merged at revision: 408
Proposed branch: lp:~ltrager/maas-images/centos_storage_curthooks
Merge into: lp:maas-images
Diff against target: 164 lines (+56/-40)
2 files modified
conf/centos.yaml (+10/-3)
curtin/centos7/curtin-hooks.py (+46/-37)
To merge this branch: bzr merge lp:~ltrager/maas-images/centos_storage_curthooks
Reviewer Review Type Date Requested Status
Andres Rodriguez (community) Approve
Review via email: mp+353351@code.launchpad.net

Commit message

Update CentOS 7 curthooks to use internal Curtin storage support.

Curtin now supports deploying custom storage on CentOS 7+. Hooks are no
longer required for CentOS but they are still included for backwards
compatibility. If CentOS is deployed with MAAS 2.5+ and a supported version of Curtin the hooks will simply call back into Curtin to configure storage.
If an older version of MAAS or Curtin is used the old hooks will still run.

Description of the change

Stream with updated CentOS 7 image: http://162.213.35.187/centos-storage/

Testing:
I tested in the CI and configured a BIOS and UEFI machine with 2 disk to have a btrfs filesystem for / on LVM on top of a RAID 1 over 2 disks

MAAS-2.4.1 + curtin-18.1-17-gae48e86f-0ubuntu1~18.04.1(stable) - Flat layout
MAAS-2.4.1 + curtin+centos-storage - Flat layout
MAAS-master + curtin-18.1-17-gae48e86f-0ubuntu1~18.04.1 - Flat layout
MAAS-master + curtin+centos-storage - Custom storage layout deployed!

* curtin+centos-storage - https://code.launchpad.net/~raharper/curtin/+git/curtin/+merge/349075

To post a comment you must log in.
Revision history for this message
Ryan Harper (raharper) :
Revision history for this message
Lee Trager (ltrager) wrote :

Thanks for the review. I've updated to use the real package name as discussed on IRC. I was also able to confirm custom storage deployment works :)

414. By Lee Trager

Cleanup package order

Revision history for this message
Andres Rodriguez (andreserl) wrote :

Question inline.

review: Needs Information
Revision history for this message
Lee Trager (ltrager) wrote :

We're not supporting custom storage on CentOS 6 so its Curtin hooks remain the same. grub2-efi-x64 is only added to the list of packages added to the CentOS 7 image.

Revision history for this message
Andres Rodriguez (andreserl) wrote :

lgtm! But this should land if and only if the hooks are exactly like those in maas-image-builder

review: Approve
Revision history for this message
Lee Trager (ltrager) wrote :

Yes they are exactly the same

$ diff -u maas-images/centos_storage_curthooks/curtin/centos7/ maas-image-builder/contrib/centos/centos7/curtin/
$

415. By Lee Trager

Don't include grub2-efi-x64

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'conf/centos.yaml'
2--- conf/centos.yaml 2018-08-16 19:38:27 +0000
3+++ conf/centos.yaml 2018-08-29 01:11:32 +0000
4@@ -17,6 +17,7 @@
5 # cloud-init depends on packages from EPEL
6 - epel-release
7 - bash-completion
8+ - linux-firmware
9 # XXX ltrager 2017-07-28 - COPR doesn't provide a URL to the latest
10 # package build. Instead it expects you to first add the repo which
11 # is what is included in this package.
12@@ -28,11 +29,16 @@
13 # Without it cloud-init will try to install it which will not work in
14 # isolated environments.
15 - bridge-utils
16- - linux-firmware
17- - grub2-efi-modules
18- - efibootmgr
19 # Tools needed to allow custom storage to be deployed without
20 # accessing the Internet.
21+ # grub2-efi-x64 contains the signed bootloader which currently does not
22+ # chainboot - RHBZ #1623296
23+ # - grub2-efi-x64
24+ # - shim-x64
25+ # RHBZ #1101352
26+ - grub2-efi-x64-modules
27+ - grub2-tools
28+ - efibootmgr
29 - dosfstools
30 - lvm2
31 - mdadm
32@@ -57,6 +63,7 @@
33 - bridge-utils
34 # Tools needed to allow custom storage to be deployed without
35 # accessing the Internet.
36+ - efibootmgr
37 - btrfs-progs
38 - dosfstools
39 - lvm2
40
41=== modified file 'curtin/centos7/curtin-hooks.py'
42--- curtin/centos7/curtin-hooks.py 2017-08-02 19:10:25 +0000
43+++ curtin/centos7/curtin-hooks.py 2018-08-29 01:11:32 +0000
44@@ -7,6 +7,7 @@
45 )
46
47 import codecs
48+import importlib
49 import os
50 import re
51 import sys
52@@ -14,41 +15,37 @@
53 from curtin import (
54 block,
55 config,
56+ log,
57 util,
58 )
59
60-try:
61- from curtin import FEATURES as curtin_features
62-except ImportError:
63- curtin_features = []
64-
65-write_files = None
66-try:
67- from curtin.futil import write_files
68-except ImportError:
69- pass
70-
71-centos_apply_network_config = None
72-try:
73- if 'CENTOS_APPLY_NETWORK_CONFIG' in curtin_features:
74- from curtin.commands.curthooks import centos_apply_network_config
75-except ImportError:
76- pass
77-
78-"""
79-CentOS 7
80-
81-Currently Support:
82-
83-- Legacy boot
84-- UEFI boot
85-- DHCP of BOOTIF
86-
87-Not Supported:
88-
89-- Multiple network configration
90-- IPv6
91-"""
92+
93+def try_import(path, unavailable=None, condition=True):
94+ """Import path (module.submodule.attribute) and return it.
95+ return unavailable if
96+ - condition is not true
97+ - import fails
98+ - attribute does not exist in module."""
99+ if not condition:
100+ return unavailable
101+ modname, _, attrname = path.rpartition(".")
102+ try:
103+ mod = importlib.import_module(modname)
104+ imported = getattr(mod, attrname, None)
105+ except ImportError:
106+ imported = None
107+ return imported if imported else unavailable
108+
109+
110+curtin_features = try_import("curtin.FEATURES", [])
111+write_files = try_import("curtin.futil.write_files")
112+centos_apply_network_config = try_import(
113+ "curtin.commands.curthooks.centos_apply_network_config",
114+ condition='CENTOS_APPLY_NETWORK_CONFIG' in curtin_features)
115+centos_curthooks = try_import(
116+ "curtin.commands.curthooks.builtin_curthooks",
117+ condition='CENTOS_CURTHOOK_SUPPORT' in curtin_features)
118+
119
120 FSTAB_PREPEND = """\
121 #
122@@ -342,6 +339,10 @@
123
124
125 def main():
126+ # Enable logging to stdout for curtin functions
127+ verbosity = int(os.environ.get("CURTIN_VERBOSITY", 1))
128+ log.basicConfig(stream=sys.stdout, verbosity=verbosity)
129+
130 state = util.load_command_environment()
131 target = state['target']
132 if target is None:
133@@ -360,6 +361,19 @@
134 print("Unable to find block device for: %s" % target)
135 sys.exit(1)
136
137+ if state.get('config'):
138+ cfg = config.load_config(state['config'])
139+ else:
140+ cfg = {}
141+
142+ # Run builtin hooks if curtin has the feature and maas sent storage cfg
143+ if centos_curthooks and cfg.get('storage'):
144+ # centos_curthooks will complete successfully or raise an exception.
145+ # Not handling the exception here means we will exit this program with
146+ # a non-zero value
147+ centos_curthooks(cfg, target, state)
148+ return
149+
150 write_fstab(target, fstab)
151
152 update_grub_default(
153@@ -373,11 +387,6 @@
154
155 set_autorelabel(target)
156
157- if state.get('config'):
158- cfg = config.load_config(state['config'])
159- else:
160- cfg = {}
161-
162 handle_cloudconfig(cfg, target)
163
164 apply_networking(cfg, target, bootmac)

Subscribers

People subscribed via source and target branches