Merge lp:~marius-zalinauskas/maas-images/centos-hooks into lp:maas-images

Proposed by Marius Žalinauskas
Status: Rejected
Rejected by: Scott Moser
Proposed branch: lp:~marius-zalinauskas/maas-images/centos-hooks
Merge into: lp:maas-images
Diff against target: 1739 lines (+1124/-462)
2 files modified
curtin/centos6/curtin-hooks.py (+622/-224)
curtin/centos7/curtin-hooks.py (+502/-238)
To merge this branch: bzr merge lp:~marius-zalinauskas/maas-images/centos-hooks
Reviewer Review Type Date Requested Status
Andres Rodriguez (community) Needs Fixing
Tom Mullaney (community) Approve
Review via email: mp+313684@code.launchpad.net

Description of the change

Heavily modified CentOS 6/7 hooks. For the most part it is curtin/commands/curthooks.py ported to CentOS with some code from old CentOS hooks.

Added functionality:

- UEFI boot. It was already kinda working on CentOS 7, but was tricky on CentOS 6.
- Simple and MD RAID layouts. No need to force the simple layout on MAAS anymore.
- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces. Much better than original DHCP only.

To post a comment you must log in.
353. By Marius Žalinauskas <email address hidden>

Bugfix, see https://bugs.launchpad.net/maas-images/+bug/1654853

Revision history for this message
Tom Mullaney (tpmullan) wrote :

These changes work great for me. They make Maas much more usable for people that don't only work with Ubuntu.

review: Approve
Revision history for this message
Andres Rodriguez (andreserl) wrote :

Hi Marius,

Thank you for your contribution!

In order to be able to accept your code you need to sign the Canonical Contributors Agreement. Can you please do that in [1].

That said, we have a few things that need fixing before we are able to review / land branches, to follow the procedures MAAS uses, Please:

 - Separate the UEFI changes into is own branch
 - Seprate the Simple/MD Raid into its own branch
 - IPv4 DHCP/Static Networking, also it its own branch.

Thank you!

[1]: https://www.ubuntu.com/legal/contributors/submit

review: Needs Fixing
Revision history for this message
Scott Moser (smoser) wrote :

Marius,
Thank you for your contribution. Sorry for the confusion or delay in responding.

I'm going to mark this 'rejected' as of right now. You are welcome to re-submit it, but do so against the git repo now.

https://code.launchpad.net/maas-images

thanks!

Unmerged revisions

353. By Marius Žalinauskas <email address hidden>

Bugfix, see https://bugs.launchpad.net/maas-images/+bug/1654853

352. By Marius Žalinauskas <email address hidden>

Extensibly modified CentOS 6/7 Curtin hooks.

For the most part it is curtin/commands/curthooks.py ported to CentOS. Some code from old CentOS hooks is still used.

Implemented:

- Legacy boot
- UEFI boot
- Simple and MD RAID layouts
- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces

Not implemented:

- Bond, bridge, route configurations
- IPv6
- Disk encryption
- bcache
- LVM
- Multipath

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'curtin/centos6/curtin-hooks.py'
--- curtin/centos6/curtin-hooks.py 2016-05-11 18:47:46 +0000
+++ curtin/centos6/curtin-hooks.py 2017-01-09 15:19:25 +0000
@@ -1,20 +1,19 @@
1#!/usr/bin/env python1#!/usr/bin/env python
22# coding: utf-8
3from __future__ import (
4 absolute_import,
5 print_function,
6 unicode_literals,
7 )
83
9import codecs4import codecs
5import ipaddress
10import os6import os
11import re7import re
8import shutil
12import sys9import sys
1310
14from curtin import (11from curtin import block, config, util
15 block,12from curtin.block import mdadm
16 util,13from curtin.commands import curthooks
17 )14from curtin.log import LOG
15from curtin.reporter import events
16
1817
19"""18"""
20CentOS 619CentOS 6
@@ -22,15 +21,21 @@
22Currently Support:21Currently Support:
2322
24- Legacy boot23- Legacy boot
25- DHCP of BOOTIF24- UEFI boot
25- Simple and MD RAID layouts
26- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces
2627
27Not Supported:28Not Supported:
2829
29- UEFI boot (*Bad support, most likely wont support)30- Bond, bridge, route configurations
30- Multiple network configration
31- IPv631- IPv6
32- Disk encryption
33- bcache
34- LVM
35- Multipath
32"""36"""
3337
38
34FSTAB_PREPEND = """\39FSTAB_PREPEND = """\
35#40#
36# /etc/fstab41# /etc/fstab
@@ -54,36 +59,95 @@
54# Created by MAAS fast-path installer.59# Created by MAAS fast-path installer.
55#60#
56default 061default 0
57timeout 062timeout 1
58title MAAS63title CentOS
59 root {grub_root}64 root {grub_root}
60 kernel /boot/{vmlinuz} root=UUID={root_uuid} {extra_opts}65 kernel /{vmlinuz} root=UUID={root_uuid} {extra_opts}
61 initrd /boot/{initrd}66 initrd /{initrd}
62"""67"""
6368
6469
65def get_block_devices(target):70def get_installed_packages(target=None):
66 """Returns list of block devices for the given target."""71 (out, _) = util.subp(['rpm', '-qa', '--qf', '"%{NAME}\n"'],
67 devs = block.get_devices_for_mp(target)72 target=target, capture=True)
68 blockdevs = set()73 return set(out.splitlines())
69 for maybepart in devs:74
70 (blockdev, part) = block.get_blockdev_for_partition(maybepart)75
71 blockdevs.add(blockdev)76def install_packages(pkglist, target):
72 return list(blockdevs)77 if isinstance(pkglist, str):
7378 pkglist = [pkglist]
7479 # Does not work – hosts are not resolved
75def get_root_info(target):80 # return util.subp(['yum', '-q', '-y', 'install'] + pkglist, target=target, capture=True)
76 """Returns the root partitions information."""81 with util.RunInChroot(target) as in_chroot:
77 rootpath = block.get_devices_for_mp(target)[0]82 in_chroot(['yum', '-q', '-y', 'install'] + pkglist)
78 rootdev = os.path.basename(rootpath)83
79 blocks = block._lsblock()84
80 return blocks[rootdev]85def install_missing_packages(cfg, target):
8186 ''' describe which operation types will require specific packages
8287
83def read_file(path):88 'custom_config_key': {
84 """Returns content of a file."""89 'pkg1': ['op_name_1', 'op_name_2', ...]
85 with codecs.open(path, encoding='utf-8') as stream:90 }
86 return stream.read()91 '''
92 custom_configs = {
93 'storage': {
94 'lvm2': ['lvm_volgroup', 'lvm_partition'],
95 'mdadm': ['raid'],
96 # TODO. At the moment there are no official packages for bcache on CentOS
97 #'bcache-tools': ['bcache']
98 },
99 'network': {
100 'bridge-utils': ['bridge']
101 },
102 }
103
104 format_configs = {
105 'xfsprogs': ['xfs'],
106 'e2fsprogs': ['ext2', 'ext3', 'ext4'],
107 'btrfs-tools': ['btrfs'],
108 }
109
110 needed_packages = []
111 installed_packages = get_installed_packages(target)
112 for cust_cfg, pkg_reqs in custom_configs.items():
113 if cust_cfg not in cfg:
114 continue
115
116 all_types = set(
117 operation['type']
118 for operation in cfg[cust_cfg]['config']
119 )
120 for pkg, types in pkg_reqs.items():
121 if set(types).intersection(all_types) and \
122 pkg not in installed_packages:
123 needed_packages.append(pkg)
124
125 format_types = set(
126 [operation['fstype']
127 for operation in cfg[cust_cfg]['config']
128 if operation['type'] == 'format'])
129 for pkg, fstypes in format_configs.items():
130 if set(fstypes).intersection(format_types) and \
131 pkg not in installed_packages:
132 needed_packages.append(pkg)
133
134 if needed_packages:
135 state = util.load_command_environment()
136 with events.ReportEventStack(
137 name=state.get('report_stack_prefix'),
138 reporting_enabled=True, level="INFO",
139 description="Installing packages on target system: " +
140 str(needed_packages)):
141 install_packages(needed_packages, target=target)
142
143
144def copy_mdadm_conf(mdadm_conf, target):
145 if not mdadm_conf:
146 LOG.warn("mdadm config must be specified, not copying")
147 return
148
149 LOG.info("copying mdadm.conf into target")
150 shutil.copy(mdadm_conf, os.path.sep.join([target, 'etc/mdadm.conf']))
87151
88152
89def write_fstab(target, curtin_fstab):153def write_fstab(target, curtin_fstab):
@@ -97,11 +161,289 @@
97 stream.write(FSTAB_APPEND)161 stream.write(FSTAB_APPEND)
98162
99163
100def extract_kernel_params(data):164def update_initramfs(target=None):
101 """Extracts the kernel parametes from the provided165 path = os.path.join(
102 grub config data."""166 target, 'etc', 'dracut.conf.d', 'local.conf')
103 match = re.search('^\s+kernel (.+?)$', data, re.MULTILINE)167 with open(path, 'w') as stream:
104 return match.group(0)168 stream.write('mdadmconf="yes"' + '\n'
169 'lvmconf="yes"' + '\n')
170
171 initrd = get_boot_file(target, 'initramfs')
172 version = initrd.replace('initramfs-', '').replace('.img', '')
173 initrd_path = os.path.join(os.sep, 'boot', initrd)
174 with util.RunInChroot(target) as in_chroot:
175 in_chroot(['dracut', '-f', initrd_path, version])
176
177
178def system_upgrade(cfg, target):
179 """run yum upgrade in target.
180
181 config:
182 system_upgrade:
183 enabled: False
184
185 """
186 mycfg = {'system_upgrade': {'enabled': False}}
187 config.merge_config(mycfg, cfg)
188 mycfg = mycfg.get('system_upgrade')
189 if not isinstance(mycfg, dict):
190 LOG.debug("system_upgrade disabled by config. entry not a dict.")
191 return
192
193 if not config.value_as_boolean(mycfg.get('enabled', True)):
194 LOG.debug("system_upgrade disabled by config.")
195 return
196
197 util.subp(['yum', '-q', '-y', 'upgrade'], target=target, capture=True)
198
199
200def read_file(path):
201 """Returns content of a file."""
202 with codecs.open(path, encoding='utf-8') as stream:
203 return stream.read()
204
205
206def get_boot_mac():
207 """Return the mac address of the booting interface."""
208 cmdline = read_file('/proc/cmdline')
209 cmdline = cmdline.split()
210 try:
211 bootif = [
212 option
213 for option in cmdline
214 if option.startswith('BOOTIF')
215 ][0]
216 except IndexError:
217 return None
218 _, mac = bootif.split('=')
219 mac = mac.split('-')[1:]
220 return ':'.join(mac)
221
222
223def get_interface_names():
224 """Return a dictionary mapping mac addresses to interface names."""
225 sys_path = "/sys/class/net"
226 ifaces = {}
227 for iname in os.listdir(sys_path):
228 mac = read_file(os.path.join(sys_path, iname, "address"))
229 mac = mac.strip().lower()
230 ifaces[mac] = iname
231 return ifaces
232
233
234def find_legacy_iface_name(consistent_iface_name):
235 """Returns legacy network interface name by running biosdevname (needs)
236 to be installed beforehand)"""
237 out, err = util.subp(['biosdevname', '--policy', 'all_ethN',
238 '--interface', consistent_iface_name],
239 capture=True)
240 return out.strip()
241
242
243def rename_ifaces(network_config, target):
244 """CentOS 6 is not using a consistent network device naming and even we
245 would enable it, interface names would differ from names they would get
246 on new systems with systemd. Will change every occurence of consistent
247 network device name to legacy name ethN and create an appropriate
248 /etc/udev/rules.d/70-persistent-net.rules"""
249
250 # Will need biosdevname to find out legacy names
251 util.install_packages(['biosdevname'])
252
253 # Build the cache as {'consistent_name': 'legacy_name'}
254 cache = {}
255 for cfg in network_config['config']:
256 if cfg['type'] == 'physical':
257 consistent_name = cfg['name']
258 cache[consistent_name] = find_legacy_iface_name(consistent_name)
259
260 rules = ''
261 rule_template = ('SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", '
262 'ATTR{{address}}=="{mac_address}", KERNEL=="eth*", '
263 'NAME="{iface_name}"\n')
264
265 # Rename devices and generate an appropriate content for
266 # /etc/udev/rules.d/70-persistent-net.rules
267 for cfg in network_config['config']:
268 if cfg['type'] == 'physical':
269 consistent_phys_name = cfg['name']
270 legacy_phys_name = cache[consistent_phys_name]
271 cfg['name'] = legacy_phys_name
272 rules += rule_template.format(iface_name=legacy_phys_name,
273 mac_address=cfg['mac_address'])
274 if cfg['type'] == 'vlan':
275 consistent_phys_name = cfg['vlan_link']
276 legacy_phys_name = cache[consistent_phys_name]
277 cfg['name'] = '{}.{}'.format(legacy_phys_name, cfg['vlan_id'])
278 cfg['vlan_link'] = legacy_phys_name
279
280 path = os.path.join(target, 'etc', 'udev', 'rules.d',
281 '70-persistent-net.rules')
282 util.write_file(path, rules)
283
284
285
286def get_subnet_config(subnets):
287 content = ''
288 for idx, subnet in enumerate(subnets):
289 if subnet['type'] in ('static', 'static4'):
290 if idx == 0:
291 idx = ''
292 iface = ipaddress.ip_interface(subnet['address'])
293 content += 'IPADDR{}={}\n'.format(idx, str(iface.ip))
294 content += 'NETMASK{}={}\n'.format(
295 idx, subnet.get('netmask', str(iface.netmask)))
296 return content
297
298
299def get_common_network_config(cfg):
300 # You can not have more than one GATEWAY, BOOTPROTO, DNS1, DNS2 and DOMAIN
301 # settings per interface in RHEL and clones. Will run through subnets
302 # section ensuring that only one of these makes to interface configuration.
303 content = ''
304 bootproto = 'dhcp'
305 defaultgw = None
306 resolvers = []
307 search_domains = []
308 for subnet in cfg['subnets']:
309 if subnet['type'] in ('dhcp', 'dhcp4'):
310 # OK, so we have a DHCP subnet. Let's assume all other subnets
311 # are also DHCP and just stop looking
312 bootproto = 'dhcp'
313 break
314 elif subnet['type'] in ('static', 'static4'):
315 # If this is a static subnet, then there might be a default
316 # gateway and resolvers set and if there aren't we'll keep
317 # looking for them in the next subnet configuration
318 bootproto = 'none'
319 if not defaultgw:
320 defaultgw = subnet.get('gateway')
321 if not resolvers:
322 resolvers = subnet.get('dns_nameservers')
323 if not search_domains:
324 search_domains = subnet.get('dns_search')
325 else:
326 # Let's log the lack of support and continue - there still
327 # migth be a supported subnet configuration
328 LOG.warn('Configuration of subnet type {} '
329 'not supported'.format(subnet['type']))
330
331 content += 'BOOTPROTO={}\n'.format(bootproto)
332 if bootproto != 'dhcp':
333 if defaultgw:
334 content += 'GATEWAY={}\n'.format(defaultgw)
335 for idx, resolver in enumerate(resolvers):
336 content += 'DNS{}={}\n'.format(idx+1, resolver)
337 if search_domains:
338 content += 'DOMAIN="{}"\n'.format(' '.join(search_domains))
339
340 return content
341
342
343def write_resolv_conf(cfg, target):
344 content = '# Generated by MAAS fast-path installer\n'
345
346 for resolver in cfg.get('address', []):
347 content += 'nameserver {}\n'.format(resolver)
348
349 search_domains = cfg.get('search', [])
350 if search_domains:
351 content += 'search {}\n'.format(' '.join(search_domains))
352
353 path = os.path.join(target, 'etc', 'resolv.conf')
354 util.write_file(path, content)
355
356
357def write_physical_iface_config(cfg, target):
358 content = (
359 '# Generated by MAAS fast-path installer\n'
360 'DEVICE={device}\n'
361 'HWADDR={hwaddr}\n'
362 'MTU={mtu}\n'
363 'ONBOOT=yes\n'
364 ).format(
365 device=cfg['name'],
366 hwaddr=cfg['mac_address'],
367 mtu=cfg.get('mtu', 1500)
368 )
369
370 content += get_common_network_config(cfg) + \
371 get_subnet_config(cfg['subnets'])
372
373 path = os.path.join(target, 'etc', 'sysconfig',
374 'network-scripts', 'ifcfg-%s' % cfg['name'])
375 util.write_file(path, content)
376
377
378def write_vlan_iface_config(cfg, target):
379 content = (
380 '# Generated by MAAS fast-path installer\n'
381 'DEVICE={device}\n'
382 'MTU={mtu}\n'
383 'ONBOOT=yes\n'
384 'VLAN=yes\n'
385 ).format(
386 device=cfg['name'],
387 mtu=cfg.get('mtu', 1500)
388 )
389
390 content += get_common_network_config(cfg) + \
391 get_subnet_config(cfg['subnets'])
392
393 path = os.path.join(target, 'etc', 'sysconfig',
394 'network-scripts', 'ifcfg-%s' % cfg['name'])
395 util.write_file(path, content)
396
397
398def apply_networking(config, target):
399 network_config = config.get('network')
400 if not network_config:
401 LOG.warn("Unable to retrieve network configuration, will fallback "
402 "to configuring DHCP on boot device")
403 bootmac = get_boot_mac()
404 inames = get_interface_names()
405 iname = inames[bootmac.lower()]
406 network_config = {
407 'config': [
408 {'mac_address': bootmac,
409 'name': iname,
410 'type': 'physical',
411 'subnets': [{'type': 'dhcp'}]}],
412 'version': 1}
413
414 # Bring back ethN device naming on CentOS 6
415 rename_ifaces(network_config, target)
416
417 for cfg in network_config['config']:
418 if cfg['type'] == 'nameserver':
419 write_resolv_conf(cfg, target)
420 if cfg['type'] == 'physical':
421 write_physical_iface_config(cfg, target)
422 if cfg['type'] == 'vlan':
423 write_vlan_iface_config(cfg, target)
424
425
426def get_grub_root(target, dedicated_boot):
427 """Extracts the grub root (hdX,X) from the grub command.
428
429 This is used so the correct root device is used to install
430 stage1/stage2 boot loader.
431
432 Note: grub-install normally does all of this for you, but
433 since the grub is older, it has an issue with the ISCSI
434 target as /dev/sda and cannot enumarate it with the BIOS.
435 """
436 stage1 = '/boot/grub/stage1'
437 if dedicated_boot:
438 stage1 = '/grub/stage1'
439
440 with util.RunInChroot(target) as in_chroot:
441 data = ('find {}\n'
442 'quit\n').format(stage1).encode('utf-8')
443 out, err = in_chroot(['grub', '--batch'],
444 data=data, capture=True)
445 regex = re.search(r'^\s+(\(.+?\))$', out, re.MULTILINE)
446 return regex.groups()[0]
105447
106448
107def strip_kernel_params(params, strip_params=[]):449def strip_kernel_params(params, strip_params=[]):
@@ -118,6 +460,33 @@
118 return new_params460 return new_params
119461
120462
463def get_extra_kernel_parameters():
464 """Extracts the extra kernel commands from /proc/cmdline
465 that should be placed onto the host.
466
467 Any command following the '---' entry should be placed
468 onto the host.
469 """
470 cmdline = read_file('/proc/cmdline')
471 cmdline = cmdline.split()
472 if '---' not in cmdline:
473 return []
474 idx = cmdline.index('---') + 1
475 if idx >= len(cmdline) + 1:
476 return []
477 return strip_kernel_params(
478 cmdline[idx:],
479 strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF='])
480
481
482def get_root_info(target):
483 """Returns the root partitions information."""
484 rootpath = block.get_devices_for_mp(target)[0]
485 rootdev = os.path.basename(rootpath)
486 blocks = block._lsblock()
487 return blocks[rootdev]
488
489
121def get_boot_file(target, filename):490def get_boot_file(target, filename):
122 """Return the full filename of file in /boot on target."""491 """Return the full filename of file in /boot on target."""
123 boot_dir = os.path.join(target, 'boot')492 boot_dir = os.path.join(target, 'boot')
@@ -125,81 +494,163 @@
125 fname494 fname
126 for fname in os.listdir(boot_dir)495 for fname in os.listdir(boot_dir)
127 if fname.startswith(filename)496 if fname.startswith(filename)
128 ]497 ]
129 if not files:498 if not files:
130 return None499 return None
131 return files[0]500 return files[0]
132501
133502
134def write_grub_conf(target, grub_root, extra=[]):503def write_grub_conf(target, dedicated_boot, extra=[]):
135 """Writes a new /boot/grub/grub.conf with the correct504 """Writes a new /boot/grub/grub.conf with the correct
136 boot arguments."""505 boot arguments."""
506 grub_path = os.path.join(target, 'boot', 'grub', 'grub.conf')
507 util.write_file(grub_path, '# UEFI system, see /etc/grub.conf\n')
508
509 if util.is_uefi_bootable():
510 grub_path = os.path.join(target, 'boot', 'efi',
511 'EFI', 'redhat', 'grub.conf')
512
513 grub_root = get_grub_root(target, dedicated_boot)
137 root_info = get_root_info(target)514 root_info = get_root_info(target)
138 grub_path = os.path.join(target, 'boot', 'grub', 'grub.conf')
139 extra_opts = ' '.join(extra)515 extra_opts = ' '.join(extra)
140 vmlinuz = get_boot_file(target, 'vmlinuz')516 vmlinuz = get_boot_file(target, 'vmlinuz')
141 initrd = get_boot_file(target, 'initramfs')517 initrd = get_boot_file(target, 'initramfs')
142 with open(grub_path, 'w') as stream:518
143 stream.write(519 if not dedicated_boot:
144 GRUB_CONF.format(520 vmlinuz = 'boot/' + vmlinuz
145 grub_root=grub_root,521 initrd = 'boot/' + initrd
146 vmlinuz=vmlinuz,522
147 initrd=initrd,523 util.write_file(grub_path,
148 root_uuid=root_info['UUID'],524 GRUB_CONF.format(
149 extra_opts=extra_opts) + '\n')525 grub_root=grub_root,
150526 vmlinuz=vmlinuz,
151527 initrd=initrd,
152def get_extra_kernel_parameters():528 root_uuid=root_info['UUID'],
153 """Extracts the extra kernel commands from /proc/cmdline529 extra_opts=extra_opts) + '\n')
154 that should be placed onto the host.530
155531 # FIXME. replace() is unreliable here if target ends with /
156 Any command following the '--' entry should be placed532 # Update /etc/grub.conf link as it is used by kernel install scripts
157 onto the host.533 util.subp(['ln', '-sf', grub_path.replace(target, '..'),
158 """534 '/etc/grub.conf'], target=target)
159 cmdline = read_file('/proc/cmdline')535
160 cmdline = cmdline.split()536
161 if '--' not in cmdline:537def get_uefi_partition():
162 return []538 """Return the UEFI partition."""
163 idx = cmdline.index('--') + 1539 for _, value in block._lsblock().items():
164 if idx >= len(cmdline) + 1:540 if value['LABEL'] == 'uefi-boot':
165 return []541 return value
166 return strip_kernel_params(542 return None
167 cmdline[idx:],543
168 strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF='])544
169545def install_bootloader(target, device=None):
170546 """Installs bootloader to the device."""
171def get_grub_root(target):547 cmd = []
172 """Extracts the grub root (hdX,X) from the grub command.548 if util.is_uefi_bootable():
173549 uefi_dev = get_uefi_partition()['device_path']
174 This is used so the correct root device is used to install550 disk, part = block.get_blockdev_for_partition(uefi_dev)
175 stage1/stage2 boot loader.551 cmd = ['efibootmgr', '-v', '-c', '-w', '-L', 'centos',
176552 '-d', disk, '-p', part, '-l', r'\EFI\redhat\grub.efi']
177 Note: grub-install normally does all of this for you, but553 else:
178 since the grub is older, it has an issue with the ISCSI554 # Legacy grub-install uses df output, which itself uses /etc/mtab
179 target as /dev/sda and cannot enumarate it with the BIOS.555 with util.RunInChroot(target) as in_chroot:
180 """556 in_chroot(['cp', '-f', '/proc/mounts', '/etc/mtab'])
181 with util.RunInChroot(target) as in_chroot:557 cmd = ['grub-install', '--recheck', device]
182 data = '\n'.join([558
183 'find /boot/grub/stage1',559 with util.RunInChroot(target) as in_chroot:
184 'quit',560 in_chroot(cmd)
185 ]).encode('utf-8')561
186 out, err = in_chroot(['grub', '--batch'],562
187 data=data, capture=True)563def setup_grub(cfg, target):
188 regex = re.search('^\s+(\(.+?\))$', out, re.MULTILINE)564 # target is the path to the mounted filesystem
189 return regex.groups()[0]565
190566 # FIXME: these methods need moving to curtin.block
191567 # and using them from there rather than commands.block_meta
192def grub_install(target, root):568 from curtin.commands.block_meta import (extract_storage_ordered_dict,
193 """Installs grub onto the root."""569 get_path_to_storage_volume)
194 root_dev = root.split(',')[0] + ')'570
195 with util.RunInChroot(target) as in_chroot:571 grubcfg = cfg.get('grub', {})
196 data = '\n'.join([572
197 'root %s' % root,573 # copy legacy top level name
198 'setup %s' % root_dev,574 if 'grub_install_devices' in cfg and 'install_devices' not in grubcfg:
199 'quit',575 grubcfg['install_devices'] = cfg['grub_install_devices']
200 ]).encode('utf-8')576
201 in_chroot(['grub', '--batch'],577 LOG.debug("setup grub on target %s", target)
202 data=data)578 # if there is storage config, look for devices tagged with 'grub_device'
579 storage_cfg_odict = None
580 try:
581 storage_cfg_odict = extract_storage_ordered_dict(cfg)
582 except ValueError as e:
583 pass
584
585 if storage_cfg_odict:
586 storage_grub_devices = []
587 for item_id, item in storage_cfg_odict.items():
588 if not item.get('grub_device'):
589 continue
590 LOG.debug("checking: %s", item)
591 storage_grub_devices.append(
592 get_path_to_storage_volume(item_id, storage_cfg_odict))
593 if len(storage_grub_devices) > 0:
594 grubcfg['install_devices'] = storage_grub_devices
595
596 LOG.debug("install_devices: %s", grubcfg.get('install_devices'))
597 if 'install_devices' in grubcfg:
598 instdevs = grubcfg.get('install_devices')
599 if isinstance(instdevs, str):
600 instdevs = [instdevs]
601 if instdevs is None:
602 LOG.debug("grub installation disabled by config")
603 else:
604 # If there were no install_devices found then we try to do the right
605 # thing. That right thing is basically installing on all block
606 # devices that are mounted. On powerpc, though it means finding PrEP
607 # partitions.
608 devs = block.get_devices_for_mp(target)
609 blockdevs = set()
610 for maybepart in devs:
611 try:
612 (blockdev, part) = block.get_blockdev_for_partition(maybepart)
613 blockdevs.add(blockdev)
614 except ValueError as e:
615 # if there is no syspath for this device such as a lvm
616 # or raid device, then a ValueError is raised here.
617 LOG.debug("failed to find block device for %s", maybepart)
618 instdevs = list(blockdevs)
619
620 # UEFI requires additional packages
621 #if util.is_uefi_bootable():
622 # pkgs = ['efibootmgr']
623 # install_packages(pkgs, target=target)
624
625 # CentOS will not assemble MD devices on boot without rd.md.uuid=MD_UUID
626 # kernel parameters
627 mdmap = {}
628 rdmduuids = []
629 try:
630 mdmap = mdadm.md_read_run_mdadm_map()
631 for md in mdmap:
632 rdmduuids.append("rd.md.uuid=%s" % mdadm.md_get_uuid(mdmap[md][2]))
633 except ValueError as e:
634 pass
635
636 dedicated_boot = False
637 if block.get_devices_for_mp(os.path.join(target, 'boot')):
638 dedicated_boot = True
639 write_grub_conf(target, dedicated_boot, extra=get_extra_kernel_parameters() + rdmduuids)
640
641 if instdevs:
642 instdevs = [block.get_dev_name_entry(i)[1] for i in instdevs]
643 else:
644 instdevs = ["none"]
645
646 LOG.debug("installing grub to %s", instdevs)
647
648 if util.is_uefi_bootable():
649 if grubcfg.get('update_nvram', False):
650 install_bootloader(target)
651 else:
652 for dev in instdevs:
653 install_bootloader(target, dev)
203654
204655
205def set_autorelabel(target):656def set_autorelabel(target):
@@ -214,128 +665,75 @@
214 open(path, 'a').close()665 open(path, 'a').close()
215666
216667
217def get_boot_mac():
218 """Return the mac address of the booting interface."""
219 cmdline = read_file('/proc/cmdline')
220 cmdline = cmdline.split()
221 try:
222 bootif = [
223 option
224 for option in cmdline
225 if option.startswith('BOOTIF')
226 ][0]
227 except IndexError:
228 return None
229 _, mac = bootif.split('=')
230 mac = mac.split('-')[1:]
231 return ':'.join(mac)
232
233
234def get_interface_names():
235 """Return a dictionary mapping mac addresses to interface names."""
236 sys_path = "/sys/class/net"
237 ifaces = {}
238 for iname in os.listdir(sys_path):
239 mac = read_file(os.path.join(sys_path, iname, "address"))
240 mac = mac.strip().lower()
241 ifaces[mac] = iname
242 return ifaces
243
244
245def get_ipv4_config(iface, data):
246 """Returns the contents of the interface file for ipv4."""
247 config = [
248 'TYPE="Ethernet"',
249 'NM_CONTROLLED="no"',
250 'USERCTL="yes"',
251 ]
252 if 'hwaddress' in data:
253 config.append('HWADDR="%s"' % data['hwaddress'])
254 else:
255 # Last ditch effort, use the device name, it probably won't match
256 # though!
257 config.append('DEVICE="%s"' % iface)
258 if data['auto']:
259 config.append('ONBOOT="yes"')
260 else:
261 config.append('ONBOOT="no"')
262
263 method = data['method']
264 if method == 'dhcp':
265 config.append('BOOTPROTO="dhcp"')
266 config.append('PEERDNS="yes"')
267 config.append('PERSISTENT_DHCLIENT="1"')
268 if 'hostname' in data:
269 config.append('DHCP_HOSTNAME="%s"' % data['hostname'])
270 elif method == 'static':
271 config.append('BOOTPROTO="none"')
272 config.append('IPADDR="%s"' % data['address'])
273 config.append('NETMASK="%s"' % data['netmask'])
274 if 'broadcast' in data:
275 config.append('BROADCAST="%s"' % data['broadcast'])
276 if 'gateway' in data:
277 config.append('GATEWAY="%s"' % data['gateway'])
278 elif method == 'manual':
279 config.append('BOOTPROTO="none"')
280 return '\n'.join(config)
281
282
283def write_interface_config(target, iface, data):
284 """Writes config for interface."""
285 family = data['family']
286 if family != "inet":
287 # Only supporting ipv4 currently
288 print(
289 "WARN: unsupported family %s, "
290 "failed to configure interface: %s" (family, iface))
291 return
292 config = get_ipv4_config(iface, data)
293 path = os.path.join(
294 target, 'etc', 'sysconfig', 'network-scripts', 'ifcfg-%s' % iface)
295 with open(path, 'w') as stream:
296 stream.write(config + '\n')
297
298
299def write_network_config(target, mac):
300 """Write network configuration for the given MAC address."""
301 inames = get_interface_names()
302 iname = inames[mac.lower()]
303 write_interface_config(
304 target, iname, {
305 'family': 'inet',
306 'hwaddress': mac.upper(),
307 'auto': True,
308 'method': 'dhcp'
309 })
310
311
312def main():668def main():
313 state = util.load_command_environment()669 state = util.load_command_environment()
670
314 target = state['target']671 target = state['target']
315 if target is None:672 if target is None:
316 print("Target was not provided in the environment.")673 sys.stderr.write("Target was not provided in the environment.")
317 sys.exit(1)674 sys.exit(1)
318 fstab = state['fstab']675
319 if fstab is None:676 cfg = config.load_command_config(None, state)
320 print("/etc/fstab output was not provided in the environment.")677 stack_prefix = state.get('report_stack_prefix', '')
321 sys.exit(1)678
322 bootmac = get_boot_mac()679 with events.ReportEventStack(
323 if bootmac is None:680 name=stack_prefix, reporting_enabled=True, level="INFO",
324 print("Unable to determine boot interface.")681 description="writing config files"):
325 sys.exit(1)682 curthooks.write_files(cfg, target)
326 devices = get_block_devices(target)683
327 if not devices:684 # Default CentOS image does not contain some packages that may be necessary
328 print("Unable to find block device for: %s" % target)685 install_missing_packages(cfg, target)
329 sys.exit(1)686
330687 # If a mdadm.conf file was created by block_meta then it needs to be copied
331 write_fstab(target, fstab)688 # onto the target system
332689 mdadm_location = os.path.join(os.path.split(state['fstab'])[0],
333 grub_root = get_grub_root(target)690 "mdadm.conf")
334 write_grub_conf(target, grub_root, extra=get_extra_kernel_parameters())691 if os.path.exists(mdadm_location):
335 grub_install(target, grub_root)692 copy_mdadm_conf(mdadm_location, target)
693
694 with events.ReportEventStack(
695 name=stack_prefix, reporting_enabled=True, level="INFO",
696 description="setting up swap"):
697 curthooks.add_swap(cfg, target, state.get('fstab'))
698
699 with events.ReportEventStack(
700 name=stack_prefix, reporting_enabled=True, level="INFO",
701 description="writing etc/fstab"):
702 write_fstab(target, state.get('fstab'))
703
704 # TODO. There's a multipath implementation on Ubuntu in curtin/curthooks.py
705 # that should be reimplemented here. Skipping for now.
706
707 with events.ReportEventStack(
708 name=stack_prefix, reporting_enabled=True, level="INFO",
709 description="updating packages on target system"):
710 system_upgrade(cfg, target)
711
712 # TODO. If a crypttab file was created by block_meta than it needs to be
713 # copied onto the target system, and update_initramfs() (well, alternative
714 # for CentOS actually) needs to be run, so that the cryptsetup hooks are
715 # properly configured on the installed system and it will be able to open
716 # encrypted volumes at boot. Skipping for now.
717
718 with events.ReportEventStack(
719 name=stack_prefix, reporting_enabled=True, level="INFO",
720 description="apply networking"):
721 apply_networking(cfg, target)
722
723 # If udev dname rules were created, copy them to target
724 udev_rules_d = os.path.join(state['scratch'], "rules.d")
725 if os.path.isdir(udev_rules_d):
726 curthooks.copy_dname_rules(udev_rules_d, target)
727
728 with events.ReportEventStack(
729 name=stack_prefix, reporting_enabled=True, level="INFO",
730 description="updating packages on target system"):
731 setup_grub(cfg, target)
336732
337 set_autorelabel(target)733 set_autorelabel(target)
338 write_network_config(target, bootmac)734 update_initramfs(target)
735
736 sys.exit(0)
339737
340738
341if __name__ == "__main__":739if __name__ == "__main__":
342740
=== modified file 'curtin/centos7/curtin-hooks.py'
--- curtin/centos7/curtin-hooks.py 2016-05-11 18:47:46 +0000
+++ curtin/centos7/curtin-hooks.py 2017-01-09 15:19:25 +0000
@@ -1,20 +1,17 @@
1#!/usr/bin/env python1#!/usr/bin/env python
22# coding: utf-8
3from __future__ import (
4 absolute_import,
5 print_function,
6 unicode_literals,
7 )
83
9import codecs4import codecs
5import ipaddress
10import os6import os
7import shutil
11import sys8import sys
12import shutil
139
14from curtin import (10from curtin import block, config, util
15 block,11from curtin.block import mdadm
16 util,12from curtin.commands import curthooks
17 )13from curtin.log import LOG
14from curtin.reporter import events
1815
1916
20"""17"""
@@ -24,60 +21,142 @@
2421
25- Legacy boot22- Legacy boot
26- UEFI boot23- UEFI boot
27- DHCP of BOOTIF24- Simple and MD RAID layouts
25- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces
2826
29Not Supported:27Not Supported:
3028
31- Multiple network configration29- Bond, bridge, route configurations
32- IPv630- IPv6
33"""31- Disk encryption
3432- bcache
35FSTAB_PREPEND = """\33- LVM
36#34- Multipath
37# /etc/fstab35"""
38# Created by MAAS fast-path installer.36
39#
40# Accessible filesystems, by reference, are maintained under '/dev/disk'
41# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
42#
43"""
44
45FSTAB_UEFI = """\
46LABEL=uefi-boot /boot/efi vfat defaults 0 0
47"""
4837
49GRUB_PREPEND = """\38GRUB_PREPEND = """\
50# Set by MAAS fast-path installer.39# Set by MAAS fast-path installer.
51GRUB_TIMEOUT=040GRUB_TIMEOUT=1
52GRUB_TERMINAL_OUTPUT=console41GRUB_TERMINAL_OUTPUT=console
53GRUB_DISABLE_OS_PROBER=true42GRUB_DISABLE_OS_PROBER=true
54"""43"""
5544
5645
57def get_block_devices(target):46def get_installed_packages(target=None):
58 """Returns list of block devices for the given target."""47 (out, _) = util.subp(['rpm', '-qa', '--qf', '"%{NAME}\n"'],
59 devs = block.get_devices_for_mp(target)48 target=target, capture=True)
60 blockdevs = set()49 return set(out.splitlines())
61 for maybepart in devs:50
62 (blockdev, part) = block.get_blockdev_for_partition(maybepart)51
63 blockdevs.add(blockdev)52def install_packages(pkglist, target):
64 return list(blockdevs)53 if isinstance(pkglist, str):
6554 pkglist = [pkglist]
6655 # Does not work – hosts are not resolved
67def get_root_info(target):56 # return util.subp(['yum', '-q', '-y', 'install'] + pkglist, target=target, capture=True)
68 """Returns the root partitions information."""57 with util.RunInChroot(target) as in_chroot:
69 rootpath = block.get_devices_for_mp(target)[0]58 in_chroot(['yum', '-q', '-y', 'install'] + pkglist)
70 rootdev = os.path.basename(rootpath)59
71 blocks = block._lsblock()60
72 return blocks[rootdev]61def install_missing_packages(cfg, target):
7362 ''' describe which operation types will require specific packages
7463
75def get_uefi_partition():64 'custom_config_key': {
76 """Return the UEFI partition."""65 'pkg1': ['op_name_1', 'op_name_2', ...]
77 for _, value in block._lsblock().items():66 }
78 if value['LABEL'] == 'uefi-boot':67 '''
79 return value68 custom_configs = {
80 return None69 'storage': {
70 'lvm2': ['lvm_volgroup', 'lvm_partition'],
71 'mdadm': ['raid'],
72 # XXX. At the moment there are no official packages for bcache on CentOS
73 #'bcache-tools': ['bcache']
74 },
75 'network': {
76 'bridge-utils': ['bridge']
77 },
78 }
79
80 format_configs = {
81 'xfsprogs': ['xfs'],
82 'e2fsprogs': ['ext2', 'ext3', 'ext4'],
83 'btrfs-tools': ['btrfs'],
84 }
85
86 needed_packages = []
87 installed_packages = get_installed_packages(target)
88 for cust_cfg, pkg_reqs in custom_configs.items():
89 if cust_cfg not in cfg:
90 continue
91
92 all_types = set(
93 operation['type']
94 for operation in cfg[cust_cfg]['config']
95 )
96 for pkg, types in pkg_reqs.items():
97 if set(types).intersection(all_types) and \
98 pkg not in installed_packages:
99 needed_packages.append(pkg)
100
101 format_types = set(
102 [operation['fstype']
103 for operation in cfg[cust_cfg]['config']
104 if operation['type'] == 'format'])
105 for pkg, fstypes in format_configs.items():
106 if set(fstypes).intersection(format_types) and \
107 pkg not in installed_packages:
108 needed_packages.append(pkg)
109
110 if needed_packages:
111 state = util.load_command_environment()
112 with events.ReportEventStack(
113 name=state.get('report_stack_prefix'),
114 reporting_enabled=True, level="INFO",
115 description="Installing packages on target system: " +
116 str(needed_packages)):
117 install_packages(needed_packages, target=target)
118
119
120def copy_mdadm_conf(mdadm_conf, target):
121 if not mdadm_conf:
122 LOG.warn("mdadm config must be specified, not copying")
123 return
124
125 LOG.info("copying mdadm.conf into target")
126 shutil.copy(mdadm_conf, os.path.sep.join([target, 'etc/mdadm.conf']))
127
128
129def update_initramfs(target=None):
130 path = os.path.join(
131 target, 'etc', 'dracut.conf.d', 'local.conf')
132 with open(path, 'w') as stream:
133 stream.write('mdadmconf="yes"' + '\n'
134 'lvmconf="yes"' + '\n')
135
136 with util.RunInChroot(target) as in_chroot:
137 in_chroot(['dracut', '-f', '--regenerate-all'])
138
139
140def system_upgrade(cfg, target):
141 """run yum upgrade in target.
142
143 config:
144 system_upgrade:
145 enabled: False
146
147 """
148 mycfg = {'system_upgrade': {'enabled': False}}
149 config.merge_config(mycfg, cfg)
150 mycfg = mycfg.get('system_upgrade')
151 if not isinstance(mycfg, dict):
152 LOG.debug("system_upgrade disabled by config. entry not a dict.")
153 return
154
155 if not config.value_as_boolean(mycfg.get('enabled', True)):
156 LOG.debug("system_upgrade disabled by config.")
157 return
158
159 util.subp(['yum', '-q', '-y', 'upgrade'], target=target, capture=True)
81160
82161
83def read_file(path):162def read_file(path):
@@ -86,16 +165,178 @@
86 return stream.read()165 return stream.read()
87166
88167
89def write_fstab(target, curtin_fstab):168def get_boot_mac():
90 """Writes the new fstab, using the fstab provided169 """Return the mac address of the booting interface."""
91 from curtin."""170 cmdline = read_file('/proc/cmdline')
92 fstab_path = os.path.join(target, 'etc', 'fstab')171 cmdline = cmdline.split()
93 fstab_data = read_file(curtin_fstab)172 try:
94 with open(fstab_path, 'w') as stream:173 bootif = [
95 stream.write(FSTAB_PREPEND)174 option
96 stream.write(fstab_data)175 for option in cmdline
97 if util.is_uefi_bootable():176 if option.startswith('BOOTIF')
98 stream.write(FSTAB_UEFI)177 ][0]
178 except IndexError:
179 return None
180 _, mac = bootif.split('=')
181 mac = mac.split('-')[1:]
182 return ':'.join(mac)
183
184
185def get_interface_names():
186 """Return a dictionary mapping mac addresses to interface names."""
187 sys_path = "/sys/class/net"
188 ifaces = {}
189 for iname in os.listdir(sys_path):
190 mac = read_file(os.path.join(sys_path, iname, "address"))
191 mac = mac.strip().lower()
192 ifaces[mac] = iname
193 return ifaces
194
195
196def get_subnet_config(subnets):
197 content = ''
198 for idx, subnet in enumerate(subnets):
199 if subnet['type'] in ('static', 'static4'):
200 if idx == 0:
201 idx = ''
202 iface = ipaddress.ip_interface(subnet['address'])
203 content += 'IPADDR{}={}\n'.format(idx, str(iface.ip))
204 content += 'NETMASK{}={}\n'.format(
205 idx, subnet.get('netmask', str(iface.netmask)))
206 return content
207
208
209def get_common_network_config(cfg):
210 # You can not have more than one GATEWAY, BOOTPROTO, DNS1, DNS2 and DOMAIN
211 # settings per interface in RHEL and clones. Will run through subnets
212 # section ensuring that only one of these makes to interface configuration.
213 content = ''
214 bootproto = 'dhcp'
215 defaultgw = None
216 resolvers = []
217 search_domains = []
218 for subnet in cfg['subnets']:
219 if subnet['type'] in ('dhcp', 'dhcp4'):
220 # OK, so we have a DHCP subnet. Let's assume all other subnets
221 # are also DHCP and just stop looking
222 bootproto = 'dhcp'
223 break
224 elif subnet['type'] in ('static', 'static4'):
225 # If this is a static subnet, then there might be a default
226 # gateway and resolvers set and if there aren't we'll keep
227 # looking for them in the next subnet configuration
228 bootproto = 'none'
229 if not defaultgw:
230 defaultgw = subnet.get('gateway')
231 if not resolvers:
232 resolvers = subnet.get('dns_nameservers')
233 if not search_domains:
234 search_domains = subnet.get('dns_search')
235 else:
236 # Let's log the lack of support and continue - there still
237 # migth be a supported subnet configuration
238 LOG.warn('Configuration of subnet type {} '
239 'not supported'.format(subnet['type']))
240
241 content += 'BOOTPROTO={}\n'.format(bootproto)
242 if bootproto != 'dhcp':
243 if defaultgw:
244 content += 'GATEWAY={}\n'.format(defaultgw)
245 for idx, resolver in enumerate(resolvers):
246 content += 'DNS{}={}\n'.format(idx+1, resolver)
247 if search_domains:
248 content += 'DOMAIN="{}"\n'.format(' '.join(search_domains))
249
250 return content
251
252
253def write_resolv_conf(cfg, target):
254 content = '# Generated by MAAS fast-path installer\n'
255
256 for resolver in cfg.get('address', []):
257 content += 'nameserver {}\n'.format(resolver)
258
259 search_domains = cfg.get('search', [])
260 if search_domains:
261 content += 'search {}\n'.format(' '.join(search_domains))
262
263 path = os.path.join(target, 'etc', 'resolv.conf')
264 util.write_file(path, content)
265
266
267def write_physical_iface_config(cfg, target):
268 content = (
269 '# Generated by MAAS fast-path installer\n'
270 'DEVICE={device}\n'
271 'HWADDR={hwaddr}\n'
272 'MTU={mtu}\n'
273 'ONBOOT=yes\n'
274 ).format(
275 device=cfg['name'],
276 hwaddr=cfg['mac_address'],
277 mtu=cfg.get('mtu', 1500)
278 )
279
280 content += get_common_network_config(cfg) + \
281 get_subnet_config(cfg['subnets'])
282
283 path = os.path.join(target, 'etc', 'sysconfig',
284 'network-scripts', 'ifcfg-%s' % cfg['name'])
285 util.write_file(path, content)
286
287
288def write_vlan_iface_config(cfg, target):
289 content = (
290 '# Generated by MAAS fast-path installer\n'
291 'DEVICE={device}\n'
292 'MTU={mtu}\n'
293 'ONBOOT=yes\n'
294 'VLAN=yes\n'
295 ).format(
296 device=cfg['name'],
297 mtu=cfg.get('mtu', 1500)
298 )
299
300 content += get_common_network_config(cfg) + \
301 get_subnet_config(cfg['subnets'])
302
303 path = os.path.join(target, 'etc', 'sysconfig',
304 'network-scripts', 'ifcfg-%s' % cfg['name'])
305 util.write_file(path, content)
306
307
308def apply_networking(config, target):
309 network_config = config.get('network')
310 if not network_config:
311 LOG.warn("Unable to retrieve network configuration, will fallback "
312 "to configuring DHCP on boot device")
313 bootmac = get_boot_mac()
314 inames = get_interface_names()
315 iname = inames[bootmac.lower()]
316 network_config = {
317 'config': [
318 {'mac_address': bootmac,
319 'name': iname,
320 'type': 'physical',
321 'subnets': [{'type': 'dhcp'}]}],
322 'version': 1}
323
324 for cfg in network_config['config']:
325 if cfg['type'] == 'nameserver':
326 write_resolv_conf(cfg, target)
327 if cfg['type'] == 'physical':
328 write_physical_iface_config(cfg, target)
329 if cfg['type'] == 'vlan':
330 write_vlan_iface_config(cfg, target)
331
332
333def update_grub_default(target, extra=[]):
334 """Updates /etc/default/grub with the correct options."""
335 grub_default_path = os.path.join(target, 'etc', 'default', 'grub')
336 kernel_cmdline = ' '.join(extra)
337 with open(grub_default_path, 'a') as stream:
338 stream.write(GRUB_PREPEND)
339 stream.write('GRUB_CMDLINE_LINUX=\"%s\"\n' % kernel_cmdline)
99340
100341
101def strip_kernel_params(params, strip_params=[]):342def strip_kernel_params(params, strip_params=[]):
@@ -121,9 +362,9 @@
121 """362 """
122 cmdline = read_file('/proc/cmdline')363 cmdline = read_file('/proc/cmdline')
123 cmdline = cmdline.split()364 cmdline = cmdline.split()
124 if '--' not in cmdline:365 if '---' not in cmdline:
125 return []366 return []
126 idx = cmdline.index('--') + 1367 idx = cmdline.index('---') + 1
127 if idx >= len(cmdline) + 1:368 if idx >= len(cmdline) + 1:
128 return []369 return []
129 return strip_kernel_params(370 return strip_kernel_params(
@@ -131,57 +372,141 @@
131 strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF='])372 strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF='])
132373
133374
134def update_grub_default(target, extra=[]):
135 """Updates /etc/default/grub with the correct options."""
136 grub_default_path = os.path.join(target, 'etc', 'default', 'grub')
137 kernel_cmdline = ' '.join(extra)
138 with open(grub_default_path, 'a') as stream:
139 stream.write(GRUB_PREPEND)
140 stream.write('GRUB_CMDLINE_LINUX=\"%s\"\n' % kernel_cmdline)
141
142
143def grub2_install(target, root):
144 """Installs grub2 to the root."""
145 with util.RunInChroot(target) as in_chroot:
146 in_chroot(['grub2-install', '--recheck', root])
147
148
149def grub2_mkconfig(target):375def grub2_mkconfig(target):
150 """Writes the new grub2 config."""376 """Writes the new grub2 config."""
151 with util.RunInChroot(target) as in_chroot:377 # NOTE. CentOS kernel packages expect /etc/grub2.cfg symlink in
152 in_chroot(['grub2-mkconfig', '-o', '/boot/grub2/grub.cfg'])378 # BIOS mode and /etc/grub2-efi.cfg symlink in UEFI mode pointing
153379 # to an actual grub.cfg
154380
155def install_efi(target, uefi_path):381 grub_cfg = '/boot/grub2/grub.cfg'
156 """Install the EFI data from /boot into efi partition."""382 grub_link = '/etc/grub2.cfg'
157 # Create temp mount point for uefi partition.383 grub_unlink = os.path.join(target, 'etc', 'grub2-efi.cfg')
158 tmp_efi = os.path.join(target, 'boot', 'efi_part')384
159 os.mkdir(tmp_efi)385 if util.is_uefi_bootable():
160 util.subp(['mount', uefi_path, tmp_efi])386 grub_cfg = '/boot/efi/EFI/centos/grub.cfg'
161387 grub_link = '/etc/grub2-efi.cfg'
162 # Copy the data over.388 grub_unlink = os.path.join(target, 'etc', 'grub2.cfg')
163 try:389 util.write_file(os.path.join(target, 'boot', 'grub2', 'grub.cfg'),
164 efi_path = os.path.join(target, 'boot', 'efi')390 '# UEFI system, see /etc/grub2-efi.cfg\n')
165 if os.path.exists(os.path.join(tmp_efi, 'EFI')):391
166 shutil.rmtree(os.path.join(tmp_efi, 'EFI'))392 with util.RunInChroot(target) as in_chroot:
167 shutil.copytree(393 in_chroot(['grub2-mkconfig', '-o', grub_cfg])
168 os.path.join(efi_path, 'EFI'),394 in_chroot(['ln', '-sf', grub_cfg, grub_link])
169 os.path.join(tmp_efi, 'EFI'))395 util.del_file(grub_unlink)
170 finally:396
171 # Clean up tmp mount397
172 util.subp(['umount', tmp_efi])398def get_uefi_partition():
173 os.rmdir(tmp_efi)399 """Return the UEFI partition."""
174400 for _, value in block._lsblock().items():
175 # Mount and do grub install401 if value['LABEL'] == 'uefi-boot':
176 util.subp(['mount', uefi_path, efi_path])402 return value
177 try:403 return None
178 with util.RunInChroot(target) as in_chroot:404
179 in_chroot([405
180 'grub2-install', '--target=x86_64-efi',406def install_bootloader(target, device=None):
181 '--efi-directory', '/boot/efi',407 """Installs bootloader to the device."""
182 '--recheck'])408 cmd = []
183 finally:409 if util.is_uefi_bootable():
184 util.subp(['umount', efi_path])410 uefi_dev = get_uefi_partition()['device_path']
411 disk, part = block.get_blockdev_for_partition(uefi_dev)
412 cmd = ['efibootmgr', '-v', '-c', '-w', '-L', 'centos',
413 '-d', disk, '-p', part, '-l', r'\EFI\centos\shim.efi']
414 else:
415 cmd = ['grub2-install', '--recheck', device]
416
417 with util.RunInChroot(target) as in_chroot:
418 in_chroot(cmd)
419
420
421def setup_grub(cfg, target):
422 # target is the path to the mounted filesystem
423
424 # FIXME: these methods need moving to curtin.block
425 # and using them from there rather than commands.block_meta
426 from curtin.commands.block_meta import (extract_storage_ordered_dict,
427 get_path_to_storage_volume)
428
429 grubcfg = cfg.get('grub', {})
430
431 # copy legacy top level name
432 if 'grub_install_devices' in cfg and 'install_devices' not in grubcfg:
433 grubcfg['install_devices'] = cfg['grub_install_devices']
434
435 LOG.debug("setup grub on target %s", target)
436 # if there is storage config, look for devices tagged with 'grub_device'
437 storage_cfg_odict = None
438 try:
439 storage_cfg_odict = extract_storage_ordered_dict(cfg)
440 except ValueError as e:
441 pass
442
443 if storage_cfg_odict:
444 storage_grub_devices = []
445 for item_id, item in storage_cfg_odict.items():
446 if not item.get('grub_device'):
447 continue
448 LOG.debug("checking: %s", item)
449 storage_grub_devices.append(
450 get_path_to_storage_volume(item_id, storage_cfg_odict))
451 if len(storage_grub_devices) > 0:
452 grubcfg['install_devices'] = storage_grub_devices
453
454 LOG.debug("install_devices: %s", grubcfg.get('install_devices'))
455 if 'install_devices' in grubcfg:
456 instdevs = grubcfg.get('install_devices')
457 if isinstance(instdevs, str):
458 instdevs = [instdevs]
459 if instdevs is None:
460 LOG.debug("grub installation disabled by config")
461 else:
462 # If there were no install_devices found then we try to do the right
463 # thing. That right thing is basically installing on all block
464 # devices that are mounted. On powerpc, though it means finding PrEP
465 # partitions.
466 devs = block.get_devices_for_mp(target)
467 blockdevs = set()
468 for maybepart in devs:
469 try:
470 (blockdev, part) = block.get_blockdev_for_partition(maybepart)
471 blockdevs.add(blockdev)
472 except ValueError as e:
473 # if there is no syspath for this device such as a lvm
474 # or raid device, then a ValueError is raised here.
475 LOG.debug("failed to find block device for %s", maybepart)
476 instdevs = list(blockdevs)
477
478 # UEFI requires additional packages
479 if util.is_uefi_bootable():
480 pkgs = ['grub2-efi', 'shim', 'efibootmgr']
481 install_packages(pkgs, target=target)
482
483 # CentOS will not assemble MD devices on boot without rd.md.uuid=MD_UUID
484 # kernel parameters
485 mdmap = {}
486 rdmduuids = []
487 try:
488 mdmap = mdadm.md_read_run_mdadm_map()
489 for md in mdmap:
490 rdmduuids.append("rd.md.uuid=%s" % mdadm.md_get_uuid(mdmap[md][2]))
491 except ValueError as e:
492 pass
493
494 update_grub_default(target, extra=get_extra_kernel_parameters() + rdmduuids)
495 grub2_mkconfig(target)
496
497 if instdevs:
498 instdevs = [block.get_dev_name_entry(i)[1] for i in instdevs]
499 else:
500 instdevs = ["none"]
501
502 LOG.debug("installing grub to %s", instdevs)
503
504 if util.is_uefi_bootable():
505 if grubcfg.get('update_nvram', False):
506 install_bootloader(target)
507 else:
508 for dev in instdevs:
509 install_bootloader(target, dev)
185510
186511
187def set_autorelabel(target):512def set_autorelabel(target):
@@ -196,136 +521,75 @@
196 open(path, 'a').close()521 open(path, 'a').close()
197522
198523
199def get_boot_mac():
200 """Return the mac address of the booting interface."""
201 cmdline = read_file('/proc/cmdline')
202 cmdline = cmdline.split()
203 try:
204 bootif = [
205 option
206 for option in cmdline
207 if option.startswith('BOOTIF')
208 ][0]
209 except IndexError:
210 return None
211 _, mac = bootif.split('=')
212 mac = mac.split('-')[1:]
213 return ':'.join(mac)
214
215
216def get_interface_names():
217 """Return a dictionary mapping mac addresses to interface names."""
218 sys_path = "/sys/class/net"
219 ifaces = {}
220 for iname in os.listdir(sys_path):
221 mac = read_file(os.path.join(sys_path, iname, "address"))
222 mac = mac.strip().lower()
223 ifaces[mac] = iname
224 return ifaces
225
226
227def get_ipv4_config(iface, data):
228 """Returns the contents of the interface file for ipv4."""
229 config = [
230 'TYPE="Ethernet"',
231 'NM_CONTROLLED="no"',
232 'USERCTL="yes"',
233 ]
234 if 'hwaddress' in data:
235 config.append('HWADDR="%s"' % data['hwaddress'])
236 # Fallback to using device name
237 else:
238 config.append('DEVICE="%"' % iface)
239 if data['auto']:
240 config.append('ONBOOT="yes"')
241 else:
242 config.append('ONBOOT="no"')
243
244 method = data['method']
245 if method == 'dhcp':
246 config.append('BOOTPROTO="dhcp"')
247 config.append('PEERDNS="yes"')
248 config.append('PERSISTENT_DHCLIENT="1"')
249 if 'hostname' in data:
250 config.append('DHCP_HOSTNAME="%s"' % data['hostname'])
251 elif method == 'static':
252 config.append('BOOTPROTO="none"')
253 config.append('IPADDR="%s"' % data['address'])
254 config.append('NETMASK="%s"' % data['netmask'])
255 if 'broadcast' in data:
256 config.append('BROADCAST="%s"' % data['broadcast'])
257 if 'gateway' in data:
258 config.append('GATEWAY="%s"' % data['gateway'])
259 elif method == 'manual':
260 config.append('BOOTPROTO="none"')
261 return '\n'.join(config)
262
263
264def write_interface_config(target, iface, data):
265 """Writes config for interface."""
266 family = data['family']
267 if family != "inet":
268 # Only supporting ipv4 currently
269 print(
270 "WARN: unsupported family %s, "
271 "failed to configure interface: %s" (family, iface))
272 return
273 config = get_ipv4_config(iface, data)
274 path = os.path.join(
275 target, 'etc', 'sysconfig', 'network-scripts', 'ifcfg-%s' % iface)
276 with open(path, 'w') as stream:
277 stream.write(config + '\n')
278
279
280def write_network_config(target, mac):
281 """Write network configuration for the given MAC address."""
282 inames = get_interface_names()
283 iname = inames[mac.lower()]
284 write_interface_config(
285 target, iname, {
286 'family': 'inet',
287 'hwaddress': mac.upper(),
288 'auto': True,
289 'method': 'dhcp'
290 })
291
292
293def main():524def main():
294 state = util.load_command_environment()525 state = util.load_command_environment()
526
295 target = state['target']527 target = state['target']
296 if target is None:528 if target is None:
297 print("Target was not provided in the environment.")529 sys.stderr.write("Target was not provided in the environment.")
298 sys.exit(1)530 sys.exit(1)
299 fstab = state['fstab']531
300 if fstab is None:532 cfg = config.load_command_config(None, state)
301 print("/etc/fstab output was not provided in the environment.")533 stack_prefix = state.get('report_stack_prefix', '')
302 sys.exit(1)534
303 bootmac = get_boot_mac()535 with events.ReportEventStack(
304 if bootmac is None:536 name=stack_prefix, reporting_enabled=True, level="INFO",
305 print("Unable to determine boot interface.")537 description="writing config files"):
306 sys.exit(1)538 curthooks.write_files(cfg, target)
307 devices = get_block_devices(target)539
308 if not devices:540 # Default CentOS image does not contain some packages that may be necessary
309 print("Unable to find block device for: %s" % target)541 install_missing_packages(cfg, target)
310 sys.exit(1)542
311543 # If a mdadm.conf file was created by block_meta then it needs to be copied
312 write_fstab(target, fstab)544 # onto the target system
313545 mdadm_location = os.path.join(os.path.split(state['fstab'])[0],
314 update_grub_default(546 "mdadm.conf")
315 target, extra=get_extra_kernel_parameters())547 if os.path.exists(mdadm_location):
316 grub2_mkconfig(target)548 copy_mdadm_conf(mdadm_location, target)
317 if util.is_uefi_bootable():549
318 uefi_part = get_uefi_partition()550 with events.ReportEventStack(
319 if uefi_part is None:551 name=stack_prefix, reporting_enabled=True, level="INFO",
320 print('Unable to determine UEFI parition.')552 description="setting up swap"):
321 sys.exit(1)553 curthooks.add_swap(cfg, target, state.get('fstab'))
322 install_efi(target, uefi_part['device_path'])554
323 else:555 with events.ReportEventStack(
324 for dev in devices:556 name=stack_prefix, reporting_enabled=True, level="INFO",
325 grub2_install(target, dev)557 description="writing etc/fstab"):
558 curthooks.copy_fstab(state.get('fstab'), target)
559
560 # TODO. There's a multipath implementation on Ubuntu in curtin/curthooks.py
561 # that should be reimplemented here. Skipping for now.
562
563 with events.ReportEventStack(
564 name=stack_prefix, reporting_enabled=True, level="INFO",
565 description="updating packages on target system"):
566 system_upgrade(cfg, target)
567
568 # TODO. If a crypttab file was created by block_meta than it needs to be
569 # copied onto the target system, and update_initramfs() (well, alternative
570 # for CentOS actually) needs to be run, so that the cryptsetup hooks are
571 # properly configured on the installed system and it will be able to open
572 # encrypted volumes at boot. Skipping for now.
573
574 with events.ReportEventStack(
575 name=stack_prefix, reporting_enabled=True, level="INFO",
576 description="apply networking"):
577 apply_networking(cfg, target)
578
579 # If udev dname rules were created, copy them to target
580 udev_rules_d = os.path.join(state['scratch'], "rules.d")
581 if os.path.isdir(udev_rules_d):
582 curthooks.copy_dname_rules(udev_rules_d, target)
583
584 with events.ReportEventStack(
585 name=stack_prefix, reporting_enabled=True, level="INFO",
586 description="updating packages on target system"):
587 setup_grub(cfg, target)
326588
327 set_autorelabel(target)589 set_autorelabel(target)
328 write_network_config(target, bootmac)590 update_initramfs(target)
591
592 sys.exit(0)
329593
330594
331if __name__ == "__main__":595if __name__ == "__main__":

Subscribers

People subscribed via source and target branches