Merge lp:~marius-zalinauskas/maas-images/centos-hooks into lp:maas-images

Proposed by Marius Žalinauskas
Status: Rejected
Rejected by: Scott Moser
Proposed branch: lp:~marius-zalinauskas/maas-images/centos-hooks
Merge into: lp:maas-images
Diff against target: 1739 lines (+1124/-462)
2 files modified
curtin/centos6/curtin-hooks.py (+622/-224)
curtin/centos7/curtin-hooks.py (+502/-238)
To merge this branch: bzr merge lp:~marius-zalinauskas/maas-images/centos-hooks
Reviewer Review Type Date Requested Status
Andres Rodriguez (community) Needs Fixing
Tom Mullaney (community) Approve
Review via email: mp+313684@code.launchpad.net

Description of the change

Heavily modified CentOS 6/7 hooks. For the most part it is curtin/commands/curthooks.py ported to CentOS with some code from old CentOS hooks.

Added functionality:

- UEFI boot. It was already kinda working on CentOS 7, but was tricky on CentOS 6.
- Simple and MD RAID layouts. No need to force the simple layout on MAAS anymore.
- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces. Much better than original DHCP only.

To post a comment you must log in.
353. By Marius Žalinauskas <email address hidden>

Bugfix, see https://bugs.launchpad.net/maas-images/+bug/1654853

Revision history for this message
Tom Mullaney (tpmullan) wrote :

These changes work great for me. They make Maas much more usable for people that don't only work with Ubuntu.

review: Approve
Revision history for this message
Andres Rodriguez (andreserl) wrote :

Hi Marius,

Thank you for your contribution!

In order to be able to accept your code you need to sign the Canonical Contributors Agreement. Can you please do that in [1].

That said, we have a few things that need fixing before we are able to review / land branches, to follow the procedures MAAS uses, Please:

 - Separate the UEFI changes into is own branch
 - Seprate the Simple/MD Raid into its own branch
 - IPv4 DHCP/Static Networking, also it its own branch.

Thank you!

[1]: https://www.ubuntu.com/legal/contributors/submit

review: Needs Fixing
Revision history for this message
Scott Moser (smoser) wrote :

Marius,
Thank you for your contribution. Sorry for the confusion or delay in responding.

I'm going to mark this 'rejected' as of right now. You are welcome to re-submit it, but do so against the git repo now.

https://code.launchpad.net/maas-images

thanks!

Unmerged revisions

353. By Marius Žalinauskas <email address hidden>

Bugfix, see https://bugs.launchpad.net/maas-images/+bug/1654853

352. By Marius Žalinauskas <email address hidden>

Extensibly modified CentOS 6/7 Curtin hooks.

For the most part it is curtin/commands/curthooks.py ported to CentOS. Some code from old CentOS hooks is still used.

Implemented:

- Legacy boot
- UEFI boot
- Simple and MD RAID layouts
- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces

Not implemented:

- Bond, bridge, route configurations
- IPv6
- Disk encryption
- bcache
- LVM
- Multipath

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'curtin/centos6/curtin-hooks.py'
2--- curtin/centos6/curtin-hooks.py 2016-05-11 18:47:46 +0000
3+++ curtin/centos6/curtin-hooks.py 2017-01-09 15:19:25 +0000
4@@ -1,20 +1,19 @@
5 #!/usr/bin/env python
6-
7-from __future__ import (
8- absolute_import,
9- print_function,
10- unicode_literals,
11- )
12+# coding: utf-8
13
14 import codecs
15+import ipaddress
16 import os
17 import re
18+import shutil
19 import sys
20
21-from curtin import (
22- block,
23- util,
24- )
25+from curtin import block, config, util
26+from curtin.block import mdadm
27+from curtin.commands import curthooks
28+from curtin.log import LOG
29+from curtin.reporter import events
30+
31
32 """
33 CentOS 6
34@@ -22,15 +21,21 @@
35 Currently Support:
36
37 - Legacy boot
38-- DHCP of BOOTIF
39+- UEFI boot
40+- Simple and MD RAID layouts
41+- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces
42
43 Not Supported:
44
45-- UEFI boot (*Bad support, most likely wont support)
46-- Multiple network configration
47+- Bond, bridge, route configurations
48 - IPv6
49+- Disk encryption
50+- bcache
51+- LVM
52+- Multipath
53 """
54
55+
56 FSTAB_PREPEND = """\
57 #
58 # /etc/fstab
59@@ -54,36 +59,95 @@
60 # Created by MAAS fast-path installer.
61 #
62 default 0
63-timeout 0
64-title MAAS
65+timeout 1
66+title CentOS
67 root {grub_root}
68- kernel /boot/{vmlinuz} root=UUID={root_uuid} {extra_opts}
69- initrd /boot/{initrd}
70+ kernel /{vmlinuz} root=UUID={root_uuid} {extra_opts}
71+ initrd /{initrd}
72 """
73
74
75-def get_block_devices(target):
76- """Returns list of block devices for the given target."""
77- devs = block.get_devices_for_mp(target)
78- blockdevs = set()
79- for maybepart in devs:
80- (blockdev, part) = block.get_blockdev_for_partition(maybepart)
81- blockdevs.add(blockdev)
82- return list(blockdevs)
83-
84-
85-def get_root_info(target):
86- """Returns the root partitions information."""
87- rootpath = block.get_devices_for_mp(target)[0]
88- rootdev = os.path.basename(rootpath)
89- blocks = block._lsblock()
90- return blocks[rootdev]
91-
92-
93-def read_file(path):
94- """Returns content of a file."""
95- with codecs.open(path, encoding='utf-8') as stream:
96- return stream.read()
97+def get_installed_packages(target=None):
98+ (out, _) = util.subp(['rpm', '-qa', '--qf', '"%{NAME}\n"'],
99+ target=target, capture=True)
100+ return set(out.splitlines())
101+
102+
103+def install_packages(pkglist, target):
104+ if isinstance(pkglist, str):
105+ pkglist = [pkglist]
106+ # Does not work – hosts are not resolved
107+ # return util.subp(['yum', '-q', '-y', 'install'] + pkglist, target=target, capture=True)
108+ with util.RunInChroot(target) as in_chroot:
109+ in_chroot(['yum', '-q', '-y', 'install'] + pkglist)
110+
111+
112+def install_missing_packages(cfg, target):
113+ ''' describe which operation types will require specific packages
114+
115+ 'custom_config_key': {
116+ 'pkg1': ['op_name_1', 'op_name_2', ...]
117+ }
118+ '''
119+ custom_configs = {
120+ 'storage': {
121+ 'lvm2': ['lvm_volgroup', 'lvm_partition'],
122+ 'mdadm': ['raid'],
123+ # TODO. At the moment there are no official packages for bcache on CentOS
124+ #'bcache-tools': ['bcache']
125+ },
126+ 'network': {
127+ 'bridge-utils': ['bridge']
128+ },
129+ }
130+
131+ format_configs = {
132+ 'xfsprogs': ['xfs'],
133+ 'e2fsprogs': ['ext2', 'ext3', 'ext4'],
134+ 'btrfs-tools': ['btrfs'],
135+ }
136+
137+ needed_packages = []
138+ installed_packages = get_installed_packages(target)
139+ for cust_cfg, pkg_reqs in custom_configs.items():
140+ if cust_cfg not in cfg:
141+ continue
142+
143+ all_types = set(
144+ operation['type']
145+ for operation in cfg[cust_cfg]['config']
146+ )
147+ for pkg, types in pkg_reqs.items():
148+ if set(types).intersection(all_types) and \
149+ pkg not in installed_packages:
150+ needed_packages.append(pkg)
151+
152+ format_types = set(
153+ [operation['fstype']
154+ for operation in cfg[cust_cfg]['config']
155+ if operation['type'] == 'format'])
156+ for pkg, fstypes in format_configs.items():
157+ if set(fstypes).intersection(format_types) and \
158+ pkg not in installed_packages:
159+ needed_packages.append(pkg)
160+
161+ if needed_packages:
162+ state = util.load_command_environment()
163+ with events.ReportEventStack(
164+ name=state.get('report_stack_prefix'),
165+ reporting_enabled=True, level="INFO",
166+ description="Installing packages on target system: " +
167+ str(needed_packages)):
168+ install_packages(needed_packages, target=target)
169+
170+
171+def copy_mdadm_conf(mdadm_conf, target):
172+ if not mdadm_conf:
173+ LOG.warn("mdadm config must be specified, not copying")
174+ return
175+
176+ LOG.info("copying mdadm.conf into target")
177+ shutil.copy(mdadm_conf, os.path.sep.join([target, 'etc/mdadm.conf']))
178
179
180 def write_fstab(target, curtin_fstab):
181@@ -97,11 +161,289 @@
182 stream.write(FSTAB_APPEND)
183
184
185-def extract_kernel_params(data):
186- """Extracts the kernel parametes from the provided
187- grub config data."""
188- match = re.search('^\s+kernel (.+?)$', data, re.MULTILINE)
189- return match.group(0)
190+def update_initramfs(target=None):
191+ path = os.path.join(
192+ target, 'etc', 'dracut.conf.d', 'local.conf')
193+ with open(path, 'w') as stream:
194+ stream.write('mdadmconf="yes"' + '\n'
195+ 'lvmconf="yes"' + '\n')
196+
197+ initrd = get_boot_file(target, 'initramfs')
198+ version = initrd.replace('initramfs-', '').replace('.img', '')
199+ initrd_path = os.path.join(os.sep, 'boot', initrd)
200+ with util.RunInChroot(target) as in_chroot:
201+ in_chroot(['dracut', '-f', initrd_path, version])
202+
203+
204+def system_upgrade(cfg, target):
205+ """run yum upgrade in target.
206+
207+ config:
208+ system_upgrade:
209+ enabled: False
210+
211+ """
212+ mycfg = {'system_upgrade': {'enabled': False}}
213+ config.merge_config(mycfg, cfg)
214+ mycfg = mycfg.get('system_upgrade')
215+ if not isinstance(mycfg, dict):
216+ LOG.debug("system_upgrade disabled by config. entry not a dict.")
217+ return
218+
219+ if not config.value_as_boolean(mycfg.get('enabled', True)):
220+ LOG.debug("system_upgrade disabled by config.")
221+ return
222+
223+ util.subp(['yum', '-q', '-y', 'upgrade'], target=target, capture=True)
224+
225+
226+def read_file(path):
227+ """Returns content of a file."""
228+ with codecs.open(path, encoding='utf-8') as stream:
229+ return stream.read()
230+
231+
232+def get_boot_mac():
233+ """Return the mac address of the booting interface."""
234+ cmdline = read_file('/proc/cmdline')
235+ cmdline = cmdline.split()
236+ try:
237+ bootif = [
238+ option
239+ for option in cmdline
240+ if option.startswith('BOOTIF')
241+ ][0]
242+ except IndexError:
243+ return None
244+ _, mac = bootif.split('=')
245+ mac = mac.split('-')[1:]
246+ return ':'.join(mac)
247+
248+
249+def get_interface_names():
250+ """Return a dictionary mapping mac addresses to interface names."""
251+ sys_path = "/sys/class/net"
252+ ifaces = {}
253+ for iname in os.listdir(sys_path):
254+ mac = read_file(os.path.join(sys_path, iname, "address"))
255+ mac = mac.strip().lower()
256+ ifaces[mac] = iname
257+ return ifaces
258+
259+
260+def find_legacy_iface_name(consistent_iface_name):
261+ """Returns legacy network interface name by running biosdevname (needs)
262+ to be installed beforehand)"""
263+ out, err = util.subp(['biosdevname', '--policy', 'all_ethN',
264+ '--interface', consistent_iface_name],
265+ capture=True)
266+ return out.strip()
267+
268+
269+def rename_ifaces(network_config, target):
270+ """CentOS 6 is not using a consistent network device naming and even we
271+ would enable it, interface names would differ from names they would get
272+ on new systems with systemd. Will change every occurence of consistent
273+ network device name to legacy name ethN and create an appropriate
274+ /etc/udev/rules.d/70-persistent-net.rules"""
275+
276+ # Will need biosdevname to find out legacy names
277+ util.install_packages(['biosdevname'])
278+
279+ # Build the cache as {'consistent_name': 'legacy_name'}
280+ cache = {}
281+ for cfg in network_config['config']:
282+ if cfg['type'] == 'physical':
283+ consistent_name = cfg['name']
284+ cache[consistent_name] = find_legacy_iface_name(consistent_name)
285+
286+ rules = ''
287+ rule_template = ('SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", '
288+ 'ATTR{{address}}=="{mac_address}", KERNEL=="eth*", '
289+ 'NAME="{iface_name}"\n')
290+
291+ # Rename devices and generate an appropriate content for
292+ # /etc/udev/rules.d/70-persistent-net.rules
293+ for cfg in network_config['config']:
294+ if cfg['type'] == 'physical':
295+ consistent_phys_name = cfg['name']
296+ legacy_phys_name = cache[consistent_phys_name]
297+ cfg['name'] = legacy_phys_name
298+ rules += rule_template.format(iface_name=legacy_phys_name,
299+ mac_address=cfg['mac_address'])
300+ if cfg['type'] == 'vlan':
301+ consistent_phys_name = cfg['vlan_link']
302+ legacy_phys_name = cache[consistent_phys_name]
303+ cfg['name'] = '{}.{}'.format(legacy_phys_name, cfg['vlan_id'])
304+ cfg['vlan_link'] = legacy_phys_name
305+
306+ path = os.path.join(target, 'etc', 'udev', 'rules.d',
307+ '70-persistent-net.rules')
308+ util.write_file(path, rules)
309+
310+
311+
312+def get_subnet_config(subnets):
313+ content = ''
314+ for idx, subnet in enumerate(subnets):
315+ if subnet['type'] in ('static', 'static4'):
316+ if idx == 0:
317+ idx = ''
318+ iface = ipaddress.ip_interface(subnet['address'])
319+ content += 'IPADDR{}={}\n'.format(idx, str(iface.ip))
320+ content += 'NETMASK{}={}\n'.format(
321+ idx, subnet.get('netmask', str(iface.netmask)))
322+ return content
323+
324+
325+def get_common_network_config(cfg):
326+ # You can not have more than one GATEWAY, BOOTPROTO, DNS1, DNS2 and DOMAIN
327+ # settings per interface in RHEL and clones. Will run through subnets
328+ # section ensuring that only one of these makes to interface configuration.
329+ content = ''
330+ bootproto = 'dhcp'
331+ defaultgw = None
332+ resolvers = []
333+ search_domains = []
334+ for subnet in cfg['subnets']:
335+ if subnet['type'] in ('dhcp', 'dhcp4'):
336+ # OK, so we have a DHCP subnet. Let's assume all other subnets
337+ # are also DHCP and just stop looking
338+ bootproto = 'dhcp'
339+ break
340+ elif subnet['type'] in ('static', 'static4'):
341+ # If this is a static subnet, then there might be a default
342+ # gateway and resolvers set and if there aren't we'll keep
343+ # looking for them in the next subnet configuration
344+ bootproto = 'none'
345+ if not defaultgw:
346+ defaultgw = subnet.get('gateway')
347+ if not resolvers:
348+ resolvers = subnet.get('dns_nameservers')
349+ if not search_domains:
350+ search_domains = subnet.get('dns_search')
351+ else:
352+ # Let's log the lack of support and continue - there still
353+ # migth be a supported subnet configuration
354+ LOG.warn('Configuration of subnet type {} '
355+ 'not supported'.format(subnet['type']))
356+
357+ content += 'BOOTPROTO={}\n'.format(bootproto)
358+ if bootproto != 'dhcp':
359+ if defaultgw:
360+ content += 'GATEWAY={}\n'.format(defaultgw)
361+ for idx, resolver in enumerate(resolvers):
362+ content += 'DNS{}={}\n'.format(idx+1, resolver)
363+ if search_domains:
364+ content += 'DOMAIN="{}"\n'.format(' '.join(search_domains))
365+
366+ return content
367+
368+
369+def write_resolv_conf(cfg, target):
370+ content = '# Generated by MAAS fast-path installer\n'
371+
372+ for resolver in cfg.get('address', []):
373+ content += 'nameserver {}\n'.format(resolver)
374+
375+ search_domains = cfg.get('search', [])
376+ if search_domains:
377+ content += 'search {}\n'.format(' '.join(search_domains))
378+
379+ path = os.path.join(target, 'etc', 'resolv.conf')
380+ util.write_file(path, content)
381+
382+
383+def write_physical_iface_config(cfg, target):
384+ content = (
385+ '# Generated by MAAS fast-path installer\n'
386+ 'DEVICE={device}\n'
387+ 'HWADDR={hwaddr}\n'
388+ 'MTU={mtu}\n'
389+ 'ONBOOT=yes\n'
390+ ).format(
391+ device=cfg['name'],
392+ hwaddr=cfg['mac_address'],
393+ mtu=cfg.get('mtu', 1500)
394+ )
395+
396+ content += get_common_network_config(cfg) + \
397+ get_subnet_config(cfg['subnets'])
398+
399+ path = os.path.join(target, 'etc', 'sysconfig',
400+ 'network-scripts', 'ifcfg-%s' % cfg['name'])
401+ util.write_file(path, content)
402+
403+
404+def write_vlan_iface_config(cfg, target):
405+ content = (
406+ '# Generated by MAAS fast-path installer\n'
407+ 'DEVICE={device}\n'
408+ 'MTU={mtu}\n'
409+ 'ONBOOT=yes\n'
410+ 'VLAN=yes\n'
411+ ).format(
412+ device=cfg['name'],
413+ mtu=cfg.get('mtu', 1500)
414+ )
415+
416+ content += get_common_network_config(cfg) + \
417+ get_subnet_config(cfg['subnets'])
418+
419+ path = os.path.join(target, 'etc', 'sysconfig',
420+ 'network-scripts', 'ifcfg-%s' % cfg['name'])
421+ util.write_file(path, content)
422+
423+
424+def apply_networking(config, target):
425+ network_config = config.get('network')
426+ if not network_config:
427+ LOG.warn("Unable to retrieve network configuration, will fallback "
428+ "to configuring DHCP on boot device")
429+ bootmac = get_boot_mac()
430+ inames = get_interface_names()
431+ iname = inames[bootmac.lower()]
432+ network_config = {
433+ 'config': [
434+ {'mac_address': bootmac,
435+ 'name': iname,
436+ 'type': 'physical',
437+ 'subnets': [{'type': 'dhcp'}]}],
438+ 'version': 1}
439+
440+ # Bring back ethN device naming on CentOS 6
441+ rename_ifaces(network_config, target)
442+
443+ for cfg in network_config['config']:
444+ if cfg['type'] == 'nameserver':
445+ write_resolv_conf(cfg, target)
446+ if cfg['type'] == 'physical':
447+ write_physical_iface_config(cfg, target)
448+ if cfg['type'] == 'vlan':
449+ write_vlan_iface_config(cfg, target)
450+
451+
452+def get_grub_root(target, dedicated_boot):
453+ """Extracts the grub root (hdX,X) from the grub command.
454+
455+ This is used so the correct root device is used to install
456+ stage1/stage2 boot loader.
457+
458+ Note: grub-install normally does all of this for you, but
459+ since the grub is older, it has an issue with the ISCSI
460+ target as /dev/sda and cannot enumarate it with the BIOS.
461+ """
462+ stage1 = '/boot/grub/stage1'
463+ if dedicated_boot:
464+ stage1 = '/grub/stage1'
465+
466+ with util.RunInChroot(target) as in_chroot:
467+ data = ('find {}\n'
468+ 'quit\n').format(stage1).encode('utf-8')
469+ out, err = in_chroot(['grub', '--batch'],
470+ data=data, capture=True)
471+ regex = re.search(r'^\s+(\(.+?\))$', out, re.MULTILINE)
472+ return regex.groups()[0]
473
474
475 def strip_kernel_params(params, strip_params=[]):
476@@ -118,6 +460,33 @@
477 return new_params
478
479
480+def get_extra_kernel_parameters():
481+ """Extracts the extra kernel commands from /proc/cmdline
482+ that should be placed onto the host.
483+
484+ Any command following the '---' entry should be placed
485+ onto the host.
486+ """
487+ cmdline = read_file('/proc/cmdline')
488+ cmdline = cmdline.split()
489+ if '---' not in cmdline:
490+ return []
491+ idx = cmdline.index('---') + 1
492+ if idx >= len(cmdline) + 1:
493+ return []
494+ return strip_kernel_params(
495+ cmdline[idx:],
496+ strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF='])
497+
498+
499+def get_root_info(target):
500+ """Returns the root partitions information."""
501+ rootpath = block.get_devices_for_mp(target)[0]
502+ rootdev = os.path.basename(rootpath)
503+ blocks = block._lsblock()
504+ return blocks[rootdev]
505+
506+
507 def get_boot_file(target, filename):
508 """Return the full filename of file in /boot on target."""
509 boot_dir = os.path.join(target, 'boot')
510@@ -125,81 +494,163 @@
511 fname
512 for fname in os.listdir(boot_dir)
513 if fname.startswith(filename)
514- ]
515+ ]
516 if not files:
517 return None
518 return files[0]
519
520
521-def write_grub_conf(target, grub_root, extra=[]):
522+def write_grub_conf(target, dedicated_boot, extra=[]):
523 """Writes a new /boot/grub/grub.conf with the correct
524 boot arguments."""
525+ grub_path = os.path.join(target, 'boot', 'grub', 'grub.conf')
526+ util.write_file(grub_path, '# UEFI system, see /etc/grub.conf\n')
527+
528+ if util.is_uefi_bootable():
529+ grub_path = os.path.join(target, 'boot', 'efi',
530+ 'EFI', 'redhat', 'grub.conf')
531+
532+ grub_root = get_grub_root(target, dedicated_boot)
533 root_info = get_root_info(target)
534- grub_path = os.path.join(target, 'boot', 'grub', 'grub.conf')
535 extra_opts = ' '.join(extra)
536 vmlinuz = get_boot_file(target, 'vmlinuz')
537 initrd = get_boot_file(target, 'initramfs')
538- with open(grub_path, 'w') as stream:
539- stream.write(
540- GRUB_CONF.format(
541- grub_root=grub_root,
542- vmlinuz=vmlinuz,
543- initrd=initrd,
544- root_uuid=root_info['UUID'],
545- extra_opts=extra_opts) + '\n')
546-
547-
548-def get_extra_kernel_parameters():
549- """Extracts the extra kernel commands from /proc/cmdline
550- that should be placed onto the host.
551-
552- Any command following the '--' entry should be placed
553- onto the host.
554- """
555- cmdline = read_file('/proc/cmdline')
556- cmdline = cmdline.split()
557- if '--' not in cmdline:
558- return []
559- idx = cmdline.index('--') + 1
560- if idx >= len(cmdline) + 1:
561- return []
562- return strip_kernel_params(
563- cmdline[idx:],
564- strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF='])
565-
566-
567-def get_grub_root(target):
568- """Extracts the grub root (hdX,X) from the grub command.
569-
570- This is used so the correct root device is used to install
571- stage1/stage2 boot loader.
572-
573- Note: grub-install normally does all of this for you, but
574- since the grub is older, it has an issue with the ISCSI
575- target as /dev/sda and cannot enumarate it with the BIOS.
576- """
577- with util.RunInChroot(target) as in_chroot:
578- data = '\n'.join([
579- 'find /boot/grub/stage1',
580- 'quit',
581- ]).encode('utf-8')
582- out, err = in_chroot(['grub', '--batch'],
583- data=data, capture=True)
584- regex = re.search('^\s+(\(.+?\))$', out, re.MULTILINE)
585- return regex.groups()[0]
586-
587-
588-def grub_install(target, root):
589- """Installs grub onto the root."""
590- root_dev = root.split(',')[0] + ')'
591- with util.RunInChroot(target) as in_chroot:
592- data = '\n'.join([
593- 'root %s' % root,
594- 'setup %s' % root_dev,
595- 'quit',
596- ]).encode('utf-8')
597- in_chroot(['grub', '--batch'],
598- data=data)
599+
600+ if not dedicated_boot:
601+ vmlinuz = 'boot/' + vmlinuz
602+ initrd = 'boot/' + initrd
603+
604+ util.write_file(grub_path,
605+ GRUB_CONF.format(
606+ grub_root=grub_root,
607+ vmlinuz=vmlinuz,
608+ initrd=initrd,
609+ root_uuid=root_info['UUID'],
610+ extra_opts=extra_opts) + '\n')
611+
612+ # FIXME. replace() is unreliable here if target ends with /
613+ # Update /etc/grub.conf link as it is used by kernel install scripts
614+ util.subp(['ln', '-sf', grub_path.replace(target, '..'),
615+ '/etc/grub.conf'], target=target)
616+
617+
618+def get_uefi_partition():
619+ """Return the UEFI partition."""
620+ for _, value in block._lsblock().items():
621+ if value['LABEL'] == 'uefi-boot':
622+ return value
623+ return None
624+
625+
626+def install_bootloader(target, device=None):
627+ """Installs bootloader to the device."""
628+ cmd = []
629+ if util.is_uefi_bootable():
630+ uefi_dev = get_uefi_partition()['device_path']
631+ disk, part = block.get_blockdev_for_partition(uefi_dev)
632+ cmd = ['efibootmgr', '-v', '-c', '-w', '-L', 'centos',
633+ '-d', disk, '-p', part, '-l', r'\EFI\redhat\grub.efi']
634+ else:
635+ # Legacy grub-install uses df output, which itself uses /etc/mtab
636+ with util.RunInChroot(target) as in_chroot:
637+ in_chroot(['cp', '-f', '/proc/mounts', '/etc/mtab'])
638+ cmd = ['grub-install', '--recheck', device]
639+
640+ with util.RunInChroot(target) as in_chroot:
641+ in_chroot(cmd)
642+
643+
644+def setup_grub(cfg, target):
645+ # target is the path to the mounted filesystem
646+
647+ # FIXME: these methods need moving to curtin.block
648+ # and using them from there rather than commands.block_meta
649+ from curtin.commands.block_meta import (extract_storage_ordered_dict,
650+ get_path_to_storage_volume)
651+
652+ grubcfg = cfg.get('grub', {})
653+
654+ # copy legacy top level name
655+ if 'grub_install_devices' in cfg and 'install_devices' not in grubcfg:
656+ grubcfg['install_devices'] = cfg['grub_install_devices']
657+
658+ LOG.debug("setup grub on target %s", target)
659+ # if there is storage config, look for devices tagged with 'grub_device'
660+ storage_cfg_odict = None
661+ try:
662+ storage_cfg_odict = extract_storage_ordered_dict(cfg)
663+ except ValueError as e:
664+ pass
665+
666+ if storage_cfg_odict:
667+ storage_grub_devices = []
668+ for item_id, item in storage_cfg_odict.items():
669+ if not item.get('grub_device'):
670+ continue
671+ LOG.debug("checking: %s", item)
672+ storage_grub_devices.append(
673+ get_path_to_storage_volume(item_id, storage_cfg_odict))
674+ if len(storage_grub_devices) > 0:
675+ grubcfg['install_devices'] = storage_grub_devices
676+
677+ LOG.debug("install_devices: %s", grubcfg.get('install_devices'))
678+ if 'install_devices' in grubcfg:
679+ instdevs = grubcfg.get('install_devices')
680+ if isinstance(instdevs, str):
681+ instdevs = [instdevs]
682+ if instdevs is None:
683+ LOG.debug("grub installation disabled by config")
684+ else:
685+ # If there were no install_devices found then we try to do the right
686+ # thing. That right thing is basically installing on all block
687+ # devices that are mounted. On powerpc, though it means finding PrEP
688+ # partitions.
689+ devs = block.get_devices_for_mp(target)
690+ blockdevs = set()
691+ for maybepart in devs:
692+ try:
693+ (blockdev, part) = block.get_blockdev_for_partition(maybepart)
694+ blockdevs.add(blockdev)
695+ except ValueError as e:
696+ # if there is no syspath for this device such as a lvm
697+ # or raid device, then a ValueError is raised here.
698+ LOG.debug("failed to find block device for %s", maybepart)
699+ instdevs = list(blockdevs)
700+
701+ # UEFI requires additional packages
702+ #if util.is_uefi_bootable():
703+ # pkgs = ['efibootmgr']
704+ # install_packages(pkgs, target=target)
705+
706+ # CentOS will not assemble MD devices on boot without rd.md.uuid=MD_UUID
707+ # kernel parameters
708+ mdmap = {}
709+ rdmduuids = []
710+ try:
711+ mdmap = mdadm.md_read_run_mdadm_map()
712+ for md in mdmap:
713+ rdmduuids.append("rd.md.uuid=%s" % mdadm.md_get_uuid(mdmap[md][2]))
714+ except ValueError as e:
715+ pass
716+
717+ dedicated_boot = False
718+ if block.get_devices_for_mp(os.path.join(target, 'boot')):
719+ dedicated_boot = True
720+ write_grub_conf(target, dedicated_boot, extra=get_extra_kernel_parameters() + rdmduuids)
721+
722+ if instdevs:
723+ instdevs = [block.get_dev_name_entry(i)[1] for i in instdevs]
724+ else:
725+ instdevs = ["none"]
726+
727+ LOG.debug("installing grub to %s", instdevs)
728+
729+ if util.is_uefi_bootable():
730+ if grubcfg.get('update_nvram', False):
731+ install_bootloader(target)
732+ else:
733+ for dev in instdevs:
734+ install_bootloader(target, dev)
735
736
737 def set_autorelabel(target):
738@@ -214,128 +665,75 @@
739 open(path, 'a').close()
740
741
742-def get_boot_mac():
743- """Return the mac address of the booting interface."""
744- cmdline = read_file('/proc/cmdline')
745- cmdline = cmdline.split()
746- try:
747- bootif = [
748- option
749- for option in cmdline
750- if option.startswith('BOOTIF')
751- ][0]
752- except IndexError:
753- return None
754- _, mac = bootif.split('=')
755- mac = mac.split('-')[1:]
756- return ':'.join(mac)
757-
758-
759-def get_interface_names():
760- """Return a dictionary mapping mac addresses to interface names."""
761- sys_path = "/sys/class/net"
762- ifaces = {}
763- for iname in os.listdir(sys_path):
764- mac = read_file(os.path.join(sys_path, iname, "address"))
765- mac = mac.strip().lower()
766- ifaces[mac] = iname
767- return ifaces
768-
769-
770-def get_ipv4_config(iface, data):
771- """Returns the contents of the interface file for ipv4."""
772- config = [
773- 'TYPE="Ethernet"',
774- 'NM_CONTROLLED="no"',
775- 'USERCTL="yes"',
776- ]
777- if 'hwaddress' in data:
778- config.append('HWADDR="%s"' % data['hwaddress'])
779- else:
780- # Last ditch effort, use the device name, it probably won't match
781- # though!
782- config.append('DEVICE="%s"' % iface)
783- if data['auto']:
784- config.append('ONBOOT="yes"')
785- else:
786- config.append('ONBOOT="no"')
787-
788- method = data['method']
789- if method == 'dhcp':
790- config.append('BOOTPROTO="dhcp"')
791- config.append('PEERDNS="yes"')
792- config.append('PERSISTENT_DHCLIENT="1"')
793- if 'hostname' in data:
794- config.append('DHCP_HOSTNAME="%s"' % data['hostname'])
795- elif method == 'static':
796- config.append('BOOTPROTO="none"')
797- config.append('IPADDR="%s"' % data['address'])
798- config.append('NETMASK="%s"' % data['netmask'])
799- if 'broadcast' in data:
800- config.append('BROADCAST="%s"' % data['broadcast'])
801- if 'gateway' in data:
802- config.append('GATEWAY="%s"' % data['gateway'])
803- elif method == 'manual':
804- config.append('BOOTPROTO="none"')
805- return '\n'.join(config)
806-
807-
808-def write_interface_config(target, iface, data):
809- """Writes config for interface."""
810- family = data['family']
811- if family != "inet":
812- # Only supporting ipv4 currently
813- print(
814- "WARN: unsupported family %s, "
815- "failed to configure interface: %s" (family, iface))
816- return
817- config = get_ipv4_config(iface, data)
818- path = os.path.join(
819- target, 'etc', 'sysconfig', 'network-scripts', 'ifcfg-%s' % iface)
820- with open(path, 'w') as stream:
821- stream.write(config + '\n')
822-
823-
824-def write_network_config(target, mac):
825- """Write network configuration for the given MAC address."""
826- inames = get_interface_names()
827- iname = inames[mac.lower()]
828- write_interface_config(
829- target, iname, {
830- 'family': 'inet',
831- 'hwaddress': mac.upper(),
832- 'auto': True,
833- 'method': 'dhcp'
834- })
835-
836-
837 def main():
838 state = util.load_command_environment()
839+
840 target = state['target']
841 if target is None:
842- print("Target was not provided in the environment.")
843- sys.exit(1)
844- fstab = state['fstab']
845- if fstab is None:
846- print("/etc/fstab output was not provided in the environment.")
847- sys.exit(1)
848- bootmac = get_boot_mac()
849- if bootmac is None:
850- print("Unable to determine boot interface.")
851- sys.exit(1)
852- devices = get_block_devices(target)
853- if not devices:
854- print("Unable to find block device for: %s" % target)
855- sys.exit(1)
856-
857- write_fstab(target, fstab)
858-
859- grub_root = get_grub_root(target)
860- write_grub_conf(target, grub_root, extra=get_extra_kernel_parameters())
861- grub_install(target, grub_root)
862+ sys.stderr.write("Target was not provided in the environment.")
863+ sys.exit(1)
864+
865+ cfg = config.load_command_config(None, state)
866+ stack_prefix = state.get('report_stack_prefix', '')
867+
868+ with events.ReportEventStack(
869+ name=stack_prefix, reporting_enabled=True, level="INFO",
870+ description="writing config files"):
871+ curthooks.write_files(cfg, target)
872+
873+ # Default CentOS image does not contain some packages that may be necessary
874+ install_missing_packages(cfg, target)
875+
876+ # If a mdadm.conf file was created by block_meta then it needs to be copied
877+ # onto the target system
878+ mdadm_location = os.path.join(os.path.split(state['fstab'])[0],
879+ "mdadm.conf")
880+ if os.path.exists(mdadm_location):
881+ copy_mdadm_conf(mdadm_location, target)
882+
883+ with events.ReportEventStack(
884+ name=stack_prefix, reporting_enabled=True, level="INFO",
885+ description="setting up swap"):
886+ curthooks.add_swap(cfg, target, state.get('fstab'))
887+
888+ with events.ReportEventStack(
889+ name=stack_prefix, reporting_enabled=True, level="INFO",
890+ description="writing etc/fstab"):
891+ write_fstab(target, state.get('fstab'))
892+
893+ # TODO. There's a multipath implementation on Ubuntu in curtin/curthooks.py
894+ # that should be reimplemented here. Skipping for now.
895+
896+ with events.ReportEventStack(
897+ name=stack_prefix, reporting_enabled=True, level="INFO",
898+ description="updating packages on target system"):
899+ system_upgrade(cfg, target)
900+
901+ # TODO. If a crypttab file was created by block_meta than it needs to be
902+ # copied onto the target system, and update_initramfs() (well, alternative
903+ # for CentOS actually) needs to be run, so that the cryptsetup hooks are
904+ # properly configured on the installed system and it will be able to open
905+ # encrypted volumes at boot. Skipping for now.
906+
907+ with events.ReportEventStack(
908+ name=stack_prefix, reporting_enabled=True, level="INFO",
909+ description="apply networking"):
910+ apply_networking(cfg, target)
911+
912+ # If udev dname rules were created, copy them to target
913+ udev_rules_d = os.path.join(state['scratch'], "rules.d")
914+ if os.path.isdir(udev_rules_d):
915+ curthooks.copy_dname_rules(udev_rules_d, target)
916+
917+ with events.ReportEventStack(
918+ name=stack_prefix, reporting_enabled=True, level="INFO",
919+ description="updating packages on target system"):
920+ setup_grub(cfg, target)
921
922 set_autorelabel(target)
923- write_network_config(target, bootmac)
924+ update_initramfs(target)
925+
926+ sys.exit(0)
927
928
929 if __name__ == "__main__":
930
931=== modified file 'curtin/centos7/curtin-hooks.py'
932--- curtin/centos7/curtin-hooks.py 2016-05-11 18:47:46 +0000
933+++ curtin/centos7/curtin-hooks.py 2017-01-09 15:19:25 +0000
934@@ -1,20 +1,17 @@
935 #!/usr/bin/env python
936-
937-from __future__ import (
938- absolute_import,
939- print_function,
940- unicode_literals,
941- )
942+# coding: utf-8
943
944 import codecs
945+import ipaddress
946 import os
947+import shutil
948 import sys
949-import shutil
950
951-from curtin import (
952- block,
953- util,
954- )
955+from curtin import block, config, util
956+from curtin.block import mdadm
957+from curtin.commands import curthooks
958+from curtin.log import LOG
959+from curtin.reporter import events
960
961
962 """
963@@ -24,60 +21,142 @@
964
965 - Legacy boot
966 - UEFI boot
967-- DHCP of BOOTIF
968+- Simple and MD RAID layouts
969+- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces
970
971 Not Supported:
972
973-- Multiple network configration
974+- Bond, bridge, route configurations
975 - IPv6
976-"""
977-
978-FSTAB_PREPEND = """\
979-#
980-# /etc/fstab
981-# Created by MAAS fast-path installer.
982-#
983-# Accessible filesystems, by reference, are maintained under '/dev/disk'
984-# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
985-#
986-"""
987-
988-FSTAB_UEFI = """\
989-LABEL=uefi-boot /boot/efi vfat defaults 0 0
990-"""
991+- Disk encryption
992+- bcache
993+- LVM
994+- Multipath
995+"""
996+
997
998 GRUB_PREPEND = """\
999 # Set by MAAS fast-path installer.
1000-GRUB_TIMEOUT=0
1001+GRUB_TIMEOUT=1
1002 GRUB_TERMINAL_OUTPUT=console
1003 GRUB_DISABLE_OS_PROBER=true
1004 """
1005
1006
1007-def get_block_devices(target):
1008- """Returns list of block devices for the given target."""
1009- devs = block.get_devices_for_mp(target)
1010- blockdevs = set()
1011- for maybepart in devs:
1012- (blockdev, part) = block.get_blockdev_for_partition(maybepart)
1013- blockdevs.add(blockdev)
1014- return list(blockdevs)
1015-
1016-
1017-def get_root_info(target):
1018- """Returns the root partitions information."""
1019- rootpath = block.get_devices_for_mp(target)[0]
1020- rootdev = os.path.basename(rootpath)
1021- blocks = block._lsblock()
1022- return blocks[rootdev]
1023-
1024-
1025-def get_uefi_partition():
1026- """Return the UEFI partition."""
1027- for _, value in block._lsblock().items():
1028- if value['LABEL'] == 'uefi-boot':
1029- return value
1030- return None
1031+def get_installed_packages(target=None):
1032+ (out, _) = util.subp(['rpm', '-qa', '--qf', '"%{NAME}\n"'],
1033+ target=target, capture=True)
1034+ return set(out.splitlines())
1035+
1036+
1037+def install_packages(pkglist, target):
1038+ if isinstance(pkglist, str):
1039+ pkglist = [pkglist]
1040+ # Does not work – hosts are not resolved
1041+ # return util.subp(['yum', '-q', '-y', 'install'] + pkglist, target=target, capture=True)
1042+ with util.RunInChroot(target) as in_chroot:
1043+ in_chroot(['yum', '-q', '-y', 'install'] + pkglist)
1044+
1045+
1046+def install_missing_packages(cfg, target):
1047+ ''' describe which operation types will require specific packages
1048+
1049+ 'custom_config_key': {
1050+ 'pkg1': ['op_name_1', 'op_name_2', ...]
1051+ }
1052+ '''
1053+ custom_configs = {
1054+ 'storage': {
1055+ 'lvm2': ['lvm_volgroup', 'lvm_partition'],
1056+ 'mdadm': ['raid'],
1057+ # XXX. At the moment there are no official packages for bcache on CentOS
1058+ #'bcache-tools': ['bcache']
1059+ },
1060+ 'network': {
1061+ 'bridge-utils': ['bridge']
1062+ },
1063+ }
1064+
1065+ format_configs = {
1066+ 'xfsprogs': ['xfs'],
1067+ 'e2fsprogs': ['ext2', 'ext3', 'ext4'],
1068+ 'btrfs-tools': ['btrfs'],
1069+ }
1070+
1071+ needed_packages = []
1072+ installed_packages = get_installed_packages(target)
1073+ for cust_cfg, pkg_reqs in custom_configs.items():
1074+ if cust_cfg not in cfg:
1075+ continue
1076+
1077+ all_types = set(
1078+ operation['type']
1079+ for operation in cfg[cust_cfg]['config']
1080+ )
1081+ for pkg, types in pkg_reqs.items():
1082+ if set(types).intersection(all_types) and \
1083+ pkg not in installed_packages:
1084+ needed_packages.append(pkg)
1085+
1086+ format_types = set(
1087+ [operation['fstype']
1088+ for operation in cfg[cust_cfg]['config']
1089+ if operation['type'] == 'format'])
1090+ for pkg, fstypes in format_configs.items():
1091+ if set(fstypes).intersection(format_types) and \
1092+ pkg not in installed_packages:
1093+ needed_packages.append(pkg)
1094+
1095+ if needed_packages:
1096+ state = util.load_command_environment()
1097+ with events.ReportEventStack(
1098+ name=state.get('report_stack_prefix'),
1099+ reporting_enabled=True, level="INFO",
1100+ description="Installing packages on target system: " +
1101+ str(needed_packages)):
1102+ install_packages(needed_packages, target=target)
1103+
1104+
1105+def copy_mdadm_conf(mdadm_conf, target):
1106+ if not mdadm_conf:
1107+ LOG.warn("mdadm config must be specified, not copying")
1108+ return
1109+
1110+ LOG.info("copying mdadm.conf into target")
1111+ shutil.copy(mdadm_conf, os.path.sep.join([target, 'etc/mdadm.conf']))
1112+
1113+
1114+def update_initramfs(target=None):
1115+ path = os.path.join(
1116+ target, 'etc', 'dracut.conf.d', 'local.conf')
1117+ with open(path, 'w') as stream:
1118+ stream.write('mdadmconf="yes"' + '\n'
1119+ 'lvmconf="yes"' + '\n')
1120+
1121+ with util.RunInChroot(target) as in_chroot:
1122+ in_chroot(['dracut', '-f', '--regenerate-all'])
1123+
1124+
1125+def system_upgrade(cfg, target):
1126+ """run yum upgrade in target.
1127+
1128+ config:
1129+ system_upgrade:
1130+ enabled: False
1131+
1132+ """
1133+ mycfg = {'system_upgrade': {'enabled': False}}
1134+ config.merge_config(mycfg, cfg)
1135+ mycfg = mycfg.get('system_upgrade')
1136+ if not isinstance(mycfg, dict):
1137+ LOG.debug("system_upgrade disabled by config. entry not a dict.")
1138+ return
1139+
1140+ if not config.value_as_boolean(mycfg.get('enabled', True)):
1141+ LOG.debug("system_upgrade disabled by config.")
1142+ return
1143+
1144+ util.subp(['yum', '-q', '-y', 'upgrade'], target=target, capture=True)
1145
1146
1147 def read_file(path):
1148@@ -86,16 +165,178 @@
1149 return stream.read()
1150
1151
1152-def write_fstab(target, curtin_fstab):
1153- """Writes the new fstab, using the fstab provided
1154- from curtin."""
1155- fstab_path = os.path.join(target, 'etc', 'fstab')
1156- fstab_data = read_file(curtin_fstab)
1157- with open(fstab_path, 'w') as stream:
1158- stream.write(FSTAB_PREPEND)
1159- stream.write(fstab_data)
1160- if util.is_uefi_bootable():
1161- stream.write(FSTAB_UEFI)
1162+def get_boot_mac():
1163+ """Return the mac address of the booting interface."""
1164+ cmdline = read_file('/proc/cmdline')
1165+ cmdline = cmdline.split()
1166+ try:
1167+ bootif = [
1168+ option
1169+ for option in cmdline
1170+ if option.startswith('BOOTIF')
1171+ ][0]
1172+ except IndexError:
1173+ return None
1174+ _, mac = bootif.split('=')
1175+ mac = mac.split('-')[1:]
1176+ return ':'.join(mac)
1177+
1178+
1179+def get_interface_names():
1180+ """Return a dictionary mapping mac addresses to interface names."""
1181+ sys_path = "/sys/class/net"
1182+ ifaces = {}
1183+ for iname in os.listdir(sys_path):
1184+ mac = read_file(os.path.join(sys_path, iname, "address"))
1185+ mac = mac.strip().lower()
1186+ ifaces[mac] = iname
1187+ return ifaces
1188+
1189+
1190+def get_subnet_config(subnets):
1191+ content = ''
1192+ for idx, subnet in enumerate(subnets):
1193+ if subnet['type'] in ('static', 'static4'):
1194+ if idx == 0:
1195+ idx = ''
1196+ iface = ipaddress.ip_interface(subnet['address'])
1197+ content += 'IPADDR{}={}\n'.format(idx, str(iface.ip))
1198+ content += 'NETMASK{}={}\n'.format(
1199+ idx, subnet.get('netmask', str(iface.netmask)))
1200+ return content
1201+
1202+
1203+def get_common_network_config(cfg):
1204+ # You can not have more than one GATEWAY, BOOTPROTO, DNS1, DNS2 and DOMAIN
1205+ # settings per interface in RHEL and clones. Will run through subnets
1206+ # section ensuring that only one of these makes to interface configuration.
1207+ content = ''
1208+ bootproto = 'dhcp'
1209+ defaultgw = None
1210+ resolvers = []
1211+ search_domains = []
1212+ for subnet in cfg['subnets']:
1213+ if subnet['type'] in ('dhcp', 'dhcp4'):
1214+ # OK, so we have a DHCP subnet. Let's assume all other subnets
1215+ # are also DHCP and just stop looking
1216+ bootproto = 'dhcp'
1217+ break
1218+ elif subnet['type'] in ('static', 'static4'):
1219+ # If this is a static subnet, then there might be a default
1220+ # gateway and resolvers set and if there aren't we'll keep
1221+ # looking for them in the next subnet configuration
1222+ bootproto = 'none'
1223+ if not defaultgw:
1224+ defaultgw = subnet.get('gateway')
1225+ if not resolvers:
1226+ resolvers = subnet.get('dns_nameservers')
1227+ if not search_domains:
1228+ search_domains = subnet.get('dns_search')
1229+ else:
1230+ # Let's log the lack of support and continue - there still
1231+ # migth be a supported subnet configuration
1232+ LOG.warn('Configuration of subnet type {} '
1233+ 'not supported'.format(subnet['type']))
1234+
1235+ content += 'BOOTPROTO={}\n'.format(bootproto)
1236+ if bootproto != 'dhcp':
1237+ if defaultgw:
1238+ content += 'GATEWAY={}\n'.format(defaultgw)
1239+ for idx, resolver in enumerate(resolvers):
1240+ content += 'DNS{}={}\n'.format(idx+1, resolver)
1241+ if search_domains:
1242+ content += 'DOMAIN="{}"\n'.format(' '.join(search_domains))
1243+
1244+ return content
1245+
1246+
1247+def write_resolv_conf(cfg, target):
1248+ content = '# Generated by MAAS fast-path installer\n'
1249+
1250+ for resolver in cfg.get('address', []):
1251+ content += 'nameserver {}\n'.format(resolver)
1252+
1253+ search_domains = cfg.get('search', [])
1254+ if search_domains:
1255+ content += 'search {}\n'.format(' '.join(search_domains))
1256+
1257+ path = os.path.join(target, 'etc', 'resolv.conf')
1258+ util.write_file(path, content)
1259+
1260+
1261+def write_physical_iface_config(cfg, target):
1262+ content = (
1263+ '# Generated by MAAS fast-path installer\n'
1264+ 'DEVICE={device}\n'
1265+ 'HWADDR={hwaddr}\n'
1266+ 'MTU={mtu}\n'
1267+ 'ONBOOT=yes\n'
1268+ ).format(
1269+ device=cfg['name'],
1270+ hwaddr=cfg['mac_address'],
1271+ mtu=cfg.get('mtu', 1500)
1272+ )
1273+
1274+ content += get_common_network_config(cfg) + \
1275+ get_subnet_config(cfg['subnets'])
1276+
1277+ path = os.path.join(target, 'etc', 'sysconfig',
1278+ 'network-scripts', 'ifcfg-%s' % cfg['name'])
1279+ util.write_file(path, content)
1280+
1281+
1282+def write_vlan_iface_config(cfg, target):
1283+ content = (
1284+ '# Generated by MAAS fast-path installer\n'
1285+ 'DEVICE={device}\n'
1286+ 'MTU={mtu}\n'
1287+ 'ONBOOT=yes\n'
1288+ 'VLAN=yes\n'
1289+ ).format(
1290+ device=cfg['name'],
1291+ mtu=cfg.get('mtu', 1500)
1292+ )
1293+
1294+ content += get_common_network_config(cfg) + \
1295+ get_subnet_config(cfg['subnets'])
1296+
1297+ path = os.path.join(target, 'etc', 'sysconfig',
1298+ 'network-scripts', 'ifcfg-%s' % cfg['name'])
1299+ util.write_file(path, content)
1300+
1301+
1302+def apply_networking(config, target):
1303+ network_config = config.get('network')
1304+ if not network_config:
1305+ LOG.warn("Unable to retrieve network configuration, will fallback "
1306+ "to configuring DHCP on boot device")
1307+ bootmac = get_boot_mac()
1308+ inames = get_interface_names()
1309+ iname = inames[bootmac.lower()]
1310+ network_config = {
1311+ 'config': [
1312+ {'mac_address': bootmac,
1313+ 'name': iname,
1314+ 'type': 'physical',
1315+ 'subnets': [{'type': 'dhcp'}]}],
1316+ 'version': 1}
1317+
1318+ for cfg in network_config['config']:
1319+ if cfg['type'] == 'nameserver':
1320+ write_resolv_conf(cfg, target)
1321+ if cfg['type'] == 'physical':
1322+ write_physical_iface_config(cfg, target)
1323+ if cfg['type'] == 'vlan':
1324+ write_vlan_iface_config(cfg, target)
1325+
1326+
1327+def update_grub_default(target, extra=[]):
1328+ """Updates /etc/default/grub with the correct options."""
1329+ grub_default_path = os.path.join(target, 'etc', 'default', 'grub')
1330+ kernel_cmdline = ' '.join(extra)
1331+ with open(grub_default_path, 'a') as stream:
1332+ stream.write(GRUB_PREPEND)
1333+ stream.write('GRUB_CMDLINE_LINUX=\"%s\"\n' % kernel_cmdline)
1334
1335
1336 def strip_kernel_params(params, strip_params=[]):
1337@@ -121,9 +362,9 @@
1338 """
1339 cmdline = read_file('/proc/cmdline')
1340 cmdline = cmdline.split()
1341- if '--' not in cmdline:
1342+ if '---' not in cmdline:
1343 return []
1344- idx = cmdline.index('--') + 1
1345+ idx = cmdline.index('---') + 1
1346 if idx >= len(cmdline) + 1:
1347 return []
1348 return strip_kernel_params(
1349@@ -131,57 +372,141 @@
1350 strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF='])
1351
1352
1353-def update_grub_default(target, extra=[]):
1354- """Updates /etc/default/grub with the correct options."""
1355- grub_default_path = os.path.join(target, 'etc', 'default', 'grub')
1356- kernel_cmdline = ' '.join(extra)
1357- with open(grub_default_path, 'a') as stream:
1358- stream.write(GRUB_PREPEND)
1359- stream.write('GRUB_CMDLINE_LINUX=\"%s\"\n' % kernel_cmdline)
1360-
1361-
1362-def grub2_install(target, root):
1363- """Installs grub2 to the root."""
1364- with util.RunInChroot(target) as in_chroot:
1365- in_chroot(['grub2-install', '--recheck', root])
1366-
1367-
1368 def grub2_mkconfig(target):
1369 """Writes the new grub2 config."""
1370- with util.RunInChroot(target) as in_chroot:
1371- in_chroot(['grub2-mkconfig', '-o', '/boot/grub2/grub.cfg'])
1372-
1373-
1374-def install_efi(target, uefi_path):
1375- """Install the EFI data from /boot into efi partition."""
1376- # Create temp mount point for uefi partition.
1377- tmp_efi = os.path.join(target, 'boot', 'efi_part')
1378- os.mkdir(tmp_efi)
1379- util.subp(['mount', uefi_path, tmp_efi])
1380-
1381- # Copy the data over.
1382- try:
1383- efi_path = os.path.join(target, 'boot', 'efi')
1384- if os.path.exists(os.path.join(tmp_efi, 'EFI')):
1385- shutil.rmtree(os.path.join(tmp_efi, 'EFI'))
1386- shutil.copytree(
1387- os.path.join(efi_path, 'EFI'),
1388- os.path.join(tmp_efi, 'EFI'))
1389- finally:
1390- # Clean up tmp mount
1391- util.subp(['umount', tmp_efi])
1392- os.rmdir(tmp_efi)
1393-
1394- # Mount and do grub install
1395- util.subp(['mount', uefi_path, efi_path])
1396- try:
1397- with util.RunInChroot(target) as in_chroot:
1398- in_chroot([
1399- 'grub2-install', '--target=x86_64-efi',
1400- '--efi-directory', '/boot/efi',
1401- '--recheck'])
1402- finally:
1403- util.subp(['umount', efi_path])
1404+ # NOTE. CentOS kernel packages expect /etc/grub2.cfg symlink in
1405+ # BIOS mode and /etc/grub2-efi.cfg symlink in UEFI mode pointing
1406+ # to an actual grub.cfg
1407+
1408+ grub_cfg = '/boot/grub2/grub.cfg'
1409+ grub_link = '/etc/grub2.cfg'
1410+ grub_unlink = os.path.join(target, 'etc', 'grub2-efi.cfg')
1411+
1412+ if util.is_uefi_bootable():
1413+ grub_cfg = '/boot/efi/EFI/centos/grub.cfg'
1414+ grub_link = '/etc/grub2-efi.cfg'
1415+ grub_unlink = os.path.join(target, 'etc', 'grub2.cfg')
1416+ util.write_file(os.path.join(target, 'boot', 'grub2', 'grub.cfg'),
1417+ '# UEFI system, see /etc/grub2-efi.cfg\n')
1418+
1419+ with util.RunInChroot(target) as in_chroot:
1420+ in_chroot(['grub2-mkconfig', '-o', grub_cfg])
1421+ in_chroot(['ln', '-sf', grub_cfg, grub_link])
1422+ util.del_file(grub_unlink)
1423+
1424+
1425+def get_uefi_partition():
1426+ """Return the UEFI partition."""
1427+ for _, value in block._lsblock().items():
1428+ if value['LABEL'] == 'uefi-boot':
1429+ return value
1430+ return None
1431+
1432+
1433+def install_bootloader(target, device=None):
1434+ """Installs bootloader to the device."""
1435+ cmd = []
1436+ if util.is_uefi_bootable():
1437+ uefi_dev = get_uefi_partition()['device_path']
1438+ disk, part = block.get_blockdev_for_partition(uefi_dev)
1439+ cmd = ['efibootmgr', '-v', '-c', '-w', '-L', 'centos',
1440+ '-d', disk, '-p', part, '-l', r'\EFI\centos\shim.efi']
1441+ else:
1442+ cmd = ['grub2-install', '--recheck', device]
1443+
1444+ with util.RunInChroot(target) as in_chroot:
1445+ in_chroot(cmd)
1446+
1447+
1448+def setup_grub(cfg, target):
1449+ # target is the path to the mounted filesystem
1450+
1451+ # FIXME: these methods need moving to curtin.block
1452+ # and using them from there rather than commands.block_meta
1453+ from curtin.commands.block_meta import (extract_storage_ordered_dict,
1454+ get_path_to_storage_volume)
1455+
1456+ grubcfg = cfg.get('grub', {})
1457+
1458+ # copy legacy top level name
1459+ if 'grub_install_devices' in cfg and 'install_devices' not in grubcfg:
1460+ grubcfg['install_devices'] = cfg['grub_install_devices']
1461+
1462+ LOG.debug("setup grub on target %s", target)
1463+ # if there is storage config, look for devices tagged with 'grub_device'
1464+ storage_cfg_odict = None
1465+ try:
1466+ storage_cfg_odict = extract_storage_ordered_dict(cfg)
1467+ except ValueError as e:
1468+ pass
1469+
1470+ if storage_cfg_odict:
1471+ storage_grub_devices = []
1472+ for item_id, item in storage_cfg_odict.items():
1473+ if not item.get('grub_device'):
1474+ continue
1475+ LOG.debug("checking: %s", item)
1476+ storage_grub_devices.append(
1477+ get_path_to_storage_volume(item_id, storage_cfg_odict))
1478+ if len(storage_grub_devices) > 0:
1479+ grubcfg['install_devices'] = storage_grub_devices
1480+
1481+ LOG.debug("install_devices: %s", grubcfg.get('install_devices'))
1482+ if 'install_devices' in grubcfg:
1483+ instdevs = grubcfg.get('install_devices')
1484+ if isinstance(instdevs, str):
1485+ instdevs = [instdevs]
1486+ if instdevs is None:
1487+ LOG.debug("grub installation disabled by config")
1488+ else:
1489+ # If there were no install_devices found then we try to do the right
1490+ # thing. That right thing is basically installing on all block
1491+ # devices that are mounted. On powerpc, though it means finding PrEP
1492+ # partitions.
1493+ devs = block.get_devices_for_mp(target)
1494+ blockdevs = set()
1495+ for maybepart in devs:
1496+ try:
1497+ (blockdev, part) = block.get_blockdev_for_partition(maybepart)
1498+ blockdevs.add(blockdev)
1499+ except ValueError as e:
1500+ # if there is no syspath for this device such as a lvm
1501+ # or raid device, then a ValueError is raised here.
1502+ LOG.debug("failed to find block device for %s", maybepart)
1503+ instdevs = list(blockdevs)
1504+
1505+ # UEFI requires additional packages
1506+ if util.is_uefi_bootable():
1507+ pkgs = ['grub2-efi', 'shim', 'efibootmgr']
1508+ install_packages(pkgs, target=target)
1509+
1510+ # CentOS will not assemble MD devices on boot without rd.md.uuid=MD_UUID
1511+ # kernel parameters
1512+ mdmap = {}
1513+ rdmduuids = []
1514+ try:
1515+ mdmap = mdadm.md_read_run_mdadm_map()
1516+ for md in mdmap:
1517+ rdmduuids.append("rd.md.uuid=%s" % mdadm.md_get_uuid(mdmap[md][2]))
1518+ except ValueError as e:
1519+ pass
1520+
1521+ update_grub_default(target, extra=get_extra_kernel_parameters() + rdmduuids)
1522+ grub2_mkconfig(target)
1523+
1524+ if instdevs:
1525+ instdevs = [block.get_dev_name_entry(i)[1] for i in instdevs]
1526+ else:
1527+ instdevs = ["none"]
1528+
1529+ LOG.debug("installing grub to %s", instdevs)
1530+
1531+ if util.is_uefi_bootable():
1532+ if grubcfg.get('update_nvram', False):
1533+ install_bootloader(target)
1534+ else:
1535+ for dev in instdevs:
1536+ install_bootloader(target, dev)
1537
1538
1539 def set_autorelabel(target):
1540@@ -196,136 +521,75 @@
1541 open(path, 'a').close()
1542
1543
1544-def get_boot_mac():
1545- """Return the mac address of the booting interface."""
1546- cmdline = read_file('/proc/cmdline')
1547- cmdline = cmdline.split()
1548- try:
1549- bootif = [
1550- option
1551- for option in cmdline
1552- if option.startswith('BOOTIF')
1553- ][0]
1554- except IndexError:
1555- return None
1556- _, mac = bootif.split('=')
1557- mac = mac.split('-')[1:]
1558- return ':'.join(mac)
1559-
1560-
1561-def get_interface_names():
1562- """Return a dictionary mapping mac addresses to interface names."""
1563- sys_path = "/sys/class/net"
1564- ifaces = {}
1565- for iname in os.listdir(sys_path):
1566- mac = read_file(os.path.join(sys_path, iname, "address"))
1567- mac = mac.strip().lower()
1568- ifaces[mac] = iname
1569- return ifaces
1570-
1571-
1572-def get_ipv4_config(iface, data):
1573- """Returns the contents of the interface file for ipv4."""
1574- config = [
1575- 'TYPE="Ethernet"',
1576- 'NM_CONTROLLED="no"',
1577- 'USERCTL="yes"',
1578- ]
1579- if 'hwaddress' in data:
1580- config.append('HWADDR="%s"' % data['hwaddress'])
1581- # Fallback to using device name
1582- else:
1583- config.append('DEVICE="%"' % iface)
1584- if data['auto']:
1585- config.append('ONBOOT="yes"')
1586- else:
1587- config.append('ONBOOT="no"')
1588-
1589- method = data['method']
1590- if method == 'dhcp':
1591- config.append('BOOTPROTO="dhcp"')
1592- config.append('PEERDNS="yes"')
1593- config.append('PERSISTENT_DHCLIENT="1"')
1594- if 'hostname' in data:
1595- config.append('DHCP_HOSTNAME="%s"' % data['hostname'])
1596- elif method == 'static':
1597- config.append('BOOTPROTO="none"')
1598- config.append('IPADDR="%s"' % data['address'])
1599- config.append('NETMASK="%s"' % data['netmask'])
1600- if 'broadcast' in data:
1601- config.append('BROADCAST="%s"' % data['broadcast'])
1602- if 'gateway' in data:
1603- config.append('GATEWAY="%s"' % data['gateway'])
1604- elif method == 'manual':
1605- config.append('BOOTPROTO="none"')
1606- return '\n'.join(config)
1607-
1608-
1609-def write_interface_config(target, iface, data):
1610- """Writes config for interface."""
1611- family = data['family']
1612- if family != "inet":
1613- # Only supporting ipv4 currently
1614- print(
1615- "WARN: unsupported family %s, "
1616- "failed to configure interface: %s" (family, iface))
1617- return
1618- config = get_ipv4_config(iface, data)
1619- path = os.path.join(
1620- target, 'etc', 'sysconfig', 'network-scripts', 'ifcfg-%s' % iface)
1621- with open(path, 'w') as stream:
1622- stream.write(config + '\n')
1623-
1624-
1625-def write_network_config(target, mac):
1626- """Write network configuration for the given MAC address."""
1627- inames = get_interface_names()
1628- iname = inames[mac.lower()]
1629- write_interface_config(
1630- target, iname, {
1631- 'family': 'inet',
1632- 'hwaddress': mac.upper(),
1633- 'auto': True,
1634- 'method': 'dhcp'
1635- })
1636-
1637-
1638 def main():
1639 state = util.load_command_environment()
1640+
1641 target = state['target']
1642 if target is None:
1643- print("Target was not provided in the environment.")
1644- sys.exit(1)
1645- fstab = state['fstab']
1646- if fstab is None:
1647- print("/etc/fstab output was not provided in the environment.")
1648- sys.exit(1)
1649- bootmac = get_boot_mac()
1650- if bootmac is None:
1651- print("Unable to determine boot interface.")
1652- sys.exit(1)
1653- devices = get_block_devices(target)
1654- if not devices:
1655- print("Unable to find block device for: %s" % target)
1656- sys.exit(1)
1657-
1658- write_fstab(target, fstab)
1659-
1660- update_grub_default(
1661- target, extra=get_extra_kernel_parameters())
1662- grub2_mkconfig(target)
1663- if util.is_uefi_bootable():
1664- uefi_part = get_uefi_partition()
1665- if uefi_part is None:
1666- print('Unable to determine UEFI parition.')
1667- sys.exit(1)
1668- install_efi(target, uefi_part['device_path'])
1669- else:
1670- for dev in devices:
1671- grub2_install(target, dev)
1672+ sys.stderr.write("Target was not provided in the environment.")
1673+ sys.exit(1)
1674+
1675+ cfg = config.load_command_config(None, state)
1676+ stack_prefix = state.get('report_stack_prefix', '')
1677+
1678+ with events.ReportEventStack(
1679+ name=stack_prefix, reporting_enabled=True, level="INFO",
1680+ description="writing config files"):
1681+ curthooks.write_files(cfg, target)
1682+
1683+ # Default CentOS image does not contain some packages that may be necessary
1684+ install_missing_packages(cfg, target)
1685+
1686+ # If a mdadm.conf file was created by block_meta then it needs to be copied
1687+ # onto the target system
1688+ mdadm_location = os.path.join(os.path.split(state['fstab'])[0],
1689+ "mdadm.conf")
1690+ if os.path.exists(mdadm_location):
1691+ copy_mdadm_conf(mdadm_location, target)
1692+
1693+ with events.ReportEventStack(
1694+ name=stack_prefix, reporting_enabled=True, level="INFO",
1695+ description="setting up swap"):
1696+ curthooks.add_swap(cfg, target, state.get('fstab'))
1697+
1698+ with events.ReportEventStack(
1699+ name=stack_prefix, reporting_enabled=True, level="INFO",
1700+ description="writing etc/fstab"):
1701+ curthooks.copy_fstab(state.get('fstab'), target)
1702+
1703+ # TODO. There's a multipath implementation on Ubuntu in curtin/curthooks.py
1704+ # that should be reimplemented here. Skipping for now.
1705+
1706+ with events.ReportEventStack(
1707+ name=stack_prefix, reporting_enabled=True, level="INFO",
1708+ description="updating packages on target system"):
1709+ system_upgrade(cfg, target)
1710+
1711+ # TODO. If a crypttab file was created by block_meta than it needs to be
1712+ # copied onto the target system, and update_initramfs() (well, alternative
1713+ # for CentOS actually) needs to be run, so that the cryptsetup hooks are
1714+ # properly configured on the installed system and it will be able to open
1715+ # encrypted volumes at boot. Skipping for now.
1716+
1717+ with events.ReportEventStack(
1718+ name=stack_prefix, reporting_enabled=True, level="INFO",
1719+ description="apply networking"):
1720+ apply_networking(cfg, target)
1721+
1722+ # If udev dname rules were created, copy them to target
1723+ udev_rules_d = os.path.join(state['scratch'], "rules.d")
1724+ if os.path.isdir(udev_rules_d):
1725+ curthooks.copy_dname_rules(udev_rules_d, target)
1726+
1727+ with events.ReportEventStack(
1728+ name=stack_prefix, reporting_enabled=True, level="INFO",
1729+ description="updating packages on target system"):
1730+ setup_grub(cfg, target)
1731
1732 set_autorelabel(target)
1733- write_network_config(target, bootmac)
1734+ update_initramfs(target)
1735+
1736+ sys.exit(0)
1737
1738
1739 if __name__ == "__main__":

Subscribers

People subscribed via source and target branches