Merge lp:~marius-zalinauskas/maas-images/centos-hooks into lp:maas-images
- centos-hooks
- Merge into maas-ephemerals
Status: | Rejected |
---|---|
Rejected by: | Scott Moser |
Proposed branch: | lp:~marius-zalinauskas/maas-images/centos-hooks |
Merge into: | lp:maas-images |
Diff against target: |
1739 lines (+1124/-462) 2 files modified
curtin/centos6/curtin-hooks.py (+622/-224) curtin/centos7/curtin-hooks.py (+502/-238) |
To merge this branch: | bzr merge lp:~marius-zalinauskas/maas-images/centos-hooks |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Andres Rodriguez (community) | Needs Fixing | ||
Tom Mullaney (community) | Approve | ||
Review via email: mp+313684@code.launchpad.net |
Commit message
Description of the change
Heavily modified CentOS 6/7 hooks. For the most part it is curtin/
Added functionality:
- UEFI boot. It was already kinda working on CentOS 7, but was tricky on CentOS 6.
- Simple and MD RAID layouts. No need to force the simple layout on MAAS anymore.
- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces. Much better than original DHCP only.
- 353. By Marius Žalinauskas <email address hidden>
-
Bugfix, see https:/
/bugs.launchpad .net/maas- images/ +bug/1654853
Andres Rodriguez (andreserl) wrote : | # |
Hi Marius,
Thank you for your contribution!
In order to be able to accept your code you need to sign the Canonical Contributors Agreement. Can you please do that in [1].
That said, we have a few things that need fixing before we are able to review / land branches, to follow the procedures MAAS uses, Please:
- Separate the UEFI changes into is own branch
- Seprate the Simple/MD Raid into its own branch
- IPv4 DHCP/Static Networking, also it its own branch.
Thank you!
Scott Moser (smoser) wrote : | # |
Marius,
Thank you for your contribution. Sorry for the confusion or delay in responding.
I'm going to mark this 'rejected' as of right now. You are welcome to re-submit it, but do so against the git repo now.
https:/
thanks!
Unmerged revisions
- 353. By Marius Žalinauskas <email address hidden>
-
Bugfix, see https:/
/bugs.launchpad .net/maas- images/ +bug/1654853 - 352. By Marius Žalinauskas <email address hidden>
-
Extensibly modified CentOS 6/7 Curtin hooks.
For the most part it is curtin/
commands/ curthooks. py ported to CentOS. Some code from old CentOS hooks is still used. Implemented:
- Legacy boot
- UEFI boot
- Simple and MD RAID layouts
- IPv4 DHCP and static networking, IP aliases on physical and VLAN interfacesNot implemented:
- Bond, bridge, route configurations
- IPv6
- Disk encryption
- bcache
- LVM
- Multipath
Preview Diff
1 | === modified file 'curtin/centos6/curtin-hooks.py' | |||
2 | --- curtin/centos6/curtin-hooks.py 2016-05-11 18:47:46 +0000 | |||
3 | +++ curtin/centos6/curtin-hooks.py 2017-01-09 15:19:25 +0000 | |||
4 | @@ -1,20 +1,19 @@ | |||
5 | 1 | #!/usr/bin/env python | 1 | #!/usr/bin/env python |
12 | 2 | 2 | # coding: utf-8 | |
7 | 3 | from __future__ import ( | ||
8 | 4 | absolute_import, | ||
9 | 5 | print_function, | ||
10 | 6 | unicode_literals, | ||
11 | 7 | ) | ||
13 | 8 | 3 | ||
14 | 9 | import codecs | 4 | import codecs |
15 | 5 | import ipaddress | ||
16 | 10 | import os | 6 | import os |
17 | 11 | import re | 7 | import re |
18 | 8 | import shutil | ||
19 | 12 | import sys | 9 | import sys |
20 | 13 | 10 | ||
25 | 14 | from curtin import ( | 11 | from curtin import block, config, util |
26 | 15 | block, | 12 | from curtin.block import mdadm |
27 | 16 | util, | 13 | from curtin.commands import curthooks |
28 | 17 | ) | 14 | from curtin.log import LOG |
29 | 15 | from curtin.reporter import events | ||
30 | 16 | |||
31 | 18 | 17 | ||
32 | 19 | """ | 18 | """ |
33 | 20 | CentOS 6 | 19 | CentOS 6 |
34 | @@ -22,15 +21,21 @@ | |||
35 | 22 | Currently Support: | 21 | Currently Support: |
36 | 23 | 22 | ||
37 | 24 | - Legacy boot | 23 | - Legacy boot |
39 | 25 | - DHCP of BOOTIF | 24 | - UEFI boot |
40 | 25 | - Simple and MD RAID layouts | ||
41 | 26 | - IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces | ||
42 | 26 | 27 | ||
43 | 27 | Not Supported: | 28 | Not Supported: |
44 | 28 | 29 | ||
47 | 29 | - UEFI boot (*Bad support, most likely wont support) | 30 | - Bond, bridge, route configurations |
46 | 30 | - Multiple network configration | ||
48 | 31 | - IPv6 | 31 | - IPv6 |
49 | 32 | - Disk encryption | ||
50 | 33 | - bcache | ||
51 | 34 | - LVM | ||
52 | 35 | - Multipath | ||
53 | 32 | """ | 36 | """ |
54 | 33 | 37 | ||
55 | 38 | |||
56 | 34 | FSTAB_PREPEND = """\ | 39 | FSTAB_PREPEND = """\ |
57 | 35 | # | 40 | # |
58 | 36 | # /etc/fstab | 41 | # /etc/fstab |
59 | @@ -54,36 +59,95 @@ | |||
60 | 54 | # Created by MAAS fast-path installer. | 59 | # Created by MAAS fast-path installer. |
61 | 55 | # | 60 | # |
62 | 56 | default 0 | 61 | default 0 |
65 | 57 | timeout 0 | 62 | timeout 1 |
66 | 58 | title MAAS | 63 | title CentOS |
67 | 59 | root {grub_root} | 64 | root {grub_root} |
70 | 60 | kernel /boot/{vmlinuz} root=UUID={root_uuid} {extra_opts} | 65 | kernel /{vmlinuz} root=UUID={root_uuid} {extra_opts} |
71 | 61 | initrd /boot/{initrd} | 66 | initrd /{initrd} |
72 | 62 | """ | 67 | """ |
73 | 63 | 68 | ||
74 | 64 | 69 | ||
97 | 65 | def get_block_devices(target): | 70 | def get_installed_packages(target=None): |
98 | 66 | """Returns list of block devices for the given target.""" | 71 | (out, _) = util.subp(['rpm', '-qa', '--qf', '"%{NAME}\n"'], |
99 | 67 | devs = block.get_devices_for_mp(target) | 72 | target=target, capture=True) |
100 | 68 | blockdevs = set() | 73 | return set(out.splitlines()) |
101 | 69 | for maybepart in devs: | 74 | |
102 | 70 | (blockdev, part) = block.get_blockdev_for_partition(maybepart) | 75 | |
103 | 71 | blockdevs.add(blockdev) | 76 | def install_packages(pkglist, target): |
104 | 72 | return list(blockdevs) | 77 | if isinstance(pkglist, str): |
105 | 73 | 78 | pkglist = [pkglist] | |
106 | 74 | 79 | # Does not work – hosts are not resolved | |
107 | 75 | def get_root_info(target): | 80 | # return util.subp(['yum', '-q', '-y', 'install'] + pkglist, target=target, capture=True) |
108 | 76 | """Returns the root partitions information.""" | 81 | with util.RunInChroot(target) as in_chroot: |
109 | 77 | rootpath = block.get_devices_for_mp(target)[0] | 82 | in_chroot(['yum', '-q', '-y', 'install'] + pkglist) |
110 | 78 | rootdev = os.path.basename(rootpath) | 83 | |
111 | 79 | blocks = block._lsblock() | 84 | |
112 | 80 | return blocks[rootdev] | 85 | def install_missing_packages(cfg, target): |
113 | 81 | 86 | ''' describe which operation types will require specific packages | |
114 | 82 | 87 | ||
115 | 83 | def read_file(path): | 88 | 'custom_config_key': { |
116 | 84 | """Returns content of a file.""" | 89 | 'pkg1': ['op_name_1', 'op_name_2', ...] |
117 | 85 | with codecs.open(path, encoding='utf-8') as stream: | 90 | } |
118 | 86 | return stream.read() | 91 | ''' |
119 | 92 | custom_configs = { | ||
120 | 93 | 'storage': { | ||
121 | 94 | 'lvm2': ['lvm_volgroup', 'lvm_partition'], | ||
122 | 95 | 'mdadm': ['raid'], | ||
123 | 96 | # TODO. At the moment there are no official packages for bcache on CentOS | ||
124 | 97 | #'bcache-tools': ['bcache'] | ||
125 | 98 | }, | ||
126 | 99 | 'network': { | ||
127 | 100 | 'bridge-utils': ['bridge'] | ||
128 | 101 | }, | ||
129 | 102 | } | ||
130 | 103 | |||
131 | 104 | format_configs = { | ||
132 | 105 | 'xfsprogs': ['xfs'], | ||
133 | 106 | 'e2fsprogs': ['ext2', 'ext3', 'ext4'], | ||
134 | 107 | 'btrfs-tools': ['btrfs'], | ||
135 | 108 | } | ||
136 | 109 | |||
137 | 110 | needed_packages = [] | ||
138 | 111 | installed_packages = get_installed_packages(target) | ||
139 | 112 | for cust_cfg, pkg_reqs in custom_configs.items(): | ||
140 | 113 | if cust_cfg not in cfg: | ||
141 | 114 | continue | ||
142 | 115 | |||
143 | 116 | all_types = set( | ||
144 | 117 | operation['type'] | ||
145 | 118 | for operation in cfg[cust_cfg]['config'] | ||
146 | 119 | ) | ||
147 | 120 | for pkg, types in pkg_reqs.items(): | ||
148 | 121 | if set(types).intersection(all_types) and \ | ||
149 | 122 | pkg not in installed_packages: | ||
150 | 123 | needed_packages.append(pkg) | ||
151 | 124 | |||
152 | 125 | format_types = set( | ||
153 | 126 | [operation['fstype'] | ||
154 | 127 | for operation in cfg[cust_cfg]['config'] | ||
155 | 128 | if operation['type'] == 'format']) | ||
156 | 129 | for pkg, fstypes in format_configs.items(): | ||
157 | 130 | if set(fstypes).intersection(format_types) and \ | ||
158 | 131 | pkg not in installed_packages: | ||
159 | 132 | needed_packages.append(pkg) | ||
160 | 133 | |||
161 | 134 | if needed_packages: | ||
162 | 135 | state = util.load_command_environment() | ||
163 | 136 | with events.ReportEventStack( | ||
164 | 137 | name=state.get('report_stack_prefix'), | ||
165 | 138 | reporting_enabled=True, level="INFO", | ||
166 | 139 | description="Installing packages on target system: " + | ||
167 | 140 | str(needed_packages)): | ||
168 | 141 | install_packages(needed_packages, target=target) | ||
169 | 142 | |||
170 | 143 | |||
171 | 144 | def copy_mdadm_conf(mdadm_conf, target): | ||
172 | 145 | if not mdadm_conf: | ||
173 | 146 | LOG.warn("mdadm config must be specified, not copying") | ||
174 | 147 | return | ||
175 | 148 | |||
176 | 149 | LOG.info("copying mdadm.conf into target") | ||
177 | 150 | shutil.copy(mdadm_conf, os.path.sep.join([target, 'etc/mdadm.conf'])) | ||
178 | 87 | 151 | ||
179 | 88 | 152 | ||
180 | 89 | def write_fstab(target, curtin_fstab): | 153 | def write_fstab(target, curtin_fstab): |
181 | @@ -97,11 +161,289 @@ | |||
182 | 97 | stream.write(FSTAB_APPEND) | 161 | stream.write(FSTAB_APPEND) |
183 | 98 | 162 | ||
184 | 99 | 163 | ||
190 | 100 | def extract_kernel_params(data): | 164 | def update_initramfs(target=None): |
191 | 101 | """Extracts the kernel parametes from the provided | 165 | path = os.path.join( |
192 | 102 | grub config data.""" | 166 | target, 'etc', 'dracut.conf.d', 'local.conf') |
193 | 103 | match = re.search('^\s+kernel (.+?)$', data, re.MULTILINE) | 167 | with open(path, 'w') as stream: |
194 | 104 | return match.group(0) | 168 | stream.write('mdadmconf="yes"' + '\n' |
195 | 169 | 'lvmconf="yes"' + '\n') | ||
196 | 170 | |||
197 | 171 | initrd = get_boot_file(target, 'initramfs') | ||
198 | 172 | version = initrd.replace('initramfs-', '').replace('.img', '') | ||
199 | 173 | initrd_path = os.path.join(os.sep, 'boot', initrd) | ||
200 | 174 | with util.RunInChroot(target) as in_chroot: | ||
201 | 175 | in_chroot(['dracut', '-f', initrd_path, version]) | ||
202 | 176 | |||
203 | 177 | |||
204 | 178 | def system_upgrade(cfg, target): | ||
205 | 179 | """run yum upgrade in target. | ||
206 | 180 | |||
207 | 181 | config: | ||
208 | 182 | system_upgrade: | ||
209 | 183 | enabled: False | ||
210 | 184 | |||
211 | 185 | """ | ||
212 | 186 | mycfg = {'system_upgrade': {'enabled': False}} | ||
213 | 187 | config.merge_config(mycfg, cfg) | ||
214 | 188 | mycfg = mycfg.get('system_upgrade') | ||
215 | 189 | if not isinstance(mycfg, dict): | ||
216 | 190 | LOG.debug("system_upgrade disabled by config. entry not a dict.") | ||
217 | 191 | return | ||
218 | 192 | |||
219 | 193 | if not config.value_as_boolean(mycfg.get('enabled', True)): | ||
220 | 194 | LOG.debug("system_upgrade disabled by config.") | ||
221 | 195 | return | ||
222 | 196 | |||
223 | 197 | util.subp(['yum', '-q', '-y', 'upgrade'], target=target, capture=True) | ||
224 | 198 | |||
225 | 199 | |||
226 | 200 | def read_file(path): | ||
227 | 201 | """Returns content of a file.""" | ||
228 | 202 | with codecs.open(path, encoding='utf-8') as stream: | ||
229 | 203 | return stream.read() | ||
230 | 204 | |||
231 | 205 | |||
232 | 206 | def get_boot_mac(): | ||
233 | 207 | """Return the mac address of the booting interface.""" | ||
234 | 208 | cmdline = read_file('/proc/cmdline') | ||
235 | 209 | cmdline = cmdline.split() | ||
236 | 210 | try: | ||
237 | 211 | bootif = [ | ||
238 | 212 | option | ||
239 | 213 | for option in cmdline | ||
240 | 214 | if option.startswith('BOOTIF') | ||
241 | 215 | ][0] | ||
242 | 216 | except IndexError: | ||
243 | 217 | return None | ||
244 | 218 | _, mac = bootif.split('=') | ||
245 | 219 | mac = mac.split('-')[1:] | ||
246 | 220 | return ':'.join(mac) | ||
247 | 221 | |||
248 | 222 | |||
249 | 223 | def get_interface_names(): | ||
250 | 224 | """Return a dictionary mapping mac addresses to interface names.""" | ||
251 | 225 | sys_path = "/sys/class/net" | ||
252 | 226 | ifaces = {} | ||
253 | 227 | for iname in os.listdir(sys_path): | ||
254 | 228 | mac = read_file(os.path.join(sys_path, iname, "address")) | ||
255 | 229 | mac = mac.strip().lower() | ||
256 | 230 | ifaces[mac] = iname | ||
257 | 231 | return ifaces | ||
258 | 232 | |||
259 | 233 | |||
260 | 234 | def find_legacy_iface_name(consistent_iface_name): | ||
261 | 235 | """Returns legacy network interface name by running biosdevname (needs) | ||
262 | 236 | to be installed beforehand)""" | ||
263 | 237 | out, err = util.subp(['biosdevname', '--policy', 'all_ethN', | ||
264 | 238 | '--interface', consistent_iface_name], | ||
265 | 239 | capture=True) | ||
266 | 240 | return out.strip() | ||
267 | 241 | |||
268 | 242 | |||
269 | 243 | def rename_ifaces(network_config, target): | ||
270 | 244 | """CentOS 6 is not using a consistent network device naming and even we | ||
271 | 245 | would enable it, interface names would differ from names they would get | ||
272 | 246 | on new systems with systemd. Will change every occurence of consistent | ||
273 | 247 | network device name to legacy name ethN and create an appropriate | ||
274 | 248 | /etc/udev/rules.d/70-persistent-net.rules""" | ||
275 | 249 | |||
276 | 250 | # Will need biosdevname to find out legacy names | ||
277 | 251 | util.install_packages(['biosdevname']) | ||
278 | 252 | |||
279 | 253 | # Build the cache as {'consistent_name': 'legacy_name'} | ||
280 | 254 | cache = {} | ||
281 | 255 | for cfg in network_config['config']: | ||
282 | 256 | if cfg['type'] == 'physical': | ||
283 | 257 | consistent_name = cfg['name'] | ||
284 | 258 | cache[consistent_name] = find_legacy_iface_name(consistent_name) | ||
285 | 259 | |||
286 | 260 | rules = '' | ||
287 | 261 | rule_template = ('SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ' | ||
288 | 262 | 'ATTR{{address}}=="{mac_address}", KERNEL=="eth*", ' | ||
289 | 263 | 'NAME="{iface_name}"\n') | ||
290 | 264 | |||
291 | 265 | # Rename devices and generate an appropriate content for | ||
292 | 266 | # /etc/udev/rules.d/70-persistent-net.rules | ||
293 | 267 | for cfg in network_config['config']: | ||
294 | 268 | if cfg['type'] == 'physical': | ||
295 | 269 | consistent_phys_name = cfg['name'] | ||
296 | 270 | legacy_phys_name = cache[consistent_phys_name] | ||
297 | 271 | cfg['name'] = legacy_phys_name | ||
298 | 272 | rules += rule_template.format(iface_name=legacy_phys_name, | ||
299 | 273 | mac_address=cfg['mac_address']) | ||
300 | 274 | if cfg['type'] == 'vlan': | ||
301 | 275 | consistent_phys_name = cfg['vlan_link'] | ||
302 | 276 | legacy_phys_name = cache[consistent_phys_name] | ||
303 | 277 | cfg['name'] = '{}.{}'.format(legacy_phys_name, cfg['vlan_id']) | ||
304 | 278 | cfg['vlan_link'] = legacy_phys_name | ||
305 | 279 | |||
306 | 280 | path = os.path.join(target, 'etc', 'udev', 'rules.d', | ||
307 | 281 | '70-persistent-net.rules') | ||
308 | 282 | util.write_file(path, rules) | ||
309 | 283 | |||
310 | 284 | |||
311 | 285 | |||
312 | 286 | def get_subnet_config(subnets): | ||
313 | 287 | content = '' | ||
314 | 288 | for idx, subnet in enumerate(subnets): | ||
315 | 289 | if subnet['type'] in ('static', 'static4'): | ||
316 | 290 | if idx == 0: | ||
317 | 291 | idx = '' | ||
318 | 292 | iface = ipaddress.ip_interface(subnet['address']) | ||
319 | 293 | content += 'IPADDR{}={}\n'.format(idx, str(iface.ip)) | ||
320 | 294 | content += 'NETMASK{}={}\n'.format( | ||
321 | 295 | idx, subnet.get('netmask', str(iface.netmask))) | ||
322 | 296 | return content | ||
323 | 297 | |||
324 | 298 | |||
325 | 299 | def get_common_network_config(cfg): | ||
326 | 300 | # You can not have more than one GATEWAY, BOOTPROTO, DNS1, DNS2 and DOMAIN | ||
327 | 301 | # settings per interface in RHEL and clones. Will run through subnets | ||
328 | 302 | # section ensuring that only one of these makes to interface configuration. | ||
329 | 303 | content = '' | ||
330 | 304 | bootproto = 'dhcp' | ||
331 | 305 | defaultgw = None | ||
332 | 306 | resolvers = [] | ||
333 | 307 | search_domains = [] | ||
334 | 308 | for subnet in cfg['subnets']: | ||
335 | 309 | if subnet['type'] in ('dhcp', 'dhcp4'): | ||
336 | 310 | # OK, so we have a DHCP subnet. Let's assume all other subnets | ||
337 | 311 | # are also DHCP and just stop looking | ||
338 | 312 | bootproto = 'dhcp' | ||
339 | 313 | break | ||
340 | 314 | elif subnet['type'] in ('static', 'static4'): | ||
341 | 315 | # If this is a static subnet, then there might be a default | ||
342 | 316 | # gateway and resolvers set and if there aren't we'll keep | ||
343 | 317 | # looking for them in the next subnet configuration | ||
344 | 318 | bootproto = 'none' | ||
345 | 319 | if not defaultgw: | ||
346 | 320 | defaultgw = subnet.get('gateway') | ||
347 | 321 | if not resolvers: | ||
348 | 322 | resolvers = subnet.get('dns_nameservers') | ||
349 | 323 | if not search_domains: | ||
350 | 324 | search_domains = subnet.get('dns_search') | ||
351 | 325 | else: | ||
352 | 326 | # Let's log the lack of support and continue - there still | ||
353 | 327 | # migth be a supported subnet configuration | ||
354 | 328 | LOG.warn('Configuration of subnet type {} ' | ||
355 | 329 | 'not supported'.format(subnet['type'])) | ||
356 | 330 | |||
357 | 331 | content += 'BOOTPROTO={}\n'.format(bootproto) | ||
358 | 332 | if bootproto != 'dhcp': | ||
359 | 333 | if defaultgw: | ||
360 | 334 | content += 'GATEWAY={}\n'.format(defaultgw) | ||
361 | 335 | for idx, resolver in enumerate(resolvers): | ||
362 | 336 | content += 'DNS{}={}\n'.format(idx+1, resolver) | ||
363 | 337 | if search_domains: | ||
364 | 338 | content += 'DOMAIN="{}"\n'.format(' '.join(search_domains)) | ||
365 | 339 | |||
366 | 340 | return content | ||
367 | 341 | |||
368 | 342 | |||
369 | 343 | def write_resolv_conf(cfg, target): | ||
370 | 344 | content = '# Generated by MAAS fast-path installer\n' | ||
371 | 345 | |||
372 | 346 | for resolver in cfg.get('address', []): | ||
373 | 347 | content += 'nameserver {}\n'.format(resolver) | ||
374 | 348 | |||
375 | 349 | search_domains = cfg.get('search', []) | ||
376 | 350 | if search_domains: | ||
377 | 351 | content += 'search {}\n'.format(' '.join(search_domains)) | ||
378 | 352 | |||
379 | 353 | path = os.path.join(target, 'etc', 'resolv.conf') | ||
380 | 354 | util.write_file(path, content) | ||
381 | 355 | |||
382 | 356 | |||
383 | 357 | def write_physical_iface_config(cfg, target): | ||
384 | 358 | content = ( | ||
385 | 359 | '# Generated by MAAS fast-path installer\n' | ||
386 | 360 | 'DEVICE={device}\n' | ||
387 | 361 | 'HWADDR={hwaddr}\n' | ||
388 | 362 | 'MTU={mtu}\n' | ||
389 | 363 | 'ONBOOT=yes\n' | ||
390 | 364 | ).format( | ||
391 | 365 | device=cfg['name'], | ||
392 | 366 | hwaddr=cfg['mac_address'], | ||
393 | 367 | mtu=cfg.get('mtu', 1500) | ||
394 | 368 | ) | ||
395 | 369 | |||
396 | 370 | content += get_common_network_config(cfg) + \ | ||
397 | 371 | get_subnet_config(cfg['subnets']) | ||
398 | 372 | |||
399 | 373 | path = os.path.join(target, 'etc', 'sysconfig', | ||
400 | 374 | 'network-scripts', 'ifcfg-%s' % cfg['name']) | ||
401 | 375 | util.write_file(path, content) | ||
402 | 376 | |||
403 | 377 | |||
404 | 378 | def write_vlan_iface_config(cfg, target): | ||
405 | 379 | content = ( | ||
406 | 380 | '# Generated by MAAS fast-path installer\n' | ||
407 | 381 | 'DEVICE={device}\n' | ||
408 | 382 | 'MTU={mtu}\n' | ||
409 | 383 | 'ONBOOT=yes\n' | ||
410 | 384 | 'VLAN=yes\n' | ||
411 | 385 | ).format( | ||
412 | 386 | device=cfg['name'], | ||
413 | 387 | mtu=cfg.get('mtu', 1500) | ||
414 | 388 | ) | ||
415 | 389 | |||
416 | 390 | content += get_common_network_config(cfg) + \ | ||
417 | 391 | get_subnet_config(cfg['subnets']) | ||
418 | 392 | |||
419 | 393 | path = os.path.join(target, 'etc', 'sysconfig', | ||
420 | 394 | 'network-scripts', 'ifcfg-%s' % cfg['name']) | ||
421 | 395 | util.write_file(path, content) | ||
422 | 396 | |||
423 | 397 | |||
424 | 398 | def apply_networking(config, target): | ||
425 | 399 | network_config = config.get('network') | ||
426 | 400 | if not network_config: | ||
427 | 401 | LOG.warn("Unable to retrieve network configuration, will fallback " | ||
428 | 402 | "to configuring DHCP on boot device") | ||
429 | 403 | bootmac = get_boot_mac() | ||
430 | 404 | inames = get_interface_names() | ||
431 | 405 | iname = inames[bootmac.lower()] | ||
432 | 406 | network_config = { | ||
433 | 407 | 'config': [ | ||
434 | 408 | {'mac_address': bootmac, | ||
435 | 409 | 'name': iname, | ||
436 | 410 | 'type': 'physical', | ||
437 | 411 | 'subnets': [{'type': 'dhcp'}]}], | ||
438 | 412 | 'version': 1} | ||
439 | 413 | |||
440 | 414 | # Bring back ethN device naming on CentOS 6 | ||
441 | 415 | rename_ifaces(network_config, target) | ||
442 | 416 | |||
443 | 417 | for cfg in network_config['config']: | ||
444 | 418 | if cfg['type'] == 'nameserver': | ||
445 | 419 | write_resolv_conf(cfg, target) | ||
446 | 420 | if cfg['type'] == 'physical': | ||
447 | 421 | write_physical_iface_config(cfg, target) | ||
448 | 422 | if cfg['type'] == 'vlan': | ||
449 | 423 | write_vlan_iface_config(cfg, target) | ||
450 | 424 | |||
451 | 425 | |||
452 | 426 | def get_grub_root(target, dedicated_boot): | ||
453 | 427 | """Extracts the grub root (hdX,X) from the grub command. | ||
454 | 428 | |||
455 | 429 | This is used so the correct root device is used to install | ||
456 | 430 | stage1/stage2 boot loader. | ||
457 | 431 | |||
458 | 432 | Note: grub-install normally does all of this for you, but | ||
459 | 433 | since the grub is older, it has an issue with the ISCSI | ||
460 | 434 | target as /dev/sda and cannot enumarate it with the BIOS. | ||
461 | 435 | """ | ||
462 | 436 | stage1 = '/boot/grub/stage1' | ||
463 | 437 | if dedicated_boot: | ||
464 | 438 | stage1 = '/grub/stage1' | ||
465 | 439 | |||
466 | 440 | with util.RunInChroot(target) as in_chroot: | ||
467 | 441 | data = ('find {}\n' | ||
468 | 442 | 'quit\n').format(stage1).encode('utf-8') | ||
469 | 443 | out, err = in_chroot(['grub', '--batch'], | ||
470 | 444 | data=data, capture=True) | ||
471 | 445 | regex = re.search(r'^\s+(\(.+?\))$', out, re.MULTILINE) | ||
472 | 446 | return regex.groups()[0] | ||
473 | 105 | 447 | ||
474 | 106 | 448 | ||
475 | 107 | def strip_kernel_params(params, strip_params=[]): | 449 | def strip_kernel_params(params, strip_params=[]): |
476 | @@ -118,6 +460,33 @@ | |||
477 | 118 | return new_params | 460 | return new_params |
478 | 119 | 461 | ||
479 | 120 | 462 | ||
480 | 463 | def get_extra_kernel_parameters(): | ||
481 | 464 | """Extracts the extra kernel commands from /proc/cmdline | ||
482 | 465 | that should be placed onto the host. | ||
483 | 466 | |||
484 | 467 | Any command following the '---' entry should be placed | ||
485 | 468 | onto the host. | ||
486 | 469 | """ | ||
487 | 470 | cmdline = read_file('/proc/cmdline') | ||
488 | 471 | cmdline = cmdline.split() | ||
489 | 472 | if '---' not in cmdline: | ||
490 | 473 | return [] | ||
491 | 474 | idx = cmdline.index('---') + 1 | ||
492 | 475 | if idx >= len(cmdline) + 1: | ||
493 | 476 | return [] | ||
494 | 477 | return strip_kernel_params( | ||
495 | 478 | cmdline[idx:], | ||
496 | 479 | strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF=']) | ||
497 | 480 | |||
498 | 481 | |||
499 | 482 | def get_root_info(target): | ||
500 | 483 | """Returns the root partitions information.""" | ||
501 | 484 | rootpath = block.get_devices_for_mp(target)[0] | ||
502 | 485 | rootdev = os.path.basename(rootpath) | ||
503 | 486 | blocks = block._lsblock() | ||
504 | 487 | return blocks[rootdev] | ||
505 | 488 | |||
506 | 489 | |||
507 | 121 | def get_boot_file(target, filename): | 490 | def get_boot_file(target, filename): |
508 | 122 | """Return the full filename of file in /boot on target.""" | 491 | """Return the full filename of file in /boot on target.""" |
509 | 123 | boot_dir = os.path.join(target, 'boot') | 492 | boot_dir = os.path.join(target, 'boot') |
510 | @@ -125,81 +494,163 @@ | |||
511 | 125 | fname | 494 | fname |
512 | 126 | for fname in os.listdir(boot_dir) | 495 | for fname in os.listdir(boot_dir) |
513 | 127 | if fname.startswith(filename) | 496 | if fname.startswith(filename) |
515 | 128 | ] | 497 | ] |
516 | 129 | if not files: | 498 | if not files: |
517 | 130 | return None | 499 | return None |
518 | 131 | return files[0] | 500 | return files[0] |
519 | 132 | 501 | ||
520 | 133 | 502 | ||
522 | 134 | def write_grub_conf(target, grub_root, extra=[]): | 503 | def write_grub_conf(target, dedicated_boot, extra=[]): |
523 | 135 | """Writes a new /boot/grub/grub.conf with the correct | 504 | """Writes a new /boot/grub/grub.conf with the correct |
524 | 136 | boot arguments.""" | 505 | boot arguments.""" |
525 | 506 | grub_path = os.path.join(target, 'boot', 'grub', 'grub.conf') | ||
526 | 507 | util.write_file(grub_path, '# UEFI system, see /etc/grub.conf\n') | ||
527 | 508 | |||
528 | 509 | if util.is_uefi_bootable(): | ||
529 | 510 | grub_path = os.path.join(target, 'boot', 'efi', | ||
530 | 511 | 'EFI', 'redhat', 'grub.conf') | ||
531 | 512 | |||
532 | 513 | grub_root = get_grub_root(target, dedicated_boot) | ||
533 | 137 | root_info = get_root_info(target) | 514 | root_info = get_root_info(target) |
534 | 138 | grub_path = os.path.join(target, 'boot', 'grub', 'grub.conf') | ||
535 | 139 | extra_opts = ' '.join(extra) | 515 | extra_opts = ' '.join(extra) |
536 | 140 | vmlinuz = get_boot_file(target, 'vmlinuz') | 516 | vmlinuz = get_boot_file(target, 'vmlinuz') |
537 | 141 | initrd = get_boot_file(target, 'initramfs') | 517 | initrd = get_boot_file(target, 'initramfs') |
599 | 142 | with open(grub_path, 'w') as stream: | 518 | |
600 | 143 | stream.write( | 519 | if not dedicated_boot: |
601 | 144 | GRUB_CONF.format( | 520 | vmlinuz = 'boot/' + vmlinuz |
602 | 145 | grub_root=grub_root, | 521 | initrd = 'boot/' + initrd |
603 | 146 | vmlinuz=vmlinuz, | 522 | |
604 | 147 | initrd=initrd, | 523 | util.write_file(grub_path, |
605 | 148 | root_uuid=root_info['UUID'], | 524 | GRUB_CONF.format( |
606 | 149 | extra_opts=extra_opts) + '\n') | 525 | grub_root=grub_root, |
607 | 150 | 526 | vmlinuz=vmlinuz, | |
608 | 151 | 527 | initrd=initrd, | |
609 | 152 | def get_extra_kernel_parameters(): | 528 | root_uuid=root_info['UUID'], |
610 | 153 | """Extracts the extra kernel commands from /proc/cmdline | 529 | extra_opts=extra_opts) + '\n') |
611 | 154 | that should be placed onto the host. | 530 | |
612 | 155 | 531 | # FIXME. replace() is unreliable here if target ends with / | |
613 | 156 | Any command following the '--' entry should be placed | 532 | # Update /etc/grub.conf link as it is used by kernel install scripts |
614 | 157 | onto the host. | 533 | util.subp(['ln', '-sf', grub_path.replace(target, '..'), |
615 | 158 | """ | 534 | '/etc/grub.conf'], target=target) |
616 | 159 | cmdline = read_file('/proc/cmdline') | 535 | |
617 | 160 | cmdline = cmdline.split() | 536 | |
618 | 161 | if '--' not in cmdline: | 537 | def get_uefi_partition(): |
619 | 162 | return [] | 538 | """Return the UEFI partition.""" |
620 | 163 | idx = cmdline.index('--') + 1 | 539 | for _, value in block._lsblock().items(): |
621 | 164 | if idx >= len(cmdline) + 1: | 540 | if value['LABEL'] == 'uefi-boot': |
622 | 165 | return [] | 541 | return value |
623 | 166 | return strip_kernel_params( | 542 | return None |
624 | 167 | cmdline[idx:], | 543 | |
625 | 168 | strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF=']) | 544 | |
626 | 169 | 545 | def install_bootloader(target, device=None): | |
627 | 170 | 546 | """Installs bootloader to the device.""" | |
628 | 171 | def get_grub_root(target): | 547 | cmd = [] |
629 | 172 | """Extracts the grub root (hdX,X) from the grub command. | 548 | if util.is_uefi_bootable(): |
630 | 173 | 549 | uefi_dev = get_uefi_partition()['device_path'] | |
631 | 174 | This is used so the correct root device is used to install | 550 | disk, part = block.get_blockdev_for_partition(uefi_dev) |
632 | 175 | stage1/stage2 boot loader. | 551 | cmd = ['efibootmgr', '-v', '-c', '-w', '-L', 'centos', |
633 | 176 | 552 | '-d', disk, '-p', part, '-l', r'\EFI\redhat\grub.efi'] | |
634 | 177 | Note: grub-install normally does all of this for you, but | 553 | else: |
635 | 178 | since the grub is older, it has an issue with the ISCSI | 554 | # Legacy grub-install uses df output, which itself uses /etc/mtab |
636 | 179 | target as /dev/sda and cannot enumarate it with the BIOS. | 555 | with util.RunInChroot(target) as in_chroot: |
637 | 180 | """ | 556 | in_chroot(['cp', '-f', '/proc/mounts', '/etc/mtab']) |
638 | 181 | with util.RunInChroot(target) as in_chroot: | 557 | cmd = ['grub-install', '--recheck', device] |
639 | 182 | data = '\n'.join([ | 558 | |
640 | 183 | 'find /boot/grub/stage1', | 559 | with util.RunInChroot(target) as in_chroot: |
641 | 184 | 'quit', | 560 | in_chroot(cmd) |
642 | 185 | ]).encode('utf-8') | 561 | |
643 | 186 | out, err = in_chroot(['grub', '--batch'], | 562 | |
644 | 187 | data=data, capture=True) | 563 | def setup_grub(cfg, target): |
645 | 188 | regex = re.search('^\s+(\(.+?\))$', out, re.MULTILINE) | 564 | # target is the path to the mounted filesystem |
646 | 189 | return regex.groups()[0] | 565 | |
647 | 190 | 566 | # FIXME: these methods need moving to curtin.block | |
648 | 191 | 567 | # and using them from there rather than commands.block_meta | |
649 | 192 | def grub_install(target, root): | 568 | from curtin.commands.block_meta import (extract_storage_ordered_dict, |
650 | 193 | """Installs grub onto the root.""" | 569 | get_path_to_storage_volume) |
651 | 194 | root_dev = root.split(',')[0] + ')' | 570 | |
652 | 195 | with util.RunInChroot(target) as in_chroot: | 571 | grubcfg = cfg.get('grub', {}) |
653 | 196 | data = '\n'.join([ | 572 | |
654 | 197 | 'root %s' % root, | 573 | # copy legacy top level name |
655 | 198 | 'setup %s' % root_dev, | 574 | if 'grub_install_devices' in cfg and 'install_devices' not in grubcfg: |
656 | 199 | 'quit', | 575 | grubcfg['install_devices'] = cfg['grub_install_devices'] |
657 | 200 | ]).encode('utf-8') | 576 | |
658 | 201 | in_chroot(['grub', '--batch'], | 577 | LOG.debug("setup grub on target %s", target) |
659 | 202 | data=data) | 578 | # if there is storage config, look for devices tagged with 'grub_device' |
660 | 579 | storage_cfg_odict = None | ||
661 | 580 | try: | ||
662 | 581 | storage_cfg_odict = extract_storage_ordered_dict(cfg) | ||
663 | 582 | except ValueError as e: | ||
664 | 583 | pass | ||
665 | 584 | |||
666 | 585 | if storage_cfg_odict: | ||
667 | 586 | storage_grub_devices = [] | ||
668 | 587 | for item_id, item in storage_cfg_odict.items(): | ||
669 | 588 | if not item.get('grub_device'): | ||
670 | 589 | continue | ||
671 | 590 | LOG.debug("checking: %s", item) | ||
672 | 591 | storage_grub_devices.append( | ||
673 | 592 | get_path_to_storage_volume(item_id, storage_cfg_odict)) | ||
674 | 593 | if len(storage_grub_devices) > 0: | ||
675 | 594 | grubcfg['install_devices'] = storage_grub_devices | ||
676 | 595 | |||
677 | 596 | LOG.debug("install_devices: %s", grubcfg.get('install_devices')) | ||
678 | 597 | if 'install_devices' in grubcfg: | ||
679 | 598 | instdevs = grubcfg.get('install_devices') | ||
680 | 599 | if isinstance(instdevs, str): | ||
681 | 600 | instdevs = [instdevs] | ||
682 | 601 | if instdevs is None: | ||
683 | 602 | LOG.debug("grub installation disabled by config") | ||
684 | 603 | else: | ||
685 | 604 | # If there were no install_devices found then we try to do the right | ||
686 | 605 | # thing. That right thing is basically installing on all block | ||
687 | 606 | # devices that are mounted. On powerpc, though it means finding PrEP | ||
688 | 607 | # partitions. | ||
689 | 608 | devs = block.get_devices_for_mp(target) | ||
690 | 609 | blockdevs = set() | ||
691 | 610 | for maybepart in devs: | ||
692 | 611 | try: | ||
693 | 612 | (blockdev, part) = block.get_blockdev_for_partition(maybepart) | ||
694 | 613 | blockdevs.add(blockdev) | ||
695 | 614 | except ValueError as e: | ||
696 | 615 | # if there is no syspath for this device such as a lvm | ||
697 | 616 | # or raid device, then a ValueError is raised here. | ||
698 | 617 | LOG.debug("failed to find block device for %s", maybepart) | ||
699 | 618 | instdevs = list(blockdevs) | ||
700 | 619 | |||
701 | 620 | # UEFI requires additional packages | ||
702 | 621 | #if util.is_uefi_bootable(): | ||
703 | 622 | # pkgs = ['efibootmgr'] | ||
704 | 623 | # install_packages(pkgs, target=target) | ||
705 | 624 | |||
706 | 625 | # CentOS will not assemble MD devices on boot without rd.md.uuid=MD_UUID | ||
707 | 626 | # kernel parameters | ||
708 | 627 | mdmap = {} | ||
709 | 628 | rdmduuids = [] | ||
710 | 629 | try: | ||
711 | 630 | mdmap = mdadm.md_read_run_mdadm_map() | ||
712 | 631 | for md in mdmap: | ||
713 | 632 | rdmduuids.append("rd.md.uuid=%s" % mdadm.md_get_uuid(mdmap[md][2])) | ||
714 | 633 | except ValueError as e: | ||
715 | 634 | pass | ||
716 | 635 | |||
717 | 636 | dedicated_boot = False | ||
718 | 637 | if block.get_devices_for_mp(os.path.join(target, 'boot')): | ||
719 | 638 | dedicated_boot = True | ||
720 | 639 | write_grub_conf(target, dedicated_boot, extra=get_extra_kernel_parameters() + rdmduuids) | ||
721 | 640 | |||
722 | 641 | if instdevs: | ||
723 | 642 | instdevs = [block.get_dev_name_entry(i)[1] for i in instdevs] | ||
724 | 643 | else: | ||
725 | 644 | instdevs = ["none"] | ||
726 | 645 | |||
727 | 646 | LOG.debug("installing grub to %s", instdevs) | ||
728 | 647 | |||
729 | 648 | if util.is_uefi_bootable(): | ||
730 | 649 | if grubcfg.get('update_nvram', False): | ||
731 | 650 | install_bootloader(target) | ||
732 | 651 | else: | ||
733 | 652 | for dev in instdevs: | ||
734 | 653 | install_bootloader(target, dev) | ||
735 | 203 | 654 | ||
736 | 204 | 655 | ||
737 | 205 | def set_autorelabel(target): | 656 | def set_autorelabel(target): |
738 | @@ -214,128 +665,75 @@ | |||
739 | 214 | open(path, 'a').close() | 665 | open(path, 'a').close() |
740 | 215 | 666 | ||
741 | 216 | 667 | ||
742 | 217 | def get_boot_mac(): | ||
743 | 218 | """Return the mac address of the booting interface.""" | ||
744 | 219 | cmdline = read_file('/proc/cmdline') | ||
745 | 220 | cmdline = cmdline.split() | ||
746 | 221 | try: | ||
747 | 222 | bootif = [ | ||
748 | 223 | option | ||
749 | 224 | for option in cmdline | ||
750 | 225 | if option.startswith('BOOTIF') | ||
751 | 226 | ][0] | ||
752 | 227 | except IndexError: | ||
753 | 228 | return None | ||
754 | 229 | _, mac = bootif.split('=') | ||
755 | 230 | mac = mac.split('-')[1:] | ||
756 | 231 | return ':'.join(mac) | ||
757 | 232 | |||
758 | 233 | |||
759 | 234 | def get_interface_names(): | ||
760 | 235 | """Return a dictionary mapping mac addresses to interface names.""" | ||
761 | 236 | sys_path = "/sys/class/net" | ||
762 | 237 | ifaces = {} | ||
763 | 238 | for iname in os.listdir(sys_path): | ||
764 | 239 | mac = read_file(os.path.join(sys_path, iname, "address")) | ||
765 | 240 | mac = mac.strip().lower() | ||
766 | 241 | ifaces[mac] = iname | ||
767 | 242 | return ifaces | ||
768 | 243 | |||
769 | 244 | |||
770 | 245 | def get_ipv4_config(iface, data): | ||
771 | 246 | """Returns the contents of the interface file for ipv4.""" | ||
772 | 247 | config = [ | ||
773 | 248 | 'TYPE="Ethernet"', | ||
774 | 249 | 'NM_CONTROLLED="no"', | ||
775 | 250 | 'USERCTL="yes"', | ||
776 | 251 | ] | ||
777 | 252 | if 'hwaddress' in data: | ||
778 | 253 | config.append('HWADDR="%s"' % data['hwaddress']) | ||
779 | 254 | else: | ||
780 | 255 | # Last ditch effort, use the device name, it probably won't match | ||
781 | 256 | # though! | ||
782 | 257 | config.append('DEVICE="%s"' % iface) | ||
783 | 258 | if data['auto']: | ||
784 | 259 | config.append('ONBOOT="yes"') | ||
785 | 260 | else: | ||
786 | 261 | config.append('ONBOOT="no"') | ||
787 | 262 | |||
788 | 263 | method = data['method'] | ||
789 | 264 | if method == 'dhcp': | ||
790 | 265 | config.append('BOOTPROTO="dhcp"') | ||
791 | 266 | config.append('PEERDNS="yes"') | ||
792 | 267 | config.append('PERSISTENT_DHCLIENT="1"') | ||
793 | 268 | if 'hostname' in data: | ||
794 | 269 | config.append('DHCP_HOSTNAME="%s"' % data['hostname']) | ||
795 | 270 | elif method == 'static': | ||
796 | 271 | config.append('BOOTPROTO="none"') | ||
797 | 272 | config.append('IPADDR="%s"' % data['address']) | ||
798 | 273 | config.append('NETMASK="%s"' % data['netmask']) | ||
799 | 274 | if 'broadcast' in data: | ||
800 | 275 | config.append('BROADCAST="%s"' % data['broadcast']) | ||
801 | 276 | if 'gateway' in data: | ||
802 | 277 | config.append('GATEWAY="%s"' % data['gateway']) | ||
803 | 278 | elif method == 'manual': | ||
804 | 279 | config.append('BOOTPROTO="none"') | ||
805 | 280 | return '\n'.join(config) | ||
806 | 281 | |||
807 | 282 | |||
808 | 283 | def write_interface_config(target, iface, data): | ||
809 | 284 | """Writes config for interface.""" | ||
810 | 285 | family = data['family'] | ||
811 | 286 | if family != "inet": | ||
812 | 287 | # Only supporting ipv4 currently | ||
813 | 288 | print( | ||
814 | 289 | "WARN: unsupported family %s, " | ||
815 | 290 | "failed to configure interface: %s" (family, iface)) | ||
816 | 291 | return | ||
817 | 292 | config = get_ipv4_config(iface, data) | ||
818 | 293 | path = os.path.join( | ||
819 | 294 | target, 'etc', 'sysconfig', 'network-scripts', 'ifcfg-%s' % iface) | ||
820 | 295 | with open(path, 'w') as stream: | ||
821 | 296 | stream.write(config + '\n') | ||
822 | 297 | |||
823 | 298 | |||
824 | 299 | def write_network_config(target, mac): | ||
825 | 300 | """Write network configuration for the given MAC address.""" | ||
826 | 301 | inames = get_interface_names() | ||
827 | 302 | iname = inames[mac.lower()] | ||
828 | 303 | write_interface_config( | ||
829 | 304 | target, iname, { | ||
830 | 305 | 'family': 'inet', | ||
831 | 306 | 'hwaddress': mac.upper(), | ||
832 | 307 | 'auto': True, | ||
833 | 308 | 'method': 'dhcp' | ||
834 | 309 | }) | ||
835 | 310 | |||
836 | 311 | |||
837 | 312 | def main(): | 668 | def main(): |
838 | 313 | state = util.load_command_environment() | 669 | state = util.load_command_environment() |
839 | 670 | |||
840 | 314 | target = state['target'] | 671 | target = state['target'] |
841 | 315 | if target is None: | 672 | if target is None: |
862 | 316 | print("Target was not provided in the environment.") | 673 | sys.stderr.write("Target was not provided in the environment.") |
863 | 317 | sys.exit(1) | 674 | sys.exit(1) |
864 | 318 | fstab = state['fstab'] | 675 | |
865 | 319 | if fstab is None: | 676 | cfg = config.load_command_config(None, state) |
866 | 320 | print("/etc/fstab output was not provided in the environment.") | 677 | stack_prefix = state.get('report_stack_prefix', '') |
867 | 321 | sys.exit(1) | 678 | |
868 | 322 | bootmac = get_boot_mac() | 679 | with events.ReportEventStack( |
869 | 323 | if bootmac is None: | 680 | name=stack_prefix, reporting_enabled=True, level="INFO", |
870 | 324 | print("Unable to determine boot interface.") | 681 | description="writing config files"): |
871 | 325 | sys.exit(1) | 682 | curthooks.write_files(cfg, target) |
872 | 326 | devices = get_block_devices(target) | 683 | |
873 | 327 | if not devices: | 684 | # Default CentOS image does not contain some packages that may be necessary |
874 | 328 | print("Unable to find block device for: %s" % target) | 685 | install_missing_packages(cfg, target) |
875 | 329 | sys.exit(1) | 686 | |
876 | 330 | 687 | # If a mdadm.conf file was created by block_meta then it needs to be copied | |
877 | 331 | write_fstab(target, fstab) | 688 | # onto the target system |
878 | 332 | 689 | mdadm_location = os.path.join(os.path.split(state['fstab'])[0], | |
879 | 333 | grub_root = get_grub_root(target) | 690 | "mdadm.conf") |
880 | 334 | write_grub_conf(target, grub_root, extra=get_extra_kernel_parameters()) | 691 | if os.path.exists(mdadm_location): |
881 | 335 | grub_install(target, grub_root) | 692 | copy_mdadm_conf(mdadm_location, target) |
882 | 693 | |||
883 | 694 | with events.ReportEventStack( | ||
884 | 695 | name=stack_prefix, reporting_enabled=True, level="INFO", | ||
885 | 696 | description="setting up swap"): | ||
886 | 697 | curthooks.add_swap(cfg, target, state.get('fstab')) | ||
887 | 698 | |||
888 | 699 | with events.ReportEventStack( | ||
889 | 700 | name=stack_prefix, reporting_enabled=True, level="INFO", | ||
890 | 701 | description="writing etc/fstab"): | ||
891 | 702 | write_fstab(target, state.get('fstab')) | ||
892 | 703 | |||
893 | 704 | # TODO. There's a multipath implementation on Ubuntu in curtin/curthooks.py | ||
894 | 705 | # that should be reimplemented here. Skipping for now. | ||
895 | 706 | |||
896 | 707 | with events.ReportEventStack( | ||
897 | 708 | name=stack_prefix, reporting_enabled=True, level="INFO", | ||
898 | 709 | description="updating packages on target system"): | ||
899 | 710 | system_upgrade(cfg, target) | ||
900 | 711 | |||
901 | 712 | # TODO. If a crypttab file was created by block_meta than it needs to be | ||
902 | 713 | # copied onto the target system, and update_initramfs() (well, alternative | ||
903 | 714 | # for CentOS actually) needs to be run, so that the cryptsetup hooks are | ||
904 | 715 | # properly configured on the installed system and it will be able to open | ||
905 | 716 | # encrypted volumes at boot. Skipping for now. | ||
906 | 717 | |||
907 | 718 | with events.ReportEventStack( | ||
908 | 719 | name=stack_prefix, reporting_enabled=True, level="INFO", | ||
909 | 720 | description="apply networking"): | ||
910 | 721 | apply_networking(cfg, target) | ||
911 | 722 | |||
912 | 723 | # If udev dname rules were created, copy them to target | ||
913 | 724 | udev_rules_d = os.path.join(state['scratch'], "rules.d") | ||
914 | 725 | if os.path.isdir(udev_rules_d): | ||
915 | 726 | curthooks.copy_dname_rules(udev_rules_d, target) | ||
916 | 727 | |||
917 | 728 | with events.ReportEventStack( | ||
918 | 729 | name=stack_prefix, reporting_enabled=True, level="INFO", | ||
919 | 730 | description="updating packages on target system"): | ||
920 | 731 | setup_grub(cfg, target) | ||
921 | 336 | 732 | ||
922 | 337 | set_autorelabel(target) | 733 | set_autorelabel(target) |
924 | 338 | write_network_config(target, bootmac) | 734 | update_initramfs(target) |
925 | 735 | |||
926 | 736 | sys.exit(0) | ||
927 | 339 | 737 | ||
928 | 340 | 738 | ||
929 | 341 | if __name__ == "__main__": | 739 | if __name__ == "__main__": |
930 | 342 | 740 | ||
931 | === modified file 'curtin/centos7/curtin-hooks.py' | |||
932 | --- curtin/centos7/curtin-hooks.py 2016-05-11 18:47:46 +0000 | |||
933 | +++ curtin/centos7/curtin-hooks.py 2017-01-09 15:19:25 +0000 | |||
934 | @@ -1,20 +1,17 @@ | |||
935 | 1 | #!/usr/bin/env python | 1 | #!/usr/bin/env python |
942 | 2 | 2 | # coding: utf-8 | |
937 | 3 | from __future__ import ( | ||
938 | 4 | absolute_import, | ||
939 | 5 | print_function, | ||
940 | 6 | unicode_literals, | ||
941 | 7 | ) | ||
943 | 8 | 3 | ||
944 | 9 | import codecs | 4 | import codecs |
945 | 5 | import ipaddress | ||
946 | 10 | import os | 6 | import os |
947 | 7 | import shutil | ||
948 | 11 | import sys | 8 | import sys |
949 | 12 | import shutil | ||
950 | 13 | 9 | ||
955 | 14 | from curtin import ( | 10 | from curtin import block, config, util |
956 | 15 | block, | 11 | from curtin.block import mdadm |
957 | 16 | util, | 12 | from curtin.commands import curthooks |
958 | 17 | ) | 13 | from curtin.log import LOG |
959 | 14 | from curtin.reporter import events | ||
960 | 18 | 15 | ||
961 | 19 | 16 | ||
962 | 20 | """ | 17 | """ |
963 | @@ -24,60 +21,142 @@ | |||
964 | 24 | 21 | ||
965 | 25 | - Legacy boot | 22 | - Legacy boot |
966 | 26 | - UEFI boot | 23 | - UEFI boot |
968 | 27 | - DHCP of BOOTIF | 24 | - Simple and MD RAID layouts |
969 | 25 | - IPv4 DHCP and static networking, IP aliases on physical and VLAN interfaces | ||
970 | 28 | 26 | ||
971 | 29 | Not Supported: | 27 | Not Supported: |
972 | 30 | 28 | ||
974 | 31 | - Multiple network configration | 29 | - Bond, bridge, route configurations |
975 | 32 | - IPv6 | 30 | - IPv6 |
991 | 33 | """ | 31 | - Disk encryption |
992 | 34 | 32 | - bcache | |
993 | 35 | FSTAB_PREPEND = """\ | 33 | - LVM |
994 | 36 | # | 34 | - Multipath |
995 | 37 | # /etc/fstab | 35 | """ |
996 | 38 | # Created by MAAS fast-path installer. | 36 | |
982 | 39 | # | ||
983 | 40 | # Accessible filesystems, by reference, are maintained under '/dev/disk' | ||
984 | 41 | # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info | ||
985 | 42 | # | ||
986 | 43 | """ | ||
987 | 44 | |||
988 | 45 | FSTAB_UEFI = """\ | ||
989 | 46 | LABEL=uefi-boot /boot/efi vfat defaults 0 0 | ||
990 | 47 | """ | ||
997 | 48 | 37 | ||
998 | 49 | GRUB_PREPEND = """\ | 38 | GRUB_PREPEND = """\ |
999 | 50 | # Set by MAAS fast-path installer. | 39 | # Set by MAAS fast-path installer. |
1001 | 51 | GRUB_TIMEOUT=0 | 40 | GRUB_TIMEOUT=1 |
1002 | 52 | GRUB_TERMINAL_OUTPUT=console | 41 | GRUB_TERMINAL_OUTPUT=console |
1003 | 53 | GRUB_DISABLE_OS_PROBER=true | 42 | GRUB_DISABLE_OS_PROBER=true |
1004 | 54 | """ | 43 | """ |
1005 | 55 | 44 | ||
1006 | 56 | 45 | ||
1031 | 57 | def get_block_devices(target): | 46 | def get_installed_packages(target=None): |
1032 | 58 | """Returns list of block devices for the given target.""" | 47 | (out, _) = util.subp(['rpm', '-qa', '--qf', '"%{NAME}\n"'], |
1033 | 59 | devs = block.get_devices_for_mp(target) | 48 | target=target, capture=True) |
1034 | 60 | blockdevs = set() | 49 | return set(out.splitlines()) |
1035 | 61 | for maybepart in devs: | 50 | |
1036 | 62 | (blockdev, part) = block.get_blockdev_for_partition(maybepart) | 51 | |
1037 | 63 | blockdevs.add(blockdev) | 52 | def install_packages(pkglist, target): |
1038 | 64 | return list(blockdevs) | 53 | if isinstance(pkglist, str): |
1039 | 65 | 54 | pkglist = [pkglist] | |
1040 | 66 | 55 | # Does not work – hosts are not resolved | |
1041 | 67 | def get_root_info(target): | 56 | # return util.subp(['yum', '-q', '-y', 'install'] + pkglist, target=target, capture=True) |
1042 | 68 | """Returns the root partitions information.""" | 57 | with util.RunInChroot(target) as in_chroot: |
1043 | 69 | rootpath = block.get_devices_for_mp(target)[0] | 58 | in_chroot(['yum', '-q', '-y', 'install'] + pkglist) |
1044 | 70 | rootdev = os.path.basename(rootpath) | 59 | |
1045 | 71 | blocks = block._lsblock() | 60 | |
1046 | 72 | return blocks[rootdev] | 61 | def install_missing_packages(cfg, target): |
1047 | 73 | 62 | ''' describe which operation types will require specific packages | |
1048 | 74 | 63 | ||
1049 | 75 | def get_uefi_partition(): | 64 | 'custom_config_key': { |
1050 | 76 | """Return the UEFI partition.""" | 65 | 'pkg1': ['op_name_1', 'op_name_2', ...] |
1051 | 77 | for _, value in block._lsblock().items(): | 66 | } |
1052 | 78 | if value['LABEL'] == 'uefi-boot': | 67 | ''' |
1053 | 79 | return value | 68 | custom_configs = { |
1054 | 80 | return None | 69 | 'storage': { |
1055 | 70 | 'lvm2': ['lvm_volgroup', 'lvm_partition'], | ||
1056 | 71 | 'mdadm': ['raid'], | ||
1057 | 72 | # XXX. At the moment there are no official packages for bcache on CentOS | ||
1058 | 73 | #'bcache-tools': ['bcache'] | ||
1059 | 74 | }, | ||
1060 | 75 | 'network': { | ||
1061 | 76 | 'bridge-utils': ['bridge'] | ||
1062 | 77 | }, | ||
1063 | 78 | } | ||
1064 | 79 | |||
1065 | 80 | format_configs = { | ||
1066 | 81 | 'xfsprogs': ['xfs'], | ||
1067 | 82 | 'e2fsprogs': ['ext2', 'ext3', 'ext4'], | ||
1068 | 83 | 'btrfs-tools': ['btrfs'], | ||
1069 | 84 | } | ||
1070 | 85 | |||
1071 | 86 | needed_packages = [] | ||
1072 | 87 | installed_packages = get_installed_packages(target) | ||
1073 | 88 | for cust_cfg, pkg_reqs in custom_configs.items(): | ||
1074 | 89 | if cust_cfg not in cfg: | ||
1075 | 90 | continue | ||
1076 | 91 | |||
1077 | 92 | all_types = set( | ||
1078 | 93 | operation['type'] | ||
1079 | 94 | for operation in cfg[cust_cfg]['config'] | ||
1080 | 95 | ) | ||
1081 | 96 | for pkg, types in pkg_reqs.items(): | ||
1082 | 97 | if set(types).intersection(all_types) and \ | ||
1083 | 98 | pkg not in installed_packages: | ||
1084 | 99 | needed_packages.append(pkg) | ||
1085 | 100 | |||
1086 | 101 | format_types = set( | ||
1087 | 102 | [operation['fstype'] | ||
1088 | 103 | for operation in cfg[cust_cfg]['config'] | ||
1089 | 104 | if operation['type'] == 'format']) | ||
1090 | 105 | for pkg, fstypes in format_configs.items(): | ||
1091 | 106 | if set(fstypes).intersection(format_types) and \ | ||
1092 | 107 | pkg not in installed_packages: | ||
1093 | 108 | needed_packages.append(pkg) | ||
1094 | 109 | |||
1095 | 110 | if needed_packages: | ||
1096 | 111 | state = util.load_command_environment() | ||
1097 | 112 | with events.ReportEventStack( | ||
1098 | 113 | name=state.get('report_stack_prefix'), | ||
1099 | 114 | reporting_enabled=True, level="INFO", | ||
1100 | 115 | description="Installing packages on target system: " + | ||
1101 | 116 | str(needed_packages)): | ||
1102 | 117 | install_packages(needed_packages, target=target) | ||
1103 | 118 | |||
1104 | 119 | |||
1105 | 120 | def copy_mdadm_conf(mdadm_conf, target): | ||
1106 | 121 | if not mdadm_conf: | ||
1107 | 122 | LOG.warn("mdadm config must be specified, not copying") | ||
1108 | 123 | return | ||
1109 | 124 | |||
1110 | 125 | LOG.info("copying mdadm.conf into target") | ||
1111 | 126 | shutil.copy(mdadm_conf, os.path.sep.join([target, 'etc/mdadm.conf'])) | ||
1112 | 127 | |||
1113 | 128 | |||
1114 | 129 | def update_initramfs(target=None): | ||
1115 | 130 | path = os.path.join( | ||
1116 | 131 | target, 'etc', 'dracut.conf.d', 'local.conf') | ||
1117 | 132 | with open(path, 'w') as stream: | ||
1118 | 133 | stream.write('mdadmconf="yes"' + '\n' | ||
1119 | 134 | 'lvmconf="yes"' + '\n') | ||
1120 | 135 | |||
1121 | 136 | with util.RunInChroot(target) as in_chroot: | ||
1122 | 137 | in_chroot(['dracut', '-f', '--regenerate-all']) | ||
1123 | 138 | |||
1124 | 139 | |||
1125 | 140 | def system_upgrade(cfg, target): | ||
1126 | 141 | """run yum upgrade in target. | ||
1127 | 142 | |||
1128 | 143 | config: | ||
1129 | 144 | system_upgrade: | ||
1130 | 145 | enabled: False | ||
1131 | 146 | |||
1132 | 147 | """ | ||
1133 | 148 | mycfg = {'system_upgrade': {'enabled': False}} | ||
1134 | 149 | config.merge_config(mycfg, cfg) | ||
1135 | 150 | mycfg = mycfg.get('system_upgrade') | ||
1136 | 151 | if not isinstance(mycfg, dict): | ||
1137 | 152 | LOG.debug("system_upgrade disabled by config. entry not a dict.") | ||
1138 | 153 | return | ||
1139 | 154 | |||
1140 | 155 | if not config.value_as_boolean(mycfg.get('enabled', True)): | ||
1141 | 156 | LOG.debug("system_upgrade disabled by config.") | ||
1142 | 157 | return | ||
1143 | 158 | |||
1144 | 159 | util.subp(['yum', '-q', '-y', 'upgrade'], target=target, capture=True) | ||
1145 | 81 | 160 | ||
1146 | 82 | 161 | ||
1147 | 83 | def read_file(path): | 162 | def read_file(path): |
1148 | @@ -86,16 +165,178 @@ | |||
1149 | 86 | return stream.read() | 165 | return stream.read() |
1150 | 87 | 166 | ||
1151 | 88 | 167 | ||
1162 | 89 | def write_fstab(target, curtin_fstab): | 168 | def get_boot_mac(): |
1163 | 90 | """Writes the new fstab, using the fstab provided | 169 | """Return the mac address of the booting interface.""" |
1164 | 91 | from curtin.""" | 170 | cmdline = read_file('/proc/cmdline') |
1165 | 92 | fstab_path = os.path.join(target, 'etc', 'fstab') | 171 | cmdline = cmdline.split() |
1166 | 93 | fstab_data = read_file(curtin_fstab) | 172 | try: |
1167 | 94 | with open(fstab_path, 'w') as stream: | 173 | bootif = [ |
1168 | 95 | stream.write(FSTAB_PREPEND) | 174 | option |
1169 | 96 | stream.write(fstab_data) | 175 | for option in cmdline |
1170 | 97 | if util.is_uefi_bootable(): | 176 | if option.startswith('BOOTIF') |
1171 | 98 | stream.write(FSTAB_UEFI) | 177 | ][0] |
1172 | 178 | except IndexError: | ||
1173 | 179 | return None | ||
1174 | 180 | _, mac = bootif.split('=') | ||
1175 | 181 | mac = mac.split('-')[1:] | ||
1176 | 182 | return ':'.join(mac) | ||
1177 | 183 | |||
1178 | 184 | |||
1179 | 185 | def get_interface_names(): | ||
1180 | 186 | """Return a dictionary mapping mac addresses to interface names.""" | ||
1181 | 187 | sys_path = "/sys/class/net" | ||
1182 | 188 | ifaces = {} | ||
1183 | 189 | for iname in os.listdir(sys_path): | ||
1184 | 190 | mac = read_file(os.path.join(sys_path, iname, "address")) | ||
1185 | 191 | mac = mac.strip().lower() | ||
1186 | 192 | ifaces[mac] = iname | ||
1187 | 193 | return ifaces | ||
1188 | 194 | |||
1189 | 195 | |||
1190 | 196 | def get_subnet_config(subnets): | ||
1191 | 197 | content = '' | ||
1192 | 198 | for idx, subnet in enumerate(subnets): | ||
1193 | 199 | if subnet['type'] in ('static', 'static4'): | ||
1194 | 200 | if idx == 0: | ||
1195 | 201 | idx = '' | ||
1196 | 202 | iface = ipaddress.ip_interface(subnet['address']) | ||
1197 | 203 | content += 'IPADDR{}={}\n'.format(idx, str(iface.ip)) | ||
1198 | 204 | content += 'NETMASK{}={}\n'.format( | ||
1199 | 205 | idx, subnet.get('netmask', str(iface.netmask))) | ||
1200 | 206 | return content | ||
1201 | 207 | |||
1202 | 208 | |||
1203 | 209 | def get_common_network_config(cfg): | ||
1204 | 210 | # You can not have more than one GATEWAY, BOOTPROTO, DNS1, DNS2 and DOMAIN | ||
1205 | 211 | # settings per interface in RHEL and clones. Will run through subnets | ||
1206 | 212 | # section ensuring that only one of these makes to interface configuration. | ||
1207 | 213 | content = '' | ||
1208 | 214 | bootproto = 'dhcp' | ||
1209 | 215 | defaultgw = None | ||
1210 | 216 | resolvers = [] | ||
1211 | 217 | search_domains = [] | ||
1212 | 218 | for subnet in cfg['subnets']: | ||
1213 | 219 | if subnet['type'] in ('dhcp', 'dhcp4'): | ||
1214 | 220 | # OK, so we have a DHCP subnet. Let's assume all other subnets | ||
1215 | 221 | # are also DHCP and just stop looking | ||
1216 | 222 | bootproto = 'dhcp' | ||
1217 | 223 | break | ||
1218 | 224 | elif subnet['type'] in ('static', 'static4'): | ||
1219 | 225 | # If this is a static subnet, then there might be a default | ||
1220 | 226 | # gateway and resolvers set and if there aren't we'll keep | ||
1221 | 227 | # looking for them in the next subnet configuration | ||
1222 | 228 | bootproto = 'none' | ||
1223 | 229 | if not defaultgw: | ||
1224 | 230 | defaultgw = subnet.get('gateway') | ||
1225 | 231 | if not resolvers: | ||
1226 | 232 | resolvers = subnet.get('dns_nameservers') | ||
1227 | 233 | if not search_domains: | ||
1228 | 234 | search_domains = subnet.get('dns_search') | ||
1229 | 235 | else: | ||
1230 | 236 | # Let's log the lack of support and continue - there still | ||
1231 | 237 | # migth be a supported subnet configuration | ||
1232 | 238 | LOG.warn('Configuration of subnet type {} ' | ||
1233 | 239 | 'not supported'.format(subnet['type'])) | ||
1234 | 240 | |||
1235 | 241 | content += 'BOOTPROTO={}\n'.format(bootproto) | ||
1236 | 242 | if bootproto != 'dhcp': | ||
1237 | 243 | if defaultgw: | ||
1238 | 244 | content += 'GATEWAY={}\n'.format(defaultgw) | ||
1239 | 245 | for idx, resolver in enumerate(resolvers): | ||
1240 | 246 | content += 'DNS{}={}\n'.format(idx+1, resolver) | ||
1241 | 247 | if search_domains: | ||
1242 | 248 | content += 'DOMAIN="{}"\n'.format(' '.join(search_domains)) | ||
1243 | 249 | |||
1244 | 250 | return content | ||
1245 | 251 | |||
1246 | 252 | |||
1247 | 253 | def write_resolv_conf(cfg, target): | ||
1248 | 254 | content = '# Generated by MAAS fast-path installer\n' | ||
1249 | 255 | |||
1250 | 256 | for resolver in cfg.get('address', []): | ||
1251 | 257 | content += 'nameserver {}\n'.format(resolver) | ||
1252 | 258 | |||
1253 | 259 | search_domains = cfg.get('search', []) | ||
1254 | 260 | if search_domains: | ||
1255 | 261 | content += 'search {}\n'.format(' '.join(search_domains)) | ||
1256 | 262 | |||
1257 | 263 | path = os.path.join(target, 'etc', 'resolv.conf') | ||
1258 | 264 | util.write_file(path, content) | ||
1259 | 265 | |||
1260 | 266 | |||
1261 | 267 | def write_physical_iface_config(cfg, target): | ||
1262 | 268 | content = ( | ||
1263 | 269 | '# Generated by MAAS fast-path installer\n' | ||
1264 | 270 | 'DEVICE={device}\n' | ||
1265 | 271 | 'HWADDR={hwaddr}\n' | ||
1266 | 272 | 'MTU={mtu}\n' | ||
1267 | 273 | 'ONBOOT=yes\n' | ||
1268 | 274 | ).format( | ||
1269 | 275 | device=cfg['name'], | ||
1270 | 276 | hwaddr=cfg['mac_address'], | ||
1271 | 277 | mtu=cfg.get('mtu', 1500) | ||
1272 | 278 | ) | ||
1273 | 279 | |||
1274 | 280 | content += get_common_network_config(cfg) + \ | ||
1275 | 281 | get_subnet_config(cfg['subnets']) | ||
1276 | 282 | |||
1277 | 283 | path = os.path.join(target, 'etc', 'sysconfig', | ||
1278 | 284 | 'network-scripts', 'ifcfg-%s' % cfg['name']) | ||
1279 | 285 | util.write_file(path, content) | ||
1280 | 286 | |||
1281 | 287 | |||
1282 | 288 | def write_vlan_iface_config(cfg, target): | ||
1283 | 289 | content = ( | ||
1284 | 290 | '# Generated by MAAS fast-path installer\n' | ||
1285 | 291 | 'DEVICE={device}\n' | ||
1286 | 292 | 'MTU={mtu}\n' | ||
1287 | 293 | 'ONBOOT=yes\n' | ||
1288 | 294 | 'VLAN=yes\n' | ||
1289 | 295 | ).format( | ||
1290 | 296 | device=cfg['name'], | ||
1291 | 297 | mtu=cfg.get('mtu', 1500) | ||
1292 | 298 | ) | ||
1293 | 299 | |||
1294 | 300 | content += get_common_network_config(cfg) + \ | ||
1295 | 301 | get_subnet_config(cfg['subnets']) | ||
1296 | 302 | |||
1297 | 303 | path = os.path.join(target, 'etc', 'sysconfig', | ||
1298 | 304 | 'network-scripts', 'ifcfg-%s' % cfg['name']) | ||
1299 | 305 | util.write_file(path, content) | ||
1300 | 306 | |||
1301 | 307 | |||
1302 | 308 | def apply_networking(config, target): | ||
1303 | 309 | network_config = config.get('network') | ||
1304 | 310 | if not network_config: | ||
1305 | 311 | LOG.warn("Unable to retrieve network configuration, will fallback " | ||
1306 | 312 | "to configuring DHCP on boot device") | ||
1307 | 313 | bootmac = get_boot_mac() | ||
1308 | 314 | inames = get_interface_names() | ||
1309 | 315 | iname = inames[bootmac.lower()] | ||
1310 | 316 | network_config = { | ||
1311 | 317 | 'config': [ | ||
1312 | 318 | {'mac_address': bootmac, | ||
1313 | 319 | 'name': iname, | ||
1314 | 320 | 'type': 'physical', | ||
1315 | 321 | 'subnets': [{'type': 'dhcp'}]}], | ||
1316 | 322 | 'version': 1} | ||
1317 | 323 | |||
1318 | 324 | for cfg in network_config['config']: | ||
1319 | 325 | if cfg['type'] == 'nameserver': | ||
1320 | 326 | write_resolv_conf(cfg, target) | ||
1321 | 327 | if cfg['type'] == 'physical': | ||
1322 | 328 | write_physical_iface_config(cfg, target) | ||
1323 | 329 | if cfg['type'] == 'vlan': | ||
1324 | 330 | write_vlan_iface_config(cfg, target) | ||
1325 | 331 | |||
1326 | 332 | |||
1327 | 333 | def update_grub_default(target, extra=[]): | ||
1328 | 334 | """Updates /etc/default/grub with the correct options.""" | ||
1329 | 335 | grub_default_path = os.path.join(target, 'etc', 'default', 'grub') | ||
1330 | 336 | kernel_cmdline = ' '.join(extra) | ||
1331 | 337 | with open(grub_default_path, 'a') as stream: | ||
1332 | 338 | stream.write(GRUB_PREPEND) | ||
1333 | 339 | stream.write('GRUB_CMDLINE_LINUX=\"%s\"\n' % kernel_cmdline) | ||
1334 | 99 | 340 | ||
1335 | 100 | 341 | ||
1336 | 101 | def strip_kernel_params(params, strip_params=[]): | 342 | def strip_kernel_params(params, strip_params=[]): |
1337 | @@ -121,9 +362,9 @@ | |||
1338 | 121 | """ | 362 | """ |
1339 | 122 | cmdline = read_file('/proc/cmdline') | 363 | cmdline = read_file('/proc/cmdline') |
1340 | 123 | cmdline = cmdline.split() | 364 | cmdline = cmdline.split() |
1342 | 124 | if '--' not in cmdline: | 365 | if '---' not in cmdline: |
1343 | 125 | return [] | 366 | return [] |
1345 | 126 | idx = cmdline.index('--') + 1 | 367 | idx = cmdline.index('---') + 1 |
1346 | 127 | if idx >= len(cmdline) + 1: | 368 | if idx >= len(cmdline) + 1: |
1347 | 128 | return [] | 369 | return [] |
1348 | 129 | return strip_kernel_params( | 370 | return strip_kernel_params( |
1349 | @@ -131,57 +372,141 @@ | |||
1350 | 131 | strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF=']) | 372 | strip_params=['initrd=', 'BOOT_IMAGE=', 'BOOTIF=']) |
1351 | 132 | 373 | ||
1352 | 133 | 374 | ||
1353 | 134 | def update_grub_default(target, extra=[]): | ||
1354 | 135 | """Updates /etc/default/grub with the correct options.""" | ||
1355 | 136 | grub_default_path = os.path.join(target, 'etc', 'default', 'grub') | ||
1356 | 137 | kernel_cmdline = ' '.join(extra) | ||
1357 | 138 | with open(grub_default_path, 'a') as stream: | ||
1358 | 139 | stream.write(GRUB_PREPEND) | ||
1359 | 140 | stream.write('GRUB_CMDLINE_LINUX=\"%s\"\n' % kernel_cmdline) | ||
1360 | 141 | |||
1361 | 142 | |||
1362 | 143 | def grub2_install(target, root): | ||
1363 | 144 | """Installs grub2 to the root.""" | ||
1364 | 145 | with util.RunInChroot(target) as in_chroot: | ||
1365 | 146 | in_chroot(['grub2-install', '--recheck', root]) | ||
1366 | 147 | |||
1367 | 148 | |||
1368 | 149 | def grub2_mkconfig(target): | 375 | def grub2_mkconfig(target): |
1369 | 150 | """Writes the new grub2 config.""" | 376 | """Writes the new grub2 config.""" |
1404 | 151 | with util.RunInChroot(target) as in_chroot: | 377 | # NOTE. CentOS kernel packages expect /etc/grub2.cfg symlink in |
1405 | 152 | in_chroot(['grub2-mkconfig', '-o', '/boot/grub2/grub.cfg']) | 378 | # BIOS mode and /etc/grub2-efi.cfg symlink in UEFI mode pointing |
1406 | 153 | 379 | # to an actual grub.cfg | |
1407 | 154 | 380 | ||
1408 | 155 | def install_efi(target, uefi_path): | 381 | grub_cfg = '/boot/grub2/grub.cfg' |
1409 | 156 | """Install the EFI data from /boot into efi partition.""" | 382 | grub_link = '/etc/grub2.cfg' |
1410 | 157 | # Create temp mount point for uefi partition. | 383 | grub_unlink = os.path.join(target, 'etc', 'grub2-efi.cfg') |
1411 | 158 | tmp_efi = os.path.join(target, 'boot', 'efi_part') | 384 | |
1412 | 159 | os.mkdir(tmp_efi) | 385 | if util.is_uefi_bootable(): |
1413 | 160 | util.subp(['mount', uefi_path, tmp_efi]) | 386 | grub_cfg = '/boot/efi/EFI/centos/grub.cfg' |
1414 | 161 | 387 | grub_link = '/etc/grub2-efi.cfg' | |
1415 | 162 | # Copy the data over. | 388 | grub_unlink = os.path.join(target, 'etc', 'grub2.cfg') |
1416 | 163 | try: | 389 | util.write_file(os.path.join(target, 'boot', 'grub2', 'grub.cfg'), |
1417 | 164 | efi_path = os.path.join(target, 'boot', 'efi') | 390 | '# UEFI system, see /etc/grub2-efi.cfg\n') |
1418 | 165 | if os.path.exists(os.path.join(tmp_efi, 'EFI')): | 391 | |
1419 | 166 | shutil.rmtree(os.path.join(tmp_efi, 'EFI')) | 392 | with util.RunInChroot(target) as in_chroot: |
1420 | 167 | shutil.copytree( | 393 | in_chroot(['grub2-mkconfig', '-o', grub_cfg]) |
1421 | 168 | os.path.join(efi_path, 'EFI'), | 394 | in_chroot(['ln', '-sf', grub_cfg, grub_link]) |
1422 | 169 | os.path.join(tmp_efi, 'EFI')) | 395 | util.del_file(grub_unlink) |
1423 | 170 | finally: | 396 | |
1424 | 171 | # Clean up tmp mount | 397 | |
1425 | 172 | util.subp(['umount', tmp_efi]) | 398 | def get_uefi_partition(): |
1426 | 173 | os.rmdir(tmp_efi) | 399 | """Return the UEFI partition.""" |
1427 | 174 | 400 | for _, value in block._lsblock().items(): | |
1428 | 175 | # Mount and do grub install | 401 | if value['LABEL'] == 'uefi-boot': |
1429 | 176 | util.subp(['mount', uefi_path, efi_path]) | 402 | return value |
1430 | 177 | try: | 403 | return None |
1431 | 178 | with util.RunInChroot(target) as in_chroot: | 404 | |
1432 | 179 | in_chroot([ | 405 | |
1433 | 180 | 'grub2-install', '--target=x86_64-efi', | 406 | def install_bootloader(target, device=None): |
1434 | 181 | '--efi-directory', '/boot/efi', | 407 | """Installs bootloader to the device.""" |
1435 | 182 | '--recheck']) | 408 | cmd = [] |
1436 | 183 | finally: | 409 | if util.is_uefi_bootable(): |
1437 | 184 | util.subp(['umount', efi_path]) | 410 | uefi_dev = get_uefi_partition()['device_path'] |
1438 | 411 | disk, part = block.get_blockdev_for_partition(uefi_dev) | ||
1439 | 412 | cmd = ['efibootmgr', '-v', '-c', '-w', '-L', 'centos', | ||
1440 | 413 | '-d', disk, '-p', part, '-l', r'\EFI\centos\shim.efi'] | ||
1441 | 414 | else: | ||
1442 | 415 | cmd = ['grub2-install', '--recheck', device] | ||
1443 | 416 | |||
1444 | 417 | with util.RunInChroot(target) as in_chroot: | ||
1445 | 418 | in_chroot(cmd) | ||
1446 | 419 | |||
1447 | 420 | |||
1448 | 421 | def setup_grub(cfg, target): | ||
1449 | 422 | # target is the path to the mounted filesystem | ||
1450 | 423 | |||
1451 | 424 | # FIXME: these methods need moving to curtin.block | ||
1452 | 425 | # and using them from there rather than commands.block_meta | ||
1453 | 426 | from curtin.commands.block_meta import (extract_storage_ordered_dict, | ||
1454 | 427 | get_path_to_storage_volume) | ||
1455 | 428 | |||
1456 | 429 | grubcfg = cfg.get('grub', {}) | ||
1457 | 430 | |||
1458 | 431 | # copy legacy top level name | ||
1459 | 432 | if 'grub_install_devices' in cfg and 'install_devices' not in grubcfg: | ||
1460 | 433 | grubcfg['install_devices'] = cfg['grub_install_devices'] | ||
1461 | 434 | |||
1462 | 435 | LOG.debug("setup grub on target %s", target) | ||
1463 | 436 | # if there is storage config, look for devices tagged with 'grub_device' | ||
1464 | 437 | storage_cfg_odict = None | ||
1465 | 438 | try: | ||
1466 | 439 | storage_cfg_odict = extract_storage_ordered_dict(cfg) | ||
1467 | 440 | except ValueError as e: | ||
1468 | 441 | pass | ||
1469 | 442 | |||
1470 | 443 | if storage_cfg_odict: | ||
1471 | 444 | storage_grub_devices = [] | ||
1472 | 445 | for item_id, item in storage_cfg_odict.items(): | ||
1473 | 446 | if not item.get('grub_device'): | ||
1474 | 447 | continue | ||
1475 | 448 | LOG.debug("checking: %s", item) | ||
1476 | 449 | storage_grub_devices.append( | ||
1477 | 450 | get_path_to_storage_volume(item_id, storage_cfg_odict)) | ||
1478 | 451 | if len(storage_grub_devices) > 0: | ||
1479 | 452 | grubcfg['install_devices'] = storage_grub_devices | ||
1480 | 453 | |||
1481 | 454 | LOG.debug("install_devices: %s", grubcfg.get('install_devices')) | ||
1482 | 455 | if 'install_devices' in grubcfg: | ||
1483 | 456 | instdevs = grubcfg.get('install_devices') | ||
1484 | 457 | if isinstance(instdevs, str): | ||
1485 | 458 | instdevs = [instdevs] | ||
1486 | 459 | if instdevs is None: | ||
1487 | 460 | LOG.debug("grub installation disabled by config") | ||
1488 | 461 | else: | ||
1489 | 462 | # If there were no install_devices found then we try to do the right | ||
1490 | 463 | # thing. That right thing is basically installing on all block | ||
1491 | 464 | # devices that are mounted. On powerpc, though it means finding PrEP | ||
1492 | 465 | # partitions. | ||
1493 | 466 | devs = block.get_devices_for_mp(target) | ||
1494 | 467 | blockdevs = set() | ||
1495 | 468 | for maybepart in devs: | ||
1496 | 469 | try: | ||
1497 | 470 | (blockdev, part) = block.get_blockdev_for_partition(maybepart) | ||
1498 | 471 | blockdevs.add(blockdev) | ||
1499 | 472 | except ValueError as e: | ||
1500 | 473 | # if there is no syspath for this device such as a lvm | ||
1501 | 474 | # or raid device, then a ValueError is raised here. | ||
1502 | 475 | LOG.debug("failed to find block device for %s", maybepart) | ||
1503 | 476 | instdevs = list(blockdevs) | ||
1504 | 477 | |||
1505 | 478 | # UEFI requires additional packages | ||
1506 | 479 | if util.is_uefi_bootable(): | ||
1507 | 480 | pkgs = ['grub2-efi', 'shim', 'efibootmgr'] | ||
1508 | 481 | install_packages(pkgs, target=target) | ||
1509 | 482 | |||
1510 | 483 | # CentOS will not assemble MD devices on boot without rd.md.uuid=MD_UUID | ||
1511 | 484 | # kernel parameters | ||
1512 | 485 | mdmap = {} | ||
1513 | 486 | rdmduuids = [] | ||
1514 | 487 | try: | ||
1515 | 488 | mdmap = mdadm.md_read_run_mdadm_map() | ||
1516 | 489 | for md in mdmap: | ||
1517 | 490 | rdmduuids.append("rd.md.uuid=%s" % mdadm.md_get_uuid(mdmap[md][2])) | ||
1518 | 491 | except ValueError as e: | ||
1519 | 492 | pass | ||
1520 | 493 | |||
1521 | 494 | update_grub_default(target, extra=get_extra_kernel_parameters() + rdmduuids) | ||
1522 | 495 | grub2_mkconfig(target) | ||
1523 | 496 | |||
1524 | 497 | if instdevs: | ||
1525 | 498 | instdevs = [block.get_dev_name_entry(i)[1] for i in instdevs] | ||
1526 | 499 | else: | ||
1527 | 500 | instdevs = ["none"] | ||
1528 | 501 | |||
1529 | 502 | LOG.debug("installing grub to %s", instdevs) | ||
1530 | 503 | |||
1531 | 504 | if util.is_uefi_bootable(): | ||
1532 | 505 | if grubcfg.get('update_nvram', False): | ||
1533 | 506 | install_bootloader(target) | ||
1534 | 507 | else: | ||
1535 | 508 | for dev in instdevs: | ||
1536 | 509 | install_bootloader(target, dev) | ||
1537 | 185 | 510 | ||
1538 | 186 | 511 | ||
1539 | 187 | def set_autorelabel(target): | 512 | def set_autorelabel(target): |
1540 | @@ -196,136 +521,75 @@ | |||
1541 | 196 | open(path, 'a').close() | 521 | open(path, 'a').close() |
1542 | 197 | 522 | ||
1543 | 198 | 523 | ||
1544 | 199 | def get_boot_mac(): | ||
1545 | 200 | """Return the mac address of the booting interface.""" | ||
1546 | 201 | cmdline = read_file('/proc/cmdline') | ||
1547 | 202 | cmdline = cmdline.split() | ||
1548 | 203 | try: | ||
1549 | 204 | bootif = [ | ||
1550 | 205 | option | ||
1551 | 206 | for option in cmdline | ||
1552 | 207 | if option.startswith('BOOTIF') | ||
1553 | 208 | ][0] | ||
1554 | 209 | except IndexError: | ||
1555 | 210 | return None | ||
1556 | 211 | _, mac = bootif.split('=') | ||
1557 | 212 | mac = mac.split('-')[1:] | ||
1558 | 213 | return ':'.join(mac) | ||
1559 | 214 | |||
1560 | 215 | |||
1561 | 216 | def get_interface_names(): | ||
1562 | 217 | """Return a dictionary mapping mac addresses to interface names.""" | ||
1563 | 218 | sys_path = "/sys/class/net" | ||
1564 | 219 | ifaces = {} | ||
1565 | 220 | for iname in os.listdir(sys_path): | ||
1566 | 221 | mac = read_file(os.path.join(sys_path, iname, "address")) | ||
1567 | 222 | mac = mac.strip().lower() | ||
1568 | 223 | ifaces[mac] = iname | ||
1569 | 224 | return ifaces | ||
1570 | 225 | |||
1571 | 226 | |||
1572 | 227 | def get_ipv4_config(iface, data): | ||
1573 | 228 | """Returns the contents of the interface file for ipv4.""" | ||
1574 | 229 | config = [ | ||
1575 | 230 | 'TYPE="Ethernet"', | ||
1576 | 231 | 'NM_CONTROLLED="no"', | ||
1577 | 232 | 'USERCTL="yes"', | ||
1578 | 233 | ] | ||
1579 | 234 | if 'hwaddress' in data: | ||
1580 | 235 | config.append('HWADDR="%s"' % data['hwaddress']) | ||
1581 | 236 | # Fallback to using device name | ||
1582 | 237 | else: | ||
1583 | 238 | config.append('DEVICE="%"' % iface) | ||
1584 | 239 | if data['auto']: | ||
1585 | 240 | config.append('ONBOOT="yes"') | ||
1586 | 241 | else: | ||
1587 | 242 | config.append('ONBOOT="no"') | ||
1588 | 243 | |||
1589 | 244 | method = data['method'] | ||
1590 | 245 | if method == 'dhcp': | ||
1591 | 246 | config.append('BOOTPROTO="dhcp"') | ||
1592 | 247 | config.append('PEERDNS="yes"') | ||
1593 | 248 | config.append('PERSISTENT_DHCLIENT="1"') | ||
1594 | 249 | if 'hostname' in data: | ||
1595 | 250 | config.append('DHCP_HOSTNAME="%s"' % data['hostname']) | ||
1596 | 251 | elif method == 'static': | ||
1597 | 252 | config.append('BOOTPROTO="none"') | ||
1598 | 253 | config.append('IPADDR="%s"' % data['address']) | ||
1599 | 254 | config.append('NETMASK="%s"' % data['netmask']) | ||
1600 | 255 | if 'broadcast' in data: | ||
1601 | 256 | config.append('BROADCAST="%s"' % data['broadcast']) | ||
1602 | 257 | if 'gateway' in data: | ||
1603 | 258 | config.append('GATEWAY="%s"' % data['gateway']) | ||
1604 | 259 | elif method == 'manual': | ||
1605 | 260 | config.append('BOOTPROTO="none"') | ||
1606 | 261 | return '\n'.join(config) | ||
1607 | 262 | |||
1608 | 263 | |||
1609 | 264 | def write_interface_config(target, iface, data): | ||
1610 | 265 | """Writes config for interface.""" | ||
1611 | 266 | family = data['family'] | ||
1612 | 267 | if family != "inet": | ||
1613 | 268 | # Only supporting ipv4 currently | ||
1614 | 269 | print( | ||
1615 | 270 | "WARN: unsupported family %s, " | ||
1616 | 271 | "failed to configure interface: %s" (family, iface)) | ||
1617 | 272 | return | ||
1618 | 273 | config = get_ipv4_config(iface, data) | ||
1619 | 274 | path = os.path.join( | ||
1620 | 275 | target, 'etc', 'sysconfig', 'network-scripts', 'ifcfg-%s' % iface) | ||
1621 | 276 | with open(path, 'w') as stream: | ||
1622 | 277 | stream.write(config + '\n') | ||
1623 | 278 | |||
1624 | 279 | |||
1625 | 280 | def write_network_config(target, mac): | ||
1626 | 281 | """Write network configuration for the given MAC address.""" | ||
1627 | 282 | inames = get_interface_names() | ||
1628 | 283 | iname = inames[mac.lower()] | ||
1629 | 284 | write_interface_config( | ||
1630 | 285 | target, iname, { | ||
1631 | 286 | 'family': 'inet', | ||
1632 | 287 | 'hwaddress': mac.upper(), | ||
1633 | 288 | 'auto': True, | ||
1634 | 289 | 'method': 'dhcp' | ||
1635 | 290 | }) | ||
1636 | 291 | |||
1637 | 292 | |||
1638 | 293 | def main(): | 524 | def main(): |
1639 | 294 | state = util.load_command_environment() | 525 | state = util.load_command_environment() |
1640 | 526 | |||
1641 | 295 | target = state['target'] | 527 | target = state['target'] |
1642 | 296 | if target is None: | 528 | if target is None: |
1672 | 297 | print("Target was not provided in the environment.") | 529 | sys.stderr.write("Target was not provided in the environment.") |
1673 | 298 | sys.exit(1) | 530 | sys.exit(1) |
1674 | 299 | fstab = state['fstab'] | 531 | |
1675 | 300 | if fstab is None: | 532 | cfg = config.load_command_config(None, state) |
1676 | 301 | print("/etc/fstab output was not provided in the environment.") | 533 | stack_prefix = state.get('report_stack_prefix', '') |
1677 | 302 | sys.exit(1) | 534 | |
1678 | 303 | bootmac = get_boot_mac() | 535 | with events.ReportEventStack( |
1679 | 304 | if bootmac is None: | 536 | name=stack_prefix, reporting_enabled=True, level="INFO", |
1680 | 305 | print("Unable to determine boot interface.") | 537 | description="writing config files"): |
1681 | 306 | sys.exit(1) | 538 | curthooks.write_files(cfg, target) |
1682 | 307 | devices = get_block_devices(target) | 539 | |
1683 | 308 | if not devices: | 540 | # Default CentOS image does not contain some packages that may be necessary |
1684 | 309 | print("Unable to find block device for: %s" % target) | 541 | install_missing_packages(cfg, target) |
1685 | 310 | sys.exit(1) | 542 | |
1686 | 311 | 543 | # If a mdadm.conf file was created by block_meta then it needs to be copied | |
1687 | 312 | write_fstab(target, fstab) | 544 | # onto the target system |
1688 | 313 | 545 | mdadm_location = os.path.join(os.path.split(state['fstab'])[0], | |
1689 | 314 | update_grub_default( | 546 | "mdadm.conf") |
1690 | 315 | target, extra=get_extra_kernel_parameters()) | 547 | if os.path.exists(mdadm_location): |
1691 | 316 | grub2_mkconfig(target) | 548 | copy_mdadm_conf(mdadm_location, target) |
1692 | 317 | if util.is_uefi_bootable(): | 549 | |
1693 | 318 | uefi_part = get_uefi_partition() | 550 | with events.ReportEventStack( |
1694 | 319 | if uefi_part is None: | 551 | name=stack_prefix, reporting_enabled=True, level="INFO", |
1695 | 320 | print('Unable to determine UEFI parition.') | 552 | description="setting up swap"): |
1696 | 321 | sys.exit(1) | 553 | curthooks.add_swap(cfg, target, state.get('fstab')) |
1697 | 322 | install_efi(target, uefi_part['device_path']) | 554 | |
1698 | 323 | else: | 555 | with events.ReportEventStack( |
1699 | 324 | for dev in devices: | 556 | name=stack_prefix, reporting_enabled=True, level="INFO", |
1700 | 325 | grub2_install(target, dev) | 557 | description="writing etc/fstab"): |
1701 | 558 | curthooks.copy_fstab(state.get('fstab'), target) | ||
1702 | 559 | |||
1703 | 560 | # TODO. There's a multipath implementation on Ubuntu in curtin/curthooks.py | ||
1704 | 561 | # that should be reimplemented here. Skipping for now. | ||
1705 | 562 | |||
1706 | 563 | with events.ReportEventStack( | ||
1707 | 564 | name=stack_prefix, reporting_enabled=True, level="INFO", | ||
1708 | 565 | description="updating packages on target system"): | ||
1709 | 566 | system_upgrade(cfg, target) | ||
1710 | 567 | |||
1711 | 568 | # TODO. If a crypttab file was created by block_meta than it needs to be | ||
1712 | 569 | # copied onto the target system, and update_initramfs() (well, alternative | ||
1713 | 570 | # for CentOS actually) needs to be run, so that the cryptsetup hooks are | ||
1714 | 571 | # properly configured on the installed system and it will be able to open | ||
1715 | 572 | # encrypted volumes at boot. Skipping for now. | ||
1716 | 573 | |||
1717 | 574 | with events.ReportEventStack( | ||
1718 | 575 | name=stack_prefix, reporting_enabled=True, level="INFO", | ||
1719 | 576 | description="apply networking"): | ||
1720 | 577 | apply_networking(cfg, target) | ||
1721 | 578 | |||
1722 | 579 | # If udev dname rules were created, copy them to target | ||
1723 | 580 | udev_rules_d = os.path.join(state['scratch'], "rules.d") | ||
1724 | 581 | if os.path.isdir(udev_rules_d): | ||
1725 | 582 | curthooks.copy_dname_rules(udev_rules_d, target) | ||
1726 | 583 | |||
1727 | 584 | with events.ReportEventStack( | ||
1728 | 585 | name=stack_prefix, reporting_enabled=True, level="INFO", | ||
1729 | 586 | description="updating packages on target system"): | ||
1730 | 587 | setup_grub(cfg, target) | ||
1731 | 326 | 588 | ||
1732 | 327 | set_autorelabel(target) | 589 | set_autorelabel(target) |
1734 | 328 | write_network_config(target, bootmac) | 590 | update_initramfs(target) |
1735 | 591 | |||
1736 | 592 | sys.exit(0) | ||
1737 | 329 | 593 | ||
1738 | 330 | 594 | ||
1739 | 331 | if __name__ == "__main__": | 595 | if __name__ == "__main__": |
These changes work great for me. They make Maas much more usable for people that don't only work with Ubuntu.