Merge lp:~smoser/curtin/trunk.lp1523779 into lp:~curtin-dev/curtin/trunk

Proposed by Scott Moser
Status: Merged
Merged at revision: 332
Proposed branch: lp:~smoser/curtin/trunk.lp1523779
Merge into: lp:~curtin-dev/curtin/trunk
Diff against target: 137 lines (+51/-31)
2 files modified
curtin/commands/block_meta.py (+19/-18)
curtin/commands/curthooks.py (+32/-13)
To merge this branch: bzr merge lp:~smoser/curtin/trunk.lp1523779
Reviewer Review Type Date Requested Status
Server Team CI bot continuous-integration Approve
Ryan Harper (community) Approve
Review via email: mp+281000@code.launchpad.net

Commit message

fix bug in install_grub to partition when storage_config provided.

when installing with a storage config and grub is to be installed
on a partition (as it is in powerpc) we were not able to look
up the device that was to be installed on.

This puts a temporary bit of code in place that finds that device
via imports from commands.block_meta. The result is that
storage_config now works for lvm on power8 systems.

To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:322
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~smoser/curtin/trunk.lp1523779/+merge/281000/+edit-commit-message

https://server-team-jenkins.canonical.com/job/curtin-ci/70/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/67/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/70/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Ryan Harper (raharper) :
Revision history for this message
Scott Moser (smoser) wrote :

The change as it is right now is to add a method 'extract_storage_ordered_dict' to get an ordered dictionary, where that code was being done inside 'meta_custom' directly.

The reason for doing that is that we need to use it from curthooks.
Previously, if custom storage was used and 'grub_device' existed on a partition it woudl fail.
That was because we could only look up by serial or path.

Now, we use extract_storage_ordered_dict method to get an ordered dictionary and then get a path to that volume (or partition) via get_path_to_storage_volume.

I dont like the inline import : i've done that here only because I dont like importing from block_meta at all. I'd rather move all this to curtin.block but I dont think its ready to go there yet.

I could add another method in block_meta that
  get_path_to_storage_voulme_with_config_id() or seomthing like that and then only use one method from curthooks.

Revision history for this message
Ryan Harper (raharper) wrote :

That makes sense. I don't think it'd be too much trouble to put the wrapper in... but which ever method reminds use we need to migrate that into block is fine with me.

Revision history for this message
Ryan Harper (raharper) wrote :

Via IRC, missing grub_device in storage config will fallback to grub_install on mounted devices. That's sufficient for now.

review: Approve
lp:~smoser/curtin/trunk.lp1523779 updated
324. By Scott Moser

explain the no-grub-dev found path a bit

325. By Scott Moser

some cleanup and doc and FIXME

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:325
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~smoser/curtin/trunk.lp1523779/+merge/281000/+edit-commit-message

https://server-team-jenkins.canonical.com/job/curtin-ci/72/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/69/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/72/rebuild

review: Needs Fixing (continuous-integration)
lp:~smoser/curtin/trunk.lp1523779 updated
326. By Scott Moser

fix keyerror if no install_devices

327. By Scott Moser

handle lvm devices

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

FAILED: Continuous integration, rev:327
No commit message was specified in the merge proposal. Click on the following link and set the commit message (if you want a jenkins rebuild you need to trigger it yourself):
https://code.launchpad.net/~smoser/curtin/trunk.lp1523779/+merge/281000/+edit-commit-message

https://server-team-jenkins.canonical.com/job/curtin-ci/75/
Executed test runs:
    None: https://server-team-jenkins.canonical.com/job/generic-update-mp/72/console

Click here to trigger a rebuild:
https://server-team-jenkins.canonical.com/job/curtin-ci/75/rebuild

review: Needs Fixing (continuous-integration)
Revision history for this message
Server Team CI bot (server-team-bot) wrote :
review: Approve (continuous-integration)

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'curtin/commands/block_meta.py'
--- curtin/commands/block_meta.py 2015-12-08 01:23:09 +0000
+++ curtin/commands/block_meta.py 2015-12-18 22:14:07 +0000
@@ -1070,6 +1070,21 @@
1070 not supported")1070 not supported")
10711071
10721072
1073def extract_storage_ordered_dict(config):
1074 storage_config = config.get('storage', {})
1075 if not storage_config:
1076 raise ValueError("no 'storage' entry in config")
1077 scfg = storage_config.get('config')
1078 if not scfg:
1079 raise ValueError("invalid storage config data")
1080
1081 # Since storage config will often have to be searched for a value by its
1082 # id, and this can become very inefficient as storage_config grows, a dict
1083 # will be generated with the id of each component of the storage_config as
1084 # its index and the component of storage_config as its value
1085 return OrderedDict((d["id"], d) for (i, d) in enumerate(scfg))
1086
1087
1073def meta_custom(args):1088def meta_custom(args):
1074 """Does custom partitioning based on the layout provided in the config1089 """Does custom partitioning based on the layout provided in the config
1075 file. Section with the name storage contains information on which1090 file. Section with the name storage contains information on which
@@ -1092,23 +1107,9 @@
1092 state = util.load_command_environment()1107 state = util.load_command_environment()
1093 cfg = config.load_command_config(args, state)1108 cfg = config.load_command_config(args, state)
10941109
1095 storage_config = cfg.get('storage', {})1110 storage_config_dict = extract_storage_ordered_dict(cfg)
1096 if not storage_config:1111
1097 raise Exception("storage configuration is required by mode '%s' "1112 for item_id, command in storage_config_dict.items():
1098 "but not provided in the config file" % CUSTOM)
1099 storage_config_data = storage_config.get('config')
1100
1101 if not storage_config_data:
1102 raise ValueError("invalid storage config data")
1103
1104 # Since storage config will often have to be searched for a value by its
1105 # id, and this can become very inefficient as storage_config grows, a dict
1106 # will be generated with the id of each component of the storage_config as
1107 # its index and the component of storage_config as its value
1108 storage_config_dict = OrderedDict((d["id"], d) for (i, d) in
1109 enumerate(storage_config_data))
1110
1111 for command in storage_config_data:
1112 handler = command_handlers.get(command['type'])1113 handler = command_handlers.get(command['type'])
1113 if not handler:1114 if not handler:
1114 raise ValueError("unknown command type '%s'" % command['type'])1115 raise ValueError("unknown command type '%s'" % command['type'])
@@ -1116,7 +1117,7 @@
1116 handler(command, storage_config_dict)1117 handler(command, storage_config_dict)
1117 except Exception as error:1118 except Exception as error:
1118 LOG.error("An error occured handling '%s': %s - %s" %1119 LOG.error("An error occured handling '%s': %s - %s" %
1119 (command.get('id'), type(error).__name__, error))1120 (item_id, type(error).__name__, error))
1120 raise1121 raise
11211122
1122 return 01123 return 0
11231124
=== modified file 'curtin/commands/curthooks.py'
--- curtin/commands/curthooks.py 2015-12-17 18:25:51 +0000
+++ curtin/commands/curthooks.py 2015-12-18 22:14:07 +0000
@@ -282,27 +282,37 @@
282282
283def setup_grub(cfg, target):283def setup_grub(cfg, target):
284 # target is the path to the mounted filesystem284 # target is the path to the mounted filesystem
285
286 # FIXME: these methods need moving to curtin.block
287 # and using them from there rather than commands.block_meta
288 from curtin.commands.block_meta import (extract_storage_ordered_dict,
289 get_path_to_storage_volume)
290
285 grubcfg = cfg.get('grub', {})291 grubcfg = cfg.get('grub', {})
286292
287 # copy legacy top level name293 # copy legacy top level name
288 if 'grub_install_devices' in cfg and 'install_devices' not in grubcfg:294 if 'grub_install_devices' in cfg and 'install_devices' not in grubcfg:
289 grubcfg['install_devices'] = cfg['grub_install_devices']295 grubcfg['install_devices'] = cfg['grub_install_devices']
290296
291 if 'storage' in cfg:297 # if there is storage config, look for devices tagged with 'grub_device'
298 storage_cfg_odict = None
299 try:
300 storage_cfg_odict = extract_storage_ordered_dict(cfg)
301 except ValueError as e:
302 pass
303
304 if storage_cfg_odict:
292 storage_grub_devices = []305 storage_grub_devices = []
293 for item in cfg.get('storage').get('config'):306 for item_id, item in storage_cfg_odict.items():
294 if item.get('grub_device'):307 if not item.get('grub_device'):
295 if item.get('serial'):308 continue
296 storage_grub_devices.append(309 LOG.debug("checking: %s", item)
297 block.lookup_disk(item.get('serial')))310 storage_grub_devices.append(
298 elif item.get('path'):311 get_path_to_storage_volume(item_id, storage_cfg_odict))
299 storage_grub_devices.append(item.get('path'))
300 else:
301 raise ValueError("setup_grub cannot find path to disk \
302 '%s'" % item.get('id'))
303 if len(storage_grub_devices) > 0:312 if len(storage_grub_devices) > 0:
304 grubcfg['install_devices'] = storage_grub_devices313 grubcfg['install_devices'] = storage_grub_devices
305314
315 LOG.debug("install_devices: %s", grubcfg.get('install_devices'))
306 if 'install_devices' in grubcfg:316 if 'install_devices' in grubcfg:
307 instdevs = grubcfg.get('install_devices')317 instdevs = grubcfg.get('install_devices')
308 if isinstance(instdevs, str):318 if isinstance(instdevs, str):
@@ -310,11 +320,20 @@
310 if instdevs is None:320 if instdevs is None:
311 LOG.debug("grub installation disabled by config")321 LOG.debug("grub installation disabled by config")
312 else:322 else:
323 # If there were no install_devices found then we try to do the right
324 # thing. That right thing is basically installing on all block
325 # devices that are mounted. On powerpc, though it means finding PrEP
326 # partitions.
313 devs = block.get_devices_for_mp(target)327 devs = block.get_devices_for_mp(target)
314 blockdevs = set()328 blockdevs = set()
315 for maybepart in devs:329 for maybepart in devs:
316 (blockdev, part) = block.get_blockdev_for_partition(maybepart)330 try:
317 blockdevs.add(blockdev)331 (blockdev, part) = block.get_blockdev_for_partition(maybepart)
332 blockdevs.add(blockdev)
333 except ValueError as e:
334 # if there is no syspath for this device such as a lvm
335 # or raid device, then a ValueError is raised here.
336 LOG.debug("failed to find block device for %s", maybepart)
318337
319 if platform.machine().startswith("ppc64"):338 if platform.machine().startswith("ppc64"):
320 # assume we want partitions that are 4100 (PReP). The snippet here339 # assume we want partitions that are 4100 (PReP). The snippet here

Subscribers

People subscribed via source and target branches