Merge ~dbungert/curtin:resize into curtin:master

[Dan Bungert]

Add support for resize of ext{2,3,4}

[Dan Bungert (dbungert)]

I'm not convinced the keyword 'resize' is what we want. Maybe 'move' as a keyword that permits both moves and resizes.

Needs a rebase to the final commit set, though maybe that matters less if the merge robot just squishes everything anyway.

The commit adding 'fake' to run_bm() is just debug stuff and probably isn't needed.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:60b44e964bb399ea4d46e8b7c793bcc615e6b993
https://jenkins.ubuntu.com/server/job/curtin-ci/279/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/279/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/279/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/279/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/279/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/279//rebuild

review: Needs Fixing (continuous-integration)

[Dan Bungert (dbungert)]

Also, there is some redundant partitioning between the calls to parted and the later table.apply().
Also, I'm not yet sure why resize2fs only sometimes wants a fsck, it's random and I don't like that it's random.

3e6c401... by Dan Bungert on 2022-03-30

block-meta: check fs format

[Michael Hudson-Doyle (mwhudson)]

This looks mostly OK to me. Do you need to invoke parted at all though? I guess if you let sfdisk do it all, you have to do the resizefs calls downs before the sfdisk invocation and the resize ups after. I think I'd still prefer that though. WDYT?

Also this clearly only works for ext{2,3,4} filesystems. What's the long term plan there?

47fec96... by Dan Bungert on 2022-03-31

block-meta: make format action not required

b2779a4... by Dan Bungert on 2022-03-31

block-meta: let sfdisk handle resize partitioning

[Dan Bungert (dbungert)]

> This looks mostly OK to me. Do you need to invoke parted at all though? I
> guess if you let sfdisk do it all, you have to do the resizefs calls downs
> before the sfdisk invocation and the resize ups after. I think I'd still
> prefer that though.

Agree. Done.

> Also this clearly only works for ext{2,3,4} filesystems. What's the long term
> plan there?

Let verify_format be aware of the resize, and complain if the partition to resize is in an unsupported format. Ship the initial pull with just ext{2,3,4}. Document the formats we can resize. Later PRs can be as needed and focused on just the new filesystem type we want to resize. Next is almost certainly NTFS.

What's your thoughts on the 'resize' keyword? I'm kind of leaning now for leaving it as is, and if we later support move, adding a 'move' keyword as well.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:bc841711aa099e333bed14fe32481efceb4a582a
https://jenkins.ubuntu.com/server/job/curtin-ci/280/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/280/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/280/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/280/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/280/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/280//rebuild

review: Needs Fixing (continuous-integration)

[Michael Hudson-Doyle (mwhudson)]

Looking good. But I think not quite here yet.

And there's always more scope for more tests.

26361fd... by Dan Bungert on 2022-04-01

block-meta: resize use existing device handling

3f9e591... by Dan Bungert on 2022-04-01

block-meta: only attempt resize on ext[234]

6075079... by Dan Bungert on 2022-04-05

block-meta: better resize size handling

ccc2fcb... by Dan Bungert on 2022-04-05

block-meta: verify offset if present

bb577a7... by Dan Bungert on 2022-04-05

block-meta: refactor and add resize tests

[Dan Bungert (dbungert)]

Closer to something I'm happy with.
See diff comments.
The commit history is awful, please ignore, will fix.
My plan at this point is a bit more integration tests, and to resolve the doc comment.

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:29ac44c98b907dde27cc7781a7a3714dcde7a445
https://jenkins.ubuntu.com/server/job/curtin-ci/281/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/281/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/281/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/281/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/281/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/281//rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:a75d45171d8037c4e8affea7193462c33e441e2e
https://jenkins.ubuntu.com/server/job/curtin-ci/282/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/282/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/282/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/282/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/282/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/282//rebuild

review: Approve (continuous-integration)

7eb9c64... by Dan Bungert on 2022-04-06

block-meta: integration test gpt doing many things

create, preserve, resize, delete

0b68d0c... by Dan Bungert on 2022-04-06

block-meta: check extended part size

aff746f... by Dan Bungert on 2022-04-06

block-meta: test resize of extended partition

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:053ffcfce64e17db96fc5efc2288485d40c6e44b
https://jenkins.ubuntu.com/server/job/curtin-ci/283/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/283/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/283/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/283/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/283/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/283//rebuild

review: Approve (continuous-integration)

d14db6c... by Dan Bungert on 2022-04-06

block-meta: integration test of many things

create, preserve, resize, delete including handling of primary,
extended, and logical partitions.

[Dan Bungert (dbungert)]

More or less ready, minus a short discussion on the documentation.
Please pay careful attention to commit 6d6e90c2cf26be176f31931c9df9d2ad96f45549

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:f42079e581d9920b993a3d36f01af15b4b45ccc5
https://jenkins.ubuntu.com/server/job/curtin-ci/284/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/284/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/284/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/284/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/284/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/284//rebuild

review: Approve (continuous-integration)

[Michael Hudson-Doyle (mwhudson)]

This is really really close now. I think the only gap is that I don't think this handles resizing a partition that is to be reformatted quite correctly -- there is no need to resize the filesystem in this case, so if it's too full to resize or not a format we know how to resize, we should still go ahead.

review: Needs Fixing

[Dan Bungert (dbungert)]

I need to check test_mix_of_operations_msdos for flakiness

ae221d2... by Dan Bungert on 2022-04-12

storage_config: add select_configs utility

ecc959a... by Dan Bungert on 2022-04-12

block-meta: find one format in verify_format

8fa9fd2... by Dan Bungert on 2022-04-12

doc: wording, v2 caution

03faa37... by Dan Bungert on 2022-04-12

block-meta: resize and preserve correctness

[Dan Bungert (dbungert)]

Feedback items implemented.

I can't reproduce the flaky test_mix_of_operations_msdos, and it's in the least useful spot (the bogus size reported on the extended partition by sysfs), so I think we should live with that one for now. If it shows up again I suggest we skip that particular check as it already known invalid.

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:03faa3751f0d4a4eaf1b2a98cc6918a599b411e5
https://jenkins.ubuntu.com/server/job/curtin-ci/286/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/286/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/286/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/286/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/286/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/286//rebuild

review: Approve (continuous-integration)

[Michael Hudson-Doyle (mwhudson)]

All this looks great, thanks.

review: Approve

diff --git a/curtin/block/schemas.py b/curtin/block/schemas.py
index 0a6e305..1834343 100644
--- a/curtin/block/schemas.py
+++ b/curtin/block/schemas.py
@@ -290,6 +290,7 @@ PARTITION = {
                  'pattern': _path_dev},
         'name': {'$ref': '#/definitions/name'},
         'offset': {'$ref': '#/definitions/size'},  # XXX: This is not used
+        'resize': {'type': 'boolean'},
         'preserve': {'$ref': '#/definitions/preserve'},
         'size': {'$ref': '#/definitions/size'},
         'uuid': {'$ref': '#/definitions/uuid'},    # XXX: This is not used
diff --git a/curtin/commands/block_meta.py b/curtin/commands/block_meta.py
index cdf30c5..5614883 100644
--- a/curtin/commands/block_meta.py
+++ b/curtin/commands/block_meta.py
@@ -779,12 +779,17 @@ def verify_exists(devpath):
         raise RuntimeError("Device %s does not exist" % devpath)
 
 
-def verify_size(devpath, expected_size_bytes, part_info):
+def get_part_size_bytes(devpath, part_info):
     (found_type, _code) = ptable_uuid_to_flag_entry(part_info.get('type'))
     if found_type == 'extended':
         found_size_bytes = int(part_info['size']) * 512
     else:
         found_size_bytes = block.read_sys_block_size_bytes(devpath)
+    return found_size_bytes
+
+
+def verify_size(devpath, expected_size_bytes, part_info):
+    found_size_bytes = get_part_size_bytes(devpath, part_info)
     msg = (
         'Verifying %s size, expecting %s bytes, found %s bytes' % (
          devpath, expected_size_bytes, found_size_bytes))
@@ -1124,6 +1129,12 @@ def _get_volume_type(device_path):
     return lsblock[kname]['TYPE']
 
 
+def _get_volume_fstype(device_path):
+    lsblock = block._lsblock([device_path])
+    kname = block.path_to_kname(device_path)
+    return lsblock[kname]['FSTYPE']
+
+
 def get_volume_spec(device_path):
     """
        Return the most reliable spec for a device per Ubuntu FSTAB wiki
diff --git a/curtin/commands/block_meta_v2.py b/curtin/commands/block_meta_v2.py
index 051649b..c1e3630 100644
--- a/curtin/commands/block_meta_v2.py
+++ b/curtin/commands/block_meta_v2.py
@@ -1,20 +1,24 @@
 # This file is part of curtin. See LICENSE file for copyright and license info.
 
+import os
 from typing import Optional
 
 import attr
 
 from curtin import (block, util)
 from curtin.commands.block_meta import (
+    _get_volume_fstype,
     disk_handler as disk_handler_v1,
     get_path_to_storage_volume,
     make_dname,
     partition_handler as partition_handler_v1,
-    partition_verify_sfdisk,
+    verify_ptable_flag,
+    verify_size,
     )
 from curtin.log import LOG
 from curtin.storage_config import (
     GPT_GUID_TO_CURTIN_MAP,
+    select_configs,
     )
 from curtin.udev import udevadm_settle
 
@@ -50,6 +54,28 @@ def align_down(size, block_size):
     return size & ~(block_size - 1)
 
 
+def resize_ext(path, size):
+    util.subp(['e2fsck', '-p', '-f', path])
+    size_k = size // 1024
+    util.subp(['resize2fs', path, f'{size_k}k'])
+
+
+def perform_resize(kname, size, direction):
+    path = block.kname_to_path(kname)
+    fstype = _get_volume_fstype(path)
+    if fstype:
+        LOG.debug('Resizing %s of type %s %s to %s',
+                  path, fstype, direction, size)
+        resizers[fstype](path, size)
+
+
+resizers = {
+    'ext2': resize_ext,
+    'ext3': resize_ext,
+    'ext4': resize_ext,
+}
+
+
 FLAG_TO_GUID = {
     flag: guid for (guid, (flag, typecode)) in GPT_GUID_TO_CURTIN_MAP.items()
     }
@@ -214,6 +240,74 @@ def _wipe_for_action(action):
     return 'superblock'
 
 
+def _prepare_resize(entry, part_info, table):
+    return {
+        'start': table.sectors2bytes(part_info['size']),
+        'end': table.sectors2bytes(entry.size),
+    }
+
+
+def needs_resize(storage_config, part_action, sfdisk_part_info):
+    if not part_action.get('preserve'):
+        return False
+    if not part_action.get('resize'):
+        return False
+
+    volume = part_action['id']
+    format_actions = select_configs(storage_config, type='format',
+                                    volume=volume, preserve=True)
+    if len(format_actions) < 1:
+        return False
+    if len(format_actions) > 1:
+        raise Exception(f'too many format actions for volume {volume}')
+
+    if not format_actions[0].get('preserve'):
+        return False
+
+    devpath = os.path.realpath(sfdisk_part_info['node'])
+    fstype = _get_volume_fstype(devpath)
+    target_fstype = format_actions[0]['fstype']
+    msg = (
+        'Verifying %s format, expecting %s, found %s' % (
+         devpath, fstype, target_fstype))
+    LOG.debug(msg)
+    if fstype != target_fstype:
+        raise RuntimeError(msg)
+
+    if part_action.get('resize'):
+        msg = 'Resize requested for format %s' % (fstype, )
+        LOG.debug(msg)
+        if fstype not in resizers:
+            raise RuntimeError(msg + ' is unsupported')
+
+    return True
+
+
+def verify_offset(devpath, part_action, current_info, table):
+    if 'offset' not in part_action:
+        return
+    current_offset = table.sectors2bytes(current_info['start'])
+    action_offset = int(util.human2bytes(part_action['offset']))
+    msg = (
+        'Verifying %s offset, expecting %s, found %s' % (
+         devpath, current_offset, action_offset))
+    LOG.debug(msg)
+    if current_offset != action_offset:
+        raise RuntimeError(msg)
+
+
+def partition_verify_sfdisk_v2(part_action, label, sfdisk_part_info,
+                               storage_config, table):
+    devpath = os.path.realpath(sfdisk_part_info['node'])
+    if not part_action.get('resize'):
+        verify_size(devpath, int(util.human2bytes(part_action['size'])),
+                    sfdisk_part_info)
+    verify_offset(devpath, part_action, sfdisk_part_info, table)
+    expected_flag = part_action.get('flag')
+    if expected_flag:
+        verify_ptable_flag(devpath, expected_flag, label, sfdisk_part_info)
+
+
 def disk_handler_v2(info, storage_config, handlers):
     disk_handler_v1(info, storage_config, handlers)
 
@@ -239,6 +333,7 @@ def disk_handler_v2(info, storage_config, handlers):
     table = table_cls(sector_size)
     preserved_offsets = set()
     wipes = {}
+    resizes = {}
 
     sfdisk_info = None
     for action in part_actions:
@@ -251,7 +346,10 @@ def disk_handler_v2(info, storage_config, handlers):
                 # vmtest infrastructure unhappy.
                 sfdisk_info = block.sfdisk_info(disk)
             part_info = _find_part_info(sfdisk_info, entry.start)
-            partition_verify_sfdisk(action, sfdisk_info['label'], part_info)
+            partition_verify_sfdisk_v2(action, sfdisk_info['label'], part_info,
+                                       storage_config, table)
+            if needs_resize(storage_config, action, part_info):
+                resizes[entry.start] = _prepare_resize(entry, part_info, table)
             preserved_offsets.add(entry.start)
         wipe = wipes[entry.start] = _wipe_for_action(action)
         if wipe is not None:
@@ -266,20 +364,26 @@ def disk_handler_v2(info, storage_config, handlers):
             LOG.debug('Wiping 1M on %s at offset %s', disk, wipe_offset)
             block.zero_file_at_offsets(disk, [wipe_offset], exclusive=False)
 
-    # Do a superblock wipe of any partitions that are being deleted.
-    for kname, nr, offset, sz in block.sysfs_partition_data(disk):
+    for kname, nr, offset, size in block.sysfs_partition_data(disk):
         offset_sectors = table.bytes2sectors(offset)
         if offset_sectors not in preserved_offsets:
+            # Do a superblock wipe of any partitions that are being deleted.
             block.wipe_volume(block.kname_to_path(kname), 'superblock')
+        resize = resizes.get(offset_sectors)
+        if resize and size > resize['end']:
+            perform_resize(kname, resize['end'], 'down')
 
     table.apply(disk)
 
-    # Wipe the new partitions as needed.
     for kname, number, offset, size in block.sysfs_partition_data(disk):
         offset_sectors = table.bytes2sectors(offset)
         mode = wipes[offset_sectors]
         if mode is not None:
+            # Wipe the new partitions as needed.
             block.wipe_volume(block.kname_to_path(kname), mode)
+        resize = resizes.get(offset_sectors)
+        if resize and resize['start'] < size:
+            perform_resize(kname, resize['end'], 'up')
 
     # Make the names if needed
     if 'name' in info:
diff --git a/curtin/storage_config.py b/curtin/storage_config.py
index e9a8fdd..2ede996 100644
--- a/curtin/storage_config.py
+++ b/curtin/storage_config.py
@@ -1357,4 +1357,12 @@ def extract_storage_config(probe_data, strict=False):
     return {'storage': merged_config}
 
 
+def select_configs(storage_config, **kwargs):
+    """ Given a set of key=value arguments, return a list of the configs that
+        match all specified key-value pairs.
+    """
+    return [cfg for cfg in storage_config.values()
+            if all(cfg.get(k) == v for k, v in kwargs.items())]
+
+
 # vi: ts=4 expandtab syntax=python
diff --git a/curtin/util.py b/curtin/util.py
index 5b66b55..d3c3b66 100644
--- a/curtin/util.py
+++ b/curtin/util.py
@@ -501,6 +501,13 @@ def chdir(dirname):
         os.chdir(curdir)
 
 
+@contextmanager
+def mount(src, target):
+    do_mount(src, target)
+    yield
+    do_umount(target)
+
+
 def do_mount(src, target, opts=None):
     # mount src at target with opts and return True
     # if already mounted, return False
diff --git a/doc/topics/storage.rst b/doc/topics/storage.rst
index 5fae90f..e6dd13a 100644
--- a/doc/topics/storage.rst
+++ b/doc/topics/storage.rst
@@ -34,6 +34,12 @@ only differ in the interpretation of ``partition`` actions at this
 time. ``lvm_partition`` actions will be interpreted differently at
 some point in the future.
 
+.. note::
+
+  Config version ``2`` is under active development and subject to change.
+  Users are advised to use version ``1`` unless features enabled by version
+  ``2`` are required.
+
 Configuration Types
 -------------------
 Each entry in the config list is a dictionary with several keys which vary
@@ -429,6 +435,17 @@ filesystem or be mounted anywhere on the system.
 
 If the preserve flag is set to true, curtin will verify that the partition
 exists and that  the ``size`` and ``flag`` match the configuration provided.
+See also the ``resize`` flag, which adjusts this behavior.
+
+**resize**: *true, false*
+
+Only applicable to v2 storage configuration.
+If the ``preserve`` flag is set to false, this value is not applicable.
+If the ``preserve`` flag is set to true, curtin will adjust the size of the
+partition to the new size.  When adjusting smaller, the size of the contents
+must permit that.  When adjusting larger, there must already be a gap beyond
+the partition in question.
+Resize is supported on filesystems of types ext2, ext3, ext4.
 
 **name**: *<name>*
 
diff --git a/tests/integration/test_block_meta.py b/tests/integration/test_block_meta.py
index 0c74cd6..be69bc0 100644
--- a/tests/integration/test_block_meta.py
+++ b/tests/integration/test_block_meta.py
@@ -2,6 +2,7 @@
 
 from collections import namedtuple
 import contextlib
+import json
 import sys
 import yaml
 import os
@@ -34,6 +35,32 @@ def loop_dev(image, sector_size=512):
 PartData = namedtuple("PartData", ('number', 'offset', 'size'))
 
 
+def _get_filesystem_size(dev, part_action, fstype='ext4'):
+    if fstype not in ('ext2', 'ext3', 'ext4'):
+        raise Exception(f'_get_filesystem_size: no support for {fstype}')
+    num = part_action['number']
+    cmd = ['dumpe2fs', '-h', f'{dev}p{num}']
+    out = util.subp(cmd, capture=True)[0]
+    for line in out.splitlines():
+        if line.startswith('Block count'):
+            block_count = line.split(':')[1].strip()
+        if line.startswith('Block size'):
+            block_size = line.split(':')[1].strip()
+    return int(block_count) * int(block_size)
+
+
+def _get_extended_partition_size(dev, num):
+    # sysfs reports extended partitions as having 1K size
+    # sfdisk seems to have a better idea
+    ptable_json = util.subp(['sfdisk', '-J', dev], capture=True)[0]
+    ptable = json.loads(ptable_json)
+
+    nodename = f'{dev}p{num}'
+    partitions = ptable['partitiontable']['partitions']
+    partition = [part for part in partitions if part['node'] == nodename][0]
+    return partition['size'] * 512
+
+
 def summarize_partitions(dev):
     # We don't care about the kname
     return sorted(
@@ -55,33 +82,32 @@ class StorageConfigBuilder:
                 },
             }
 
-    def add_image(self, *, path, size, create=False, **kw):
-        action = {
-            'type': 'image',
-            'id': 'id' + str(len(self.config)),
-            'path': path,
-            'size': size,
-            }
-        action.update(**kw)
-        self.cur_image = action['id']
+    def _add(self, *, type, **kw):
+        if type != 'image' and self.cur_image is None:
+            raise Exception("no current image")
+        action = {'id': 'id' + str(len(self.config))}
+        action.update(type=type, **kw)
         self.config.append(action)
+        return action
+
+    def add_image(self, *, path, size, create=False, **kw):
         if create:
             with open(path, "wb") as f:
                 f.write(b"\0" * int(util.human2bytes(size)))
+        action = self._add(type='image', path=path, size=size, **kw)
+        self.cur_image = action['id']
         return action
 
     def add_part(self, *, size, **kw):
-        if self.cur_image is None:
-            raise Exception("no current image")
-        action = {
-            'type': 'partition',
-            'id': 'id' + str(len(self.config)),
-            'device': self.cur_image,
-            'size': size,
-            }
-        action.update(**kw)
-        self.config.append(action)
-        return action
+        fstype = kw.pop('fstype', None)
+        part = self._add(type='partition', device=self.cur_image, size=size,
+                         **kw)
+        if fstype:
+            self.add_format(part=part, fstype=fstype)
+        return part
+
+    def add_format(self, *, part, fstype='ext4', **kw):
+        return self._add(type='format', volume=part['id'], fstype=fstype, **kw)
 
     def set_preserve(self):
         for action in self.config:
@@ -89,6 +115,29 @@ class StorageConfigBuilder:
 
 
 class TestBlockMeta(IntegrationTestCase):
+    def setUp(self):
+        self.data = self.random_string()
+
+    @contextlib.contextmanager
+    def mount(self, dev, partition_cfg):
+        mnt_point = self.tmp_dir()
+        num = partition_cfg['number']
+        with util.mount(f'{dev}p{num}', mnt_point):
+            yield mnt_point
+
+    @contextlib.contextmanager
+    def open_file_on_part(self, dev, part_action, mode):
+        with self.mount(dev, part_action) as mnt_point:
+            with open(f'{mnt_point}/data.txt', mode) as fp:
+                yield fp
+
+    def create_data(self, dev, part_action):
+        with self.open_file_on_part(dev, part_action, 'w') as fp:
+            fp.write(self.data)
+
+    def check_data(self, dev, part_action):
+        with self.open_file_on_part(dev, part_action, 'r') as fp:
+            self.assertEqual(self.data, fp.read())
 
     def run_bm(self, config, *args, **kwargs):
         config_path = self.tmp_path('config.yaml')
@@ -223,7 +272,10 @@ class TestBlockMeta(IntegrationTestCase):
         img = self.tmp_path('image.img')
         config = StorageConfigBuilder(version=version)
         config.add_image(path=img, size='100M', ptable='msdos')
-        config.add_part(size='50M', number=1, flag='extended')
+        # curtin adds 1MiB to the size of the extend partition per contained
+        # logical partition, but only in v1 mode
+        size = '97M' if version == 1 else '99M'
+        config.add_part(size=size, number=1, flag='extended')
         config.add_part(size='10M', number=5, flag='logical')
         config.add_part(size='10M', number=6, flag='logical')
         self.run_bm(config.render())
@@ -238,6 +290,7 @@ class TestBlockMeta(IntegrationTestCase):
                     # gap.
                     PartData(number=6, offset=13 << 20, size=10 << 20),
                 ])
+            self.assertEqual(99 << 20, _get_extended_partition_size(dev, 1))
 
             p1kname = block.partition_kname(block.path_to_kname(dev), 1)
             self.assertTrue(block.is_extended_partition('/dev/' + p1kname))
@@ -303,6 +356,7 @@ class TestBlockMeta(IntegrationTestCase):
                     PartData(number=5, offset=(2 << 20),         size=psize),
                     PartData(number=6, offset=(3 << 20) + psize, size=psize),
                 ])
+            self.assertEqual(90 << 20, _get_extended_partition_size(dev, 1))
 
         config = StorageConfigBuilder(version=2)
         config.add_image(path=img, size='100M', ptable='msdos', preserve=True)
@@ -318,6 +372,7 @@ class TestBlockMeta(IntegrationTestCase):
                     PartData(number=1, offset=1 << 20,           size=1 << 10),
                     PartData(number=5, offset=(3 << 20) + psize, size=psize),
                 ])
+            self.assertEqual(90 << 20, _get_extended_partition_size(dev, 1))
 
     def _test_wiping(self, ptable):
         # Test wiping behaviour.
@@ -415,3 +470,306 @@ class TestBlockMeta(IntegrationTestCase):
                     )
         finally:
             server.stop()
+
+    def _do_test_resize(self, start, end, fstype):
+        start <<= 20
+        end <<= 20
+        img = self.tmp_path('image.img')
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='200M', ptable='gpt')
+        p1 = config.add_part(size=start, offset=1 << 20, number=1,
+                             fstype=fstype)
+        self.run_bm(config.render())
+        with loop_dev(img) as dev:
+            self.create_data(dev, p1)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=start),
+                ])
+            fs_size = _get_filesystem_size(dev, p1, fstype)
+            self.assertEqual(start, fs_size)
+
+        config.set_preserve()
+        p1['resize'] = True
+        p1['size'] = end
+        self.run_bm(config.render())
+        with loop_dev(img) as dev:
+            self.check_data(dev, p1)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=end),
+                ])
+            fs_size = _get_filesystem_size(dev, p1, fstype)
+            self.assertEqual(end, fs_size)
+
+    def test_resize_up_ext2(self):
+        self._do_test_resize(40, 80, 'ext2')
+
+    def test_resize_down_ext2(self):
+        self._do_test_resize(80, 40, 'ext2')
+
+    def test_resize_up_ext3(self):
+        self._do_test_resize(40, 80, 'ext3')
+
+    def test_resize_down_ext3(self):
+        self._do_test_resize(80, 40, 'ext3')
+
+    def test_resize_up_ext4(self):
+        self._do_test_resize(40, 80, 'ext4')
+
+    def test_resize_down_ext4(self):
+        self._do_test_resize(80, 40, 'ext4')
+
+    def test_resize_logical(self):
+        img = self.tmp_path('image.img')
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='100M', ptable='msdos')
+        config.add_part(size='50M', number=1, flag='extended', offset=1 << 20)
+        config.add_part(size='10M', number=5, flag='logical', offset=2 << 20)
+        p6 = config.add_part(size='10M', number=6, flag='logical',
+                             offset=13 << 20, fstype='ext4')
+        self.run_bm(config.render())
+
+        with loop_dev(img) as dev:
+            self.create_data(dev, p6)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    # extended partitions get a strange size in sysfs
+                    PartData(number=1, offset=1 << 20,  size=1 << 10),
+                    PartData(number=5, offset=2 << 20,  size=10 << 20),
+                    # part 5 takes us to 12 MiB offset, curtin leaves a 1 MiB
+                    # gap.
+                    PartData(number=6, offset=13 << 20, size=10 << 20),
+                ])
+            self.assertEqual(50 << 20, _get_extended_partition_size(dev, 1))
+
+        config.set_preserve()
+        p6['resize'] = True
+        p6['size'] = '20M'
+        self.run_bm(config.render())
+
+        with loop_dev(img) as dev:
+            self.check_data(dev, p6)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20,  size=1 << 10),
+                    PartData(number=5, offset=2 << 20,  size=10 << 20),
+                    PartData(number=6, offset=13 << 20, size=20 << 20),
+                ])
+            self.assertEqual(50 << 20, _get_extended_partition_size(dev, 1))
+
+    def test_resize_extended(self):
+        img = self.tmp_path('image.img')
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='100M', ptable='msdos')
+        p1 = config.add_part(size='50M', number=1, flag='extended',
+                             offset=1 << 20)
+        p5 = config.add_part(size='49M', number=5, flag='logical',
+                             offset=2 << 20)
+        self.run_bm(config.render())
+
+        with loop_dev(img) as dev:
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    # extended partitions get a strange size in sysfs
+                    PartData(number=1, offset=1 << 20,  size=1 << 10),
+                    PartData(number=5, offset=2 << 20,  size=49 << 20),
+                ])
+            self.assertEqual(50 << 20, _get_extended_partition_size(dev, 1))
+
+        config.set_preserve()
+        p1['resize'] = True
+        p1['size'] = '99M'
+        p5['resize'] = True
+        p5['size'] = '98M'
+        self.run_bm(config.render())
+
+        with loop_dev(img) as dev:
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20,  size=1 << 10),
+                    PartData(number=5, offset=2 << 20,  size=98 << 20),
+                ])
+            self.assertEqual(99 << 20, _get_extended_partition_size(dev, 1))
+
+    def test_split(self):
+        img = self.tmp_path('image.img')
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='200M', ptable='gpt')
+        config.add_part(size=9 << 20, offset=1 << 20, number=1)
+        p2 = config.add_part(size='180M', offset=10 << 20, number=2,
+                             fstype='ext4')
+        self.run_bm(config.render())
+        with loop_dev(img) as dev:
+            self.create_data(dev, p2)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=9 << 20),
+                    PartData(number=2, offset=10 << 20, size=180 << 20),
+                ])
+            self.assertEqual(180 << 20, _get_filesystem_size(dev, p2))
+
+        config.set_preserve()
+        p2['resize'] = True
+        p2['size'] = '80M'
+        p3 = config.add_part(size='100M', offset=90 << 20, number=3,
+                             fstype='ext4')
+        self.run_bm(config.render())
+        with loop_dev(img) as dev:
+            self.check_data(dev, p2)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=9 << 20),
+                    PartData(number=2, offset=10 << 20, size=80 << 20),
+                    PartData(number=3, offset=90 << 20, size=100 << 20),
+                ])
+            self.assertEqual(80 << 20, _get_filesystem_size(dev, p2))
+            self.assertEqual(100 << 20, _get_filesystem_size(dev, p3))
+
+    def test_partition_unify(self):
+        img = self.tmp_path('image.img')
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='200M', ptable='gpt')
+        config.add_part(size=9 << 20, offset=1 << 20, number=1)
+        p2 = config.add_part(size='40M', offset=10 << 20, number=2,
+                             fstype='ext4')
+        p3 = config.add_part(size='60M', offset=50 << 20, number=3,
+                             fstype='ext4')
+        self.run_bm(config.render())
+        with loop_dev(img) as dev:
+            self.create_data(dev, p2)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=9 << 20),
+                    PartData(number=2, offset=10 << 20, size=40 << 20),
+                    PartData(number=3, offset=50 << 20, size=60 << 20),
+                ])
+            self.assertEqual(40 << 20, _get_filesystem_size(dev, p2))
+            self.assertEqual(60 << 20, _get_filesystem_size(dev, p3))
+
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='200M', ptable='gpt')
+        config.add_part(size=9 << 20, offset=1 << 20, number=1)
+        p2 = config.add_part(size='100M', offset=10 << 20, number=2,
+                             fstype='ext4', resize=True)
+        config.set_preserve()
+        self.run_bm(config.render())
+        with loop_dev(img) as dev:
+            self.check_data(dev, p2)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=9 << 20),
+                    PartData(number=2, offset=10 << 20, size=100 << 20),
+                ])
+            self.assertEqual(100 << 20, _get_filesystem_size(dev, p2))
+
+    def test_mix_of_operations_gpt(self):
+        # a test that keeps, creates, resizes, and deletes a partition
+        # 200 MiB disk, using full disk
+        #      init size preserve     final size
+        # p1 -  9 MiB    yes            9MiB
+        # p2 - 90 MiB    yes, resize  139MiB
+        # p3 - 99 MiB    no            50MiB
+        img = self.tmp_path('image.img')
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='200M', ptable='gpt')
+        config.add_part(size=9 << 20, offset=1 << 20, number=1)
+        p2 = config.add_part(size='90M', offset=10 << 20, number=2,
+                             fstype='ext4')
+        p3 = config.add_part(size='99M', offset=100 << 20, number=3,
+                             fstype='ext4')
+        self.run_bm(config.render())
+        with loop_dev(img) as dev:
+            self.create_data(dev, p2)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=9 << 20),
+                    PartData(number=2, offset=10 << 20, size=90 << 20),
+                    PartData(number=3, offset=100 << 20, size=99 << 20),
+                ])
+            self.assertEqual(90 << 20, _get_filesystem_size(dev, p2))
+            self.assertEqual(99 << 20, _get_filesystem_size(dev, p3))
+
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='200M', ptable='gpt')
+        config.add_part(size=9 << 20, offset=1 << 20, number=1)
+        p2 = config.add_part(size='139M', offset=10 << 20, number=2,
+                             fstype='ext4', resize=True)
+        config.set_preserve()
+        p3 = config.add_part(size='50M', offset=149 << 20, number=3,
+                             fstype='ext4')
+        self.run_bm(config.render())
+        with loop_dev(img) as dev:
+            self.check_data(dev, p2)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=9 << 20),
+                    PartData(number=2, offset=10 << 20, size=139 << 20),
+                    PartData(number=3, offset=149 << 20, size=50 << 20),
+                ])
+            self.assertEqual(139 << 20, _get_filesystem_size(dev, p2))
+            self.assertEqual(50 << 20, _get_filesystem_size(dev, p3))
+
+    def test_mix_of_operations_msdos(self):
+        # a test that keeps, creates, resizes, and deletes a partition
+        # including handling of extended/logical
+        # 200 MiB disk, initially only using front 100MiB
+        #      flag     init size preserve     final size
+        # p1 - primary   9MiB     yes            9MiB
+        # p2 - extended 89MiB     yes, resize  189MiB
+        # p3 - logical  37MiB     yes, resize  137MiB
+        # p4 - logical  50MiB     no            50MiB
+        img = self.tmp_path('image.img')
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='200M', ptable='msdos')
+        p1 = config.add_part(size='9M', offset=1 << 20, number=1,
+                             fstype='ext4')
+        config.add_part(size='89M', offset=10 << 20, number=2, flag='extended')
+        p5 = config.add_part(size='36M', offset=11 << 20, number=5,
+                             flag='logical', fstype='ext4')
+        p6 = config.add_part(size='50M', offset=49 << 20, number=6,
+                             flag='logical', fstype='ext4')
+        self.run_bm(config.render())
+
+        with loop_dev(img) as dev:
+            self.create_data(dev, p1)
+            self.create_data(dev, p5)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=9 << 20),
+                    PartData(number=2, offset=10 << 20, size=1 << 10),
+                    PartData(number=5, offset=11 << 20, size=36 << 20),
+                    PartData(number=6, offset=49 << 20, size=50 << 20),
+                ])
+            self.assertEqual(89 << 20, _get_extended_partition_size(dev, 2))
+            self.assertEqual(9 << 20, _get_filesystem_size(dev, p1))
+            self.assertEqual(36 << 20, _get_filesystem_size(dev, p5))
+            self.assertEqual(50 << 20, _get_filesystem_size(dev, p6))
+
+        config = StorageConfigBuilder(version=2)
+        config.add_image(path=img, size='200M', ptable='msdos')
+        p1 = config.add_part(size='9M', offset=1 << 20, number=1,
+                             fstype='ext4')
+        config.add_part(size='189M', offset=10 << 20, number=2,
+                        flag='extended', resize=True)
+        p5 = config.add_part(size='136M', offset=11 << 20, number=5,
+                             flag='logical', fstype='ext4', resize=True)
+        config.set_preserve()
+        p6 = config.add_part(size='50M', offset=149 << 20, number=6,
+                             flag='logical', fstype='ext4')
+        self.run_bm(config.render())
+
+        with loop_dev(img) as dev:
+            self.check_data(dev, p1)
+            self.check_data(dev, p5)
+            self.assertEqual(
+                summarize_partitions(dev), [
+                    PartData(number=1, offset=1 << 20, size=9 << 20),
+                    PartData(number=2, offset=10 << 20, size=1 << 10),
+                    PartData(number=5, offset=11 << 20, size=136 << 20),
+                    PartData(number=6, offset=149 << 20, size=50 << 20),
+                ])
+            self.assertEqual(189 << 20, _get_extended_partition_size(dev, 2))
+            self.assertEqual(9 << 20, _get_filesystem_size(dev, p1))
+            self.assertEqual(136 << 20, _get_filesystem_size(dev, p5))
+            self.assertEqual(50 << 20, _get_filesystem_size(dev, p6))
diff --git a/tests/unittests/test_commands_block_meta.py b/tests/unittests/test_commands_block_meta.py
index 7c8e7bf..3698d32 100644
--- a/tests/unittests/test_commands_block_meta.py
+++ b/tests/unittests/test_commands_block_meta.py
@@ -3,12 +3,16 @@
 from argparse import Namespace
 from collections import OrderedDict
 import copy
-from mock import patch, call
+from mock import (
+    call,
+    Mock,
+    patch,
+)
 import os
 import random
 
 from curtin.block import dasd
-from curtin.commands import block_meta
+from curtin.commands import block_meta, block_meta_v2
 from curtin import paths, util
 from .helpers import CiTestCase
 
@@ -2613,6 +2617,133 @@ class TestPartitionVerifySfdisk(CiTestCase):
         self.assertEqual([], self.m_verify_ptable_flag.call_args_list)
 
 
+class TestPartitionVerifySfdiskV2(CiTestCase):
+
+    def setUp(self):
+        super(TestPartitionVerifySfdiskV2, self).setUp()
+        base = 'curtin.commands.block_meta_v2.'
+        self.add_patch(base + 'verify_size', 'm_verify_size')
+        self.add_patch(base + 'verify_ptable_flag', 'm_verify_ptable_flag')
+        self.add_patch(base + 'os.path.realpath', 'm_realpath')
+        self.m_realpath.side_effect = lambda x: x
+        self.info = {
+            'id': 'disk-sda-part-2',
+            'type': 'partition',
+            'offset': '1GB',
+            'device': 'sda',
+            'number': 2,
+            'size': '5GB',
+            'flag': 'boot',
+        }
+        self.part_size = int(util.human2bytes(self.info['size']))
+        self.devpath = self.random_string()
+        self.sfdisk_part_info = {
+            'node': self.devpath,
+            'start': (1 << 30) // 512,
+        }
+        self.storage_config = {self.info['id']: self.info}
+        self.label = self.random_string()
+        self.table = Mock()
+        self.table.sectors2bytes = lambda x: x * 512
+
+    def test_partition_verify_sfdisk(self):
+        block_meta_v2.partition_verify_sfdisk_v2(self.info, self.label,
+                                                 self.sfdisk_part_info,
+                                                 self.storage_config,
+                                                 self.table)
+        self.assertEqual(
+            [call(self.devpath, self.part_size, self.sfdisk_part_info)],
+            self.m_verify_size.call_args_list)
+        self.assertEqual(
+            [call(self.devpath, self.info['flag'], self.label,
+                  self.sfdisk_part_info)],
+            self.m_verify_ptable_flag.call_args_list)
+
+    def test_partition_verify_no_moves(self):
+        self.info['preserve'] = True
+        self.info['resize'] = True
+        self.info['offset'] = '2GB'
+        with self.assertRaises(RuntimeError):
+            block_meta_v2.partition_verify_sfdisk_v2(
+                self.info, self.label, self.sfdisk_part_info,
+                self.storage_config, self.table)
+
+
+class TestPartitionNeedsResize(CiTestCase):
+
+    def setUp(self):
+        super(TestPartitionNeedsResize, self).setUp()
+        base = 'curtin.commands.block_meta_v2.'
+        self.add_patch(base + 'os.path.realpath', 'm_realpath')
+        self.add_patch(base + '_get_volume_fstype', 'm_get_volume_fstype')
+        self.m_realpath.side_effect = lambda x: x
+        self.partition = {
+            'id': 'disk-sda-part-2',
+            'type': 'partition',
+            'offset': '1GB',
+            'device': 'sda',
+            'number': 2,
+            'size': '5GB',
+            'flag': 'boot',
+        }
+        self.devpath = self.random_string()
+        self.sfdisk_part_info = {
+            'node': self.devpath,
+            'start': (1 << 30) // 512,
+        }
+        self.format = {
+            'id': 'id-format',
+            'type': 'format',
+            'fstype': 'ext4',
+            'volume': self.partition['id'],
+        }
+        self.storage_config = {
+            self.partition['id']: self.partition,
+            self.format['id']: self.format,
+        }
+
+    def test_partition_resize_change_fs(self):
+        self.partition['preserve'] = True
+        self.partition['resize'] = True
+        self.format['preserve'] = True
+        self.format['fstype'] = 'ext3'
+        self.m_get_volume_fstype.return_value = 'ext4'
+        with self.assertRaises(RuntimeError):
+            block_meta_v2.needs_resize(
+                self.storage_config, self.partition, self.sfdisk_part_info)
+
+    def test_partition_resize_unsupported_fs(self):
+        self.partition['preserve'] = True
+        self.partition['resize'] = True
+        self.format['preserve'] = True
+        self.format['fstype'] = 'reiserfs'
+        self.m_get_volume_fstype.return_value = 'resierfs'
+        with self.assertRaises(RuntimeError):
+            block_meta_v2.needs_resize(
+                self.storage_config, self.partition, self.sfdisk_part_info)
+
+    def test_partition_resize_format_preserve_false(self):
+        # though the filesystem type is not supported for resize, it's ok
+        # because with format preserve=False, we're recreating anyhow
+        self.partition['preserve'] = True
+        self.partition['resize'] = True
+        self.format['preserve'] = False
+        self.format['fstype'] = 'reiserfs'
+        self.m_get_volume_fstype.return_value = 'reiserfs'
+        block_meta_v2.needs_resize(
+            self.storage_config, self.partition, self.sfdisk_part_info)
+
+    def test_partition_resize_partition_preserve_false(self):
+        # not a resize - partition is recreated
+        self.partition['preserve'] = False
+        self.partition['resize'] = True
+        self.format['preserve'] = False
+        self.format['fstype'] = 'reiserfs'
+        self.m_get_volume_fstype.return_value = 'reiserfs'
+        block_meta_v2.needs_resize(
+            self.storage_config, self.partition, self.sfdisk_part_info)
+
+
 class TestPartitionVerifyFdasd(CiTestCase):
 
     def setUp(self):
diff --git a/tests/unittests/test_storage_config.py b/tests/unittests/test_storage_config.py
index f0f8148..d6b0a36 100644
--- a/tests/unittests/test_storage_config.py
+++ b/tests/unittests/test_storage_config.py
@@ -7,7 +7,7 @@ from curtin.storage_config import ProbertParser as baseparser
 from curtin.storage_config import (BcacheParser, BlockdevParser, DasdParser,
                                    DmcryptParser, FilesystemParser, LvmParser,
                                    RaidParser, MountParser, ZfsParser)
-from curtin.storage_config import ptable_uuid_to_flag_entry
+from curtin.storage_config import ptable_uuid_to_flag_entry, select_configs
 from curtin import util
 
 
@@ -1117,4 +1117,26 @@ class TestExtractStorageConfig(CiTestCase):
         self.assertEqual(expected_dict, bitlocker[0])
 
 
+class TestSelectConfigs(CiTestCase):
+    def test_basic(self):
+        id0 = {'a': 1, 'b': 2}
+        id1 = {'a': 1, 'c': 3}
+        sc = {'id0': id0, 'id1': id1}
+
+        self.assertEqual([id0, id1], select_configs(sc, a=1))
+
+    def test_not_found(self):
+        id0 = {'a': 1, 'b': 2}
+        id1 = {'a': 1, 'c': 3}
+        sc = {'id0': id0, 'id1': id1}
+
+        self.assertEqual([], select_configs(sc, a=4))
+
+    def test_multi_criteria(self):
+        id0 = {'a': 1, 'b': 2}
+        id1 = {'a': 1, 'c': 3}
+        sc = {'id0': id0, 'id1': id1}
+
+        self.assertEqual([id0], select_configs(sc, a=1, b=2))
+
 # vi: ts=4 expandtab syntax=python