Merge ~gyurco/curtin:imsm into curtin:master

[György Szombathelyi]

Support imsm external metadata RAID containers

LP: #1893661

[Ryan Harper (raharper)]

Thanks for working on this.

I've provided some feedback below in the code. In addition to what we have here, this will also need updates to curtin/block/schema.py as we're adding new fields to the raid structure.

diff --git a/curtin/block/schemas.py b/curtin/block/schemas.py
index 9e2c41fe..4dc2f0a7 100644
--- a/curtin/block/schemas.py
+++ b/curtin/block/schemas.py
@@ -316,12 +316,14 @@ RAID = {
         'preserve': {'$ref': '#/definitions/preserve'},
         'ptable': {'$ref': '#/definitions/ptable'},
         'spare_devices': {'$ref': '#/definitions/devices'},
+ 'container': {'$ref': '#/definitions/id'},
         'type': {'const': 'raid'},
         'raidlevel': {
             'type': ['integer', 'string'],
             'oneOf': [
                 {'enum': [0, 1, 4, 5, 6, 10]},
- {'enum': ['raid0', 'linear', '0',
+ {'enum': ['container',
+ 'raid0', 'linear', '0',
                           'raid1', 'mirror', 'stripe', '1',
                           'raid4', '4',
                           'raid5', '5',

review: Needs Fixing

[György Szombathelyi (gyurco)]

I have some minor explanations about some of your concerns, otherwise I'll try to implement as you suggested.

[Ryan Harper (raharper)]

Thanks.

0141803... by György Szombathelyi on 2020-09-10

Add md_is_container

3856b5c... by György Szombathelyi on 2020-09-10

Use container instead of container_devcnt

6168dbb... by György Szombathelyi on 2020-09-10

Add container to RAID schema

341a674... by György Szombathelyi on 2020-09-10

Clarify comment

e3158e4... by György Szombathelyi on 2020-09-10

Revert unnecessary change

[György Szombathelyi (gyurco)]

I think I've addressed most(all) of the concerns. Can you review it, please? I have to commission the test servers into production this week, after that I cannot continue the work until I receive another batch of machines.

[Ryan Harper (raharper)]

One suggestion in line.

review: Needs Fixing

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:1d466b3cb87ca26c173f0aadfc41ecce4cd3340a
https://jenkins.ubuntu.com/server/job/curtin-ci/187/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/187/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/187/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/187/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/187/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/187//rebuild

review: Approve (continuous-integration)

cd11d1a... by György Szombathelyi on 2020-09-14

md_is_container -> md_is_in container (and return bool)

[Paride Legovini (paride)]

I'll trigger another CI run after this lands:

https://code.launchpad.net/~paride/curtin/+git/curtin/+merge/390749

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:1d466b3cb87ca26c173f0aadfc41ecce4cd3340a
https://jenkins.ubuntu.com/server/job/curtin-ci/189/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/189/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/189/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/189/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/189/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/189//rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:a0b1e8c5f68ae2538b67e88349135cdfab22ef70
https://jenkins.ubuntu.com/server/job/curtin-ci/192/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/192/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/192/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/192/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/192/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/192//rebuild

review: Approve (continuous-integration)

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:a0b1e8c5f68ae2538b67e88349135cdfab22ef70
https://jenkins.ubuntu.com/server/job/curtin-ci/193/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/193/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/193/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/193/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/193/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/193//rebuild

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

FAILED: Continuous integration, rev:a0b1e8c5f68ae2538b67e88349135cdfab22ef70
https://jenkins.ubuntu.com/server/job/curtin-ci/194/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/194/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/194/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/194/
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/194/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/194//rebuild

review: Needs Fixing (continuous-integration)

[Paride Legovini (paride)]

Hi György,

I ran the CI on the wrong revision, my bad. We actually have some failures, see the full run log [1]. The errors are just linting issues:

py3-pylint:

curtin/block/clear_holders.py:170:80: E501 line too long (100 > 79 characters)
curtin/block/mdadm.py:149:80: E501 line too long (89 > 79 characters)
curtin/block/mdadm.py:163:80: E501 line too long (83 > 79 characters)
curtin/block/mdadm.py:165:80: E501 line too long (89 > 79 characters)
curtin/block/mdadm.py:362:1: E302 expected 2 blank lines, found 1
curtin/block/mdadm.py:519:80: E501 line too long (84 > 79 characters)
curtin/block/mdadm.py:615:41: E703 statement ends with a semicolon
curtin/block/mdadm.py:853:1: E302 expected 2 blank lines, found 1
curtin/commands/block_meta.py:1495:80: E501 line too long (88 > 79 characters)
curtin/commands/block_meta.py:1509:21: E128 continuation line under-indented for visual indent
tests/unittests/test_block_mdadm.py:274:80: E501 line too long (98 > 79 characters)
tests/unittests/test_block_mdadm.py:277:1: E302 expected 2 blank lines, found 1
tests/unittests/test_block_mdadm.py:1340:1: E302 expected 2 blank lines, found 1
tests/unittests/test_clear_holders.py:268:29: E128 continuation line under-indented for visual indent
tests/unittests/test_clear_holders.py:272:9: F841 local variable 'md_devs' is assigned to but never used
tests/unittests/test_clear_holders.py:278:69: E231 missing whitespace after ':'
tests/unittests/test_clear_holders.py:278:80: E501 line too long (85 > 79 characters)

py3-pyflakes:

tests/unittests/test_clear_holders.py:272: local variable 'md_devs' is assigned to but never used

Please rebase your branch on master (mainly to pickup [2]), run `tox` and fix all the linting issues it will spot. In general I think the MP is in good shape, thanks for working on it!

Paride

[1] https://paste.ubuntu.com/p/s3zCszGCMy/
[2] https://git.launchpad.net/curtin/commit/?id=ac7d5e5c5f6b14fb9a1429594a4d28abe9448793

review: Needs Fixing

0e2b8ff... by György Szombathelyi on 2020-09-22

Fix lint issues

[Server Team CI bot (server-team-bot)]

PASSED: Continuous integration, rev:0e2b8ff3711d178d2a5c4ad3ff1c78e9ecc827b6
https://jenkins.ubuntu.com/server/job/curtin-ci/197/
Executed test runs:
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-amd64/197/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-arm64/197/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-ppc64el/197/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-ci/nodes=metal-s390x/197/

Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/curtin-ci/197//rebuild

review: Approve (continuous-integration)

[Paride Legovini (paride)]

I can't test on actual hardware but the code LGTM. Thanks for this MP!

review: Approve

[Server Team CI bot (server-team-bot)]

Autolanding: FAILED
More details in the following jenkins job:
https://jenkins.ubuntu.com/server/job/curtin-autoland-test/248/
Executed test runs:
    FAILURE: https://jenkins.ubuntu.com/server/job/curtin-autoland-test/nodes=metal-amd64/248/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-autoland-test/nodes=metal-arm64/248/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-autoland-test/nodes=metal-ppc64el/248/
    SUCCESS: https://jenkins.ubuntu.com/server/job/curtin-autoland-test/nodes=metal-s390x/248/

review: Needs Fixing (continuous-integration)

[Server Team CI bot (server-team-bot)]

review: Approve (continuous-integration)

diff --git a/curtin/block/clear_holders.py b/curtin/block/clear_holders.py
index 116ee81..9553db6 100644
--- a/curtin/block/clear_holders.py
+++ b/curtin/block/clear_holders.py
@@ -166,10 +166,16 @@ def shutdown_mdadm(device):
 
     blockdev = block.sysfs_to_devpath(device)
 
-    LOG.info('Discovering raid devices and spares for %s', device)
-    md_devs = (
-        mdadm.md_get_devices_list(blockdev) +
-        mdadm.md_get_spares_list(blockdev))
+    if mdadm.md_is_in_container(blockdev):
+        LOG.info('Array is in a container, skip discovering ' +
+                 'raid devices and spares for %s', device)
+        md_devs = []
+    else:
+        LOG.info('Discovering raid devices and spares for %s', device)
+        md_devs = (
+            mdadm.md_get_devices_list(blockdev) +
+            mdadm.md_get_spares_list(blockdev))
+
     mdadm.set_sync_action(blockdev, action="idle")
     mdadm.set_sync_action(blockdev, action="frozen")
 
diff --git a/curtin/block/mdadm.py b/curtin/block/mdadm.py
index 32b467c..a6ac970 100644
--- a/curtin/block/mdadm.py
+++ b/curtin/block/mdadm.py
@@ -26,6 +26,7 @@ from curtin.log import LOG
 
 NOSPARE_RAID_LEVELS = [
     'linear', 'raid0', '0', 0,
+    'container'
 ]
 
 SPARE_RAID_LEVELS = [
@@ -145,8 +146,8 @@ def mdadm_assemble(md_devname=None, devices=[], spares=[], scan=False,
     udev.udevadm_settle()
 
 
-def mdadm_create(md_devname, raidlevel, devices, spares=None, md_name="",
-                 metadata=None):
+def mdadm_create(md_devname, raidlevel, devices, spares=None, container=None,
+                 md_name="", metadata=None):
     LOG.debug('mdadm_create: ' +
               'md_name=%s raidlevel=%s ' % (md_devname, raidlevel) +
               ' devices=%s spares=%s name=%s' % (devices, spares, md_name))
@@ -159,8 +160,11 @@ def mdadm_create(md_devname, raidlevel, devices, spares=None, md_name="",
         raise ValueError('Invalid raidlevel: [{}]'.format(raidlevel))
 
     min_devices = md_minimum_devices(raidlevel)
-    if len(devices) < min_devices:
-        err = 'Not enough devices for raidlevel: ' + str(raidlevel)
+    devcnt = len(devices) if not container else \
+        len(md_get_devices_list(container))
+    if devcnt < min_devices:
+        err = 'Not enough devices (' + str(devcnt) + ') '
+        err += 'for raidlevel: ' + str(raidlevel)
         err += ' minimum devices needed: ' + str(min_devices)
         raise ValueError(err)
 
@@ -171,13 +175,20 @@ def mdadm_create(md_devname, raidlevel, devices, spares=None, md_name="",
     (hostname, _err) = util.subp(["hostname", "-s"], rcs=[0], capture=True)
 
     cmd = ["mdadm", "--create", md_devname, "--run",
-           "--metadata=%s" % metadata,
            "--homehost=%s" % hostname.strip(),
-           "--level=%s" % raidlevel,
-           "--raid-devices=%s" % len(devices)]
+           "--raid-devices=%s" % devcnt]
+
+    if not container:
+        cmd.append("--metadata=%s" % metadata)
+    if raidlevel != 'container':
+        cmd.append("--level=%s" % raidlevel)
+
     if md_name:
         cmd.append("--name=%s" % md_name)
 
+    if container:
+        cmd.append(container)
+
     for device in devices:
         holders = get_holders(device)
         if len(holders) > 0:
@@ -508,7 +519,8 @@ def md_sysfs_attr(md_devname, attrname):
 
 
 def md_raidlevel_short(raidlevel):
-    if isinstance(raidlevel, int) or raidlevel in ['linear', 'stripe']:
+    if isinstance(raidlevel, int) or \
+       raidlevel in ['linear', 'stripe', 'container']:
         return raidlevel
 
     return int(raidlevel.replace('raid', ''))
@@ -517,7 +529,7 @@ def md_raidlevel_short(raidlevel):
 def md_minimum_devices(raidlevel):
     ''' return the minimum number of devices for a given raid level '''
     rl = md_raidlevel_short(raidlevel)
-    if rl in [0, 1, 'linear', 'stripe']:
+    if rl in [0, 1, 'linear', 'stripe', 'container']:
         return 2
     if rl in [5]:
         return 3
@@ -603,6 +615,11 @@ def __mdadm_detail_to_dict(input):
     # start after the first newline
     remainder = input[input.find('\n')+1:]
 
+    # keep only the first section (imsm container)
+    arraysection = remainder.find('\n[')
+    if arraysection != -1:
+        remainder = remainder[:arraysection]
+
     #  FIXME: probably could do a better regex to match the LHS which
     #         has one, two or three words
     rem = r'(\w+|\w+\ \w+|\w+\ \w+\ \w+)\ \:\ ([a-zA-Z0-9\-\.,: \(\)=\']+)'
@@ -837,4 +854,8 @@ def md_check(md_devname, raidlevel, devices=[], spares=[]):
     LOG.debug('RAID array OK: ' + md_devname)
     return True
 
+
+def md_is_in_container(md_devname):
+    return 'MD_CONTAINER' in mdadm_query_detail(md_devname)
+
 # vi: ts=4 expandtab syntax=python
diff --git a/curtin/block/schemas.py b/curtin/block/schemas.py
index 9e2c41f..4dc2f0a 100644
--- a/curtin/block/schemas.py
+++ b/curtin/block/schemas.py
@@ -316,12 +316,14 @@ RAID = {
         'preserve': {'$ref': '#/definitions/preserve'},
         'ptable': {'$ref': '#/definitions/ptable'},
         'spare_devices': {'$ref': '#/definitions/devices'},
+        'container': {'$ref': '#/definitions/id'},
         'type': {'const': 'raid'},
         'raidlevel': {
             'type': ['integer', 'string'],
             'oneOf': [
                 {'enum': [0, 1, 4, 5, 6, 10]},
-                {'enum': ['raid0', 'linear', '0',
+                {'enum': ['container',
+                          'raid0', 'linear', '0',
                           'raid1', 'mirror', 'stripe', '1',
                           'raid4', '4',
                           'raid5', '5',
diff --git a/curtin/commands/block_meta.py b/curtin/commands/block_meta.py
index dee73b1..30fc817 100644
--- a/curtin/commands/block_meta.py
+++ b/curtin/commands/block_meta.py
@@ -1486,21 +1486,30 @@ def raid_handler(info, storage_config):
     raidlevel = info.get('raidlevel')
     spare_devices = info.get('spare_devices')
     md_devname = block.md_path(info.get('name'))
+    container = info.get('container')
+    metadata = info.get('metadata')
     preserve = config.value_as_boolean(info.get('preserve'))
-    if not devices:
-        raise ValueError("devices for raid must be specified")
+    if not devices and not container:
+        raise ValueError("devices or container for raid must be specified")
     if raidlevel not in ['linear', 'raid0', 0, 'stripe', 'raid1', 1, 'mirror',
-                         'raid4', 4, 'raid5', 5, 'raid6', 6, 'raid10', 10]:
+                         'raid4', 4, 'raid5', 5, 'raid6', 6, 'raid10', 10,
+                         'container']:
         raise ValueError("invalid raidlevel '%s'" % raidlevel)
-    if raidlevel in ['linear', 'raid0', 0, 'stripe']:
+    if raidlevel in ['linear', 'raid0', 0, 'stripe', 'container']:
         if spare_devices:
             raise ValueError("spareunsupported in raidlevel '%s'" % raidlevel)
 
     LOG.debug('raid: cfg: %s', util.json_dumps(info))
-    device_paths = list(get_path_to_storage_volume(dev, storage_config) for
-                        dev in devices)
-    LOG.debug('raid: device path mapping: %s',
-              list(zip(devices, device_paths)))
+
+    container_dev = None
+    device_paths = []
+    if container:
+        container_dev = get_path_to_storage_volume(container, storage_config)
+    else:
+        device_paths = list(get_path_to_storage_volume(dev, storage_config) for
+                            dev in devices)
+        LOG.debug('raid: device path mapping: {}'.format(
+            zip(devices, device_paths)))
 
     spare_device_paths = []
     if spare_devices:
@@ -1517,8 +1526,8 @@ def raid_handler(info, storage_config):
 
     if create_raid:
         mdadm.mdadm_create(md_devname, raidlevel,
-                           device_paths, spare_device_paths,
-                           info.get('mdname', ''))
+                           device_paths, spare_device_paths, container_dev,
+                           info.get('mdname', ''), metadata)
 
     wipe_mode = info.get('wipe')
     if wipe_mode:
diff --git a/tests/unittests/test_block_mdadm.py b/tests/unittests/test_block_mdadm.py
index dba0f74..b04cf82 100644
--- a/tests/unittests/test_block_mdadm.py
+++ b/tests/unittests/test_block_mdadm.py
@@ -97,7 +97,7 @@ class TestBlockMdadmCreate(CiTestCase):
         self.mock_holders.return_value = []
 
     def prepare_mock(self, md_devname, raidlevel, devices, spares,
-                     metadata=None):
+                     container=None, metadata=None):
         side_effects = []
         expected_calls = []
         hostname = 'ubuntu'
@@ -120,10 +120,15 @@ class TestBlockMdadmCreate(CiTestCase):
         side_effects.append(("", ""))  # mdadm create
         # build command how mdadm_create does
         cmd = (["mdadm", "--create", md_devname, "--run",
-                "--metadata=%s" % metadata,
-                "--homehost=%s" % hostname, "--level=%s" % raidlevel,
-                "--raid-devices=%s" % len(devices)] +
-               devices)
+                "--homehost=%s" % hostname,
+                "--raid-devices=%s" % (len(devices) if not container else 4)])
+        if not container:
+            cmd += ["--metadata=%s" % metadata]
+        if raidlevel != 'container':
+            cmd += ["--level=%s" % raidlevel]
+        if container:
+            cmd += [container]
+        cmd += devices
         if spares:
             cmd += ["--spare-devices=%s" % len(spares)] + spares
 
@@ -228,6 +233,48 @@ class TestBlockMdadmCreate(CiTestCase):
                            devices=devices, spares=spares)
         self.mock_util.subp.assert_has_calls(expected_calls)
 
+    def test_mdadm_create_imsm_container(self):
+        md_devname = "/dev/md/imsm"
+        raidlevel = 'container'
+        devices = ['/dev/nvme0n1', '/dev/nvme1n1', '/dev/nvme2n1']
+        metadata = 'imsm'
+        spares = []
+        (side_effects, expected_calls) = self.prepare_mock(md_devname,
+                                                           raidlevel,
+                                                           devices,
+                                                           spares,
+                                                           None,
+                                                           metadata)
+
+        self.mock_util.subp.side_effect = side_effects
+        mdadm.mdadm_create(md_devname=md_devname, raidlevel=raidlevel,
+                           devices=devices, spares=spares, metadata=metadata)
+        self.mock_util.subp.assert_has_calls(expected_calls)
+
+    @patch("curtin.block._md_get_members_list")
+    def test_mdadm_create_array_in_imsm_container(self, mock_get_members):
+        md_devname = "/dev/md126"
+        raidlevel = 5
+        devices = []
+        metadata = 'imsm'
+        spares = []
+        container = "/dev/md/imsm"
+        (side_effects, expected_calls) = self.prepare_mock(md_devname,
+                                                           raidlevel,
+                                                           devices,
+                                                           spares,
+                                                           container,
+                                                           metadata)
+
+        self.mock_util.subp.side_effect = side_effects
+        mock_get_members.return_value = [
+            '/dev/nvme0n1', '/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1'
+        ]
+        mdadm.mdadm_create(md_devname=md_devname, raidlevel=raidlevel,
+                           devices=devices, spares=spares,
+                           container=container, metadata=metadata)
+        self.mock_util.subp.assert_has_calls(expected_calls)
+
 
 class TestBlockMdadmExamine(CiTestCase):
     def setUp(self):
@@ -315,6 +362,70 @@ class TestBlockMdadmExamine(CiTestCase):
         self.mock_util.subp.assert_has_calls(expected_calls)
         self.assertEqual(data, {})
 
+    def test_mdadm_examine_no_export_imsm(self):
+        self.mock_util.subp.return_value = ("""/dev/nvme0n1:
+          Magic : Intel Raid ISM Cfg Sig.
+        Version : 1.3.00
+    Orig Family : 6f8c68e3
+         Family : 6f8c68e3
+     Generation : 00000112
+     Attributes : All supported
+           UUID : 7ec12162:ee5cd20b:0ac8b069:cfbd93ec
+       Checksum : 4a5cebe2 correct
+    MPB Sectors : 2
+          Disks : 4
+   RAID Devices : 1
+
+  Disk03 Serial : LJ910504Q41P0FGN
+          State : active
+             Id : 00000000
+    Usable Size : 1953514766 (931.51 GiB 1000.20 GB)
+
+[126]:
+           UUID : f9792759:7f61d0c7:e7313d5a:2e7c2e22
+     RAID Level : 5
+        Members : 4
+          Slots : [UUUU]
+    Failed disk : none
+      This Slot : 3
+    Sector Size : 512
+     Array Size : 5860540416 (2794.52 GiB 3000.60 GB)
+   Per Dev Size : 1953515520 (931.51 GiB 1000.20 GB)
+  Sector Offset : 0
+    Num Stripes : 7630912
+     Chunk Size : 128 KiB
+       Reserved : 0
+  Migrate State : idle
+      Map State : normal
+    Dirty State : dirty
+     RWH Policy : off
+
+  Disk00 Serial : LJ91040H2Y1P0FGN
+          State : active
+             Id : 00000003
+    Usable Size : 1953514766 (931.51 GiB 1000.20 GB)
+
+  Disk01 Serial : LJ916308CZ1P0FGN
+          State : active
+             Id : 00000002
+    Usable Size : 1953514766 (931.51 GiB 1000.20 GB)
+
+  Disk02 Serial : LJ916308RF1P0FGN
+          State : active
+             Id : 00000001
+    Usable Size : 1953514766 (931.51 GiB 1000.20 GB)
+        """, "")   # mdadm --examine /dev/nvme0n1
+
+        device = "/dev/nvme0n1"
+        data = mdadm.mdadm_examine(device, export=False)
+
+        expected_calls = [
+            call(["mdadm", "--examine", device], capture=True),
+        ]
+        self.mock_util.subp.assert_has_calls(expected_calls)
+        self.assertEqual(data['uuid'],
+                         '7ec12162:ee5cd20b:0ac8b069:cfbd93ec')
+
 
 class TestBlockMdadmStop(CiTestCase):
     def setUp(self):
@@ -1171,6 +1282,63 @@ class TestBlockMdadmMdHelpers(CiTestCase):
         self.assertFalse(md_is_present)
         self.mock_util.load_file.assert_called_with('/proc/mdstat')
 
+    def test_md_is_in_container_false(self):
+        self.mock_util.subp.return_value = (
+            """
+            MD_LEVEL=raid1
+            MD_DEVICES=2
+            MD_METADATA=1.2
+            MD_UUID=93a73e10:427f280b:b7076c02:204b8f7a
+            MD_NAME=wily-foobar:0
+            MD_DEVICE_vdc_ROLE=0
+            MD_DEVICE_vdc_DEV=/dev/vdc
+            MD_DEVICE_vdd_ROLE=1
+            MD_DEVICE_vdd_DEV=/dev/vdd
+            MD_DEVICE_vde_ROLE=spare
+            MD_DEVICE_vde_DEV=/dev/vde
+            """, "")
+
+        device = "/dev/md0"
+        self.mock_valid.return_value = True
+        is_in_container = mdadm.md_is_in_container(device)
+
+        expected_calls = [
+            call(["mdadm", "--query", "--detail", "--export", device],
+                 capture=True),
+        ]
+        self.mock_util.subp.assert_has_calls(expected_calls)
+        self.assertEqual(is_in_container, False)
+
+    def test_md_is_in_container_true(self):
+        self.mock_util.subp.return_value = (
+            """
+            MD_LEVEL=raid5
+            MD_DEVICES=4
+            MD_CONTAINER=/dev/md/imsm0
+            MD_MEMBER=0
+            MD_UUID=5fa06b36:53e67142:37ff9ad6:44ef0e89
+            MD_DEVNAME=126
+            MD_DEVICE_ev_nvme2n1_ROLE=3
+            MD_DEVICE_ev_nvme2n1_DEV=/dev/nvme2n1
+            MD_DEVICE_ev_nvme1n1_ROLE=0
+            MD_DEVICE_ev_nvme1n1_DEV=/dev/nvme1n1
+            MD_DEVICE_ev_nvme0n1_ROLE=1
+            MD_DEVICE_ev_nvme0n1_DEV=/dev/nvme0n1
+            MD_DEVICE_ev_nvme3n1_ROLE=2
+            MD_DEVICE_ev_nvme3n1_DEV=/dev/nvme3n1
+            """, "")
+
+        device = "/dev/md0"
+        self.mock_valid.return_value = True
+        is_in_container = mdadm.md_is_in_container(device)
+
+        expected_calls = [
+            call(["mdadm", "--query", "--detail", "--export", device],
+                 capture=True),
+        ]
+        self.mock_util.subp.assert_has_calls(expected_calls)
+        self.assertEqual(is_in_container, True)
+
 
 class TestBlockMdadmZeroDevice(CiTestCase):
 
@@ -1243,6 +1411,4 @@ class TestBlockMdadmZeroDevice(CiTestCase):
         self.mock_examine.assert_called_with(device, export=False)
         self.m_zero.assert_called_with(device, expected_offsets,
                                        buflen=1024, count=1024, strict=True)
-
-
 # vi: ts=4 expandtab syntax=python
diff --git a/tests/unittests/test_clear_holders.py b/tests/unittests/test_clear_holders.py
index 25e9e79..d1c2590 100644
--- a/tests/unittests/test_clear_holders.py
+++ b/tests/unittests/test_clear_holders.py
@@ -238,6 +238,7 @@ class TestClearHolders(CiTestCase):
         mock_mdadm.md_present.return_value = False
         mock_mdadm.md_get_devices_list.return_value = devices
         mock_mdadm.md_get_spares_list.return_value = spares
+        mock_mdadm.md_is_in_container.return_value = False
 
         clear_holders.shutdown_mdadm(self.test_syspath)
 
@@ -256,6 +257,38 @@ class TestClearHolders(CiTestCase):
         mock_mdadm.md_present.assert_called_with(self.test_blockdev)
         self.assertTrue(mock_log.debug.called)
 
+    @mock.patch('curtin.block.wipe_volume')
+    @mock.patch('curtin.block.path_to_kname')
+    @mock.patch('curtin.block.sysfs_to_devpath')
+    @mock.patch('curtin.block.clear_holders.time')
+    @mock.patch('curtin.block.clear_holders.util')
+    @mock.patch('curtin.block.clear_holders.LOG')
+    @mock.patch('curtin.block.clear_holders.mdadm')
+    def test_shutdown_mdadm_in_container(self, mock_mdadm, mock_log, mock_util,
+                                         mock_time, mock_sysdev, mock_path,
+                                         mock_wipe):
+        """test clear_holders.shutdown_mdadm"""
+        devices = ['/dev/wda1', '/dev/wda2']
+        spares = ['/dev/wdb1']
+        mock_sysdev.return_value = self.test_blockdev
+        mock_path.return_value = self.test_blockdev
+        mock_mdadm.md_present.return_value = False
+        mock_mdadm.md_get_devices_list.return_value = devices
+        mock_mdadm.md_get_spares_list.return_value = spares
+        mock_mdadm.mdadm_query_detail.return_value = \
+            {'MD_CONTAINER': '/dev/md/imsm0'}
+
+        clear_holders.shutdown_mdadm(self.test_syspath)
+
+        mock_wipe.assert_called_with(self.test_blockdev, exclusive=False,
+                                     mode='superblock', strict=True)
+        mock_mdadm.set_sync_action.assert_has_calls([
+                mock.call(self.test_blockdev, action="idle"),
+                mock.call(self.test_blockdev, action="frozen")])
+        mock_mdadm.mdadm_stop.assert_called_with(self.test_blockdev)
+        mock_mdadm.md_present.assert_called_with(self.test_blockdev)
+        self.assertTrue(mock_log.debug.called)
+
     @mock.patch('curtin.block.clear_holders.os')
     @mock.patch('curtin.block.clear_holders.time')
     @mock.patch('curtin.block.clear_holders.util')
@@ -271,6 +304,7 @@ class TestClearHolders(CiTestCase):
         mock_mdadm.md_present.return_value = True
         mock_util.subp.return_value = ("", "")
         mock_os.path.exists.return_value = True
+        mock_mdadm.mdadm_query_detail.return_value = {}
 
         with self.assertRaises(OSError):
             clear_holders.shutdown_mdadm(self.test_syspath)
@@ -295,6 +329,7 @@ class TestClearHolders(CiTestCase):
         mock_block.path_to_kname.return_value = self.test_blockdev
         mock_mdadm.md_present.return_value = True
         mock_os.path.exists.return_value = False
+        mock_mdadm.mdadm_query_detail.return_value = {}
 
         with self.assertRaises(OSError):
             clear_holders.shutdown_mdadm(self.test_syspath)
diff --git a/tests/unittests/test_commands_block_meta.py b/tests/unittests/test_commands_block_meta.py
index d954296..98be573 100644
--- a/tests/unittests/test_commands_block_meta.py
+++ b/tests/unittests/test_commands_block_meta.py
@@ -1892,7 +1892,7 @@ class TestRaidHandler(CiTestCase):
         self.m_getpath.side_effect = iter(devices)
         block_meta.raid_handler(self.storage_config['mddevice'],
                                 self.storage_config)
-        self.assertEqual([call(md_devname, 5, devices, [], '')],
+        self.assertEqual([call(md_devname, 5, devices, [], None, '', None)],
                          self.m_mdadm.mdadm_create.call_args_list)
 
     @patch('curtin.commands.block_meta.raid_verify')