Merge ~gpiccoli/maas:nvme_secure_erase into maas:master

Proposed by Guilherme G. Piccoli
Status: Superseded
Proposed branch: ~gpiccoli/maas:nvme_secure_erase
Merge into: maas:master
Diff against target: 1765 lines (+1120/-242)
6 files modified
src/maasserver/compose_preseed.py (+3/-0)
src/maasserver/tests/test_compose_preseed.py (+1/-0)
src/maasserver/tests/test_preseed.py (+1/-0)
src/metadataserver/user_data/templates/snippets/maas_wipe.py (+289/-63)
src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py (+594/-179)
src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe_defs.py (+232/-0)
Reviewer Review Type Date Requested Status
MAAS Lander Needs Fixing
Lee Trager Pending
Adam Collard Pending
Review via email: mp+386894@code.launchpad.net

This proposal supersedes a proposal from 2020-07-06.

This proposal has been superseded by a proposal from 2020-07-06.

Commit message

This is the NVMe secure erase implementation (+ one more patch, to fix/improve quick erase).
I tried to explain/detail everything in the commit messages; also, code/tests were validated against Flake8 to prevent style issues.

Description of the change

V4:
(Sorry for re-iterations, I'm unable to run the full test suite on my system, for some reason, so I can't validate all tests are working before submitting).
- Fixed 2 tests to fit with the nvme-cli preseed change (thanks again d0ugal for CI logs)

V3:
- Fixed compose_preseed (forgot to account for Node == None, thanks d0ugal for helping me with the CI log!)

V2:

- Fixed style issues by running Black (thanks Adam!)
- Implemented the great suggestion from Lee about using vendor_data to install nvme-cli instead of reinventing the wheel
- Improved some exception messages (removed '\n', better formatting, etc)
- Implemented more exceptions for nvme-cli (OSError) and their respective unit tests

Thanks in advance for the re-review!
Cheers,

Guilherme

To post a comment you must log in.
Revision history for this message
MAAS Lander (maas-lander) wrote : Posted in a previous version of this proposal

UNIT TESTS
-b nvme_secure_erase lp:~gpiccoli/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/7873/console
COMMIT: d6c02a76af681afcaf8b8116c0d66b9726ba21ac

review: Needs Fixing
Revision history for this message
Adam Collard (adam-collard) wrote : Posted in a previous version of this proposal

 + sudo -u ubuntu -E -H make lint
 ERROR: /run/build/maas/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py Imports are incorrectly sorted.
 --- /run/build/maas/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py:before 2020-06-30 17:06:30.431234
 +++ /run/build/maas/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py:after 2020-06-30 17:09:32.020722
 @@ -6,11 +6,12 @@
  __all__ = []

  import argparse
 +import builtins
  import subprocess
 -import builtins
  from textwrap import dedent
  from unittest.mock import call, MagicMock

 +# hdparm and nvme-cli outputs used in the tests
  from maastesting.factory import factory
  from maastesting.matchers import (
      MockCalledOnceWith,
 @@ -22,29 +23,31 @@
  from snippets.maas_wipe import (
      get_disk_info,
      get_disk_security_info,
 + install_nvme_cli,
      list_disks,
 + nvme_write_zeroes,
      secure_erase_hdparm,
      try_secure_erase,
 - install_nvme_cli,
 - nvme_write_zeroes,
      wipe_quickly,
      WipeError,
      zero_disk,
  )
 -# hdparm and nvme-cli outputs used in the tests
  from snippets.tests.test_maas_wipe_defs import (
 - HDPARM_BEFORE_SECURITY, HDPARM_AFTER_SECURITY,
 + HDPARM_AFTER_SECURITY,
 + HDPARM_BEFORE_SECURITY,
 + HDPARM_SECURITY_ALL_TRUE,
      HDPARM_SECURITY_NOT_SUPPORTED,
 + HDPARM_SECURITY_SUPPORTED_ENABLED,
      HDPARM_SECURITY_SUPPORTED_NOT_ENABLED,
 - HDPARM_SECURITY_SUPPORTED_ENABLED,
 - HDPARM_SECURITY_ALL_TRUE, NVME_IDCTRL_PROLOGUE,
 + NVME_IDCTRL_EPILOGUE,
 + NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED,
 + NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED,
      NVME_IDCTRL_OACS_FORMAT_SUPPORTED,
      NVME_IDCTRL_OACS_FORMAT_UNSUPPORTED,
      NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED,
      NVME_IDCTRL_ONCS_WRITEZ_UNSUPPORTED,
 - NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED,
 - NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED,
 - NVME_IDCTRL_EPILOGUE)
 + NVME_IDCTRL_PROLOGUE,
 +)

Revision history for this message
Lee Trager (ltrager) wrote : Posted in a previous version of this proposal

Thanks for submitting things, two small things below in addition to what Adam posted.

review: Needs Fixing
Revision history for this message
Lee Trager (ltrager) : Posted in a previous version of this proposal
Revision history for this message
Igor Gnip (igorgnip) wrote : Posted in a previous version of this proposal

Hello,

This still does not fail if secure erase is not supported.
Other than that, I don't see any verification that data on the disks changed after tool reported that wipe was done. Do we trust the tool ? Do we flush the buffers and force re-read from storage device ?

Also, as long as there is no way to force this script to fail if secure erase was not done, it is useless for our use case. I am happy that other use cases got coverage for NVME but consider environments where zero wiping is considered not secure enough.

Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal

Thanks Adam, I'll fix that!

Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal

Thanks Lee, take a look in the diff comments for the inline replies!

Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal

Hi Igor, thanks again for you feedback. I think we are talking about 2 different features!

(a) Supporting NVMe secure erase. Currently it is not *supported*. So, users with NVMe drives cannot at all have them securely erased. This is what we are addressing here, AFAIK.

(b) To fail the erase operation if a secure erase is not performed well (or not supported). I think for this item, the best would be to have a new checkbox in MAAS user interface, something like "Fail if not secure erased".

Right now, the way secure erase is implemented is an attempt to do the secure erase operation and if it does not complete properly or if it's not supported, we fallback to zeroing the device. I guess we can discuss the change to a more strict mode with MAAS team, but I consider it a different feature, which is not in the scope for this proposal.

Finally, regarding the validation if secure erase was correct, like by writing data in the device and checking back after the secure operation: in a first moment that seems a good idea. But..how can we be sure?

Imagine we have a 1TB NVMe device , and due to the way FW works, it returns secure erase as completed but it took some more time and FW continue working in background. What if we write 1G of data, secure erase the drive and read back 1G, all "wrong" data (meaning secure erase worked), but...after the 500G offset in the disk, the data is not secure erased yet? Then if an user hard power-off the system (like removing the power cable), it is still prone to a cold-boot attack.
I know I'm a bit "philosophical" here, and that hypothesis I suggested above is almost nonsense, but by not trusting the tool, we should (for consistency) not trust the firmware, so...what to do?

I vote to trust the tool/firmware in a first moment, then we could improve that (although I'm not sure how to be fully guarded that secure erase worked). Perhaps nvme sanitize for devices that support that?

Cheers,

Guilherme

Revision history for this message
Lee Trager (ltrager) wrote : Posted in a previous version of this proposal

The way MAAS runs disk erasing is by sending the script below as user_data. cloud-init runs the script and shuts down the system when it is done. MAAS does not capture logs(besides syslog) of this process and there is no way for maas-wipe to signal any sort of failure. MAAS just assumes everything went well and does not validate that disk erasing actually happened.

I've wanted to change this for awhile. We could pretty easily leverage the commissioning/testing framework. This would give us logging per disk(like the smartctl-validate test), signaling of status, the ability to run disk erasing in parallel instead of being run serially, and an agent that validates maas-wipe successfully ran. I haven't been able to get it on our roadmap which is why it hasn't happened yet.

To independently validate that erasure happened would be very time consuming. Secure erase itself should be almost instant. The firmware tells every memory cell on the SSD to either goto 0 or a random value. However to validate to validate that worked maas-wipe would have to write a repeating pattern that fills the block device, use secure erase, then check the pattern isn't found anywhere on the block device. You have to check the entire block device as many SSDs now have an internal cache.

I don't think maas-wipe should be validating that firmware is doing what it claims it supports. MAAS supports hardware testing which is where firmware should be validated.

Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal

> [...]
> Secure erase itself should be almost instant. The firmware tells every memory
> cell on the SSD to either goto 0 or a random value.

Thanks Lee! To add on that, by using the cryptographic secure erase in NVMe (which is our first option if available) is even faster, the FW just recreates the internal key, which invalidates all previously written data heheh

Cheers,

Guilherme

Revision history for this message
Lee Trager (ltrager) : Posted in a previous version of this proposal
Revision history for this message
Guilherme G. Piccoli (gpiccoli) : Posted in a previous version of this proposal
Revision history for this message
MAAS Lander (maas-lander) wrote : Posted in a previous version of this proposal

UNIT TESTS
-b nvme_secure_erase lp:~gpiccoli/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/7930/console
COMMIT: 5a4fc8024a3825da8504169cd6a42aa734810806

review: Needs Fixing
Revision history for this message
Igor Gnip (igorgnip) wrote : Posted in a previous version of this proposal

Cryptographic erase just deletes the key
which in fact leaves data intact
for someone to bruteforce the key (theoretically), all data is still accessible.

In our implementation, we do both cryptographic erase (change key) and plain zero erase. Just to be sure that both key is changed and data is gone.

As far as controlling success of operation, I firmly believe that release should just release machine from maas database and the ui buttons on maas ui should actually produce a TESTING call with using the chosen testing modules to perform actual erase / test.
Release should check for result of TEST and if all passed, release the machine.
But there are many ways that can be good or bad depending on use case.

We currently run a test and attach a script which erases drives. If the test script fails, erase was not done.
Release api call is always called just to release the machine (no erase box checked).
This seems to work better and provides more granular control of the process.

Regards,
Igor

Revision history for this message
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal

Igor, I guess a lot of firmware implementations of NVMe devices use cryptographic erase *even if you don't set that*, because (a) it's secure, no way to recover the data, (b) it's quite faster, (c) prevents the wear of disk memories, by not writing in the full disk.

I think there's a limit on what secure erase can do if you'll count brute-force recover attempts and whatnot, and if you'll not trust the device's firmware.
Cheers,

Guilherme

Revision history for this message
MAAS Lander (maas-lander) wrote : Posted in a previous version of this proposal

UNIT TESTS
-b nvme_secure_erase lp:~gpiccoli/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/7936/console
COMMIT: f6c8240f466c5d06a26b5e735800d5da1725b4fe

review: Needs Fixing
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b nvme_secure_erase lp:~gpiccoli/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/7937/console
COMMIT: e5308f902db5529d5633a0da9775e981880b2756

review: Needs Fixing

Unmerged commits

e5308f9... by Guilherme G. Piccoli

Fix error in quick erase if OS throws an exception and add wipefs cleaning

Currently, if quick erase is used but OS throws any exception, the operation
is aborted and all subsequent disks to be quickly erased are ignored. Also,
the code currently only performs a 2MB write in the beginning and end of the
disk as the quick erase. This is usually enough, but we have more proper ways
of getting rid of old disk layouts for example (the wipefs command being the
de facto Linux standard way).

This patch fixes the exception problem by using a try block in which the
OS-related functions are executed; exceptions don't completely break the
method anymore. Also, we add the wipefs call before the disk write, so it
catches other partition layouts that potentially aren't within the 2MB range.
Notice that we consider a fail if *both* the 2MB write and the wipefs command
fail - in case only one works, it is still a success.

Unit tests were worked to take the wipefs and the exceptions into account.
We also validated the files against Flake8/Black to prevent style issues.

Signed-off-by: Guilherme G. Piccoli <email address hidden>

e8e93e5... by Guilherme G. Piccoli

LP: #1835954: Add NVMe secure erase / write zeroes support

MAAS currently allows 2 types of disk erase when releasing a node:
secure erase and quick erase. Secure erase fallback to zeroing the
disk in case the secure functionality doesn't work.

There is a limitation though: hdparm is currently used to perform
secure erasing, regardless if the disk is a SCSI/ATA device or a NVMe
device. This is not a good idea mainly for 2 reasons:
(a) Secure erase obviously never works for NVMe, relying in the "slow"
procedure of fully zeroing the disk;
(b) Zeroing a NVMe device is not a 100% secure guaranteed operation;
some firmwares may return a completion but physically the blocks may
still contain data (prone to "cold boot"-like attacks).

This patch proposes a solution to this problem by using nvme-cli tool
instead of hdparm if the disk is NVMe. Secure erase is attempted,
with cryptographic erase as a preferred mode (faster and less harmful
for the device). In case secure erase is not available or the operation
fails, zeroing the disk is still the fallback, but we try first to rely
on write-zeroes when available (much faster than fully writing zeroes
to the entire device). Only if both secure erase and write zeroes fail
we go with the full disk zeroing approach.

This patch was tested in real NVMe device and all source files created
or modified here were validated against Flake8/Black for style issues.
I'd like to thank specially the following colleagues due to their help
and suggestions during this work: Dan Streetman, Igor Gnip, Rodrigo
"Ganso" Barbieri and Lee Trager (for his idea of using the vendor_data
to install nvme-cli).

Signed-off-by: Guilherme G. Piccoli <email address hidden>

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/src/maasserver/compose_preseed.py b/src/maasserver/compose_preseed.py
2index 8acf2f5..244d87f 100644
3--- a/src/maasserver/compose_preseed.py
4+++ b/src/maasserver/compose_preseed.py
5@@ -391,6 +391,9 @@ def get_base_preseed(node=None):
6 # jq is used during enlistment to read the JSON string containing
7 # the system_id of the newly created machine.
8 cloud_config["packages"] += ["jq"]
9+ # On disk erasing, we need nvme-cli
10+ if node is None or node.status == NODE_STATUS.DISK_ERASING:
11+ cloud_config["packages"] += ["nvme-cli"]
12
13 return cloud_config
14
15diff --git a/src/maasserver/tests/test_compose_preseed.py b/src/maasserver/tests/test_compose_preseed.py
16index 576fbb2..64d102b 100644
17--- a/src/maasserver/tests/test_compose_preseed.py
18+++ b/src/maasserver/tests/test_compose_preseed.py
19@@ -1086,6 +1086,7 @@ class TestComposePreseed(MAASServerTestCase):
20 "sshpass",
21 "archdetect-deb",
22 "jq",
23+ "nvme-cli",
24 ],
25 preseed["packages"],
26 )
27diff --git a/src/maasserver/tests/test_preseed.py b/src/maasserver/tests/test_preseed.py
28index 077a215..4e455c1 100644
29--- a/src/maasserver/tests/test_preseed.py
30+++ b/src/maasserver/tests/test_preseed.py
31@@ -677,6 +677,7 @@ class TestRenderEnlistmentPreseed(MAASServerTestCase):
32 "sshpass",
33 "archdetect-deb",
34 "jq",
35+ "nvme-cli",
36 ],
37 preseed["packages"],
38 )
39diff --git a/src/metadataserver/user_data/templates/snippets/maas_wipe.py b/src/metadataserver/user_data/templates/snippets/maas_wipe.py
40old mode 100644
41new mode 100755
42index 317c62b..f7e611a
43--- a/src/metadataserver/user_data/templates/snippets/maas_wipe.py
44+++ b/src/metadataserver/user_data/templates/snippets/maas_wipe.py
45@@ -29,12 +29,95 @@ def list_disks():
46 return disks
47
48
49-def get_disk_security_info(disk):
50- """Get the disk security information.
51+def get_nvme_security_info(disk):
52+ """Gather NVMe information from the NVMe disks using the
53+ nvme-cli tool. Info from id-ctrl and id-ns is needed for
54+ secure erase (nvme format) and write zeroes."""
55+
56+ # Grab the relevant info from nvme id-ctrl. We need to check the
57+ # following bits:
58+ #
59+ # OACS (Optional Admin Command Support) bit 1: Format supported
60+ # ONCS (Optional NVM Command Support) bit 3: Write Zeroes supported
61+ # FNA (Format NVM Attributes) bit 2: Cryptographic format supported
62+
63+ security_info = {
64+ "format_supported": False,
65+ "writez_supported": False,
66+ "crypto_format": False,
67+ "nsze": 0,
68+ "lbaf": 0,
69+ "ms": 0,
70+ }
71+
72+ try:
73+ output = subprocess.check_output(["nvme", "id-ctrl", DEV_PATH % disk])
74+ except subprocess.CalledProcessError as exc:
75+ print_flush("Error on nvme id-ctrl (%s)" % exc.returncode)
76+ return security_info
77+ except OSError as exc:
78+ print_flush("OS error when running nvme-cli (%s)" % exc.strerror)
79+ return security_info
80+
81+ output = output.decode()
82+
83+ for line in output.split("\n"):
84+ if "oacs" in line:
85+ oacs = line.split(":")[1]
86+ if int(oacs, 16) & 0x2:
87+ security_info["format_supported"] = True
88+
89+ if "oncs" in line:
90+ oncs = line.split(":")[1]
91+ if int(oncs, 16) & 0x8:
92+ security_info["writez_supported"] = True
93+
94+ if "fna" in line:
95+ fna = line.split(":")[1]
96+ if int(fna, 16) & 0x4:
97+ security_info["crypto_format"] = True
98+
99+ # Next step: collect LBAF (LBA Format), MS (Metadata Setting) and
100+ # NSZE (Namespace Size) from id-ns. According to NVMe spec, bits 0:3
101+ # from FLBAS corresponds to the LBAF value, whereas bit 4 is MS.
102+
103+ try:
104+ output = subprocess.check_output(["nvme", "id-ns", DEV_PATH % disk])
105+ except subprocess.CalledProcessError as exc:
106+ print_flush("Error on nvme id-ns (%s)" % exc.returncode)
107+ security_info["format_supported"] = False
108+ security_info["writez_supported"] = False
109+ return security_info
110+ except OSError as exc:
111+ print_flush("OS error when running nvme-cli (%s)" % exc.strerror)
112+ security_info["format_supported"] = False
113+ security_info["writez_supported"] = False
114+ return security_info
115+
116+ output = output.decode()
117+
118+ for line in output.split("\n"):
119+ if "nsze" in line:
120+ # According to spec., this should be used as 0-based value.
121+ nsze = line.split(":")[1]
122+ security_info["nsze"] = int(nsze, 16) - 1
123+
124+ if "flbas" in line:
125+ flbas = line.split(":")[1]
126+ flbas = int(flbas, 16)
127+ security_info["lbaf"] = flbas & 0xF
128+
129+ if flbas & 0x10:
130+ security_info["ms"] = 1
131+
132+ return security_info
133
134- Uses `hdparam` to get security information about the disk. Sadly hdparam
135- doesn't provide an output that makes it easy to parse.
136+
137+def get_hdparm_security_info(disk):
138+ """Get SCSI/ATA disk security info from hdparm.
139+ Sadly hdparam doesn't provide an output that makes it easy to parse.
140 """
141+
142 # Grab the security section for hdparam.
143 security_section = []
144 output = subprocess.check_output([b"hdparm", b"-I", DEV_PATH % disk])
145@@ -62,57 +145,25 @@ def get_disk_security_info(disk):
146 return security_info
147
148
149+def get_disk_security_info(disk):
150+ """Get the disk security information.
151+
152+ Uses `hdparam` to get security information about the SCSI/ATA disks.
153+ If NVMe, nvme-cli is used instead.
154+ """
155+
156+ if b"nvme" in disk:
157+ return get_nvme_security_info(disk)
158+
159+ return get_hdparm_security_info(disk)
160+
161+
162 def get_disk_info():
163 """Return dictionary of wipeable disks and thier security information."""
164 return {kname: get_disk_security_info(kname) for kname in list_disks()}
165
166
167-def try_secure_erase(kname, info):
168- """Try to wipe the disk with secure erase."""
169- if info[b"supported"]:
170- if info[b"frozen"]:
171- print_flush(
172- "%s: not using secure erase; "
173- "drive is currently frozen." % kname.decode("ascii")
174- )
175- return False
176- elif info[b"locked"]:
177- print_flush(
178- "%s: not using secure erase; "
179- "drive is currently locked." % kname.decode("ascii")
180- )
181- return False
182- elif info[b"enabled"]:
183- print_flush(
184- "%s: not using secure erase; "
185- "drive security is already enabled." % kname.decode("ascii")
186- )
187- return False
188- else:
189- # Wiping using secure erase.
190- try:
191- secure_erase(kname)
192- except Exception as e:
193- print_flush(
194- "%s: failed to be securely erased: %s"
195- % (kname.decode("ascii"), e)
196- )
197- return False
198- else:
199- print_flush(
200- "%s: successfully securely erased."
201- % (kname.decode("ascii"))
202- )
203- return True
204- else:
205- print_flush(
206- "%s: drive does not support secure erase."
207- % (kname.decode("ascii"))
208- )
209- return False
210-
211-
212-def secure_erase(kname):
213+def secure_erase_hdparm(kname):
214 """Securely wipe the device."""
215 # First write 1 MiB of known data to the beginning of the block device.
216 # This is used to check at the end of the secure erase that it worked
217@@ -142,7 +193,7 @@ def secure_erase(kname):
218
219 # Now that the user password is set the device should have its
220 # security mode enabled.
221- info = get_disk_security_info(kname)
222+ info = get_hdparm_security_info(kname)
223 if not info[b"enabled"]:
224 # If not enabled that means the password did not take, so it does not
225 # need to be cleared.
226@@ -167,7 +218,7 @@ def secure_erase(kname):
227 failed_exc = exc
228
229 # Make sure that the device is now not enabled.
230- info = get_disk_security_info(kname)
231+ info = get_hdparm_security_info(kname)
232 if info[b"enabled"]:
233 # Wipe failed since security is still enabled.
234 subprocess.check_output(
235@@ -184,23 +235,197 @@ def secure_erase(kname):
236 )
237
238
239-def wipe_quickly(kname):
240- """Quickly wipe the disk by zeroing the beginning and end of the disk.
241+def try_secure_erase_hdparm(kname, info):
242+ """Try to wipe the disk with secure erase."""
243+ if info[b"supported"]:
244+ if info[b"frozen"]:
245+ print_flush(
246+ "%s: not using secure erase; "
247+ "drive is currently frozen." % kname.decode("ascii")
248+ )
249+ return False
250+ elif info[b"locked"]:
251+ print_flush(
252+ "%s: not using secure erase; "
253+ "drive is currently locked." % kname.decode("ascii")
254+ )
255+ return False
256+ elif info[b"enabled"]:
257+ print_flush(
258+ "%s: not using secure erase; "
259+ "drive security is already enabled." % kname.decode("ascii")
260+ )
261+ return False
262+ else:
263+ # Wiping using secure erase.
264+ try:
265+ secure_erase_hdparm(kname)
266+ except Exception as e:
267+ print_flush(
268+ "%s: failed to be securely erased: %s"
269+ % (kname.decode("ascii"), e)
270+ )
271+ return False
272+ else:
273+ print_flush(
274+ "%s: successfully securely erased."
275+ % (kname.decode("ascii"))
276+ )
277+ return True
278+ else:
279+ print_flush(
280+ "%s: drive does not support secure erase."
281+ % (kname.decode("ascii"))
282+ )
283+ return False
284+
285+
286+def try_secure_erase_nvme(kname, info):
287+ """Perform a secure-erase on NVMe disk if that feature is
288+ available. Prefer cryptographic erase, when available."""
289+
290+ if not info["format_supported"]:
291+ print_flush(
292+ "Device %s does not support formatting" % kname.decode("ascii")
293+ )
294+ return False
295+
296+ if info["crypto_format"]:
297+ ses = 2
298+ else:
299+ ses = 1
300+
301+ try:
302+ subprocess.check_output(
303+ [
304+ "nvme",
305+ "format",
306+ "-s",
307+ str(ses),
308+ "-l",
309+ str(info["lbaf"]),
310+ "-m",
311+ str(info["ms"]),
312+ DEV_PATH % kname,
313+ ]
314+ )
315+ except subprocess.CalledProcessError as exc:
316+ print_flush("Error with format command (%s)" % exc.returncode)
317+ return False
318+ except OSError as exc:
319+ print_flush("OS error when running nvme-cli (%s)" % exc.strerror)
320+ return False
321+
322+ print_flush(
323+ "Secure erase was successful on NVMe drive %s" % kname.decode("ascii")
324+ )
325+ return True
326+
327+
328+def try_secure_erase(kname, info):
329+ """Entry-point for secure-erase for SCSI/ATA or NVMe disks."""
330+
331+ if b"nvme" in kname:
332+ return try_secure_erase_nvme(kname, info)
333+
334+ return try_secure_erase_hdparm(kname, info)
335
336- This is not a secure erase but does make it harder to get the data from
337- the device.
338+
339+def wipe_quickly(kname):
340+ """Quickly wipe the disk by using wipefs and zeroing the beginning
341+ and end of the disk. This is not a secure erase but does make it
342+ harder to get the data from the device and also clears previous layouts.
343 """
344+
345+ wipe_error = 0
346 print_flush("%s: starting quick wipe." % kname.decode("ascii"))
347+ try:
348+ subprocess.check_output(["wipefs", "-f", "-a", DEV_PATH % kname])
349+ wipe_error -= 1
350+ except subprocess.CalledProcessError as exc:
351+ print_flush(
352+ "%s: wipefs failed (%s)" % (kname.decode("ascii"), exc.returncode)
353+ )
354+ wipe_error += 1
355+
356 buf = b"\0" * 1024 * 1024 * 2 # 2 MiB
357- with open(DEV_PATH % kname, "wb") as fp:
358+ try:
359+ fp = open(DEV_PATH % kname, "wb")
360 fp.write(buf)
361 fp.seek(-len(buf), 2)
362 fp.write(buf)
363- print_flush("%s: successfully quickly wiped." % kname.decode("ascii"))
364+ wipe_error -= 1
365+ except OSError as exc:
366+ print_flush(
367+ "%s: OS error while wiping beginning/end of disk (%s)"
368+ % (kname.decode("ascii"), exc.strerror)
369+ )
370+ wipe_error += 1
371+
372+ if wipe_error > 0:
373+ print_flush("%s: failed to be quickly wiped." % kname.decode("ascii"))
374+ else:
375+ print_flush("%s: successfully quickly wiped." % kname.decode("ascii"))
376+
377+
378+def nvme_write_zeroes(kname, info):
379+ """Perform a write-zeroes operation on NVMe device instead of
380+ dd'ing 0 to the entire disk if secure erase is not available.
381+ Write-zeroes is a faster way to clean a NVMe disk."""
382+
383+ fallback = False
384+
385+ if not info["writez_supported"]:
386+ print(
387+ "NVMe drive %s does not support write-zeroes"
388+ % kname.decode("ascii")
389+ )
390+ fallback = True
391+
392+ if info["nsze"] <= 0:
393+ print(
394+ "Bad namespace information collected on NVMe drive %s"
395+ % kname.decode("ascii")
396+ )
397+ fallback = True
398+
399+ if fallback:
400+ print_flush("Will fallback to regular drive zeroing.")
401+ return False
402+
403+ try:
404+ subprocess.check_output(
405+ [
406+ "nvme",
407+ "write-zeroes",
408+ "-f",
409+ "-s",
410+ "0",
411+ "-c",
412+ str(hex(info["nsze"])[2:]),
413+ DEV_PATH % kname,
414+ ]
415+ )
416+ except subprocess.CalledProcessError as exc:
417+ print_flush("Error with write-zeroes command (%s)" % exc.returncode)
418+ return False
419+ except OSError as exc:
420+ print_flush("OS error when running nvme-cli (%s)" % exc.strerror)
421+ return False
422+
423+ print_flush(
424+ "%s: successfully zeroed (using write-zeroes)." % kname.decode("ascii")
425+ )
426+ return True
427+
428+
429+def zero_disk(kname, info):
430+ """Zero the entire disk, trying write-zeroes first if NVMe disk."""
431
432+ if b"nvme" in kname:
433+ if nvme_write_zeroes(kname, info):
434+ return
435
436-def zero_disk(kname):
437- """Zero the entire disk."""
438 # Get the total size of the device.
439 size = 0
440 with open(DEV_PATH % kname, "rb") as fp:
441@@ -266,8 +491,9 @@ def main():
442 action="store_true",
443 default=False,
444 help=(
445- "Wipe 1MiB at the start and at the end of the drive to make data "
446- "recovery inconvenient and unlikely to happen by accident. This "
447+ "Wipe 2MiB at the start and at the end of the drive to make data "
448+ "recovery inconvenient and unlikely to happen by accident. Also, "
449+ "it runs wipefs to clear known partition/layout signatures. This "
450 "is not secure."
451 ),
452 )
453@@ -288,7 +514,7 @@ def main():
454 if args.quick_erase:
455 wipe_quickly(kname)
456 else:
457- zero_disk(kname)
458+ zero_disk(kname, info)
459
460 print_flush("All disks have been successfully wiped.")
461
462diff --git a/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py b/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py
463index b5edcad..300d643 100644
464--- a/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py
465+++ b/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py
466@@ -6,6 +6,7 @@
467 __all__ = []
468
469 import argparse
470+import builtins
471 import subprocess
472 from textwrap import dedent
473 from unittest.mock import call, MagicMock
474@@ -22,148 +23,29 @@ from snippets.maas_wipe import (
475 get_disk_info,
476 get_disk_security_info,
477 list_disks,
478- secure_erase,
479+ nvme_write_zeroes,
480+ secure_erase_hdparm,
481 try_secure_erase,
482 wipe_quickly,
483 WipeError,
484 zero_disk,
485 )
486-
487-HDPARM_BEFORE_SECURITY = b"""\
488-/dev/sda:
489-
490-ATA device, with non-removable media
491- Model Number: INTEL SSDSC2CT240A4
492- Serial Number: CVKI3206029X240DGN
493- Firmware Revision: 335u
494- Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions
495-Standards:
496- Used: unknown (minor revision code 0xffff)
497- Supported: 9 8 7 6 5
498- Likely used: 9
499-Configuration:
500- Logical max current
501- cylinders 16383 16383
502- heads 16 16
503- sectors/track 63 63
504- --
505- CHS current addressable sectors: 16514064
506- LBA user addressable sectors: 268435455
507- LBA48 user addressable sectors: 468862128
508- Logical Sector size: 512 bytes
509- Physical Sector size: 512 bytes
510- Logical Sector-0 offset: 0 bytes
511- device size with M = 1024*1024: 228936 MBytes
512- device size with M = 1000*1000: 240057 MBytes (240 GB)
513- cache/buffer size = unknown
514- Nominal Media Rotation Rate: Solid State Device
515-Capabilities:
516- LBA, IORDY(can be disabled)
517- Queue depth: 32
518- Standby timer values: spec'd by Standard, no device specific minimum
519- R/W multiple sector transfer: Max = 16 Current = 16
520- Advanced power management level: 254
521- DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
522- Cycle time: min=120ns recommended=120ns
523- PIO: pio0 pio1 pio2 pio3 pio4
524- Cycle time: no flow control=120ns IORDY flow control=120ns
525-Commands/features:
526- Enabled Supported:
527- * SMART feature set
528- Security Mode feature set
529- * Power Management feature set
530- * Write cache
531- * Look-ahead
532- * Host Protected Area feature set
533- * WRITE_BUFFER command
534- * READ_BUFFER command
535- * NOP cmd
536- * DOWNLOAD_MICROCODE
537- * Advanced Power Management feature set
538- Power-Up In Standby feature set
539- * 48-bit Address feature set
540- * Mandatory FLUSH_CACHE
541- * FLUSH_CACHE_EXT
542- * SMART error logging
543- * SMART self-test
544- * General Purpose Logging feature set
545- * WRITE_{DMA|MULTIPLE}_FUA_EXT
546- * 64-bit World wide name
547- * IDLE_IMMEDIATE with UNLOAD
548- * WRITE_UNCORRECTABLE_EXT command
549- * {READ,WRITE}_DMA_EXT_GPL commands
550- * Segmented DOWNLOAD_MICROCODE
551- * Gen1 signaling speed (1.5Gb/s)
552- * Gen2 signaling speed (3.0Gb/s)
553- * Gen3 signaling speed (6.0Gb/s)
554- * Native Command Queueing (NCQ)
555- * Host-initiated interface power management
556- * Phy event counters
557- * DMA Setup Auto-Activate optimization
558- Device-initiated interface power management
559- * Software settings preservation
560- * SMART Command Transport (SCT) feature set
561- * SCT Data Tables (AC5)
562- * reserved 69[4]
563- * Data Set Management TRIM supported (limit 1 block)
564- * Deterministic read data after TRIM
565-"""
566-
567-HDPARM_AFTER_SECURITY = b"""\
568-Logical Unit WWN Device Identifier: 55cd2e40002643cf
569- NAA : 5
570- IEEE OUI : 5cd2e4
571- Unique ID : 0002643cf
572-Checksum: correct
573-"""
574-
575-HDPARM_SECURITY_NOT_SUPPORTED = b"""\
576-Security:
577- Master password revision code = 65534
578- not supported
579- not enabled
580- not locked
581- not frozen
582- not expired: security count
583- supported: enhanced erase
584- 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
585-"""
586-
587-HDPARM_SECURITY_SUPPORTED_NOT_ENABLED = b"""\
588-Security:
589- Master password revision code = 65534
590- supported
591- not enabled
592- not locked
593- not frozen
594- not expired: security count
595- supported: enhanced erase
596- 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
597-"""
598-
599-HDPARM_SECURITY_SUPPORTED_ENABLED = b"""\
600-Security:
601- Master password revision code = 65534
602- supported
603- enabled
604- not locked
605- not frozen
606- not expired: security count
607- supported: enhanced erase
608- 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
609-"""
610-
611-HDPARM_SECURITY_ALL_TRUE = b"""\
612-Security:
613- Master password revision code = 65534
614- supported
615- enabled
616- locked
617- frozen
618- not expired: security count
619- supported: enhanced erase
620- 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
621-"""
622+from snippets.tests.test_maas_wipe_defs import (
623+ HDPARM_AFTER_SECURITY,
624+ HDPARM_BEFORE_SECURITY,
625+ HDPARM_SECURITY_ALL_TRUE,
626+ HDPARM_SECURITY_NOT_SUPPORTED,
627+ HDPARM_SECURITY_SUPPORTED_ENABLED,
628+ HDPARM_SECURITY_SUPPORTED_NOT_ENABLED,
629+ NVME_IDCTRL_EPILOGUE,
630+ NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED,
631+ NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED,
632+ NVME_IDCTRL_OACS_FORMAT_SUPPORTED,
633+ NVME_IDCTRL_OACS_FORMAT_UNSUPPORTED,
634+ NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED,
635+ NVME_IDCTRL_ONCS_WRITEZ_UNSUPPORTED,
636+ NVME_IDCTRL_PROLOGUE,
637+)
638
639
640 class TestMAASWipe(MAASTestCase):
641@@ -196,11 +78,13 @@ class TestMAASWipe(MAASTestCase):
642 sdb disk 1
643 sr0 rom 0
644 sr1 rom 0
645+ nvme0n1 disk 0
646+ nvme1n1 disk 1
647 """
648 ).encode("ascii")
649- self.assertEqual([b"sda"], list_disks())
650+ self.assertEqual([b"sda", b"nvme0n1"], list_disks())
651
652- def test_get_disk_security_info_missing(self):
653+ def test_get_disk_security_info_missing_hdparm(self):
654 hdparm_output = HDPARM_BEFORE_SECURITY + HDPARM_AFTER_SECURITY
655 mock_check_output = self.patch(subprocess, "check_output")
656 mock_check_output.return_value = hdparm_output
657@@ -220,7 +104,7 @@ class TestMAASWipe(MAASTestCase):
658 observered,
659 )
660
661- def test_get_disk_security_info_not_supported(self):
662+ def test_get_disk_security_info_not_supported_hdparm(self):
663 hdparm_output = (
664 HDPARM_BEFORE_SECURITY
665 + HDPARM_SECURITY_NOT_SUPPORTED
666@@ -244,7 +128,7 @@ class TestMAASWipe(MAASTestCase):
667 observered,
668 )
669
670- def test_get_disk_security_info_supported_not_enabled(self):
671+ def test_get_disk_security_info_supported_not_enabled_hdparm(self):
672 hdparm_output = (
673 HDPARM_BEFORE_SECURITY
674 + HDPARM_SECURITY_SUPPORTED_NOT_ENABLED
675@@ -268,7 +152,7 @@ class TestMAASWipe(MAASTestCase):
676 observered,
677 )
678
679- def test_get_disk_security_info_supported_enabled(self):
680+ def test_get_disk_security_info_supported_enabled_hdparm(self):
681 hdparm_output = (
682 HDPARM_BEFORE_SECURITY
683 + HDPARM_SECURITY_SUPPORTED_ENABLED
684@@ -292,7 +176,7 @@ class TestMAASWipe(MAASTestCase):
685 observered,
686 )
687
688- def test_get_disk_security_info_all_true(self):
689+ def test_get_disk_security_info_all_true_hdparm(self):
690 hdparm_output = (
691 HDPARM_BEFORE_SECURITY
692 + HDPARM_SECURITY_ALL_TRUE
693@@ -316,7 +200,7 @@ class TestMAASWipe(MAASTestCase):
694 observered,
695 )
696
697- def test_get_disk_info(self):
698+ def test_get_disk_info_hdparm(self):
699 disk_names = [
700 factory.make_name("disk").encode("ascii") for _ in range(3)
701 ]
702@@ -331,14 +215,243 @@ class TestMAASWipe(MAASTestCase):
703 for _ in range(3)
704 ]
705 self.patch(
706- maas_wipe, "get_disk_security_info"
707+ maas_wipe, "get_hdparm_security_info"
708+ ).side_effect = security_info
709+ observed = get_disk_info()
710+ self.assertEqual(
711+ {disk_names[i]: security_info[i] for i in range(3)}, observed
712+ )
713+
714+ def test_get_disk_security_info_crypt_format_writez_nvme(self):
715+ nvme_cli_output = (
716+ NVME_IDCTRL_PROLOGUE
717+ + NVME_IDCTRL_OACS_FORMAT_SUPPORTED
718+ + NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED
719+ + NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED
720+ + NVME_IDCTRL_EPILOGUE
721+ )
722+ mock_check_output = self.patch(subprocess, "check_output")
723+ mock_check_output.return_value = nvme_cli_output
724+ disk_name = factory.make_name("nvme").encode("ascii")
725+ observered = get_disk_security_info(disk_name)
726+ self.assertThat(
727+ mock_check_output,
728+ MockCallsMatch(
729+ call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]),
730+ call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]),
731+ ),
732+ )
733+ self.assertEqual(
734+ {
735+ "format_supported": True,
736+ "writez_supported": True,
737+ "crypto_format": True,
738+ "nsze": 0,
739+ "lbaf": 0,
740+ "ms": 0,
741+ },
742+ observered,
743+ )
744+
745+ def test_get_disk_security_info_nocrypt_format_writez_nvme(self):
746+ nvme_cli_output = (
747+ NVME_IDCTRL_PROLOGUE
748+ + NVME_IDCTRL_OACS_FORMAT_SUPPORTED
749+ + NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED
750+ + NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED
751+ + NVME_IDCTRL_EPILOGUE
752+ )
753+ mock_check_output = self.patch(subprocess, "check_output")
754+ mock_check_output.return_value = nvme_cli_output
755+ disk_name = factory.make_name("nvme").encode("ascii")
756+ observered = get_disk_security_info(disk_name)
757+ self.assertThat(
758+ mock_check_output,
759+ MockCallsMatch(
760+ call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]),
761+ call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]),
762+ ),
763+ )
764+ self.assertEqual(
765+ {
766+ "format_supported": True,
767+ "writez_supported": True,
768+ "crypto_format": False,
769+ "nsze": 0,
770+ "lbaf": 0,
771+ "ms": 0,
772+ },
773+ observered,
774+ )
775+
776+ def test_get_disk_security_info_crypt_format_nowritez_nvme(self):
777+ nvme_cli_output = (
778+ NVME_IDCTRL_PROLOGUE
779+ + NVME_IDCTRL_OACS_FORMAT_SUPPORTED
780+ + NVME_IDCTRL_ONCS_WRITEZ_UNSUPPORTED
781+ + NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED
782+ + NVME_IDCTRL_EPILOGUE
783+ )
784+ mock_check_output = self.patch(subprocess, "check_output")
785+ mock_check_output.return_value = nvme_cli_output
786+ disk_name = factory.make_name("nvme").encode("ascii")
787+ observered = get_disk_security_info(disk_name)
788+ self.assertThat(
789+ mock_check_output,
790+ MockCallsMatch(
791+ call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]),
792+ call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]),
793+ ),
794+ )
795+ self.assertEqual(
796+ {
797+ "format_supported": True,
798+ "writez_supported": False,
799+ "crypto_format": True,
800+ "nsze": 0,
801+ "lbaf": 0,
802+ "ms": 0,
803+ },
804+ observered,
805+ )
806+
807+ def test_get_disk_security_info_noformat_writez_nvme(self):
808+ nvme_cli_output = (
809+ NVME_IDCTRL_PROLOGUE
810+ + NVME_IDCTRL_OACS_FORMAT_UNSUPPORTED
811+ + NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED
812+ + NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED
813+ + NVME_IDCTRL_EPILOGUE
814+ )
815+ mock_check_output = self.patch(subprocess, "check_output")
816+ mock_check_output.return_value = nvme_cli_output
817+ disk_name = factory.make_name("nvme").encode("ascii")
818+ observered = get_disk_security_info(disk_name)
819+ self.assertThat(
820+ mock_check_output,
821+ MockCallsMatch(
822+ call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]),
823+ call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]),
824+ ),
825+ )
826+ self.assertEqual(
827+ {
828+ "format_supported": False,
829+ "writez_supported": True,
830+ "crypto_format": True,
831+ "nsze": 0,
832+ "lbaf": 0,
833+ "ms": 0,
834+ },
835+ observered,
836+ )
837+
838+ def test_get_disk_security_info_noformat_nowritez_nvme(self):
839+ nvme_cli_output = (
840+ NVME_IDCTRL_PROLOGUE
841+ + NVME_IDCTRL_OACS_FORMAT_UNSUPPORTED
842+ + NVME_IDCTRL_ONCS_WRITEZ_UNSUPPORTED
843+ + NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED
844+ + NVME_IDCTRL_EPILOGUE
845+ )
846+ mock_check_output = self.patch(subprocess, "check_output")
847+ mock_check_output.return_value = nvme_cli_output
848+ disk_name = factory.make_name("nvme").encode("ascii")
849+ observered = get_disk_security_info(disk_name)
850+ self.assertThat(
851+ mock_check_output,
852+ MockCallsMatch(
853+ call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]),
854+ call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]),
855+ ),
856+ )
857+ self.assertEqual(
858+ {
859+ "format_supported": False,
860+ "writez_supported": False,
861+ "crypto_format": False,
862+ "nsze": 0,
863+ "lbaf": 0,
864+ "ms": 0,
865+ },
866+ observered,
867+ )
868+
869+ def test_get_disk_security_info_failed_cmd_nvme(self):
870+ mock_check_output = self.patch(subprocess, "check_output")
871+ mock_check_output.side_effect = subprocess.CalledProcessError(
872+ 1, "nvme id-ctrl"
873+ )
874+ disk_name = factory.make_name("nvme").encode("ascii")
875+ observered = get_disk_security_info(disk_name)
876+
877+ self.assertThat(
878+ self.print_flush,
879+ MockCalledOnceWith("Error on nvme id-ctrl (%s)" % "1"),
880+ )
881+ self.assertEqual(
882+ {
883+ "format_supported": False,
884+ "writez_supported": False,
885+ "crypto_format": False,
886+ "nsze": 0,
887+ "lbaf": 0,
888+ "ms": 0,
889+ },
890+ observered,
891+ )
892+
893+ def test_get_disk_security_info_failed_os_nvme(self):
894+ mock_check_output = self.patch(subprocess, "check_output")
895+ mock_check_output.side_effect = OSError(
896+ -2, "No such file or directory"
897+ )
898+ disk_name = factory.make_name("nvme").encode("ascii")
899+ observered = get_disk_security_info(disk_name)
900+
901+ self.assertThat(
902+ self.print_flush,
903+ MockCalledOnceWith(
904+ "OS error when running nvme-cli (No such file or directory)"
905+ ),
906+ )
907+ self.assertEqual(
908+ {
909+ "format_supported": False,
910+ "writez_supported": False,
911+ "crypto_format": False,
912+ "nsze": 0,
913+ "lbaf": 0,
914+ "ms": 0,
915+ },
916+ observered,
917+ )
918+
919+ def test_get_disk_info_nvme(self):
920+ disk_names = [
921+ factory.make_name("nvme").encode("ascii") for _ in range(3)
922+ ]
923+ self.patch(maas_wipe, "list_disks").return_value = disk_names
924+ security_info = [
925+ {
926+ "format_supported": True,
927+ "writez_supported": True,
928+ "crypto_format": True,
929+ "nsze": 0,
930+ "lbaf": 0,
931+ "ms": 0,
932+ }
933+ for _ in range(3)
934+ ]
935+ self.patch(
936+ maas_wipe, "get_nvme_security_info"
937 ).side_effect = security_info
938 observed = get_disk_info()
939 self.assertEqual(
940 {disk_names[i]: security_info[i] for i in range(3)}, observed
941 )
942
943- def test_try_secure_erase_not_supported(self):
944+ def test_try_secure_erase_not_supported_hdparm(self):
945 disk_name = factory.make_name("disk").encode("ascii")
946 disk_info = {
947 b"supported": False,
948@@ -355,7 +468,7 @@ class TestMAASWipe(MAASTestCase):
949 ),
950 )
951
952- def test_try_secure_erase_frozen(self):
953+ def test_try_secure_erase_frozen_hdparm(self):
954 disk_name = factory.make_name("disk").encode("ascii")
955 disk_info = {
956 b"supported": True,
957@@ -372,7 +485,7 @@ class TestMAASWipe(MAASTestCase):
958 ),
959 )
960
961- def test_try_secure_erase_locked(self):
962+ def test_try_secure_erase_locked_hdparm(self):
963 disk_name = factory.make_name("disk").encode("ascii")
964 disk_info = {
965 b"supported": True,
966@@ -389,7 +502,7 @@ class TestMAASWipe(MAASTestCase):
967 ),
968 )
969
970- def test_try_secure_erase_enabled(self):
971+ def test_try_secure_erase_enabled_hdparm(self):
972 disk_name = factory.make_name("disk").encode("ascii")
973 disk_info = {
974 b"supported": True,
975@@ -406,7 +519,7 @@ class TestMAASWipe(MAASTestCase):
976 ),
977 )
978
979- def test_try_secure_erase_failed_erase(self):
980+ def test_try_secure_erase_failed_erase_hdparm(self):
981 disk_name = factory.make_name("disk").encode("ascii")
982 disk_info = {
983 b"supported": True,
984@@ -415,7 +528,7 @@ class TestMAASWipe(MAASTestCase):
985 b"frozen": False,
986 }
987 exception = factory.make_exception()
988- self.patch(maas_wipe, "secure_erase").side_effect = exception
989+ self.patch(maas_wipe, "secure_erase_hdparm").side_effect = exception
990 self.assertFalse(try_secure_erase(disk_name, disk_info))
991 self.assertThat(
992 self.print_flush,
993@@ -425,7 +538,7 @@ class TestMAASWipe(MAASTestCase):
994 ),
995 )
996
997- def test_try_secure_erase_successful_erase(self):
998+ def test_try_secure_erase_successful_erase_hdparm(self):
999 disk_name = factory.make_name("disk").encode("ascii")
1000 disk_info = {
1001 b"supported": True,
1002@@ -433,7 +546,7 @@ class TestMAASWipe(MAASTestCase):
1003 b"locked": False,
1004 b"frozen": False,
1005 }
1006- self.patch(maas_wipe, "secure_erase")
1007+ self.patch(maas_wipe, "secure_erase_hdparm")
1008 self.assertTrue(try_secure_erase(disk_name, disk_info))
1009 self.assertThat(
1010 self.print_flush,
1011@@ -443,7 +556,223 @@ class TestMAASWipe(MAASTestCase):
1012 ),
1013 )
1014
1015- def test_secure_erase_writes_known_data(self):
1016+ def test_try_secure_erase_not_supported_nvme(self):
1017+ disk_name = factory.make_name("nvme").encode("ascii")
1018+ sec_info = {
1019+ "format_supported": False,
1020+ "writez_supported": True,
1021+ "crypto_format": True,
1022+ "nsze": 0,
1023+ "lbaf": 0,
1024+ "ms": 0,
1025+ }
1026+ self.assertFalse(try_secure_erase(disk_name, sec_info))
1027+ self.assertThat(
1028+ self.print_flush,
1029+ MockCalledOnceWith(
1030+ "Device %s does not support formatting"
1031+ % disk_name.decode("ascii")
1032+ ),
1033+ )
1034+
1035+ def test_try_secure_erase_successful_cryto_nvme(self):
1036+ disk_name = factory.make_name("nvme").encode("ascii")
1037+ sec_info = {
1038+ "format_supported": True,
1039+ "writez_supported": True,
1040+ "crypto_format": True,
1041+ "nsze": 0,
1042+ "lbaf": 0,
1043+ "ms": 0,
1044+ }
1045+ mock_check_output = self.patch(subprocess, "check_output")
1046+ self.assertTrue(try_secure_erase(disk_name, sec_info))
1047+ self.assertThat(
1048+ mock_check_output,
1049+ MockCalledOnceWith(
1050+ [
1051+ "nvme",
1052+ "format",
1053+ "-s",
1054+ "2",
1055+ "-l",
1056+ "0",
1057+ "-m",
1058+ "0",
1059+ maas_wipe.DEV_PATH % disk_name,
1060+ ]
1061+ ),
1062+ )
1063+ self.assertThat(
1064+ self.print_flush,
1065+ MockCalledOnceWith(
1066+ "Secure erase was successful on NVMe drive %s"
1067+ % disk_name.decode("ascii")
1068+ ),
1069+ )
1070+
1071+ def test_try_secure_erase_successful_nocryto_nvme(self):
1072+ disk_name = factory.make_name("nvme").encode("ascii")
1073+ sec_info = {
1074+ "format_supported": True,
1075+ "writez_supported": True,
1076+ "crypto_format": False,
1077+ "nsze": 0,
1078+ "lbaf": 0,
1079+ "ms": 0,
1080+ }
1081+ mock_check_output = self.patch(subprocess, "check_output")
1082+ self.assertTrue(try_secure_erase(disk_name, sec_info))
1083+ self.assertThat(
1084+ mock_check_output,
1085+ MockCalledOnceWith(
1086+ [
1087+ "nvme",
1088+ "format",
1089+ "-s",
1090+ "1",
1091+ "-l",
1092+ "0",
1093+ "-m",
1094+ "0",
1095+ maas_wipe.DEV_PATH % disk_name,
1096+ ]
1097+ ),
1098+ )
1099+ self.assertThat(
1100+ self.print_flush,
1101+ MockCalledOnceWith(
1102+ "Secure erase was successful on NVMe drive %s"
1103+ % disk_name.decode("ascii")
1104+ ),
1105+ )
1106+
1107+ def test_try_secure_erase_failed_nvme(self):
1108+ disk_name = factory.make_name("nvme").encode("ascii")
1109+ sec_info = {
1110+ "format_supported": True,
1111+ "writez_supported": True,
1112+ "crypto_format": True,
1113+ "nsze": 0,
1114+ "lbaf": 0,
1115+ "ms": 0,
1116+ }
1117+ mock_check_output = self.patch(subprocess, "check_output")
1118+ mock_check_output.side_effect = subprocess.CalledProcessError(
1119+ 1, "nvme format"
1120+ )
1121+
1122+ self.assertFalse(try_secure_erase(disk_name, sec_info))
1123+ self.assertThat(
1124+ self.print_flush,
1125+ MockCalledOnceWith("Error with format command (%s)" % "1"),
1126+ )
1127+
1128+ def test_try_write_zeroes_not_supported_nvme(self):
1129+ disk_name = factory.make_name("nvme").encode("ascii")
1130+ sec_info = {
1131+ "format_supported": False,
1132+ "writez_supported": False,
1133+ "crypto_format": False,
1134+ "nsze": 1,
1135+ "lbaf": 0,
1136+ "ms": 0,
1137+ }
1138+ mock_print = self.patch(builtins, "print")
1139+ self.assertFalse(nvme_write_zeroes(disk_name, sec_info))
1140+ self.assertThat(
1141+ mock_print,
1142+ MockCalledOnceWith(
1143+ "NVMe drive %s does not support write-zeroes"
1144+ % disk_name.decode("ascii")
1145+ ),
1146+ )
1147+ self.assertThat(
1148+ self.print_flush,
1149+ MockCalledOnceWith("Will fallback to regular drive zeroing."),
1150+ )
1151+
1152+ def test_try_write_zeroes_supported_invalid_nsze_nvme(self):
1153+ disk_name = factory.make_name("nvme").encode("ascii")
1154+ sec_info = {
1155+ "format_supported": False,
1156+ "writez_supported": True,
1157+ "crypto_format": False,
1158+ "nsze": 0,
1159+ "lbaf": 0,
1160+ "ms": 0,
1161+ }
1162+ mock_print = self.patch(builtins, "print")
1163+ self.assertFalse(nvme_write_zeroes(disk_name, sec_info))
1164+ self.assertThat(
1165+ mock_print,
1166+ MockCalledOnceWith(
1167+ "Bad namespace information collected on NVMe drive %s"
1168+ % disk_name.decode("ascii")
1169+ ),
1170+ )
1171+ self.assertThat(
1172+ self.print_flush,
1173+ MockCalledOnceWith("Will fallback to regular drive zeroing."),
1174+ )
1175+
1176+ def test_try_write_zeroes_successful_nvme(self):
1177+ disk_name = factory.make_name("nvme").encode("ascii")
1178+ sec_info = {
1179+ "format_supported": False,
1180+ "writez_supported": True,
1181+ "crypto_format": False,
1182+ "nsze": 0x100A,
1183+ "lbaf": 0,
1184+ "ms": 0,
1185+ }
1186+ mock_check_output = self.patch(subprocess, "check_output")
1187+ self.assertTrue(nvme_write_zeroes(disk_name, sec_info))
1188+ self.assertThat(
1189+ mock_check_output,
1190+ MockCalledOnceWith(
1191+ [
1192+ "nvme",
1193+ "write-zeroes",
1194+ "-f",
1195+ "-s",
1196+ "0",
1197+ "-c",
1198+ "100a",
1199+ maas_wipe.DEV_PATH % disk_name,
1200+ ]
1201+ ),
1202+ )
1203+ self.assertThat(
1204+ self.print_flush,
1205+ MockCalledOnceWith(
1206+ "%s: successfully zeroed (using write-zeroes)."
1207+ % disk_name.decode("ascii")
1208+ ),
1209+ )
1210+
1211+ def test_try_write_zeroes_failed_nvme(self):
1212+ disk_name = factory.make_name("nvme").encode("ascii")
1213+ sec_info = {
1214+ "format_supported": False,
1215+ "writez_supported": True,
1216+ "crypto_format": False,
1217+ "nsze": 100,
1218+ "lbaf": 0,
1219+ "ms": 0,
1220+ }
1221+ mock_check_output = self.patch(subprocess, "check_output")
1222+ mock_check_output.side_effect = subprocess.CalledProcessError(
1223+ 1, "nvme write-zeroes"
1224+ )
1225+
1226+ self.assertFalse(nvme_write_zeroes(disk_name, sec_info))
1227+ self.assertThat(
1228+ self.print_flush,
1229+ MockCalledOnceWith("Error with write-zeroes command (%s)" % "1"),
1230+ )
1231+
1232+ def test_secure_erase_writes_known_data_hdparm(self):
1233 tmp_dir = self.make_dir()
1234 dev_path = (tmp_dir + "/%s").encode("ascii")
1235 self.patch(maas_wipe, "DEV_PATH", dev_path)
1236@@ -455,7 +784,7 @@ class TestMAASWipe(MAASTestCase):
1237 mock_check_output = self.patch(subprocess, "check_output")
1238 mock_check_output.side_effect = factory.make_exception()
1239
1240- self.assertRaises(WipeError, secure_erase, dev_name)
1241+ self.assertRaises(WipeError, secure_erase_hdparm, dev_name)
1242 expected_buf = b"M" * 1024 * 1024
1243 with open(file_path, "rb") as fp:
1244 read_buf = fp.read(len(expected_buf))
1245@@ -463,7 +792,7 @@ class TestMAASWipe(MAASTestCase):
1246 expected_buf, read_buf, "First 1 MiB of file was not written."
1247 )
1248
1249- def test_secure_erase_sets_security_password(self):
1250+ def test_secure_erase_sets_security_password_hdparm(self):
1251 tmp_dir = self.make_dir()
1252 dev_path = (tmp_dir + "/%s").encode("ascii")
1253 self.patch(maas_wipe, "DEV_PATH", dev_path)
1254@@ -476,10 +805,10 @@ class TestMAASWipe(MAASTestCase):
1255 # Fail to get disk info just to exit early.
1256 exception_type = factory.make_exception_type()
1257 self.patch(
1258- maas_wipe, "get_disk_security_info"
1259+ maas_wipe, "get_hdparm_security_info"
1260 ).side_effect = exception_type()
1261
1262- self.assertRaises(exception_type, secure_erase, dev_name)
1263+ self.assertRaises(exception_type, secure_erase_hdparm, dev_name)
1264 self.assertThat(
1265 mock_check_output,
1266 MockCalledOnceWith(
1267@@ -494,7 +823,7 @@ class TestMAASWipe(MAASTestCase):
1268 ),
1269 )
1270
1271- def test_secure_erase_fails_if_not_enabled(self):
1272+ def test_secure_erase_fails_if_not_enabled_hdparm(self):
1273 tmp_dir = self.make_dir()
1274 dev_path = (tmp_dir + "/%s").encode("ascii")
1275 self.patch(maas_wipe, "DEV_PATH", dev_path)
1276@@ -503,16 +832,16 @@ class TestMAASWipe(MAASTestCase):
1277 self.make_empty_file(file_path)
1278
1279 self.patch(subprocess, "check_output")
1280- self.patch(maas_wipe, "get_disk_security_info").return_value = {
1281+ self.patch(maas_wipe, "get_hdparm_security_info").return_value = {
1282 b"enabled": False
1283 }
1284
1285- error = self.assertRaises(WipeError, secure_erase, dev_name)
1286+ error = self.assertRaises(WipeError, secure_erase_hdparm, dev_name)
1287 self.assertEqual(
1288 "Failed to enable security to perform secure erase.", str(error)
1289 )
1290
1291- def test_secure_erase_fails_when_still_enabled(self):
1292+ def test_secure_erase_fails_when_still_enabled_hdparm(self):
1293 tmp_dir = self.make_dir()
1294 dev_path = (tmp_dir + "/%s").encode("ascii")
1295 self.patch(maas_wipe, "DEV_PATH", dev_path)
1296@@ -521,14 +850,14 @@ class TestMAASWipe(MAASTestCase):
1297 self.make_empty_file(file_path)
1298
1299 mock_check_output = self.patch(subprocess, "check_output")
1300- self.patch(maas_wipe, "get_disk_security_info").return_value = {
1301+ self.patch(maas_wipe, "get_hdparm_security_info").return_value = {
1302 b"enabled": True
1303 }
1304 exception = factory.make_exception()
1305 mock_check_call = self.patch(subprocess, "check_call")
1306 mock_check_call.side_effect = exception
1307
1308- error = self.assertRaises(WipeError, secure_erase, dev_name)
1309+ error = self.assertRaises(WipeError, secure_erase_hdparm, dev_name)
1310 self.assertThat(
1311 mock_check_call,
1312 MockCalledOnceWith(
1313@@ -561,7 +890,7 @@ class TestMAASWipe(MAASTestCase):
1314 self.assertEqual("Failed to securely erase.", str(error))
1315 self.assertEqual(exception, error.__cause__)
1316
1317- def test_secure_erase_fails_when_buffer_not_different(self):
1318+ def test_secure_erase_fails_when_buffer_not_different_hdparm(self):
1319 tmp_dir = self.make_dir()
1320 dev_path = (tmp_dir + "/%s").encode("ascii")
1321 self.patch(maas_wipe, "DEV_PATH", dev_path)
1322@@ -570,13 +899,13 @@ class TestMAASWipe(MAASTestCase):
1323 self.make_empty_file(file_path)
1324
1325 self.patch(subprocess, "check_output")
1326- self.patch(maas_wipe, "get_disk_security_info").side_effect = [
1327+ self.patch(maas_wipe, "get_hdparm_security_info").side_effect = [
1328 {b"enabled": True},
1329 {b"enabled": False},
1330 ]
1331 mock_check_call = self.patch(subprocess, "check_call")
1332
1333- error = self.assertRaises(WipeError, secure_erase, dev_name)
1334+ error = self.assertRaises(WipeError, secure_erase_hdparm, dev_name)
1335 self.assertThat(
1336 mock_check_call,
1337 MockCalledOnceWith(
1338@@ -595,7 +924,7 @@ class TestMAASWipe(MAASTestCase):
1339 str(error),
1340 )
1341
1342- def test_secure_erase_fails_success(self):
1343+ def test_secure_erase_fails_success_hdparm(self):
1344 tmp_dir = self.make_dir()
1345 dev_path = (tmp_dir + "/%s").encode("ascii")
1346 self.patch(maas_wipe, "DEV_PATH", dev_path)
1347@@ -604,7 +933,7 @@ class TestMAASWipe(MAASTestCase):
1348 self.make_empty_file(file_path)
1349
1350 self.patch(subprocess, "check_output")
1351- self.patch(maas_wipe, "get_disk_security_info").side_effect = [
1352+ self.patch(maas_wipe, "get_hdparm_security_info").side_effect = [
1353 {b"enabled": True},
1354 {b"enabled": False},
1355 ]
1356@@ -620,9 +949,9 @@ class TestMAASWipe(MAASTestCase):
1357 mock_check_call.side_effect = wipe_buffer
1358
1359 # No error should be raised.
1360- secure_erase(dev_name)
1361+ secure_erase_hdparm(dev_name)
1362
1363- def test_wipe_quickly(self):
1364+ def test_wipe_quickly_successful(self):
1365 tmp_dir = self.make_dir()
1366 dev_path = (tmp_dir + "/%s").encode("ascii")
1367 self.patch(maas_wipe, "DEV_PATH", dev_path)
1368@@ -630,7 +959,14 @@ class TestMAASWipe(MAASTestCase):
1369 file_path = dev_path % dev_name
1370 self.make_empty_file(file_path, content=b"T")
1371
1372+ mock_check_output = self.patch(subprocess, "check_output")
1373 wipe_quickly(dev_name)
1374+ self.assertThat(
1375+ mock_check_output,
1376+ MockCalledOnceWith(
1377+ ["wipefs", "-f", "-a", maas_wipe.DEV_PATH % dev_name]
1378+ ),
1379+ )
1380
1381 buf_size = 1024 * 1024
1382 with open(file_path, "rb") as fp:
1383@@ -641,8 +977,18 @@ class TestMAASWipe(MAASTestCase):
1384 zero_buf = b"\0" * 1024 * 1024
1385 self.assertEqual(zero_buf, first_buf, "First 1 MiB was not wiped.")
1386 self.assertEqual(zero_buf, last_buf, "Last 1 MiB was not wiped.")
1387+ self.assertThat(
1388+ self.print_flush,
1389+ MockCallsMatch(
1390+ call("%s: starting quick wipe." % dev_name.decode("ascii")),
1391+ call(
1392+ "%s: successfully quickly wiped."
1393+ % dev_name.decode("ascii")
1394+ ),
1395+ ),
1396+ )
1397
1398- def test_zero_disk(self):
1399+ def test_wipe_quickly_successful_but_wipefs_failed(self):
1400 tmp_dir = self.make_dir()
1401 dev_path = (tmp_dir + "/%s").encode("ascii")
1402 self.patch(maas_wipe, "DEV_PATH", dev_path)
1403@@ -650,13 +996,83 @@ class TestMAASWipe(MAASTestCase):
1404 file_path = dev_path % dev_name
1405 self.make_empty_file(file_path, content=b"T")
1406
1407+ mock_check_output = self.patch(subprocess, "check_output")
1408+ mock_check_output.side_effect = subprocess.CalledProcessError(
1409+ 1, "wipefs"
1410+ )
1411+ wipe_quickly(dev_name)
1412+
1413+ buf_size = 1024 * 1024
1414+ with open(file_path, "rb") as fp:
1415+ first_buf = fp.read(buf_size)
1416+ fp.seek(-buf_size, 2)
1417+ last_buf = fp.read(buf_size)
1418+
1419+ zero_buf = b"\0" * 1024 * 1024
1420+ self.assertEqual(zero_buf, first_buf, "First 1 MiB was not wiped.")
1421+ self.assertEqual(zero_buf, last_buf, "Last 1 MiB was not wiped.")
1422+ self.assertThat(
1423+ self.print_flush,
1424+ MockCallsMatch(
1425+ call("%s: starting quick wipe." % dev_name.decode("ascii")),
1426+ call("%s: wipefs failed (1)" % dev_name.decode("ascii")),
1427+ call(
1428+ "%s: successfully quickly wiped."
1429+ % dev_name.decode("ascii")
1430+ ),
1431+ ),
1432+ )
1433+
1434+ def test_wipe_quickly_failed(self):
1435+ dev_name = factory.make_name("disk").encode("ascii")
1436+
1437+ mock_check_output = self.patch(subprocess, "check_output")
1438+ mock_check_output.side_effect = subprocess.CalledProcessError(
1439+ 1, "wipefs"
1440+ )
1441+
1442+ mock_os_open = self.patch(builtins, "open")
1443+ mock_os_open.side_effect = OSError(-2, "No such file or directory")
1444+
1445+ wipe_quickly(dev_name)
1446+
1447+ self.assertThat(
1448+ self.print_flush,
1449+ MockCallsMatch(
1450+ call("%s: starting quick wipe." % dev_name.decode("ascii")),
1451+ call("%s: wipefs failed (1)" % dev_name.decode("ascii")),
1452+ call(
1453+ "%s: OS error while wiping beginning/end of disk (No such file or directory)"
1454+ % dev_name.decode("ascii")
1455+ ),
1456+ call(
1457+ "%s: failed to be quickly wiped."
1458+ % dev_name.decode("ascii")
1459+ ),
1460+ ),
1461+ )
1462+
1463+ def test_zero_disk_hdd(self):
1464+ tmp_dir = self.make_dir()
1465+ dev_path = (tmp_dir + "/%s").encode("ascii")
1466+ self.patch(maas_wipe, "DEV_PATH", dev_path)
1467+ dev_name = factory.make_name("disk").encode("ascii")
1468+ file_path = dev_path % dev_name
1469+ self.make_empty_file(file_path, content=b"T")
1470+ disk_info = {
1471+ b"supported": True,
1472+ b"enabled": False,
1473+ b"locked": False,
1474+ b"frozen": False,
1475+ }
1476+
1477 # Add a little size to the file making it not evenly
1478 # divisable by 1 MiB.
1479 extra_end = 512
1480 with open(file_path, "a+b") as fp:
1481 fp.write(b"T" * extra_end)
1482
1483- zero_disk(dev_name)
1484+ zero_disk(dev_name, disk_info)
1485
1486 zero_buf = b"\0" * 1024 * 1024
1487 with open(file_path, "rb") as fp:
1488@@ -682,7 +1098,7 @@ class TestMAASWipe(MAASTestCase):
1489 parser.parse_args.return_value = args
1490 self.patch(argparse, "ArgumentParser").return_value = parser
1491
1492- def test_main_calls_try_secure_erase_for_all_disks(self):
1493+ def test_main_calls_try_secure_erase_for_all_hdd(self):
1494 self.patch_args(True, False)
1495 disks = {
1496 factory.make_name("disk").encode("ascii"): {} for _ in range(3)
1497@@ -698,7 +1114,7 @@ class TestMAASWipe(MAASTestCase):
1498 self.assertThat(mock_try, MockCallsMatch(*calls))
1499 self.assertThat(mock_zero, MockNotCalled())
1500
1501- def test_main_calls_zero_disk_if_no_secure_erase(self):
1502+ def test_main_calls_zero_disk_if_no_secure_erase_hdd(self):
1503 self.patch_args(True, False)
1504 disks = {
1505 factory.make_name("disk").encode("ascii"): {} for _ in range(3)
1506@@ -711,11 +1127,10 @@ class TestMAASWipe(MAASTestCase):
1507 maas_wipe.main()
1508
1509 try_calls = [call(disk, info) for disk, info in disks.items()]
1510- wipe_calls = [call(disk) for disk in disks.keys()]
1511 self.assertThat(mock_try, MockCallsMatch(*try_calls))
1512- self.assertThat(mock_zero, MockCallsMatch(*wipe_calls))
1513+ self.assertThat(mock_zero, MockCallsMatch(*try_calls))
1514
1515- def test_main_calls_wipe_quickly_if_no_secure_erase(self):
1516+ def test_main_calls_wipe_quickly_if_no_secure_erase_hdd(self):
1517 self.patch_args(True, True)
1518 disks = {
1519 factory.make_name("disk").encode("ascii"): {} for _ in range(3)
1520@@ -760,6 +1175,6 @@ class TestMAASWipe(MAASTestCase):
1521 mock_try.return_value = False
1522 maas_wipe.main()
1523
1524- wipe_calls = [call(disk) for disk in disks.keys()]
1525+ wipe_calls = [call(disk, info) for disk, info in disks.items()]
1526 self.assertThat(mock_try, MockNotCalled())
1527 self.assertThat(zero_disk, MockCallsMatch(*wipe_calls))
1528diff --git a/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe_defs.py b/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe_defs.py
1529new file mode 100644
1530index 0000000..f9bb440
1531--- /dev/null
1532+++ b/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe_defs.py
1533@@ -0,0 +1,232 @@
1534+#!/usr/bin/python3
1535+# Copyright 2020 Canonical Ltd. This software is licensed under the
1536+# GNU Affero General Public License version 3 (see the file LICENSE).
1537+#
1538+# hdparm / nvme-cli outputs used on maas_wipe testing
1539+
1540+HDPARM_BEFORE_SECURITY = b"""\
1541+/dev/sda:
1542+
1543+ATA device, with non-removable media
1544+ Model Number: INTEL SSDSC2CT240A4
1545+ Serial Number: CVKI3206029X240DGN
1546+ Firmware Revision: 335u
1547+ Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions
1548+Standards:
1549+ Used: unknown (minor revision code 0xffff)
1550+ Supported: 9 8 7 6 5
1551+ Likely used: 9
1552+Configuration:
1553+ Logical max current
1554+ cylinders 16383 16383
1555+ heads 16 16
1556+ sectors/track 63 63
1557+ --
1558+ CHS current addressable sectors: 16514064
1559+ LBA user addressable sectors: 268435455
1560+ LBA48 user addressable sectors: 468862128
1561+ Logical Sector size: 512 bytes
1562+ Physical Sector size: 512 bytes
1563+ Logical Sector-0 offset: 0 bytes
1564+ device size with M = 1024*1024: 228936 MBytes
1565+ device size with M = 1000*1000: 240057 MBytes (240 GB)
1566+ cache/buffer size = unknown
1567+ Nominal Media Rotation Rate: Solid State Device
1568+Capabilities:
1569+ LBA, IORDY(can be disabled)
1570+ Queue depth: 32
1571+ Standby timer values: spec'd by Standard, no device specific minimum
1572+ R/W multiple sector transfer: Max = 16 Current = 16
1573+ Advanced power management level: 254
1574+ DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
1575+ Cycle time: min=120ns recommended=120ns
1576+ PIO: pio0 pio1 pio2 pio3 pio4
1577+ Cycle time: no flow control=120ns IORDY flow control=120ns
1578+Commands/features:
1579+ Enabled Supported:
1580+ * SMART feature set
1581+ Security Mode feature set
1582+ * Power Management feature set
1583+ * Write cache
1584+ * Look-ahead
1585+ * Host Protected Area feature set
1586+ * WRITE_BUFFER command
1587+ * READ_BUFFER command
1588+ * NOP cmd
1589+ * DOWNLOAD_MICROCODE
1590+ * Advanced Power Management feature set
1591+ Power-Up In Standby feature set
1592+ * 48-bit Address feature set
1593+ * Mandatory FLUSH_CACHE
1594+ * FLUSH_CACHE_EXT
1595+ * SMART error logging
1596+ * SMART self-test
1597+ * General Purpose Logging feature set
1598+ * WRITE_{DMA|MULTIPLE}_FUA_EXT
1599+ * 64-bit World wide name
1600+ * IDLE_IMMEDIATE with UNLOAD
1601+ * WRITE_UNCORRECTABLE_EXT command
1602+ * {READ,WRITE}_DMA_EXT_GPL commands
1603+ * Segmented DOWNLOAD_MICROCODE
1604+ * Gen1 signaling speed (1.5Gb/s)
1605+ * Gen2 signaling speed (3.0Gb/s)
1606+ * Gen3 signaling speed (6.0Gb/s)
1607+ * Native Command Queueing (NCQ)
1608+ * Host-initiated interface power management
1609+ * Phy event counters
1610+ * DMA Setup Auto-Activate optimization
1611+ Device-initiated interface power management
1612+ * Software settings preservation
1613+ * SMART Command Transport (SCT) feature set
1614+ * SCT Data Tables (AC5)
1615+ * reserved 69[4]
1616+ * Data Set Management TRIM supported (limit 1 block)
1617+ * Deterministic read data after TRIM
1618+"""
1619+
1620+HDPARM_AFTER_SECURITY = b"""\
1621+Logical Unit WWN Device Identifier: 55cd2e40002643cf
1622+ NAA : 5
1623+ IEEE OUI : 5cd2e4
1624+ Unique ID : 0002643cf
1625+Checksum: correct
1626+"""
1627+
1628+HDPARM_SECURITY_NOT_SUPPORTED = b"""\
1629+Security:
1630+ Master password revision code = 65534
1631+ not supported
1632+ not enabled
1633+ not locked
1634+ not frozen
1635+ not expired: security count
1636+ supported: enhanced erase
1637+ 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
1638+"""
1639+
1640+HDPARM_SECURITY_SUPPORTED_NOT_ENABLED = b"""\
1641+Security:
1642+ Master password revision code = 65534
1643+ supported
1644+ not enabled
1645+ not locked
1646+ not frozen
1647+ not expired: security count
1648+ supported: enhanced erase
1649+ 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
1650+"""
1651+
1652+HDPARM_SECURITY_SUPPORTED_ENABLED = b"""\
1653+Security:
1654+ Master password revision code = 65534
1655+ supported
1656+ enabled
1657+ not locked
1658+ not frozen
1659+ not expired: security count
1660+ supported: enhanced erase
1661+ 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
1662+"""
1663+
1664+HDPARM_SECURITY_ALL_TRUE = b"""\
1665+Security:
1666+ Master password revision code = 65534
1667+ supported
1668+ enabled
1669+ locked
1670+ frozen
1671+ not expired: security count
1672+ supported: enhanced erase
1673+ 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT.
1674+"""
1675+
1676+NVME_IDCTRL_PROLOGUE = b"""\
1677+NVME Identify Controller:
1678+vid : 0x8086
1679+ssvid : 0x8086
1680+sn : CVMD5066002T400AGN
1681+mn : INTEL SSDPEDME400G4
1682+fr : 8DV10131
1683+rab : 0
1684+ieee : 5cd2e4
1685+cmic : 0
1686+mdts : 5
1687+cntlid : 0
1688+ver : 0
1689+rtd3r : 0
1690+rtd3e : 0
1691+oaes : 0
1692+ctratt : 0
1693+acl : 3
1694+aerl : 3
1695+frmw : 0x2
1696+lpa : 0
1697+elpe : 63
1698+npss : 0
1699+avscc : 0
1700+apsta : 0
1701+wctemp : 0
1702+cctemp : 0
1703+mtfa : 0
1704+hmpre : 0
1705+hmmin : 0
1706+tnvmcap : 0
1707+unvmcap : 0
1708+rpmbs : 0
1709+edstt : 0
1710+dsto : 0
1711+fwug : 0
1712+kas : 0
1713+hctma : 0
1714+mntmt : 0
1715+mxtmt : 0
1716+sanicap : 0
1717+hmminds : 0
1718+hmmaxd : 0
1719+sqes : 0x66
1720+cqes : 0x44
1721+maxcmd : 0
1722+nn : 1
1723+fuses : 0
1724+"""
1725+
1726+NVME_IDCTRL_OACS_FORMAT_SUPPORTED = b"""\
1727+oacs : 0x6
1728+"""
1729+
1730+NVME_IDCTRL_OACS_FORMAT_UNSUPPORTED = b"""\
1731+oacs : 0x4
1732+"""
1733+
1734+NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED = b"""\
1735+oncs : 0xe
1736+"""
1737+
1738+NVME_IDCTRL_ONCS_WRITEZ_UNSUPPORTED = b"""\
1739+oncs : 0x6
1740+"""
1741+
1742+NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED = b"""\
1743+fna : 0x7
1744+"""
1745+
1746+NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED = b"""\
1747+fna : 0x3
1748+"""
1749+
1750+NVME_IDCTRL_EPILOGUE = b"""\
1751+vwc : 0
1752+awun : 0
1753+awupf : 0
1754+nvscc : 0
1755+acwu : 0
1756+sgls : 0
1757+subnqn :
1758+ioccsz : 0
1759+iorcsz : 0
1760+icdoff : 0
1761+ctrattr : 0
1762+msdbd : 0
1763+ps 0 : mp:25.00W operational enlat:0 exlat:0 rrt:0 rrl:0
1764+ rwt:0 rwl:0 idle_power:- active_power:-
1765+"""

Subscribers

People subscribed via source and target branches