Merge ~gpiccoli/maas:nvme_secure_erase into maas:master
- Git
- lp:~gpiccoli/maas
- nvme_secure_erase
- Merge into master
Status: | Superseded | ||||
---|---|---|---|---|---|
Proposed branch: | ~gpiccoli/maas:nvme_secure_erase | ||||
Merge into: | maas:master | ||||
Diff against target: |
1765 lines (+1120/-242) 6 files modified
src/maasserver/compose_preseed.py (+3/-0) src/maasserver/tests/test_compose_preseed.py (+1/-0) src/maasserver/tests/test_preseed.py (+1/-0) src/metadataserver/user_data/templates/snippets/maas_wipe.py (+289/-63) src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py (+594/-179) src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe_defs.py (+232/-0) |
||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
MAAS Lander | Needs Fixing | ||
Adam Collard | Pending | ||
Lee Trager | Pending | ||
Review via email: mp+386890@code.launchpad.net |
This proposal supersedes a proposal from 2020-07-03.
This proposal has been superseded by a proposal from 2020-07-06.
Commit message
This is the NVMe secure erase implementation (+ one more patch, to fix/improve quick erase).
I tried to explain/detail everything in the commit messages; also, code/tests were validated against Flake8 to prevent style issues.
Description of the change
V3:
- Fixed compose_preseed (forgot to account for Node == None, thanks d0ugal for helping me with the CI log!)
V2:
- Fixed style issues by running Black (thanks Adam!)
- Implemented the great suggestion from Lee about using vendor_data to install nvme-cli instead of reinventing the wheel
- Improved some exception messages (removed '\n', better formatting, etc)
- Implemented more exceptions for nvme-cli (OSError) and their respective unit tests
Thanks in advance for the re-review!
Cheers,
Guilherme
MAAS Lander (maas-lander) wrote : Posted in a previous version of this proposal | # |
Adam Collard (adam-collard) wrote : Posted in a previous version of this proposal | # |
+ sudo -u ubuntu -E -H make lint
ERROR: /run/build/
--- /run/build/
+++ /run/build/
@@ -6,11 +6,12 @@
__all__ = []
import argparse
+import builtins
import subprocess
-import builtins
from textwrap import dedent
from unittest.mock import call, MagicMock
+# hdparm and nvme-cli outputs used in the tests
from maastesting.factory import factory
from maastesting.
MockCalle
@@ -22,29 +23,31 @@
from snippets.maas_wipe import (
get_
get_
+ install_nvme_cli,
list_disks,
+ nvme_write_zeroes,
secure_
try_
- install_nvme_cli,
- nvme_write_zeroes,
wipe_quickly,
WipeError,
zero_disk,
)
-# hdparm and nvme-cli outputs used in the tests
from snippets.
- HDPARM_
+ HDPARM_
+ HDPARM_
+ HDPARM_
HDPARM_
+ HDPARM_
HDPARM_
- HDPARM_
- HDPARM_
+ NVME_IDCTRL_
+ NVME_IDCTRL_
+ NVME_IDCTRL_
NVME_
NVME_
NVME_
NVME_
- NVME_IDCTRL_
- NVME_IDCTRL_
- NVME_IDCTRL_
+ NVME_IDCTRL_
+)
Lee Trager (ltrager) wrote : Posted in a previous version of this proposal | # |
Thanks for submitting things, two small things below in addition to what Adam posted.
Lee Trager (ltrager) : Posted in a previous version of this proposal | # |
Igor Gnip (igorgnip) wrote : Posted in a previous version of this proposal | # |
Hello,
This still does not fail if secure erase is not supported.
Other than that, I don't see any verification that data on the disks changed after tool reported that wipe was done. Do we trust the tool ? Do we flush the buffers and force re-read from storage device ?
Also, as long as there is no way to force this script to fail if secure erase was not done, it is useless for our use case. I am happy that other use cases got coverage for NVME but consider environments where zero wiping is considered not secure enough.
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal | # |
Thanks Adam, I'll fix that!
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal | # |
Thanks Lee, take a look in the diff comments for the inline replies!
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal | # |
Hi Igor, thanks again for you feedback. I think we are talking about 2 different features!
(a) Supporting NVMe secure erase. Currently it is not *supported*. So, users with NVMe drives cannot at all have them securely erased. This is what we are addressing here, AFAIK.
(b) To fail the erase operation if a secure erase is not performed well (or not supported). I think for this item, the best would be to have a new checkbox in MAAS user interface, something like "Fail if not secure erased".
Right now, the way secure erase is implemented is an attempt to do the secure erase operation and if it does not complete properly or if it's not supported, we fallback to zeroing the device. I guess we can discuss the change to a more strict mode with MAAS team, but I consider it a different feature, which is not in the scope for this proposal.
Finally, regarding the validation if secure erase was correct, like by writing data in the device and checking back after the secure operation: in a first moment that seems a good idea. But..how can we be sure?
Imagine we have a 1TB NVMe device , and due to the way FW works, it returns secure erase as completed but it took some more time and FW continue working in background. What if we write 1G of data, secure erase the drive and read back 1G, all "wrong" data (meaning secure erase worked), but...after the 500G offset in the disk, the data is not secure erased yet? Then if an user hard power-off the system (like removing the power cable), it is still prone to a cold-boot attack.
I know I'm a bit "philosophical" here, and that hypothesis I suggested above is almost nonsense, but by not trusting the tool, we should (for consistency) not trust the firmware, so...what to do?
I vote to trust the tool/firmware in a first moment, then we could improve that (although I'm not sure how to be fully guarded that secure erase worked). Perhaps nvme sanitize for devices that support that?
Cheers,
Guilherme
Lee Trager (ltrager) wrote : Posted in a previous version of this proposal | # |
The way MAAS runs disk erasing is by sending the script below as user_data. cloud-init runs the script and shuts down the system when it is done. MAAS does not capture logs(besides syslog) of this process and there is no way for maas-wipe to signal any sort of failure. MAAS just assumes everything went well and does not validate that disk erasing actually happened.
I've wanted to change this for awhile. We could pretty easily leverage the commissioning/
To independently validate that erasure happened would be very time consuming. Secure erase itself should be almost instant. The firmware tells every memory cell on the SSD to either goto 0 or a random value. However to validate to validate that worked maas-wipe would have to write a repeating pattern that fills the block device, use secure erase, then check the pattern isn't found anywhere on the block device. You have to check the entire block device as many SSDs now have an internal cache.
I don't think maas-wipe should be validating that firmware is doing what it claims it supports. MAAS supports hardware testing which is where firmware should be validated.
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal | # |
> [...]
> Secure erase itself should be almost instant. The firmware tells every memory
> cell on the SSD to either goto 0 or a random value.
Thanks Lee! To add on that, by using the cryptographic secure erase in NVMe (which is our first option if available) is even faster, the FW just recreates the internal key, which invalidates all previously written data heheh
Cheers,
Guilherme
Lee Trager (ltrager) : Posted in a previous version of this proposal | # |
Guilherme G. Piccoli (gpiccoli) : Posted in a previous version of this proposal | # |
MAAS Lander (maas-lander) wrote : Posted in a previous version of this proposal | # |
UNIT TESTS
-b nvme_secure_erase lp:~gpiccoli/maas/+git/maas into -b master lp:~maas-committers/maas
STATUS: FAILED
LOG: http://
COMMIT: 5a4fc8024a3825d
Igor Gnip (igorgnip) wrote : Posted in a previous version of this proposal | # |
Cryptographic erase just deletes the key
which in fact leaves data intact
for someone to bruteforce the key (theoretically), all data is still accessible.
In our implementation, we do both cryptographic erase (change key) and plain zero erase. Just to be sure that both key is changed and data is gone.
As far as controlling success of operation, I firmly believe that release should just release machine from maas database and the ui buttons on maas ui should actually produce a TESTING call with using the chosen testing modules to perform actual erase / test.
Release should check for result of TEST and if all passed, release the machine.
But there are many ways that can be good or bad depending on use case.
We currently run a test and attach a script which erases drives. If the test script fails, erase was not done.
Release api call is always called just to release the machine (no erase box checked).
This seems to work better and provides more granular control of the process.
Regards,
Igor
Guilherme G. Piccoli (gpiccoli) wrote : Posted in a previous version of this proposal | # |
Igor, I guess a lot of firmware implementations of NVMe devices use cryptographic erase *even if you don't set that*, because (a) it's secure, no way to recover the data, (b) it's quite faster, (c) prevents the wear of disk memories, by not writing in the full disk.
I think there's a limit on what secure erase can do if you'll count brute-force recover attempts and whatnot, and if you'll not trust the device's firmware.
Cheers,
Guilherme
MAAS Lander (maas-lander) wrote : | # |
UNIT TESTS
-b nvme_secure_erase lp:~gpiccoli/maas/+git/maas into -b master lp:~maas-committers/maas
STATUS: FAILED
LOG: http://
COMMIT: f6c8240f466c5d0
Unmerged commits
- e5308f9... by Guilherme G. Piccoli
-
Fix error in quick erase if OS throws an exception and add wipefs cleaning
Currently, if quick erase is used but OS throws any exception, the operation
is aborted and all subsequent disks to be quickly erased are ignored. Also,
the code currently only performs a 2MB write in the beginning and end of the
disk as the quick erase. This is usually enough, but we have more proper ways
of getting rid of old disk layouts for example (the wipefs command being the
de facto Linux standard way).This patch fixes the exception problem by using a try block in which the
OS-related functions are executed; exceptions don't completely break the
method anymore. Also, we add the wipefs call before the disk write, so it
catches other partition layouts that potentially aren't within the 2MB range.
Notice that we consider a fail if *both* the 2MB write and the wipefs command
fail - in case only one works, it is still a success.Unit tests were worked to take the wipefs and the exceptions into account.
We also validated the files against Flake8/Black to prevent style issues.Signed-off-by: Guilherme G. Piccoli <email address hidden>
- e8e93e5... by Guilherme G. Piccoli
-
LP: #1835954: Add NVMe secure erase / write zeroes support
MAAS currently allows 2 types of disk erase when releasing a node:
secure erase and quick erase. Secure erase fallback to zeroing the
disk in case the secure functionality doesn't work.There is a limitation though: hdparm is currently used to perform
secure erasing, regardless if the disk is a SCSI/ATA device or a NVMe
device. This is not a good idea mainly for 2 reasons:
(a) Secure erase obviously never works for NVMe, relying in the "slow"
procedure of fully zeroing the disk;
(b) Zeroing a NVMe device is not a 100% secure guaranteed operation;
some firmwares may return a completion but physically the blocks may
still contain data (prone to "cold boot"-like attacks).This patch proposes a solution to this problem by using nvme-cli tool
instead of hdparm if the disk is NVMe. Secure erase is attempted,
with cryptographic erase as a preferred mode (faster and less harmful
for the device). In case secure erase is not available or the operation
fails, zeroing the disk is still the fallback, but we try first to rely
on write-zeroes when available (much faster than fully writing zeroes
to the entire device). Only if both secure erase and write zeroes fail
we go with the full disk zeroing approach.This patch was tested in real NVMe device and all source files created
or modified here were validated against Flake8/Black for style issues.
I'd like to thank specially the following colleagues due to their help
and suggestions during this work: Dan Streetman, Igor Gnip, Rodrigo
"Ganso" Barbieri and Lee Trager (for his idea of using the vendor_data
to install nvme-cli).Signed-off-by: Guilherme G. Piccoli <email address hidden>
Preview Diff
1 | diff --git a/src/maasserver/compose_preseed.py b/src/maasserver/compose_preseed.py |
2 | index 8acf2f5..244d87f 100644 |
3 | --- a/src/maasserver/compose_preseed.py |
4 | +++ b/src/maasserver/compose_preseed.py |
5 | @@ -391,6 +391,9 @@ def get_base_preseed(node=None): |
6 | # jq is used during enlistment to read the JSON string containing |
7 | # the system_id of the newly created machine. |
8 | cloud_config["packages"] += ["jq"] |
9 | + # On disk erasing, we need nvme-cli |
10 | + if node is None or node.status == NODE_STATUS.DISK_ERASING: |
11 | + cloud_config["packages"] += ["nvme-cli"] |
12 | |
13 | return cloud_config |
14 | |
15 | diff --git a/src/maasserver/tests/test_compose_preseed.py b/src/maasserver/tests/test_compose_preseed.py |
16 | index 576fbb2..64d102b 100644 |
17 | --- a/src/maasserver/tests/test_compose_preseed.py |
18 | +++ b/src/maasserver/tests/test_compose_preseed.py |
19 | @@ -1086,6 +1086,7 @@ class TestComposePreseed(MAASServerTestCase): |
20 | "sshpass", |
21 | "archdetect-deb", |
22 | "jq", |
23 | + "nvme-cli", |
24 | ], |
25 | preseed["packages"], |
26 | ) |
27 | diff --git a/src/maasserver/tests/test_preseed.py b/src/maasserver/tests/test_preseed.py |
28 | index 077a215..4e455c1 100644 |
29 | --- a/src/maasserver/tests/test_preseed.py |
30 | +++ b/src/maasserver/tests/test_preseed.py |
31 | @@ -677,6 +677,7 @@ class TestRenderEnlistmentPreseed(MAASServerTestCase): |
32 | "sshpass", |
33 | "archdetect-deb", |
34 | "jq", |
35 | + "nvme-cli", |
36 | ], |
37 | preseed["packages"], |
38 | ) |
39 | diff --git a/src/metadataserver/user_data/templates/snippets/maas_wipe.py b/src/metadataserver/user_data/templates/snippets/maas_wipe.py |
40 | old mode 100644 |
41 | new mode 100755 |
42 | index 317c62b..f7e611a |
43 | --- a/src/metadataserver/user_data/templates/snippets/maas_wipe.py |
44 | +++ b/src/metadataserver/user_data/templates/snippets/maas_wipe.py |
45 | @@ -29,12 +29,95 @@ def list_disks(): |
46 | return disks |
47 | |
48 | |
49 | -def get_disk_security_info(disk): |
50 | - """Get the disk security information. |
51 | +def get_nvme_security_info(disk): |
52 | + """Gather NVMe information from the NVMe disks using the |
53 | + nvme-cli tool. Info from id-ctrl and id-ns is needed for |
54 | + secure erase (nvme format) and write zeroes.""" |
55 | + |
56 | + # Grab the relevant info from nvme id-ctrl. We need to check the |
57 | + # following bits: |
58 | + # |
59 | + # OACS (Optional Admin Command Support) bit 1: Format supported |
60 | + # ONCS (Optional NVM Command Support) bit 3: Write Zeroes supported |
61 | + # FNA (Format NVM Attributes) bit 2: Cryptographic format supported |
62 | + |
63 | + security_info = { |
64 | + "format_supported": False, |
65 | + "writez_supported": False, |
66 | + "crypto_format": False, |
67 | + "nsze": 0, |
68 | + "lbaf": 0, |
69 | + "ms": 0, |
70 | + } |
71 | + |
72 | + try: |
73 | + output = subprocess.check_output(["nvme", "id-ctrl", DEV_PATH % disk]) |
74 | + except subprocess.CalledProcessError as exc: |
75 | + print_flush("Error on nvme id-ctrl (%s)" % exc.returncode) |
76 | + return security_info |
77 | + except OSError as exc: |
78 | + print_flush("OS error when running nvme-cli (%s)" % exc.strerror) |
79 | + return security_info |
80 | + |
81 | + output = output.decode() |
82 | + |
83 | + for line in output.split("\n"): |
84 | + if "oacs" in line: |
85 | + oacs = line.split(":")[1] |
86 | + if int(oacs, 16) & 0x2: |
87 | + security_info["format_supported"] = True |
88 | + |
89 | + if "oncs" in line: |
90 | + oncs = line.split(":")[1] |
91 | + if int(oncs, 16) & 0x8: |
92 | + security_info["writez_supported"] = True |
93 | + |
94 | + if "fna" in line: |
95 | + fna = line.split(":")[1] |
96 | + if int(fna, 16) & 0x4: |
97 | + security_info["crypto_format"] = True |
98 | + |
99 | + # Next step: collect LBAF (LBA Format), MS (Metadata Setting) and |
100 | + # NSZE (Namespace Size) from id-ns. According to NVMe spec, bits 0:3 |
101 | + # from FLBAS corresponds to the LBAF value, whereas bit 4 is MS. |
102 | + |
103 | + try: |
104 | + output = subprocess.check_output(["nvme", "id-ns", DEV_PATH % disk]) |
105 | + except subprocess.CalledProcessError as exc: |
106 | + print_flush("Error on nvme id-ns (%s)" % exc.returncode) |
107 | + security_info["format_supported"] = False |
108 | + security_info["writez_supported"] = False |
109 | + return security_info |
110 | + except OSError as exc: |
111 | + print_flush("OS error when running nvme-cli (%s)" % exc.strerror) |
112 | + security_info["format_supported"] = False |
113 | + security_info["writez_supported"] = False |
114 | + return security_info |
115 | + |
116 | + output = output.decode() |
117 | + |
118 | + for line in output.split("\n"): |
119 | + if "nsze" in line: |
120 | + # According to spec., this should be used as 0-based value. |
121 | + nsze = line.split(":")[1] |
122 | + security_info["nsze"] = int(nsze, 16) - 1 |
123 | + |
124 | + if "flbas" in line: |
125 | + flbas = line.split(":")[1] |
126 | + flbas = int(flbas, 16) |
127 | + security_info["lbaf"] = flbas & 0xF |
128 | + |
129 | + if flbas & 0x10: |
130 | + security_info["ms"] = 1 |
131 | + |
132 | + return security_info |
133 | |
134 | - Uses `hdparam` to get security information about the disk. Sadly hdparam |
135 | - doesn't provide an output that makes it easy to parse. |
136 | + |
137 | +def get_hdparm_security_info(disk): |
138 | + """Get SCSI/ATA disk security info from hdparm. |
139 | + Sadly hdparam doesn't provide an output that makes it easy to parse. |
140 | """ |
141 | + |
142 | # Grab the security section for hdparam. |
143 | security_section = [] |
144 | output = subprocess.check_output([b"hdparm", b"-I", DEV_PATH % disk]) |
145 | @@ -62,57 +145,25 @@ def get_disk_security_info(disk): |
146 | return security_info |
147 | |
148 | |
149 | +def get_disk_security_info(disk): |
150 | + """Get the disk security information. |
151 | + |
152 | + Uses `hdparam` to get security information about the SCSI/ATA disks. |
153 | + If NVMe, nvme-cli is used instead. |
154 | + """ |
155 | + |
156 | + if b"nvme" in disk: |
157 | + return get_nvme_security_info(disk) |
158 | + |
159 | + return get_hdparm_security_info(disk) |
160 | + |
161 | + |
162 | def get_disk_info(): |
163 | """Return dictionary of wipeable disks and thier security information.""" |
164 | return {kname: get_disk_security_info(kname) for kname in list_disks()} |
165 | |
166 | |
167 | -def try_secure_erase(kname, info): |
168 | - """Try to wipe the disk with secure erase.""" |
169 | - if info[b"supported"]: |
170 | - if info[b"frozen"]: |
171 | - print_flush( |
172 | - "%s: not using secure erase; " |
173 | - "drive is currently frozen." % kname.decode("ascii") |
174 | - ) |
175 | - return False |
176 | - elif info[b"locked"]: |
177 | - print_flush( |
178 | - "%s: not using secure erase; " |
179 | - "drive is currently locked." % kname.decode("ascii") |
180 | - ) |
181 | - return False |
182 | - elif info[b"enabled"]: |
183 | - print_flush( |
184 | - "%s: not using secure erase; " |
185 | - "drive security is already enabled." % kname.decode("ascii") |
186 | - ) |
187 | - return False |
188 | - else: |
189 | - # Wiping using secure erase. |
190 | - try: |
191 | - secure_erase(kname) |
192 | - except Exception as e: |
193 | - print_flush( |
194 | - "%s: failed to be securely erased: %s" |
195 | - % (kname.decode("ascii"), e) |
196 | - ) |
197 | - return False |
198 | - else: |
199 | - print_flush( |
200 | - "%s: successfully securely erased." |
201 | - % (kname.decode("ascii")) |
202 | - ) |
203 | - return True |
204 | - else: |
205 | - print_flush( |
206 | - "%s: drive does not support secure erase." |
207 | - % (kname.decode("ascii")) |
208 | - ) |
209 | - return False |
210 | - |
211 | - |
212 | -def secure_erase(kname): |
213 | +def secure_erase_hdparm(kname): |
214 | """Securely wipe the device.""" |
215 | # First write 1 MiB of known data to the beginning of the block device. |
216 | # This is used to check at the end of the secure erase that it worked |
217 | @@ -142,7 +193,7 @@ def secure_erase(kname): |
218 | |
219 | # Now that the user password is set the device should have its |
220 | # security mode enabled. |
221 | - info = get_disk_security_info(kname) |
222 | + info = get_hdparm_security_info(kname) |
223 | if not info[b"enabled"]: |
224 | # If not enabled that means the password did not take, so it does not |
225 | # need to be cleared. |
226 | @@ -167,7 +218,7 @@ def secure_erase(kname): |
227 | failed_exc = exc |
228 | |
229 | # Make sure that the device is now not enabled. |
230 | - info = get_disk_security_info(kname) |
231 | + info = get_hdparm_security_info(kname) |
232 | if info[b"enabled"]: |
233 | # Wipe failed since security is still enabled. |
234 | subprocess.check_output( |
235 | @@ -184,23 +235,197 @@ def secure_erase(kname): |
236 | ) |
237 | |
238 | |
239 | -def wipe_quickly(kname): |
240 | - """Quickly wipe the disk by zeroing the beginning and end of the disk. |
241 | +def try_secure_erase_hdparm(kname, info): |
242 | + """Try to wipe the disk with secure erase.""" |
243 | + if info[b"supported"]: |
244 | + if info[b"frozen"]: |
245 | + print_flush( |
246 | + "%s: not using secure erase; " |
247 | + "drive is currently frozen." % kname.decode("ascii") |
248 | + ) |
249 | + return False |
250 | + elif info[b"locked"]: |
251 | + print_flush( |
252 | + "%s: not using secure erase; " |
253 | + "drive is currently locked." % kname.decode("ascii") |
254 | + ) |
255 | + return False |
256 | + elif info[b"enabled"]: |
257 | + print_flush( |
258 | + "%s: not using secure erase; " |
259 | + "drive security is already enabled." % kname.decode("ascii") |
260 | + ) |
261 | + return False |
262 | + else: |
263 | + # Wiping using secure erase. |
264 | + try: |
265 | + secure_erase_hdparm(kname) |
266 | + except Exception as e: |
267 | + print_flush( |
268 | + "%s: failed to be securely erased: %s" |
269 | + % (kname.decode("ascii"), e) |
270 | + ) |
271 | + return False |
272 | + else: |
273 | + print_flush( |
274 | + "%s: successfully securely erased." |
275 | + % (kname.decode("ascii")) |
276 | + ) |
277 | + return True |
278 | + else: |
279 | + print_flush( |
280 | + "%s: drive does not support secure erase." |
281 | + % (kname.decode("ascii")) |
282 | + ) |
283 | + return False |
284 | + |
285 | + |
286 | +def try_secure_erase_nvme(kname, info): |
287 | + """Perform a secure-erase on NVMe disk if that feature is |
288 | + available. Prefer cryptographic erase, when available.""" |
289 | + |
290 | + if not info["format_supported"]: |
291 | + print_flush( |
292 | + "Device %s does not support formatting" % kname.decode("ascii") |
293 | + ) |
294 | + return False |
295 | + |
296 | + if info["crypto_format"]: |
297 | + ses = 2 |
298 | + else: |
299 | + ses = 1 |
300 | + |
301 | + try: |
302 | + subprocess.check_output( |
303 | + [ |
304 | + "nvme", |
305 | + "format", |
306 | + "-s", |
307 | + str(ses), |
308 | + "-l", |
309 | + str(info["lbaf"]), |
310 | + "-m", |
311 | + str(info["ms"]), |
312 | + DEV_PATH % kname, |
313 | + ] |
314 | + ) |
315 | + except subprocess.CalledProcessError as exc: |
316 | + print_flush("Error with format command (%s)" % exc.returncode) |
317 | + return False |
318 | + except OSError as exc: |
319 | + print_flush("OS error when running nvme-cli (%s)" % exc.strerror) |
320 | + return False |
321 | + |
322 | + print_flush( |
323 | + "Secure erase was successful on NVMe drive %s" % kname.decode("ascii") |
324 | + ) |
325 | + return True |
326 | + |
327 | + |
328 | +def try_secure_erase(kname, info): |
329 | + """Entry-point for secure-erase for SCSI/ATA or NVMe disks.""" |
330 | + |
331 | + if b"nvme" in kname: |
332 | + return try_secure_erase_nvme(kname, info) |
333 | + |
334 | + return try_secure_erase_hdparm(kname, info) |
335 | |
336 | - This is not a secure erase but does make it harder to get the data from |
337 | - the device. |
338 | + |
339 | +def wipe_quickly(kname): |
340 | + """Quickly wipe the disk by using wipefs and zeroing the beginning |
341 | + and end of the disk. This is not a secure erase but does make it |
342 | + harder to get the data from the device and also clears previous layouts. |
343 | """ |
344 | + |
345 | + wipe_error = 0 |
346 | print_flush("%s: starting quick wipe." % kname.decode("ascii")) |
347 | + try: |
348 | + subprocess.check_output(["wipefs", "-f", "-a", DEV_PATH % kname]) |
349 | + wipe_error -= 1 |
350 | + except subprocess.CalledProcessError as exc: |
351 | + print_flush( |
352 | + "%s: wipefs failed (%s)" % (kname.decode("ascii"), exc.returncode) |
353 | + ) |
354 | + wipe_error += 1 |
355 | + |
356 | buf = b"\0" * 1024 * 1024 * 2 # 2 MiB |
357 | - with open(DEV_PATH % kname, "wb") as fp: |
358 | + try: |
359 | + fp = open(DEV_PATH % kname, "wb") |
360 | fp.write(buf) |
361 | fp.seek(-len(buf), 2) |
362 | fp.write(buf) |
363 | - print_flush("%s: successfully quickly wiped." % kname.decode("ascii")) |
364 | + wipe_error -= 1 |
365 | + except OSError as exc: |
366 | + print_flush( |
367 | + "%s: OS error while wiping beginning/end of disk (%s)" |
368 | + % (kname.decode("ascii"), exc.strerror) |
369 | + ) |
370 | + wipe_error += 1 |
371 | + |
372 | + if wipe_error > 0: |
373 | + print_flush("%s: failed to be quickly wiped." % kname.decode("ascii")) |
374 | + else: |
375 | + print_flush("%s: successfully quickly wiped." % kname.decode("ascii")) |
376 | + |
377 | + |
378 | +def nvme_write_zeroes(kname, info): |
379 | + """Perform a write-zeroes operation on NVMe device instead of |
380 | + dd'ing 0 to the entire disk if secure erase is not available. |
381 | + Write-zeroes is a faster way to clean a NVMe disk.""" |
382 | + |
383 | + fallback = False |
384 | + |
385 | + if not info["writez_supported"]: |
386 | + print( |
387 | + "NVMe drive %s does not support write-zeroes" |
388 | + % kname.decode("ascii") |
389 | + ) |
390 | + fallback = True |
391 | + |
392 | + if info["nsze"] <= 0: |
393 | + print( |
394 | + "Bad namespace information collected on NVMe drive %s" |
395 | + % kname.decode("ascii") |
396 | + ) |
397 | + fallback = True |
398 | + |
399 | + if fallback: |
400 | + print_flush("Will fallback to regular drive zeroing.") |
401 | + return False |
402 | + |
403 | + try: |
404 | + subprocess.check_output( |
405 | + [ |
406 | + "nvme", |
407 | + "write-zeroes", |
408 | + "-f", |
409 | + "-s", |
410 | + "0", |
411 | + "-c", |
412 | + str(hex(info["nsze"])[2:]), |
413 | + DEV_PATH % kname, |
414 | + ] |
415 | + ) |
416 | + except subprocess.CalledProcessError as exc: |
417 | + print_flush("Error with write-zeroes command (%s)" % exc.returncode) |
418 | + return False |
419 | + except OSError as exc: |
420 | + print_flush("OS error when running nvme-cli (%s)" % exc.strerror) |
421 | + return False |
422 | + |
423 | + print_flush( |
424 | + "%s: successfully zeroed (using write-zeroes)." % kname.decode("ascii") |
425 | + ) |
426 | + return True |
427 | + |
428 | + |
429 | +def zero_disk(kname, info): |
430 | + """Zero the entire disk, trying write-zeroes first if NVMe disk.""" |
431 | |
432 | + if b"nvme" in kname: |
433 | + if nvme_write_zeroes(kname, info): |
434 | + return |
435 | |
436 | -def zero_disk(kname): |
437 | - """Zero the entire disk.""" |
438 | # Get the total size of the device. |
439 | size = 0 |
440 | with open(DEV_PATH % kname, "rb") as fp: |
441 | @@ -266,8 +491,9 @@ def main(): |
442 | action="store_true", |
443 | default=False, |
444 | help=( |
445 | - "Wipe 1MiB at the start and at the end of the drive to make data " |
446 | - "recovery inconvenient and unlikely to happen by accident. This " |
447 | + "Wipe 2MiB at the start and at the end of the drive to make data " |
448 | + "recovery inconvenient and unlikely to happen by accident. Also, " |
449 | + "it runs wipefs to clear known partition/layout signatures. This " |
450 | "is not secure." |
451 | ), |
452 | ) |
453 | @@ -288,7 +514,7 @@ def main(): |
454 | if args.quick_erase: |
455 | wipe_quickly(kname) |
456 | else: |
457 | - zero_disk(kname) |
458 | + zero_disk(kname, info) |
459 | |
460 | print_flush("All disks have been successfully wiped.") |
461 | |
462 | diff --git a/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py b/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py |
463 | index b5edcad..300d643 100644 |
464 | --- a/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py |
465 | +++ b/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe.py |
466 | @@ -6,6 +6,7 @@ |
467 | __all__ = [] |
468 | |
469 | import argparse |
470 | +import builtins |
471 | import subprocess |
472 | from textwrap import dedent |
473 | from unittest.mock import call, MagicMock |
474 | @@ -22,148 +23,29 @@ from snippets.maas_wipe import ( |
475 | get_disk_info, |
476 | get_disk_security_info, |
477 | list_disks, |
478 | - secure_erase, |
479 | + nvme_write_zeroes, |
480 | + secure_erase_hdparm, |
481 | try_secure_erase, |
482 | wipe_quickly, |
483 | WipeError, |
484 | zero_disk, |
485 | ) |
486 | - |
487 | -HDPARM_BEFORE_SECURITY = b"""\ |
488 | -/dev/sda: |
489 | - |
490 | -ATA device, with non-removable media |
491 | - Model Number: INTEL SSDSC2CT240A4 |
492 | - Serial Number: CVKI3206029X240DGN |
493 | - Firmware Revision: 335u |
494 | - Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions |
495 | -Standards: |
496 | - Used: unknown (minor revision code 0xffff) |
497 | - Supported: 9 8 7 6 5 |
498 | - Likely used: 9 |
499 | -Configuration: |
500 | - Logical max current |
501 | - cylinders 16383 16383 |
502 | - heads 16 16 |
503 | - sectors/track 63 63 |
504 | - -- |
505 | - CHS current addressable sectors: 16514064 |
506 | - LBA user addressable sectors: 268435455 |
507 | - LBA48 user addressable sectors: 468862128 |
508 | - Logical Sector size: 512 bytes |
509 | - Physical Sector size: 512 bytes |
510 | - Logical Sector-0 offset: 0 bytes |
511 | - device size with M = 1024*1024: 228936 MBytes |
512 | - device size with M = 1000*1000: 240057 MBytes (240 GB) |
513 | - cache/buffer size = unknown |
514 | - Nominal Media Rotation Rate: Solid State Device |
515 | -Capabilities: |
516 | - LBA, IORDY(can be disabled) |
517 | - Queue depth: 32 |
518 | - Standby timer values: spec'd by Standard, no device specific minimum |
519 | - R/W multiple sector transfer: Max = 16 Current = 16 |
520 | - Advanced power management level: 254 |
521 | - DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 |
522 | - Cycle time: min=120ns recommended=120ns |
523 | - PIO: pio0 pio1 pio2 pio3 pio4 |
524 | - Cycle time: no flow control=120ns IORDY flow control=120ns |
525 | -Commands/features: |
526 | - Enabled Supported: |
527 | - * SMART feature set |
528 | - Security Mode feature set |
529 | - * Power Management feature set |
530 | - * Write cache |
531 | - * Look-ahead |
532 | - * Host Protected Area feature set |
533 | - * WRITE_BUFFER command |
534 | - * READ_BUFFER command |
535 | - * NOP cmd |
536 | - * DOWNLOAD_MICROCODE |
537 | - * Advanced Power Management feature set |
538 | - Power-Up In Standby feature set |
539 | - * 48-bit Address feature set |
540 | - * Mandatory FLUSH_CACHE |
541 | - * FLUSH_CACHE_EXT |
542 | - * SMART error logging |
543 | - * SMART self-test |
544 | - * General Purpose Logging feature set |
545 | - * WRITE_{DMA|MULTIPLE}_FUA_EXT |
546 | - * 64-bit World wide name |
547 | - * IDLE_IMMEDIATE with UNLOAD |
548 | - * WRITE_UNCORRECTABLE_EXT command |
549 | - * {READ,WRITE}_DMA_EXT_GPL commands |
550 | - * Segmented DOWNLOAD_MICROCODE |
551 | - * Gen1 signaling speed (1.5Gb/s) |
552 | - * Gen2 signaling speed (3.0Gb/s) |
553 | - * Gen3 signaling speed (6.0Gb/s) |
554 | - * Native Command Queueing (NCQ) |
555 | - * Host-initiated interface power management |
556 | - * Phy event counters |
557 | - * DMA Setup Auto-Activate optimization |
558 | - Device-initiated interface power management |
559 | - * Software settings preservation |
560 | - * SMART Command Transport (SCT) feature set |
561 | - * SCT Data Tables (AC5) |
562 | - * reserved 69[4] |
563 | - * Data Set Management TRIM supported (limit 1 block) |
564 | - * Deterministic read data after TRIM |
565 | -""" |
566 | - |
567 | -HDPARM_AFTER_SECURITY = b"""\ |
568 | -Logical Unit WWN Device Identifier: 55cd2e40002643cf |
569 | - NAA : 5 |
570 | - IEEE OUI : 5cd2e4 |
571 | - Unique ID : 0002643cf |
572 | -Checksum: correct |
573 | -""" |
574 | - |
575 | -HDPARM_SECURITY_NOT_SUPPORTED = b"""\ |
576 | -Security: |
577 | - Master password revision code = 65534 |
578 | - not supported |
579 | - not enabled |
580 | - not locked |
581 | - not frozen |
582 | - not expired: security count |
583 | - supported: enhanced erase |
584 | - 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT. |
585 | -""" |
586 | - |
587 | -HDPARM_SECURITY_SUPPORTED_NOT_ENABLED = b"""\ |
588 | -Security: |
589 | - Master password revision code = 65534 |
590 | - supported |
591 | - not enabled |
592 | - not locked |
593 | - not frozen |
594 | - not expired: security count |
595 | - supported: enhanced erase |
596 | - 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT. |
597 | -""" |
598 | - |
599 | -HDPARM_SECURITY_SUPPORTED_ENABLED = b"""\ |
600 | -Security: |
601 | - Master password revision code = 65534 |
602 | - supported |
603 | - enabled |
604 | - not locked |
605 | - not frozen |
606 | - not expired: security count |
607 | - supported: enhanced erase |
608 | - 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT. |
609 | -""" |
610 | - |
611 | -HDPARM_SECURITY_ALL_TRUE = b"""\ |
612 | -Security: |
613 | - Master password revision code = 65534 |
614 | - supported |
615 | - enabled |
616 | - locked |
617 | - frozen |
618 | - not expired: security count |
619 | - supported: enhanced erase |
620 | - 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT. |
621 | -""" |
622 | +from snippets.tests.test_maas_wipe_defs import ( |
623 | + HDPARM_AFTER_SECURITY, |
624 | + HDPARM_BEFORE_SECURITY, |
625 | + HDPARM_SECURITY_ALL_TRUE, |
626 | + HDPARM_SECURITY_NOT_SUPPORTED, |
627 | + HDPARM_SECURITY_SUPPORTED_ENABLED, |
628 | + HDPARM_SECURITY_SUPPORTED_NOT_ENABLED, |
629 | + NVME_IDCTRL_EPILOGUE, |
630 | + NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED, |
631 | + NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED, |
632 | + NVME_IDCTRL_OACS_FORMAT_SUPPORTED, |
633 | + NVME_IDCTRL_OACS_FORMAT_UNSUPPORTED, |
634 | + NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED, |
635 | + NVME_IDCTRL_ONCS_WRITEZ_UNSUPPORTED, |
636 | + NVME_IDCTRL_PROLOGUE, |
637 | +) |
638 | |
639 | |
640 | class TestMAASWipe(MAASTestCase): |
641 | @@ -196,11 +78,13 @@ class TestMAASWipe(MAASTestCase): |
642 | sdb disk 1 |
643 | sr0 rom 0 |
644 | sr1 rom 0 |
645 | + nvme0n1 disk 0 |
646 | + nvme1n1 disk 1 |
647 | """ |
648 | ).encode("ascii") |
649 | - self.assertEqual([b"sda"], list_disks()) |
650 | + self.assertEqual([b"sda", b"nvme0n1"], list_disks()) |
651 | |
652 | - def test_get_disk_security_info_missing(self): |
653 | + def test_get_disk_security_info_missing_hdparm(self): |
654 | hdparm_output = HDPARM_BEFORE_SECURITY + HDPARM_AFTER_SECURITY |
655 | mock_check_output = self.patch(subprocess, "check_output") |
656 | mock_check_output.return_value = hdparm_output |
657 | @@ -220,7 +104,7 @@ class TestMAASWipe(MAASTestCase): |
658 | observered, |
659 | ) |
660 | |
661 | - def test_get_disk_security_info_not_supported(self): |
662 | + def test_get_disk_security_info_not_supported_hdparm(self): |
663 | hdparm_output = ( |
664 | HDPARM_BEFORE_SECURITY |
665 | + HDPARM_SECURITY_NOT_SUPPORTED |
666 | @@ -244,7 +128,7 @@ class TestMAASWipe(MAASTestCase): |
667 | observered, |
668 | ) |
669 | |
670 | - def test_get_disk_security_info_supported_not_enabled(self): |
671 | + def test_get_disk_security_info_supported_not_enabled_hdparm(self): |
672 | hdparm_output = ( |
673 | HDPARM_BEFORE_SECURITY |
674 | + HDPARM_SECURITY_SUPPORTED_NOT_ENABLED |
675 | @@ -268,7 +152,7 @@ class TestMAASWipe(MAASTestCase): |
676 | observered, |
677 | ) |
678 | |
679 | - def test_get_disk_security_info_supported_enabled(self): |
680 | + def test_get_disk_security_info_supported_enabled_hdparm(self): |
681 | hdparm_output = ( |
682 | HDPARM_BEFORE_SECURITY |
683 | + HDPARM_SECURITY_SUPPORTED_ENABLED |
684 | @@ -292,7 +176,7 @@ class TestMAASWipe(MAASTestCase): |
685 | observered, |
686 | ) |
687 | |
688 | - def test_get_disk_security_info_all_true(self): |
689 | + def test_get_disk_security_info_all_true_hdparm(self): |
690 | hdparm_output = ( |
691 | HDPARM_BEFORE_SECURITY |
692 | + HDPARM_SECURITY_ALL_TRUE |
693 | @@ -316,7 +200,7 @@ class TestMAASWipe(MAASTestCase): |
694 | observered, |
695 | ) |
696 | |
697 | - def test_get_disk_info(self): |
698 | + def test_get_disk_info_hdparm(self): |
699 | disk_names = [ |
700 | factory.make_name("disk").encode("ascii") for _ in range(3) |
701 | ] |
702 | @@ -331,14 +215,243 @@ class TestMAASWipe(MAASTestCase): |
703 | for _ in range(3) |
704 | ] |
705 | self.patch( |
706 | - maas_wipe, "get_disk_security_info" |
707 | + maas_wipe, "get_hdparm_security_info" |
708 | + ).side_effect = security_info |
709 | + observed = get_disk_info() |
710 | + self.assertEqual( |
711 | + {disk_names[i]: security_info[i] for i in range(3)}, observed |
712 | + ) |
713 | + |
714 | + def test_get_disk_security_info_crypt_format_writez_nvme(self): |
715 | + nvme_cli_output = ( |
716 | + NVME_IDCTRL_PROLOGUE |
717 | + + NVME_IDCTRL_OACS_FORMAT_SUPPORTED |
718 | + + NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED |
719 | + + NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED |
720 | + + NVME_IDCTRL_EPILOGUE |
721 | + ) |
722 | + mock_check_output = self.patch(subprocess, "check_output") |
723 | + mock_check_output.return_value = nvme_cli_output |
724 | + disk_name = factory.make_name("nvme").encode("ascii") |
725 | + observered = get_disk_security_info(disk_name) |
726 | + self.assertThat( |
727 | + mock_check_output, |
728 | + MockCallsMatch( |
729 | + call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]), |
730 | + call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]), |
731 | + ), |
732 | + ) |
733 | + self.assertEqual( |
734 | + { |
735 | + "format_supported": True, |
736 | + "writez_supported": True, |
737 | + "crypto_format": True, |
738 | + "nsze": 0, |
739 | + "lbaf": 0, |
740 | + "ms": 0, |
741 | + }, |
742 | + observered, |
743 | + ) |
744 | + |
745 | + def test_get_disk_security_info_nocrypt_format_writez_nvme(self): |
746 | + nvme_cli_output = ( |
747 | + NVME_IDCTRL_PROLOGUE |
748 | + + NVME_IDCTRL_OACS_FORMAT_SUPPORTED |
749 | + + NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED |
750 | + + NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED |
751 | + + NVME_IDCTRL_EPILOGUE |
752 | + ) |
753 | + mock_check_output = self.patch(subprocess, "check_output") |
754 | + mock_check_output.return_value = nvme_cli_output |
755 | + disk_name = factory.make_name("nvme").encode("ascii") |
756 | + observered = get_disk_security_info(disk_name) |
757 | + self.assertThat( |
758 | + mock_check_output, |
759 | + MockCallsMatch( |
760 | + call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]), |
761 | + call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]), |
762 | + ), |
763 | + ) |
764 | + self.assertEqual( |
765 | + { |
766 | + "format_supported": True, |
767 | + "writez_supported": True, |
768 | + "crypto_format": False, |
769 | + "nsze": 0, |
770 | + "lbaf": 0, |
771 | + "ms": 0, |
772 | + }, |
773 | + observered, |
774 | + ) |
775 | + |
776 | + def test_get_disk_security_info_crypt_format_nowritez_nvme(self): |
777 | + nvme_cli_output = ( |
778 | + NVME_IDCTRL_PROLOGUE |
779 | + + NVME_IDCTRL_OACS_FORMAT_SUPPORTED |
780 | + + NVME_IDCTRL_ONCS_WRITEZ_UNSUPPORTED |
781 | + + NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED |
782 | + + NVME_IDCTRL_EPILOGUE |
783 | + ) |
784 | + mock_check_output = self.patch(subprocess, "check_output") |
785 | + mock_check_output.return_value = nvme_cli_output |
786 | + disk_name = factory.make_name("nvme").encode("ascii") |
787 | + observered = get_disk_security_info(disk_name) |
788 | + self.assertThat( |
789 | + mock_check_output, |
790 | + MockCallsMatch( |
791 | + call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]), |
792 | + call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]), |
793 | + ), |
794 | + ) |
795 | + self.assertEqual( |
796 | + { |
797 | + "format_supported": True, |
798 | + "writez_supported": False, |
799 | + "crypto_format": True, |
800 | + "nsze": 0, |
801 | + "lbaf": 0, |
802 | + "ms": 0, |
803 | + }, |
804 | + observered, |
805 | + ) |
806 | + |
807 | + def test_get_disk_security_info_noformat_writez_nvme(self): |
808 | + nvme_cli_output = ( |
809 | + NVME_IDCTRL_PROLOGUE |
810 | + + NVME_IDCTRL_OACS_FORMAT_UNSUPPORTED |
811 | + + NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED |
812 | + + NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED |
813 | + + NVME_IDCTRL_EPILOGUE |
814 | + ) |
815 | + mock_check_output = self.patch(subprocess, "check_output") |
816 | + mock_check_output.return_value = nvme_cli_output |
817 | + disk_name = factory.make_name("nvme").encode("ascii") |
818 | + observered = get_disk_security_info(disk_name) |
819 | + self.assertThat( |
820 | + mock_check_output, |
821 | + MockCallsMatch( |
822 | + call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]), |
823 | + call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]), |
824 | + ), |
825 | + ) |
826 | + self.assertEqual( |
827 | + { |
828 | + "format_supported": False, |
829 | + "writez_supported": True, |
830 | + "crypto_format": True, |
831 | + "nsze": 0, |
832 | + "lbaf": 0, |
833 | + "ms": 0, |
834 | + }, |
835 | + observered, |
836 | + ) |
837 | + |
838 | + def test_get_disk_security_info_noformat_nowritez_nvme(self): |
839 | + nvme_cli_output = ( |
840 | + NVME_IDCTRL_PROLOGUE |
841 | + + NVME_IDCTRL_OACS_FORMAT_UNSUPPORTED |
842 | + + NVME_IDCTRL_ONCS_WRITEZ_UNSUPPORTED |
843 | + + NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED |
844 | + + NVME_IDCTRL_EPILOGUE |
845 | + ) |
846 | + mock_check_output = self.patch(subprocess, "check_output") |
847 | + mock_check_output.return_value = nvme_cli_output |
848 | + disk_name = factory.make_name("nvme").encode("ascii") |
849 | + observered = get_disk_security_info(disk_name) |
850 | + self.assertThat( |
851 | + mock_check_output, |
852 | + MockCallsMatch( |
853 | + call(["nvme", "id-ctrl", maas_wipe.DEV_PATH % disk_name]), |
854 | + call(["nvme", "id-ns", maas_wipe.DEV_PATH % disk_name]), |
855 | + ), |
856 | + ) |
857 | + self.assertEqual( |
858 | + { |
859 | + "format_supported": False, |
860 | + "writez_supported": False, |
861 | + "crypto_format": False, |
862 | + "nsze": 0, |
863 | + "lbaf": 0, |
864 | + "ms": 0, |
865 | + }, |
866 | + observered, |
867 | + ) |
868 | + |
869 | + def test_get_disk_security_info_failed_cmd_nvme(self): |
870 | + mock_check_output = self.patch(subprocess, "check_output") |
871 | + mock_check_output.side_effect = subprocess.CalledProcessError( |
872 | + 1, "nvme id-ctrl" |
873 | + ) |
874 | + disk_name = factory.make_name("nvme").encode("ascii") |
875 | + observered = get_disk_security_info(disk_name) |
876 | + |
877 | + self.assertThat( |
878 | + self.print_flush, |
879 | + MockCalledOnceWith("Error on nvme id-ctrl (%s)" % "1"), |
880 | + ) |
881 | + self.assertEqual( |
882 | + { |
883 | + "format_supported": False, |
884 | + "writez_supported": False, |
885 | + "crypto_format": False, |
886 | + "nsze": 0, |
887 | + "lbaf": 0, |
888 | + "ms": 0, |
889 | + }, |
890 | + observered, |
891 | + ) |
892 | + |
893 | + def test_get_disk_security_info_failed_os_nvme(self): |
894 | + mock_check_output = self.patch(subprocess, "check_output") |
895 | + mock_check_output.side_effect = OSError( |
896 | + -2, "No such file or directory" |
897 | + ) |
898 | + disk_name = factory.make_name("nvme").encode("ascii") |
899 | + observered = get_disk_security_info(disk_name) |
900 | + |
901 | + self.assertThat( |
902 | + self.print_flush, |
903 | + MockCalledOnceWith( |
904 | + "OS error when running nvme-cli (No such file or directory)" |
905 | + ), |
906 | + ) |
907 | + self.assertEqual( |
908 | + { |
909 | + "format_supported": False, |
910 | + "writez_supported": False, |
911 | + "crypto_format": False, |
912 | + "nsze": 0, |
913 | + "lbaf": 0, |
914 | + "ms": 0, |
915 | + }, |
916 | + observered, |
917 | + ) |
918 | + |
919 | + def test_get_disk_info_nvme(self): |
920 | + disk_names = [ |
921 | + factory.make_name("nvme").encode("ascii") for _ in range(3) |
922 | + ] |
923 | + self.patch(maas_wipe, "list_disks").return_value = disk_names |
924 | + security_info = [ |
925 | + { |
926 | + "format_supported": True, |
927 | + "writez_supported": True, |
928 | + "crypto_format": True, |
929 | + "nsze": 0, |
930 | + "lbaf": 0, |
931 | + "ms": 0, |
932 | + } |
933 | + for _ in range(3) |
934 | + ] |
935 | + self.patch( |
936 | + maas_wipe, "get_nvme_security_info" |
937 | ).side_effect = security_info |
938 | observed = get_disk_info() |
939 | self.assertEqual( |
940 | {disk_names[i]: security_info[i] for i in range(3)}, observed |
941 | ) |
942 | |
943 | - def test_try_secure_erase_not_supported(self): |
944 | + def test_try_secure_erase_not_supported_hdparm(self): |
945 | disk_name = factory.make_name("disk").encode("ascii") |
946 | disk_info = { |
947 | b"supported": False, |
948 | @@ -355,7 +468,7 @@ class TestMAASWipe(MAASTestCase): |
949 | ), |
950 | ) |
951 | |
952 | - def test_try_secure_erase_frozen(self): |
953 | + def test_try_secure_erase_frozen_hdparm(self): |
954 | disk_name = factory.make_name("disk").encode("ascii") |
955 | disk_info = { |
956 | b"supported": True, |
957 | @@ -372,7 +485,7 @@ class TestMAASWipe(MAASTestCase): |
958 | ), |
959 | ) |
960 | |
961 | - def test_try_secure_erase_locked(self): |
962 | + def test_try_secure_erase_locked_hdparm(self): |
963 | disk_name = factory.make_name("disk").encode("ascii") |
964 | disk_info = { |
965 | b"supported": True, |
966 | @@ -389,7 +502,7 @@ class TestMAASWipe(MAASTestCase): |
967 | ), |
968 | ) |
969 | |
970 | - def test_try_secure_erase_enabled(self): |
971 | + def test_try_secure_erase_enabled_hdparm(self): |
972 | disk_name = factory.make_name("disk").encode("ascii") |
973 | disk_info = { |
974 | b"supported": True, |
975 | @@ -406,7 +519,7 @@ class TestMAASWipe(MAASTestCase): |
976 | ), |
977 | ) |
978 | |
979 | - def test_try_secure_erase_failed_erase(self): |
980 | + def test_try_secure_erase_failed_erase_hdparm(self): |
981 | disk_name = factory.make_name("disk").encode("ascii") |
982 | disk_info = { |
983 | b"supported": True, |
984 | @@ -415,7 +528,7 @@ class TestMAASWipe(MAASTestCase): |
985 | b"frozen": False, |
986 | } |
987 | exception = factory.make_exception() |
988 | - self.patch(maas_wipe, "secure_erase").side_effect = exception |
989 | + self.patch(maas_wipe, "secure_erase_hdparm").side_effect = exception |
990 | self.assertFalse(try_secure_erase(disk_name, disk_info)) |
991 | self.assertThat( |
992 | self.print_flush, |
993 | @@ -425,7 +538,7 @@ class TestMAASWipe(MAASTestCase): |
994 | ), |
995 | ) |
996 | |
997 | - def test_try_secure_erase_successful_erase(self): |
998 | + def test_try_secure_erase_successful_erase_hdparm(self): |
999 | disk_name = factory.make_name("disk").encode("ascii") |
1000 | disk_info = { |
1001 | b"supported": True, |
1002 | @@ -433,7 +546,7 @@ class TestMAASWipe(MAASTestCase): |
1003 | b"locked": False, |
1004 | b"frozen": False, |
1005 | } |
1006 | - self.patch(maas_wipe, "secure_erase") |
1007 | + self.patch(maas_wipe, "secure_erase_hdparm") |
1008 | self.assertTrue(try_secure_erase(disk_name, disk_info)) |
1009 | self.assertThat( |
1010 | self.print_flush, |
1011 | @@ -443,7 +556,223 @@ class TestMAASWipe(MAASTestCase): |
1012 | ), |
1013 | ) |
1014 | |
1015 | - def test_secure_erase_writes_known_data(self): |
1016 | + def test_try_secure_erase_not_supported_nvme(self): |
1017 | + disk_name = factory.make_name("nvme").encode("ascii") |
1018 | + sec_info = { |
1019 | + "format_supported": False, |
1020 | + "writez_supported": True, |
1021 | + "crypto_format": True, |
1022 | + "nsze": 0, |
1023 | + "lbaf": 0, |
1024 | + "ms": 0, |
1025 | + } |
1026 | + self.assertFalse(try_secure_erase(disk_name, sec_info)) |
1027 | + self.assertThat( |
1028 | + self.print_flush, |
1029 | + MockCalledOnceWith( |
1030 | + "Device %s does not support formatting" |
1031 | + % disk_name.decode("ascii") |
1032 | + ), |
1033 | + ) |
1034 | + |
1035 | + def test_try_secure_erase_successful_cryto_nvme(self): |
1036 | + disk_name = factory.make_name("nvme").encode("ascii") |
1037 | + sec_info = { |
1038 | + "format_supported": True, |
1039 | + "writez_supported": True, |
1040 | + "crypto_format": True, |
1041 | + "nsze": 0, |
1042 | + "lbaf": 0, |
1043 | + "ms": 0, |
1044 | + } |
1045 | + mock_check_output = self.patch(subprocess, "check_output") |
1046 | + self.assertTrue(try_secure_erase(disk_name, sec_info)) |
1047 | + self.assertThat( |
1048 | + mock_check_output, |
1049 | + MockCalledOnceWith( |
1050 | + [ |
1051 | + "nvme", |
1052 | + "format", |
1053 | + "-s", |
1054 | + "2", |
1055 | + "-l", |
1056 | + "0", |
1057 | + "-m", |
1058 | + "0", |
1059 | + maas_wipe.DEV_PATH % disk_name, |
1060 | + ] |
1061 | + ), |
1062 | + ) |
1063 | + self.assertThat( |
1064 | + self.print_flush, |
1065 | + MockCalledOnceWith( |
1066 | + "Secure erase was successful on NVMe drive %s" |
1067 | + % disk_name.decode("ascii") |
1068 | + ), |
1069 | + ) |
1070 | + |
1071 | + def test_try_secure_erase_successful_nocryto_nvme(self): |
1072 | + disk_name = factory.make_name("nvme").encode("ascii") |
1073 | + sec_info = { |
1074 | + "format_supported": True, |
1075 | + "writez_supported": True, |
1076 | + "crypto_format": False, |
1077 | + "nsze": 0, |
1078 | + "lbaf": 0, |
1079 | + "ms": 0, |
1080 | + } |
1081 | + mock_check_output = self.patch(subprocess, "check_output") |
1082 | + self.assertTrue(try_secure_erase(disk_name, sec_info)) |
1083 | + self.assertThat( |
1084 | + mock_check_output, |
1085 | + MockCalledOnceWith( |
1086 | + [ |
1087 | + "nvme", |
1088 | + "format", |
1089 | + "-s", |
1090 | + "1", |
1091 | + "-l", |
1092 | + "0", |
1093 | + "-m", |
1094 | + "0", |
1095 | + maas_wipe.DEV_PATH % disk_name, |
1096 | + ] |
1097 | + ), |
1098 | + ) |
1099 | + self.assertThat( |
1100 | + self.print_flush, |
1101 | + MockCalledOnceWith( |
1102 | + "Secure erase was successful on NVMe drive %s" |
1103 | + % disk_name.decode("ascii") |
1104 | + ), |
1105 | + ) |
1106 | + |
1107 | + def test_try_secure_erase_failed_nvme(self): |
1108 | + disk_name = factory.make_name("nvme").encode("ascii") |
1109 | + sec_info = { |
1110 | + "format_supported": True, |
1111 | + "writez_supported": True, |
1112 | + "crypto_format": True, |
1113 | + "nsze": 0, |
1114 | + "lbaf": 0, |
1115 | + "ms": 0, |
1116 | + } |
1117 | + mock_check_output = self.patch(subprocess, "check_output") |
1118 | + mock_check_output.side_effect = subprocess.CalledProcessError( |
1119 | + 1, "nvme format" |
1120 | + ) |
1121 | + |
1122 | + self.assertFalse(try_secure_erase(disk_name, sec_info)) |
1123 | + self.assertThat( |
1124 | + self.print_flush, |
1125 | + MockCalledOnceWith("Error with format command (%s)" % "1"), |
1126 | + ) |
1127 | + |
1128 | + def test_try_write_zeroes_not_supported_nvme(self): |
1129 | + disk_name = factory.make_name("nvme").encode("ascii") |
1130 | + sec_info = { |
1131 | + "format_supported": False, |
1132 | + "writez_supported": False, |
1133 | + "crypto_format": False, |
1134 | + "nsze": 1, |
1135 | + "lbaf": 0, |
1136 | + "ms": 0, |
1137 | + } |
1138 | + mock_print = self.patch(builtins, "print") |
1139 | + self.assertFalse(nvme_write_zeroes(disk_name, sec_info)) |
1140 | + self.assertThat( |
1141 | + mock_print, |
1142 | + MockCalledOnceWith( |
1143 | + "NVMe drive %s does not support write-zeroes" |
1144 | + % disk_name.decode("ascii") |
1145 | + ), |
1146 | + ) |
1147 | + self.assertThat( |
1148 | + self.print_flush, |
1149 | + MockCalledOnceWith("Will fallback to regular drive zeroing."), |
1150 | + ) |
1151 | + |
1152 | + def test_try_write_zeroes_supported_invalid_nsze_nvme(self): |
1153 | + disk_name = factory.make_name("nvme").encode("ascii") |
1154 | + sec_info = { |
1155 | + "format_supported": False, |
1156 | + "writez_supported": True, |
1157 | + "crypto_format": False, |
1158 | + "nsze": 0, |
1159 | + "lbaf": 0, |
1160 | + "ms": 0, |
1161 | + } |
1162 | + mock_print = self.patch(builtins, "print") |
1163 | + self.assertFalse(nvme_write_zeroes(disk_name, sec_info)) |
1164 | + self.assertThat( |
1165 | + mock_print, |
1166 | + MockCalledOnceWith( |
1167 | + "Bad namespace information collected on NVMe drive %s" |
1168 | + % disk_name.decode("ascii") |
1169 | + ), |
1170 | + ) |
1171 | + self.assertThat( |
1172 | + self.print_flush, |
1173 | + MockCalledOnceWith("Will fallback to regular drive zeroing."), |
1174 | + ) |
1175 | + |
1176 | + def test_try_write_zeroes_successful_nvme(self): |
1177 | + disk_name = factory.make_name("nvme").encode("ascii") |
1178 | + sec_info = { |
1179 | + "format_supported": False, |
1180 | + "writez_supported": True, |
1181 | + "crypto_format": False, |
1182 | + "nsze": 0x100A, |
1183 | + "lbaf": 0, |
1184 | + "ms": 0, |
1185 | + } |
1186 | + mock_check_output = self.patch(subprocess, "check_output") |
1187 | + self.assertTrue(nvme_write_zeroes(disk_name, sec_info)) |
1188 | + self.assertThat( |
1189 | + mock_check_output, |
1190 | + MockCalledOnceWith( |
1191 | + [ |
1192 | + "nvme", |
1193 | + "write-zeroes", |
1194 | + "-f", |
1195 | + "-s", |
1196 | + "0", |
1197 | + "-c", |
1198 | + "100a", |
1199 | + maas_wipe.DEV_PATH % disk_name, |
1200 | + ] |
1201 | + ), |
1202 | + ) |
1203 | + self.assertThat( |
1204 | + self.print_flush, |
1205 | + MockCalledOnceWith( |
1206 | + "%s: successfully zeroed (using write-zeroes)." |
1207 | + % disk_name.decode("ascii") |
1208 | + ), |
1209 | + ) |
1210 | + |
1211 | + def test_try_write_zeroes_failed_nvme(self): |
1212 | + disk_name = factory.make_name("nvme").encode("ascii") |
1213 | + sec_info = { |
1214 | + "format_supported": False, |
1215 | + "writez_supported": True, |
1216 | + "crypto_format": False, |
1217 | + "nsze": 100, |
1218 | + "lbaf": 0, |
1219 | + "ms": 0, |
1220 | + } |
1221 | + mock_check_output = self.patch(subprocess, "check_output") |
1222 | + mock_check_output.side_effect = subprocess.CalledProcessError( |
1223 | + 1, "nvme write-zeroes" |
1224 | + ) |
1225 | + |
1226 | + self.assertFalse(nvme_write_zeroes(disk_name, sec_info)) |
1227 | + self.assertThat( |
1228 | + self.print_flush, |
1229 | + MockCalledOnceWith("Error with write-zeroes command (%s)" % "1"), |
1230 | + ) |
1231 | + |
1232 | + def test_secure_erase_writes_known_data_hdparm(self): |
1233 | tmp_dir = self.make_dir() |
1234 | dev_path = (tmp_dir + "/%s").encode("ascii") |
1235 | self.patch(maas_wipe, "DEV_PATH", dev_path) |
1236 | @@ -455,7 +784,7 @@ class TestMAASWipe(MAASTestCase): |
1237 | mock_check_output = self.patch(subprocess, "check_output") |
1238 | mock_check_output.side_effect = factory.make_exception() |
1239 | |
1240 | - self.assertRaises(WipeError, secure_erase, dev_name) |
1241 | + self.assertRaises(WipeError, secure_erase_hdparm, dev_name) |
1242 | expected_buf = b"M" * 1024 * 1024 |
1243 | with open(file_path, "rb") as fp: |
1244 | read_buf = fp.read(len(expected_buf)) |
1245 | @@ -463,7 +792,7 @@ class TestMAASWipe(MAASTestCase): |
1246 | expected_buf, read_buf, "First 1 MiB of file was not written." |
1247 | ) |
1248 | |
1249 | - def test_secure_erase_sets_security_password(self): |
1250 | + def test_secure_erase_sets_security_password_hdparm(self): |
1251 | tmp_dir = self.make_dir() |
1252 | dev_path = (tmp_dir + "/%s").encode("ascii") |
1253 | self.patch(maas_wipe, "DEV_PATH", dev_path) |
1254 | @@ -476,10 +805,10 @@ class TestMAASWipe(MAASTestCase): |
1255 | # Fail to get disk info just to exit early. |
1256 | exception_type = factory.make_exception_type() |
1257 | self.patch( |
1258 | - maas_wipe, "get_disk_security_info" |
1259 | + maas_wipe, "get_hdparm_security_info" |
1260 | ).side_effect = exception_type() |
1261 | |
1262 | - self.assertRaises(exception_type, secure_erase, dev_name) |
1263 | + self.assertRaises(exception_type, secure_erase_hdparm, dev_name) |
1264 | self.assertThat( |
1265 | mock_check_output, |
1266 | MockCalledOnceWith( |
1267 | @@ -494,7 +823,7 @@ class TestMAASWipe(MAASTestCase): |
1268 | ), |
1269 | ) |
1270 | |
1271 | - def test_secure_erase_fails_if_not_enabled(self): |
1272 | + def test_secure_erase_fails_if_not_enabled_hdparm(self): |
1273 | tmp_dir = self.make_dir() |
1274 | dev_path = (tmp_dir + "/%s").encode("ascii") |
1275 | self.patch(maas_wipe, "DEV_PATH", dev_path) |
1276 | @@ -503,16 +832,16 @@ class TestMAASWipe(MAASTestCase): |
1277 | self.make_empty_file(file_path) |
1278 | |
1279 | self.patch(subprocess, "check_output") |
1280 | - self.patch(maas_wipe, "get_disk_security_info").return_value = { |
1281 | + self.patch(maas_wipe, "get_hdparm_security_info").return_value = { |
1282 | b"enabled": False |
1283 | } |
1284 | |
1285 | - error = self.assertRaises(WipeError, secure_erase, dev_name) |
1286 | + error = self.assertRaises(WipeError, secure_erase_hdparm, dev_name) |
1287 | self.assertEqual( |
1288 | "Failed to enable security to perform secure erase.", str(error) |
1289 | ) |
1290 | |
1291 | - def test_secure_erase_fails_when_still_enabled(self): |
1292 | + def test_secure_erase_fails_when_still_enabled_hdparm(self): |
1293 | tmp_dir = self.make_dir() |
1294 | dev_path = (tmp_dir + "/%s").encode("ascii") |
1295 | self.patch(maas_wipe, "DEV_PATH", dev_path) |
1296 | @@ -521,14 +850,14 @@ class TestMAASWipe(MAASTestCase): |
1297 | self.make_empty_file(file_path) |
1298 | |
1299 | mock_check_output = self.patch(subprocess, "check_output") |
1300 | - self.patch(maas_wipe, "get_disk_security_info").return_value = { |
1301 | + self.patch(maas_wipe, "get_hdparm_security_info").return_value = { |
1302 | b"enabled": True |
1303 | } |
1304 | exception = factory.make_exception() |
1305 | mock_check_call = self.patch(subprocess, "check_call") |
1306 | mock_check_call.side_effect = exception |
1307 | |
1308 | - error = self.assertRaises(WipeError, secure_erase, dev_name) |
1309 | + error = self.assertRaises(WipeError, secure_erase_hdparm, dev_name) |
1310 | self.assertThat( |
1311 | mock_check_call, |
1312 | MockCalledOnceWith( |
1313 | @@ -561,7 +890,7 @@ class TestMAASWipe(MAASTestCase): |
1314 | self.assertEqual("Failed to securely erase.", str(error)) |
1315 | self.assertEqual(exception, error.__cause__) |
1316 | |
1317 | - def test_secure_erase_fails_when_buffer_not_different(self): |
1318 | + def test_secure_erase_fails_when_buffer_not_different_hdparm(self): |
1319 | tmp_dir = self.make_dir() |
1320 | dev_path = (tmp_dir + "/%s").encode("ascii") |
1321 | self.patch(maas_wipe, "DEV_PATH", dev_path) |
1322 | @@ -570,13 +899,13 @@ class TestMAASWipe(MAASTestCase): |
1323 | self.make_empty_file(file_path) |
1324 | |
1325 | self.patch(subprocess, "check_output") |
1326 | - self.patch(maas_wipe, "get_disk_security_info").side_effect = [ |
1327 | + self.patch(maas_wipe, "get_hdparm_security_info").side_effect = [ |
1328 | {b"enabled": True}, |
1329 | {b"enabled": False}, |
1330 | ] |
1331 | mock_check_call = self.patch(subprocess, "check_call") |
1332 | |
1333 | - error = self.assertRaises(WipeError, secure_erase, dev_name) |
1334 | + error = self.assertRaises(WipeError, secure_erase_hdparm, dev_name) |
1335 | self.assertThat( |
1336 | mock_check_call, |
1337 | MockCalledOnceWith( |
1338 | @@ -595,7 +924,7 @@ class TestMAASWipe(MAASTestCase): |
1339 | str(error), |
1340 | ) |
1341 | |
1342 | - def test_secure_erase_fails_success(self): |
1343 | + def test_secure_erase_fails_success_hdparm(self): |
1344 | tmp_dir = self.make_dir() |
1345 | dev_path = (tmp_dir + "/%s").encode("ascii") |
1346 | self.patch(maas_wipe, "DEV_PATH", dev_path) |
1347 | @@ -604,7 +933,7 @@ class TestMAASWipe(MAASTestCase): |
1348 | self.make_empty_file(file_path) |
1349 | |
1350 | self.patch(subprocess, "check_output") |
1351 | - self.patch(maas_wipe, "get_disk_security_info").side_effect = [ |
1352 | + self.patch(maas_wipe, "get_hdparm_security_info").side_effect = [ |
1353 | {b"enabled": True}, |
1354 | {b"enabled": False}, |
1355 | ] |
1356 | @@ -620,9 +949,9 @@ class TestMAASWipe(MAASTestCase): |
1357 | mock_check_call.side_effect = wipe_buffer |
1358 | |
1359 | # No error should be raised. |
1360 | - secure_erase(dev_name) |
1361 | + secure_erase_hdparm(dev_name) |
1362 | |
1363 | - def test_wipe_quickly(self): |
1364 | + def test_wipe_quickly_successful(self): |
1365 | tmp_dir = self.make_dir() |
1366 | dev_path = (tmp_dir + "/%s").encode("ascii") |
1367 | self.patch(maas_wipe, "DEV_PATH", dev_path) |
1368 | @@ -630,7 +959,14 @@ class TestMAASWipe(MAASTestCase): |
1369 | file_path = dev_path % dev_name |
1370 | self.make_empty_file(file_path, content=b"T") |
1371 | |
1372 | + mock_check_output = self.patch(subprocess, "check_output") |
1373 | wipe_quickly(dev_name) |
1374 | + self.assertThat( |
1375 | + mock_check_output, |
1376 | + MockCalledOnceWith( |
1377 | + ["wipefs", "-f", "-a", maas_wipe.DEV_PATH % dev_name] |
1378 | + ), |
1379 | + ) |
1380 | |
1381 | buf_size = 1024 * 1024 |
1382 | with open(file_path, "rb") as fp: |
1383 | @@ -641,8 +977,18 @@ class TestMAASWipe(MAASTestCase): |
1384 | zero_buf = b"\0" * 1024 * 1024 |
1385 | self.assertEqual(zero_buf, first_buf, "First 1 MiB was not wiped.") |
1386 | self.assertEqual(zero_buf, last_buf, "Last 1 MiB was not wiped.") |
1387 | + self.assertThat( |
1388 | + self.print_flush, |
1389 | + MockCallsMatch( |
1390 | + call("%s: starting quick wipe." % dev_name.decode("ascii")), |
1391 | + call( |
1392 | + "%s: successfully quickly wiped." |
1393 | + % dev_name.decode("ascii") |
1394 | + ), |
1395 | + ), |
1396 | + ) |
1397 | |
1398 | - def test_zero_disk(self): |
1399 | + def test_wipe_quickly_successful_but_wipefs_failed(self): |
1400 | tmp_dir = self.make_dir() |
1401 | dev_path = (tmp_dir + "/%s").encode("ascii") |
1402 | self.patch(maas_wipe, "DEV_PATH", dev_path) |
1403 | @@ -650,13 +996,83 @@ class TestMAASWipe(MAASTestCase): |
1404 | file_path = dev_path % dev_name |
1405 | self.make_empty_file(file_path, content=b"T") |
1406 | |
1407 | + mock_check_output = self.patch(subprocess, "check_output") |
1408 | + mock_check_output.side_effect = subprocess.CalledProcessError( |
1409 | + 1, "wipefs" |
1410 | + ) |
1411 | + wipe_quickly(dev_name) |
1412 | + |
1413 | + buf_size = 1024 * 1024 |
1414 | + with open(file_path, "rb") as fp: |
1415 | + first_buf = fp.read(buf_size) |
1416 | + fp.seek(-buf_size, 2) |
1417 | + last_buf = fp.read(buf_size) |
1418 | + |
1419 | + zero_buf = b"\0" * 1024 * 1024 |
1420 | + self.assertEqual(zero_buf, first_buf, "First 1 MiB was not wiped.") |
1421 | + self.assertEqual(zero_buf, last_buf, "Last 1 MiB was not wiped.") |
1422 | + self.assertThat( |
1423 | + self.print_flush, |
1424 | + MockCallsMatch( |
1425 | + call("%s: starting quick wipe." % dev_name.decode("ascii")), |
1426 | + call("%s: wipefs failed (1)" % dev_name.decode("ascii")), |
1427 | + call( |
1428 | + "%s: successfully quickly wiped." |
1429 | + % dev_name.decode("ascii") |
1430 | + ), |
1431 | + ), |
1432 | + ) |
1433 | + |
1434 | + def test_wipe_quickly_failed(self): |
1435 | + dev_name = factory.make_name("disk").encode("ascii") |
1436 | + |
1437 | + mock_check_output = self.patch(subprocess, "check_output") |
1438 | + mock_check_output.side_effect = subprocess.CalledProcessError( |
1439 | + 1, "wipefs" |
1440 | + ) |
1441 | + |
1442 | + mock_os_open = self.patch(builtins, "open") |
1443 | + mock_os_open.side_effect = OSError(-2, "No such file or directory") |
1444 | + |
1445 | + wipe_quickly(dev_name) |
1446 | + |
1447 | + self.assertThat( |
1448 | + self.print_flush, |
1449 | + MockCallsMatch( |
1450 | + call("%s: starting quick wipe." % dev_name.decode("ascii")), |
1451 | + call("%s: wipefs failed (1)" % dev_name.decode("ascii")), |
1452 | + call( |
1453 | + "%s: OS error while wiping beginning/end of disk (No such file or directory)" |
1454 | + % dev_name.decode("ascii") |
1455 | + ), |
1456 | + call( |
1457 | + "%s: failed to be quickly wiped." |
1458 | + % dev_name.decode("ascii") |
1459 | + ), |
1460 | + ), |
1461 | + ) |
1462 | + |
1463 | + def test_zero_disk_hdd(self): |
1464 | + tmp_dir = self.make_dir() |
1465 | + dev_path = (tmp_dir + "/%s").encode("ascii") |
1466 | + self.patch(maas_wipe, "DEV_PATH", dev_path) |
1467 | + dev_name = factory.make_name("disk").encode("ascii") |
1468 | + file_path = dev_path % dev_name |
1469 | + self.make_empty_file(file_path, content=b"T") |
1470 | + disk_info = { |
1471 | + b"supported": True, |
1472 | + b"enabled": False, |
1473 | + b"locked": False, |
1474 | + b"frozen": False, |
1475 | + } |
1476 | + |
1477 | # Add a little size to the file making it not evenly |
1478 | # divisable by 1 MiB. |
1479 | extra_end = 512 |
1480 | with open(file_path, "a+b") as fp: |
1481 | fp.write(b"T" * extra_end) |
1482 | |
1483 | - zero_disk(dev_name) |
1484 | + zero_disk(dev_name, disk_info) |
1485 | |
1486 | zero_buf = b"\0" * 1024 * 1024 |
1487 | with open(file_path, "rb") as fp: |
1488 | @@ -682,7 +1098,7 @@ class TestMAASWipe(MAASTestCase): |
1489 | parser.parse_args.return_value = args |
1490 | self.patch(argparse, "ArgumentParser").return_value = parser |
1491 | |
1492 | - def test_main_calls_try_secure_erase_for_all_disks(self): |
1493 | + def test_main_calls_try_secure_erase_for_all_hdd(self): |
1494 | self.patch_args(True, False) |
1495 | disks = { |
1496 | factory.make_name("disk").encode("ascii"): {} for _ in range(3) |
1497 | @@ -698,7 +1114,7 @@ class TestMAASWipe(MAASTestCase): |
1498 | self.assertThat(mock_try, MockCallsMatch(*calls)) |
1499 | self.assertThat(mock_zero, MockNotCalled()) |
1500 | |
1501 | - def test_main_calls_zero_disk_if_no_secure_erase(self): |
1502 | + def test_main_calls_zero_disk_if_no_secure_erase_hdd(self): |
1503 | self.patch_args(True, False) |
1504 | disks = { |
1505 | factory.make_name("disk").encode("ascii"): {} for _ in range(3) |
1506 | @@ -711,11 +1127,10 @@ class TestMAASWipe(MAASTestCase): |
1507 | maas_wipe.main() |
1508 | |
1509 | try_calls = [call(disk, info) for disk, info in disks.items()] |
1510 | - wipe_calls = [call(disk) for disk in disks.keys()] |
1511 | self.assertThat(mock_try, MockCallsMatch(*try_calls)) |
1512 | - self.assertThat(mock_zero, MockCallsMatch(*wipe_calls)) |
1513 | + self.assertThat(mock_zero, MockCallsMatch(*try_calls)) |
1514 | |
1515 | - def test_main_calls_wipe_quickly_if_no_secure_erase(self): |
1516 | + def test_main_calls_wipe_quickly_if_no_secure_erase_hdd(self): |
1517 | self.patch_args(True, True) |
1518 | disks = { |
1519 | factory.make_name("disk").encode("ascii"): {} for _ in range(3) |
1520 | @@ -760,6 +1175,6 @@ class TestMAASWipe(MAASTestCase): |
1521 | mock_try.return_value = False |
1522 | maas_wipe.main() |
1523 | |
1524 | - wipe_calls = [call(disk) for disk in disks.keys()] |
1525 | + wipe_calls = [call(disk, info) for disk, info in disks.items()] |
1526 | self.assertThat(mock_try, MockNotCalled()) |
1527 | self.assertThat(zero_disk, MockCallsMatch(*wipe_calls)) |
1528 | diff --git a/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe_defs.py b/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe_defs.py |
1529 | new file mode 100644 |
1530 | index 0000000..f9bb440 |
1531 | --- /dev/null |
1532 | +++ b/src/metadataserver/user_data/templates/snippets/tests/test_maas_wipe_defs.py |
1533 | @@ -0,0 +1,232 @@ |
1534 | +#!/usr/bin/python3 |
1535 | +# Copyright 2020 Canonical Ltd. This software is licensed under the |
1536 | +# GNU Affero General Public License version 3 (see the file LICENSE). |
1537 | +# |
1538 | +# hdparm / nvme-cli outputs used on maas_wipe testing |
1539 | + |
1540 | +HDPARM_BEFORE_SECURITY = b"""\ |
1541 | +/dev/sda: |
1542 | + |
1543 | +ATA device, with non-removable media |
1544 | + Model Number: INTEL SSDSC2CT240A4 |
1545 | + Serial Number: CVKI3206029X240DGN |
1546 | + Firmware Revision: 335u |
1547 | + Transport: Serial, ATA8-AST, SATA 1.0a, SATA II Extensions |
1548 | +Standards: |
1549 | + Used: unknown (minor revision code 0xffff) |
1550 | + Supported: 9 8 7 6 5 |
1551 | + Likely used: 9 |
1552 | +Configuration: |
1553 | + Logical max current |
1554 | + cylinders 16383 16383 |
1555 | + heads 16 16 |
1556 | + sectors/track 63 63 |
1557 | + -- |
1558 | + CHS current addressable sectors: 16514064 |
1559 | + LBA user addressable sectors: 268435455 |
1560 | + LBA48 user addressable sectors: 468862128 |
1561 | + Logical Sector size: 512 bytes |
1562 | + Physical Sector size: 512 bytes |
1563 | + Logical Sector-0 offset: 0 bytes |
1564 | + device size with M = 1024*1024: 228936 MBytes |
1565 | + device size with M = 1000*1000: 240057 MBytes (240 GB) |
1566 | + cache/buffer size = unknown |
1567 | + Nominal Media Rotation Rate: Solid State Device |
1568 | +Capabilities: |
1569 | + LBA, IORDY(can be disabled) |
1570 | + Queue depth: 32 |
1571 | + Standby timer values: spec'd by Standard, no device specific minimum |
1572 | + R/W multiple sector transfer: Max = 16 Current = 16 |
1573 | + Advanced power management level: 254 |
1574 | + DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 |
1575 | + Cycle time: min=120ns recommended=120ns |
1576 | + PIO: pio0 pio1 pio2 pio3 pio4 |
1577 | + Cycle time: no flow control=120ns IORDY flow control=120ns |
1578 | +Commands/features: |
1579 | + Enabled Supported: |
1580 | + * SMART feature set |
1581 | + Security Mode feature set |
1582 | + * Power Management feature set |
1583 | + * Write cache |
1584 | + * Look-ahead |
1585 | + * Host Protected Area feature set |
1586 | + * WRITE_BUFFER command |
1587 | + * READ_BUFFER command |
1588 | + * NOP cmd |
1589 | + * DOWNLOAD_MICROCODE |
1590 | + * Advanced Power Management feature set |
1591 | + Power-Up In Standby feature set |
1592 | + * 48-bit Address feature set |
1593 | + * Mandatory FLUSH_CACHE |
1594 | + * FLUSH_CACHE_EXT |
1595 | + * SMART error logging |
1596 | + * SMART self-test |
1597 | + * General Purpose Logging feature set |
1598 | + * WRITE_{DMA|MULTIPLE}_FUA_EXT |
1599 | + * 64-bit World wide name |
1600 | + * IDLE_IMMEDIATE with UNLOAD |
1601 | + * WRITE_UNCORRECTABLE_EXT command |
1602 | + * {READ,WRITE}_DMA_EXT_GPL commands |
1603 | + * Segmented DOWNLOAD_MICROCODE |
1604 | + * Gen1 signaling speed (1.5Gb/s) |
1605 | + * Gen2 signaling speed (3.0Gb/s) |
1606 | + * Gen3 signaling speed (6.0Gb/s) |
1607 | + * Native Command Queueing (NCQ) |
1608 | + * Host-initiated interface power management |
1609 | + * Phy event counters |
1610 | + * DMA Setup Auto-Activate optimization |
1611 | + Device-initiated interface power management |
1612 | + * Software settings preservation |
1613 | + * SMART Command Transport (SCT) feature set |
1614 | + * SCT Data Tables (AC5) |
1615 | + * reserved 69[4] |
1616 | + * Data Set Management TRIM supported (limit 1 block) |
1617 | + * Deterministic read data after TRIM |
1618 | +""" |
1619 | + |
1620 | +HDPARM_AFTER_SECURITY = b"""\ |
1621 | +Logical Unit WWN Device Identifier: 55cd2e40002643cf |
1622 | + NAA : 5 |
1623 | + IEEE OUI : 5cd2e4 |
1624 | + Unique ID : 0002643cf |
1625 | +Checksum: correct |
1626 | +""" |
1627 | + |
1628 | +HDPARM_SECURITY_NOT_SUPPORTED = b"""\ |
1629 | +Security: |
1630 | + Master password revision code = 65534 |
1631 | + not supported |
1632 | + not enabled |
1633 | + not locked |
1634 | + not frozen |
1635 | + not expired: security count |
1636 | + supported: enhanced erase |
1637 | + 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT. |
1638 | +""" |
1639 | + |
1640 | +HDPARM_SECURITY_SUPPORTED_NOT_ENABLED = b"""\ |
1641 | +Security: |
1642 | + Master password revision code = 65534 |
1643 | + supported |
1644 | + not enabled |
1645 | + not locked |
1646 | + not frozen |
1647 | + not expired: security count |
1648 | + supported: enhanced erase |
1649 | + 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT. |
1650 | +""" |
1651 | + |
1652 | +HDPARM_SECURITY_SUPPORTED_ENABLED = b"""\ |
1653 | +Security: |
1654 | + Master password revision code = 65534 |
1655 | + supported |
1656 | + enabled |
1657 | + not locked |
1658 | + not frozen |
1659 | + not expired: security count |
1660 | + supported: enhanced erase |
1661 | + 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT. |
1662 | +""" |
1663 | + |
1664 | +HDPARM_SECURITY_ALL_TRUE = b"""\ |
1665 | +Security: |
1666 | + Master password revision code = 65534 |
1667 | + supported |
1668 | + enabled |
1669 | + locked |
1670 | + frozen |
1671 | + not expired: security count |
1672 | + supported: enhanced erase |
1673 | + 4min for SECURITY ERASE UNIT. 2min for ENHANCED SECURITY ERASE UNIT. |
1674 | +""" |
1675 | + |
1676 | +NVME_IDCTRL_PROLOGUE = b"""\ |
1677 | +NVME Identify Controller: |
1678 | +vid : 0x8086 |
1679 | +ssvid : 0x8086 |
1680 | +sn : CVMD5066002T400AGN |
1681 | +mn : INTEL SSDPEDME400G4 |
1682 | +fr : 8DV10131 |
1683 | +rab : 0 |
1684 | +ieee : 5cd2e4 |
1685 | +cmic : 0 |
1686 | +mdts : 5 |
1687 | +cntlid : 0 |
1688 | +ver : 0 |
1689 | +rtd3r : 0 |
1690 | +rtd3e : 0 |
1691 | +oaes : 0 |
1692 | +ctratt : 0 |
1693 | +acl : 3 |
1694 | +aerl : 3 |
1695 | +frmw : 0x2 |
1696 | +lpa : 0 |
1697 | +elpe : 63 |
1698 | +npss : 0 |
1699 | +avscc : 0 |
1700 | +apsta : 0 |
1701 | +wctemp : 0 |
1702 | +cctemp : 0 |
1703 | +mtfa : 0 |
1704 | +hmpre : 0 |
1705 | +hmmin : 0 |
1706 | +tnvmcap : 0 |
1707 | +unvmcap : 0 |
1708 | +rpmbs : 0 |
1709 | +edstt : 0 |
1710 | +dsto : 0 |
1711 | +fwug : 0 |
1712 | +kas : 0 |
1713 | +hctma : 0 |
1714 | +mntmt : 0 |
1715 | +mxtmt : 0 |
1716 | +sanicap : 0 |
1717 | +hmminds : 0 |
1718 | +hmmaxd : 0 |
1719 | +sqes : 0x66 |
1720 | +cqes : 0x44 |
1721 | +maxcmd : 0 |
1722 | +nn : 1 |
1723 | +fuses : 0 |
1724 | +""" |
1725 | + |
1726 | +NVME_IDCTRL_OACS_FORMAT_SUPPORTED = b"""\ |
1727 | +oacs : 0x6 |
1728 | +""" |
1729 | + |
1730 | +NVME_IDCTRL_OACS_FORMAT_UNSUPPORTED = b"""\ |
1731 | +oacs : 0x4 |
1732 | +""" |
1733 | + |
1734 | +NVME_IDCTRL_ONCS_WRITEZ_SUPPORTED = b"""\ |
1735 | +oncs : 0xe |
1736 | +""" |
1737 | + |
1738 | +NVME_IDCTRL_ONCS_WRITEZ_UNSUPPORTED = b"""\ |
1739 | +oncs : 0x6 |
1740 | +""" |
1741 | + |
1742 | +NVME_IDCTRL_FNA_CRYPTFORMAT_SUPPORTED = b"""\ |
1743 | +fna : 0x7 |
1744 | +""" |
1745 | + |
1746 | +NVME_IDCTRL_FNA_CRYPTFORMAT_UNSUPPORTED = b"""\ |
1747 | +fna : 0x3 |
1748 | +""" |
1749 | + |
1750 | +NVME_IDCTRL_EPILOGUE = b"""\ |
1751 | +vwc : 0 |
1752 | +awun : 0 |
1753 | +awupf : 0 |
1754 | +nvscc : 0 |
1755 | +acwu : 0 |
1756 | +sgls : 0 |
1757 | +subnqn : |
1758 | +ioccsz : 0 |
1759 | +iorcsz : 0 |
1760 | +icdoff : 0 |
1761 | +ctrattr : 0 |
1762 | +msdbd : 0 |
1763 | +ps 0 : mp:25.00W operational enlat:0 exlat:0 rrt:0 rrl:0 |
1764 | + rwt:0 rwl:0 idle_power:- active_power:- |
1765 | +""" |
UNIT TESTS
-b nvme_secure_erase lp:~gpiccoli/maas/+git/maas into -b master lp:~maas-committers/maas
STATUS: FAILED maas-ci. internal: 8080/job/ maas/job/ branch- tester/ 7873/console caf8b8116c0d66b 9726ba21ac
LOG: http://
COMMIT: d6c02a76af681af