Merge ~raharper/curtin:fix/swap-files-on-btrfs into curtin:master
- Git
- lp:~raharper/curtin
- fix/swap-files-on-btrfs
- Merge into master
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Paride Legovini | ||||
Approved revision: | beccf1f0b604b9f71e018d31416c871e61919624 | ||||
Merge reported by: | Server Team CI bot | ||||
Merged at revision: | not available | ||||
Proposed branch: | ~raharper/curtin:fix/swap-files-on-btrfs | ||||
Merge into: | curtin:master | ||||
Diff against target: |
536 lines (+233/-44) 12 files modified
curtin/__init__.py (+2/-0) curtin/commands/curthooks.py (+2/-1) curtin/commands/swap.py (+4/-1) curtin/distro.py (+17/-5) curtin/swap.py (+79/-5) doc/topics/config.rst (+15/-1) examples/tests/basic.yaml (+4/-0) examples/tests/basic_scsi.yaml (+4/-0) tests/unittests/test_distro.py (+26/-6) tests/unittests/test_feature.py (+3/-0) tests/vmtests/__init__.py (+58/-20) tests/vmtests/test_basic.py (+19/-5) |
||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Server Team CI bot | continuous-integration | Approve | |
Paride Legovini | Approve | ||
Review via email: mp+386117@code.launchpad.net |
Commit message
swaps: handle swapfiles on btrfs
Special care and handling are needed for creating swap files on top
of btrfs filesystems. Curtin will attempt to disable btrfs CoW
on the target file before attempting to fallocate/dd the file.
- Btrfs swapfile requires target kernel 5.0+, older kernels cannot use.
https:/
- Query target fstype to check if we can proceed with swap file
btrfs requires newer kernel, xfs cannot use fallocate
- Update distro.
including linux-image-
- Adjust TestBasic,
- Add the swap partition/file to TestBasic's fstab unittest
- Fix test_swaps_used to use fstab data instead of storage config
since file-based swaps are created via curtin 'swap' config not
storage-config.
- Add swap config key 'force', default is false
- Add curtin feature flag, BTRFS_SWAPFILE
LP: #1884161
Description of the change
Server Team CI bot (server-team-bot) wrote : | # |
Ryan Harper (raharper) wrote : | # |
The Bionic/Xenial tests are failing due to the kernel version; prior to kernel 5.0, brtfs does not implement the required features to support a swapfile
https:/
I'm fixing this branch to detect the target linux version installed and will skip creating a swapfile if not < 5.0
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:8b1ba51091e
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Paride Legovini (paride) wrote : | # |
Just in case it's useful here is the output of the CI run:
Server Team CI bot (server-team-bot) wrote : | # |
FAILED: Continuous integration, rev:4643789b9c0
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Chad Smith (chad.smith) wrote : | # |
Latest test run https:/
Chad Smith (chad.smith) wrote : | # |
and the install.log https:/
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:0c596d8db22
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
Click here to trigger a rebuild:
https:/
Paride Legovini (paride) wrote : | # |
Nice, thanks Ryan! I left some inline comments.
Ryan Harper (raharper) wrote : | # |
Thanks for the review, I'll address your comments.
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:994563e081c
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
Click here to trigger a rebuild:
https:/
Paride Legovini (paride) wrote : | # |
LGTM modulo one too much "they".
Server Team CI bot (server-team-bot) wrote : | # |
PASSED: Continuous integration, rev:beccf1f0b60
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
Click here to trigger a rebuild:
https:/
Paride Legovini (paride) : | # |
Server Team CI bot (server-team-bot) wrote : | # |
Commit message lints:
- Line #7 has 1 too many characters. Line starts with: " https:/
Server Team CI bot (server-team-bot) wrote : | # |
Autolanding: FAILED
More details in the following jenkins job:
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
Server Team CI bot (server-team-bot) : | # |
Preview Diff
1 | diff --git a/curtin/__init__.py b/curtin/__init__.py |
2 | index 2e1a0ed..ae4ce11 100644 |
3 | --- a/curtin/__init__.py |
4 | +++ b/curtin/__init__.py |
5 | @@ -8,6 +8,8 @@ KERNEL_CMDLINE_COPY_TO_INSTALL_SEP = "---" |
6 | # can determine which features are supported. Each entry should have |
7 | # a consistent meaning. |
8 | FEATURES = [ |
9 | + # curtin supports creating swapfiles on btrfs, if possible |
10 | + 'BTRFS_SWAPFILE', |
11 | # curtin can apply centos networking via centos_apply_network_config |
12 | 'CENTOS_APPLY_NETWORK_CONFIG', |
13 | # curtin can configure centos storage devices and boot devices |
14 | diff --git a/curtin/commands/curthooks.py b/curtin/commands/curthooks.py |
15 | index d66afa7..458eb9d 100644 |
16 | --- a/curtin/commands/curthooks.py |
17 | +++ b/curtin/commands/curthooks.py |
18 | @@ -900,6 +900,7 @@ def add_swap(cfg, target, fstab): |
19 | fname = swapcfg.get('filename', None) |
20 | size = swapcfg.get('size', None) |
21 | maxsize = swapcfg.get('maxsize', None) |
22 | + force = swapcfg.get('force', False) |
23 | |
24 | if size: |
25 | size = util.human2bytes(str(size)) |
26 | @@ -907,7 +908,7 @@ def add_swap(cfg, target, fstab): |
27 | maxsize = util.human2bytes(str(maxsize)) |
28 | |
29 | swap.setup_swapfile(target=target, fstab=fstab, swapfile=fname, size=size, |
30 | - maxsize=maxsize) |
31 | + maxsize=maxsize, force=force) |
32 | |
33 | |
34 | def detect_and_handle_multipath(cfg, target, osfamily=DISTROS.debian): |
35 | diff --git a/curtin/commands/swap.py b/curtin/commands/swap.py |
36 | index f2381e6..089cd73 100644 |
37 | --- a/curtin/commands/swap.py |
38 | +++ b/curtin/commands/swap.py |
39 | @@ -40,7 +40,7 @@ def swap_main(args): |
40 | |
41 | swap.setup_swapfile(target=state['target'], fstab=state['fstab'], |
42 | swapfile=args.swapfile, size=size, |
43 | - maxsize=args.maxsize) |
44 | + maxsize=args.maxsize, force=args.force) |
45 | sys.exit(2) |
46 | |
47 | |
48 | @@ -54,6 +54,9 @@ CMD_ARGUMENTS = ( |
49 | 'default is env[TARGET_MOUNT_POINT]'), |
50 | 'action': 'store', 'metavar': 'TARGET', |
51 | 'default': os.environ.get('TARGET_MOUNT_POINT')}), |
52 | + (('-F', '--force'), |
53 | + {'help': 'force creating of swapfile even if it may fail (btrfs,xfs)', |
54 | + 'default': False, 'action': 'store_true'}), |
55 | (('-s', '--size'), |
56 | {'help': 'size of swap file (eg: 1G, 1500M, 1024K, 100000. def: "auto")', |
57 | 'default': None, 'action': 'store'}), |
58 | diff --git a/curtin/distro.py b/curtin/distro.py |
59 | index 43b0c19..e2af24a 100644 |
60 | --- a/curtin/distro.py |
61 | +++ b/curtin/distro.py |
62 | @@ -472,6 +472,7 @@ def parse_dpkg_version(raw, name=None, semx=None): |
63 | as the upstream version. |
64 | |
65 | returns a dictionary with fields: |
66 | + 'epoch' |
67 | 'major' (int), 'minor' (int), 'micro' (int), |
68 | 'semantic_version' (int), |
69 | 'extra' (string), 'raw' (string), 'upstream' (string), |
70 | @@ -484,12 +485,20 @@ def parse_dpkg_version(raw, name=None, semx=None): |
71 | if semx is None: |
72 | semx = (10000, 100, 1) |
73 | |
74 | - if "-" in raw: |
75 | - upstream = raw.rsplit('-', 1)[0] |
76 | + raw_offset = 0 |
77 | + if ':' in raw: |
78 | + epoch, _, upstream = raw.partition(':') |
79 | + raw_offset = len(epoch) + 1 |
80 | else: |
81 | - # this is a native package, package version treated as upstream. |
82 | + epoch = 0 |
83 | upstream = raw |
84 | |
85 | + if "-" in raw[raw_offset:]: |
86 | + upstream = raw[raw_offset:].rsplit('-', 1)[0] |
87 | + else: |
88 | + # this is a native package, package version treated as upstream. |
89 | + upstream = raw[raw_offset:] |
90 | + |
91 | match = re.search(r'[^0-9.]', upstream) |
92 | if match: |
93 | extra = upstream[match.start():] |
94 | @@ -498,8 +507,10 @@ def parse_dpkg_version(raw, name=None, semx=None): |
95 | upstream_base = upstream |
96 | extra = None |
97 | |
98 | - toks = upstream_base.split(".", 2) |
99 | - if len(toks) == 3: |
100 | + toks = upstream_base.split(".", 3) |
101 | + if len(toks) == 4: |
102 | + major, minor, micro, extra = toks |
103 | + elif len(toks) == 3: |
104 | major, minor, micro = toks |
105 | elif len(toks) == 2: |
106 | major, minor, micro = (toks[0], toks[1], 0) |
107 | @@ -507,6 +518,7 @@ def parse_dpkg_version(raw, name=None, semx=None): |
108 | major, minor, micro = (toks[0], 0, 0) |
109 | |
110 | version = { |
111 | + 'epoch': int(epoch), |
112 | 'major': int(major), |
113 | 'minor': int(minor), |
114 | 'micro': int(micro), |
115 | diff --git a/curtin/swap.py b/curtin/swap.py |
116 | index d3f29dc..11e95c4 100644 |
117 | --- a/curtin/swap.py |
118 | +++ b/curtin/swap.py |
119 | @@ -5,6 +5,8 @@ import resource |
120 | |
121 | from .log import LOG |
122 | from . import util |
123 | +from curtin import paths |
124 | +from curtin import distro |
125 | |
126 | |
127 | def suggested_swapsize(memsize=None, maxsize=None, fsys=None): |
128 | @@ -51,7 +53,62 @@ def suggested_swapsize(memsize=None, maxsize=None, fsys=None): |
129 | return maxsize |
130 | |
131 | |
132 | -def setup_swapfile(target, fstab=None, swapfile=None, size=None, maxsize=None): |
133 | +def get_fstype(target, source): |
134 | + target_source = paths.target_path(target, source) |
135 | + try: |
136 | + out, _ = util.subp(['findmnt', '--noheading', '--target', |
137 | + target_source, '-o', 'FSTYPE'], capture=True) |
138 | + except util.ProcessExecutionError as exc: |
139 | + LOG.warning('Failed to query %s fstype, findmnt returned error: %s', |
140 | + target_source, exc) |
141 | + return None |
142 | + |
143 | + if out: |
144 | + """ |
145 | + $ findmnt --noheading --target /btrfs -o FSTYPE |
146 | + btrfs |
147 | + """ |
148 | + return out.splitlines()[-1] |
149 | + |
150 | + return None |
151 | + |
152 | + |
153 | +def get_target_kernel_version(target): |
154 | + pkg_ver = None |
155 | + |
156 | + distro_info = distro.get_distroinfo(target=target) |
157 | + if not distro_info: |
158 | + raise RuntimeError('Failed to determine target distro') |
159 | + osfamily = distro_info.family |
160 | + if osfamily == distro.DISTROS.debian: |
161 | + try: |
162 | + # check in-target version |
163 | + pkg_ver = distro.get_package_version('linux-image-generic', |
164 | + target=target) |
165 | + except Exception as e: |
166 | + LOG.warn( |
167 | + "failed reading linux-image-generic package version, %s", e) |
168 | + return pkg_ver |
169 | + |
170 | + |
171 | +def can_use_swapfile(target, fstype): |
172 | + if fstype is None: |
173 | + raise RuntimeError( |
174 | + 'Unknown target filesystem type, may not support swapfiles') |
175 | + if fstype in ['btrfs', 'xfs']: |
176 | + # check kernel version |
177 | + pkg_ver = get_target_kernel_version(target) |
178 | + if not pkg_ver: |
179 | + raise RuntimeError('Failed to read target kernel version') |
180 | + if fstype == 'btrfs' and pkg_ver['major'] < 5: |
181 | + raise RuntimeError( |
182 | + 'btrfs requiers kernel version 5.0+ to use swapfiles') |
183 | + elif fstype in ['zfs']: |
184 | + raise RuntimeError('ZFS cannot use swapfiles') |
185 | + |
186 | + |
187 | +def setup_swapfile(target, fstab=None, swapfile=None, size=None, maxsize=None, |
188 | + force=False): |
189 | if size is None: |
190 | size = suggested_swapsize(fsys=target, maxsize=maxsize) |
191 | |
192 | @@ -65,6 +122,24 @@ def setup_swapfile(target, fstab=None, swapfile=None, size=None, maxsize=None): |
193 | if not swapfile.startswith("/"): |
194 | swapfile = "/" + swapfile |
195 | |
196 | + # query the directory in which swapfile will reside |
197 | + fstype = get_fstype(target, os.path.dirname(swapfile)) |
198 | + try: |
199 | + can_use_swapfile(target, fstype) |
200 | + except RuntimeError as err: |
201 | + if force: |
202 | + LOG.warning('swapfile may not work: %s', err) |
203 | + else: |
204 | + LOG.debug('Not creating swap: %s', err) |
205 | + return |
206 | + |
207 | + allocate_cmd = 'fallocate -l "${2}M" "$1"' |
208 | + # fallocate uses IOCTLs to allocate space in a filesystem, however it's not |
209 | + # clear (from curtin's POV) that it creates non-sparse files as required by |
210 | + # mkswap so we'll skip fallocate for now and use dd. |
211 | + if fstype in ['btrfs', 'xfs']: |
212 | + allocate_cmd = 'dd if=/dev/zero "of=$1" bs=1M "count=$2"' |
213 | + |
214 | mbsize = str(int(size / (2 ** 20))) |
215 | msg = "creating swap file '%s' of %sMB" % (swapfile, mbsize) |
216 | fpath = os.path.sep.join([target, swapfile]) |
217 | @@ -73,10 +148,9 @@ def setup_swapfile(target, fstab=None, swapfile=None, size=None, maxsize=None): |
218 | with util.LogTimer(LOG.debug, msg): |
219 | util.subp( |
220 | ['sh', '-c', |
221 | - ('rm -f "$1" && umask 0066 && ' |
222 | - '{ fallocate -l "${2}M" "$1" || ' |
223 | - ' dd if=/dev/zero "of=$1" bs=1M "count=$2"; } && ' |
224 | - 'mkswap "$1" || { r=$?; rm -f "$1"; exit $r; }'), |
225 | + ('rm -f "$1" && umask 0066 && truncate -s 0 "$1" && ' |
226 | + '{ chattr +C "$1" || true; } && ') + allocate_cmd + |
227 | + (' && mkswap "$1" || { r=$?; rm -f "$1"; exit $r; }'), |
228 | 'setup_swap', fpath, mbsize]) |
229 | except Exception: |
230 | LOG.warn("failed %s" % msg) |
231 | diff --git a/doc/topics/config.rst b/doc/topics/config.rst |
232 | index 72cd683..7f8396e 100644 |
233 | --- a/doc/topics/config.rst |
234 | +++ b/doc/topics/config.rst |
235 | @@ -752,13 +752,27 @@ Configure the max size of the swapfile, defaults to 8GB |
236 | Configure the exact size of the swapfile. Setting ``size`` to 0 will |
237 | disable swap. |
238 | |
239 | +**force**: *<boolean>* |
240 | + |
241 | +Force the creation of swapfile even if curtin detects it may not work. |
242 | +In some target filesystems, e.g. btrfs, xfs, zfs, the use of a swap file has |
243 | +restrictions. If curtin detects that there may be issues it will refuse |
244 | +to create the swapfile. Users can force creation of a swapfile by passing |
245 | +``force: true``. A forced swapfile may not be used by the target OS and could |
246 | +log cause an error. |
247 | + |
248 | **Example**:: |
249 | |
250 | swap: |
251 | filename: swap.img |
252 | - size: None |
253 | + size: 1GB |
254 | maxsize: 4GB |
255 | |
256 | + swap: |
257 | + filename: btrfs_swapfile.img |
258 | + size: 1GB |
259 | + force: true |
260 | + |
261 | |
262 | system_upgrade |
263 | ~~~~~~~~~~~~~~ |
264 | diff --git a/examples/tests/basic.yaml b/examples/tests/basic.yaml |
265 | index 71730c0..82f5ad1 100644 |
266 | --- a/examples/tests/basic.yaml |
267 | +++ b/examples/tests/basic.yaml |
268 | @@ -1,4 +1,8 @@ |
269 | showtrace: true |
270 | +swap: |
271 | + filename: /btrfs/btrfsswap.img |
272 | + size: 1GB |
273 | + maxsize: 1GB |
274 | storage: |
275 | version: 1 |
276 | config: |
277 | diff --git a/examples/tests/basic_scsi.yaml b/examples/tests/basic_scsi.yaml |
278 | index 51f5236..fd28bbe 100644 |
279 | --- a/examples/tests/basic_scsi.yaml |
280 | +++ b/examples/tests/basic_scsi.yaml |
281 | @@ -1,4 +1,8 @@ |
282 | showtrace: true |
283 | +swap: |
284 | + filename: /btrfs/btrfsswap.img |
285 | + size: 1GB |
286 | + maxsize: 1GB |
287 | storage: |
288 | version: 1 |
289 | config: |
290 | diff --git a/tests/unittests/test_distro.py b/tests/unittests/test_distro.py |
291 | index eb62dd8..23c3fba 100644 |
292 | --- a/tests/unittests/test_distro.py |
293 | +++ b/tests/unittests/test_distro.py |
294 | @@ -65,7 +65,7 @@ class TestParseDpkgVersion(CiTestCase): |
295 | def test_simple_native_package_version(self): |
296 | """dpkg versions must have a -. If not present expect value error.""" |
297 | self.assertEqual( |
298 | - {'major': 2, 'minor': 28, 'micro': 0, 'extra': None, |
299 | + {'epoch': 0, 'major': 2, 'minor': 28, 'micro': 0, 'extra': None, |
300 | 'raw': '2.28', 'upstream': '2.28', 'name': 'germinate', |
301 | 'semantic_version': 22800}, |
302 | distro.parse_dpkg_version('2.28', name='germinate')) |
303 | @@ -73,7 +73,7 @@ class TestParseDpkgVersion(CiTestCase): |
304 | def test_complex_native_package_version(self): |
305 | dver = '1.0.106ubuntu2+really1.0.97ubuntu1' |
306 | self.assertEqual( |
307 | - {'major': 1, 'minor': 0, 'micro': 106, |
308 | + {'epoch': 0, 'major': 1, 'minor': 0, 'micro': 106, |
309 | 'extra': 'ubuntu2+really1.0.97ubuntu1', |
310 | 'raw': dver, 'upstream': dver, 'name': 'debootstrap', |
311 | 'semantic_version': 100106}, |
312 | @@ -82,14 +82,14 @@ class TestParseDpkgVersion(CiTestCase): |
313 | |
314 | def test_simple_valid(self): |
315 | self.assertEqual( |
316 | - {'major': 1, 'minor': 2, 'micro': 3, 'extra': None, |
317 | + {'epoch': 0, 'major': 1, 'minor': 2, 'micro': 3, 'extra': None, |
318 | 'raw': '1.2.3-0', 'upstream': '1.2.3', 'name': 'foo', |
319 | 'semantic_version': 10203}, |
320 | distro.parse_dpkg_version('1.2.3-0', name='foo')) |
321 | |
322 | def test_simple_valid_with_semx(self): |
323 | self.assertEqual( |
324 | - {'major': 1, 'minor': 2, 'micro': 3, 'extra': None, |
325 | + {'epoch': 0, 'major': 1, 'minor': 2, 'micro': 3, 'extra': None, |
326 | 'raw': '1.2.3-0', 'upstream': '1.2.3', |
327 | 'semantic_version': 123}, |
328 | distro.parse_dpkg_version('1.2.3-0', semx=(100, 10, 1))) |
329 | @@ -98,7 +98,8 @@ class TestParseDpkgVersion(CiTestCase): |
330 | """upstream versions may have a hyphen.""" |
331 | cver = '18.2-14-g6d48d265-0ubuntu1' |
332 | self.assertEqual( |
333 | - {'major': 18, 'minor': 2, 'micro': 0, 'extra': '-14-g6d48d265', |
334 | + {'epoch': 0, 'major': 18, 'minor': 2, 'micro': 0, |
335 | + 'extra': '-14-g6d48d265', |
336 | 'raw': cver, 'upstream': '18.2-14-g6d48d265', |
337 | 'name': 'cloud-init', 'semantic_version': 180200}, |
338 | distro.parse_dpkg_version(cver, name='cloud-init')) |
339 | @@ -107,11 +108,30 @@ class TestParseDpkgVersion(CiTestCase): |
340 | """multipath tools has a + in it.""" |
341 | mver = '0.5.0+git1.656f8865-5ubuntu2.5' |
342 | self.assertEqual( |
343 | - {'major': 0, 'minor': 5, 'micro': 0, 'extra': '+git1.656f8865', |
344 | + {'epoch': 0, 'major': 0, 'minor': 5, 'micro': 0, |
345 | + 'extra': '+git1.656f8865', |
346 | 'raw': mver, 'upstream': '0.5.0+git1.656f8865', |
347 | 'semantic_version': 500}, |
348 | distro.parse_dpkg_version(mver)) |
349 | |
350 | + def test_package_with_epoch(self): |
351 | + """xxd has epoch""" |
352 | + mver = '2:8.1.2269-1ubuntu5' |
353 | + self.assertEqual( |
354 | + {'epoch': 2, 'major': 8, 'minor': 1, 'micro': 2269, |
355 | + 'extra': None, 'raw': mver, 'upstream': '8.1.2269', |
356 | + 'semantic_version': 82369}, |
357 | + distro.parse_dpkg_version(mver)) |
358 | + |
359 | + def test_package_with_dot_in_extra(self): |
360 | + """linux-image-generic has multiple dots in extra""" |
361 | + mver = '5.4.0.37.40' |
362 | + self.assertEqual( |
363 | + {'epoch': 0, 'major': 5, 'minor': 4, 'micro': 0, |
364 | + 'extra': '37.40', 'raw': mver, 'upstream': '5.4.0.37.40', |
365 | + 'semantic_version': 50400}, |
366 | + distro.parse_dpkg_version(mver)) |
367 | + |
368 | |
369 | class TestDistros(CiTestCase): |
370 | |
371 | diff --git a/tests/unittests/test_feature.py b/tests/unittests/test_feature.py |
372 | index 7c55882..84325ef 100644 |
373 | --- a/tests/unittests/test_feature.py |
374 | +++ b/tests/unittests/test_feature.py |
375 | @@ -24,4 +24,7 @@ class TestExportsFeatures(CiTestCase): |
376 | def test_has_centos_curthook_support(self): |
377 | self.assertIn('CENTOS_CURTHOOK_SUPPORT', curtin.FEATURES) |
378 | |
379 | + def test_has_btrfs_swapfile_support(self): |
380 | + self.assertIn('BTRFS_SWAPFILE', curtin.FEATURES) |
381 | + |
382 | # vi: ts=4 expandtab syntax=python |
383 | diff --git a/tests/vmtests/__init__.py b/tests/vmtests/__init__.py |
384 | index 8ffb7cb..1fc3650 100644 |
385 | --- a/tests/vmtests/__init__.py |
386 | +++ b/tests/vmtests/__init__.py |
387 | @@ -1676,8 +1676,8 @@ class VMBaseClass(TestCase): |
388 | if spec in line: |
389 | fstab_entry = line |
390 | self.assertIsNotNone(fstab_entry) |
391 | - self.assertEqual(mp, fstab_entry.split(' ')[1]) |
392 | - self.assertEqual(fsopts, fstab_entry.split(' ')[3]) |
393 | + self.assertEqual(mp, fstab_entry.split()[1]) |
394 | + self.assertEqual(fsopts, fstab_entry.split()[3]) |
395 | found.append((spec, mp, fsopts)) |
396 | |
397 | self.assertEqual(sorted(expected), sorted(found)) |
398 | @@ -1810,6 +1810,36 @@ class VMBaseClass(TestCase): |
399 | self.assertEqual(len(uuid), 36) |
400 | return uuid |
401 | |
402 | + def _byuuid_to_kname(self, devpath): |
403 | + # lookup kname via /dev/disk/by-uuid symlink |
404 | + # parsing ls -al output on /dev/disk/by-uuid: |
405 | + # lrwxrwxrwx 1 root root 9 Dec 4 20:02 |
406 | + # d591e9e9-825a-4f0a-b280-3bfaf470b83c -> ../../vdg |
407 | + uuid = os.path.basename(devpath) |
408 | + self.assertIsNotNone(uuid) |
409 | + print(uuid) |
410 | + ls_uuid = self.load_collect_file("ls_al_byuuid") |
411 | + kname = [line.split()[-1] for line in ls_uuid.split('\n') |
412 | + if uuid in line.split()] |
413 | + self.assertEqual(len(kname), 1) |
414 | + kname = os.path.basename(kname.pop()) |
415 | + return kname |
416 | + |
417 | + def _bypath_to_kname(self, devpath): |
418 | + # lookup kname via /dev/disk/by-path symlink |
419 | + # parsing ls -al output on /dev/disk/by-path: |
420 | + # lrwxrwxrwx 1 root root 9 Dec 4 20:02 |
421 | + # pci-0000:00:03.0-scsi-0:0:0:0-part3 -> ../../sda3 |
422 | + dpath = os.path.basename(devpath) |
423 | + self.assertIsNotNone(dpath) |
424 | + print(dpath) |
425 | + ls_bypath = self.load_collect_file("ls_al_bypath") |
426 | + kname = [line.split()[-1] for line in ls_bypath.split('\n') |
427 | + if dpath in line.split()] |
428 | + self.assertEqual(len(kname), 1) |
429 | + kname = os.path.basename(kname.pop()) |
430 | + return kname |
431 | + |
432 | def _bcache_to_byuuid(self, kname): |
433 | # extract bcache uuid from /dev/bcache/by-uuid on /dev/<kname> |
434 | # parsing ls -al output on /dev/bcache/by-uuid |
435 | @@ -1991,25 +2021,33 @@ class VMBaseClass(TestCase): |
436 | |
437 | @skip_if_flag('expected_failure') |
438 | def test_swaps_used(self): |
439 | - if not self.has_storage_config(): |
440 | - raise SkipTest("This test does not use storage config.") |
441 | |
442 | - stgcfg = self.get_storage_config() |
443 | - swap_ids = [d["id"] for d in stgcfg if d.get("fstype") == "swap"] |
444 | - swap_mounts = [d for d in stgcfg if d.get("device") in swap_ids] |
445 | - self.assertEqual(len(swap_ids), len(swap_mounts), |
446 | - "number config swap fstypes != number swap mounts") |
447 | - |
448 | - swaps_found = [] |
449 | - for line in self.load_collect_file("proc-swaps").splitlines(): |
450 | - fname, ttype, size, used, priority = line.split() |
451 | - if ttype == "partition": |
452 | - swaps_found.append( |
453 | - {"fname": fname, ttype: "ttype", "size": int(size), |
454 | - "used": int(used), "priority": int(priority)}) |
455 | - self.assertEqual( |
456 | - len(swap_mounts), len(swaps_found), |
457 | - "Number swaps configured != number used") |
458 | + def find_fstab_swaps(): |
459 | + swaps = [] |
460 | + path = self.collect_path("fstab") |
461 | + if not os.path.exists(path): |
462 | + return swaps |
463 | + for line in util.load_file(path).splitlines(): |
464 | + if line.startswith("#"): |
465 | + continue |
466 | + (fs, mp, fstype, opts, dump, passno) = line.split() |
467 | + if fstype == 'swap': |
468 | + if fs.startswith('/dev/disk/by-uuid'): |
469 | + swaps.append('/dev/' + self._byuuid_to_kname(fs)) |
470 | + elif fs.startswith('/dev/disk/by-id'): |
471 | + kname = self._serial_to_kname(os.path.basename(fs)) |
472 | + swaps.append('/dev/' + kname) |
473 | + elif fs.startswith('/dev/disk/by-path'): |
474 | + swaps.append('/dev/' + self._bypath_to_kname(fs)) |
475 | + else: |
476 | + swaps.append(fs) |
477 | + |
478 | + return swaps |
479 | + |
480 | + expected_swaps = find_fstab_swaps() |
481 | + proc_swaps = self.load_collect_file("proc-swaps") |
482 | + for swap in expected_swaps: |
483 | + self.assertIn(swap, proc_swaps) |
484 | |
485 | |
486 | class PsuedoVMBaseClass(VMBaseClass): |
487 | diff --git a/tests/vmtests/test_basic.py b/tests/vmtests/test_basic.py |
488 | index 88b9897..4a0f427 100644 |
489 | --- a/tests/vmtests/test_basic.py |
490 | +++ b/tests/vmtests/test_basic.py |
491 | @@ -143,11 +143,16 @@ class TestBasicAbs(VMBaseClass): |
492 | def get_fstab_expected(self): |
493 | rootdev = self._serial_to_kname('disk-a') |
494 | btrfsdev = self._serial_to_kname('disk-c') |
495 | - return [ |
496 | + expected = [ |
497 | (self._kname_to_byuuid(rootdev + '1'), '/', 'defaults'), |
498 | (self._kname_to_byuuid(rootdev + '2'), '/home', 'defaults'), |
499 | - (self._kname_to_byuuid(btrfsdev), '/btrfs', 'defaults,noatime') |
500 | + (self._kname_to_byuuid(btrfsdev), '/btrfs', 'defaults,noatime'), |
501 | + (self._kname_to_byuuid(rootdev + '3'), 'none', 'sw'), |
502 | ] |
503 | + if self.target_release in ['focal']: |
504 | + expected.append(('/btrfs/btrfsswap.img', 'none', 'sw')) |
505 | + |
506 | + return expected |
507 | |
508 | def test_whole_disk_uuid(self): |
509 | self._test_whole_disk_uuid( |
510 | @@ -307,14 +312,23 @@ class TestBasicScsiAbs(TestBasicAbs): |
511 | home_kname = ( |
512 | self._serial_to_kname('0x39cc071e72c64cc4-part2')) |
513 | btrfs_kname = self._serial_to_kname('0x22dc58dc023c7008') |
514 | + swap_kname = ( |
515 | + self._serial_to_kname('0x39cc071e72c64cc4-part3')) |
516 | |
517 | map_func = self._kname_to_byuuid |
518 | if self.arch == 's390x': |
519 | map_func = self._kname_to_bypath |
520 | |
521 | - return [(map_func(root_kname), '/', 'defaults'), |
522 | - (map_func(home_kname), '/home', 'defaults'), |
523 | - (map_func(btrfs_kname), '/btrfs', 'defaults,noatime')] |
524 | + expected = [ |
525 | + (map_func(root_kname), '/', 'defaults'), |
526 | + (map_func(home_kname), '/home', 'defaults'), |
527 | + (map_func(btrfs_kname), '/btrfs', 'defaults,noatime'), |
528 | + (map_func(swap_kname), 'none', 'sw')] |
529 | + |
530 | + if self.target_release in ['focal']: |
531 | + expected.append(('/btrfs/btrfsswap.img', 'none', 'sw')) |
532 | + |
533 | + return expected |
534 | |
535 | @skip_if_arch('s390x') |
536 | def test_whole_disk_uuid(self): |
FAILED: Continuous integration, rev:e0e89ae1240 db5c2da577f9f9e 02db22401f2c70 /jenkins. ubuntu. com/server/ job/curtin- ci/151/ /jenkins. ubuntu. com/server/ job/curtin- ci/nodes= metal-amd64/ 151/ /jenkins. ubuntu. com/server/ job/curtin- ci/nodes= metal-arm64/ 151/ /jenkins. ubuntu. com/server/ job/curtin- ci/nodes= metal-ppc64el/ 151/ /jenkins. ubuntu. com/server/ job/curtin- ci/nodes= metal-s390x/ 151/
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild: /jenkins. ubuntu. com/server/ job/curtin- ci/151/ /rebuild
https:/