Merge ~chad.smith/cloud-init:ubuntu/devel into cloud-init:ubuntu/devel
- Git
- lp:~chad.smith/cloud-init
- ubuntu/devel
- Merge into ubuntu/devel
Proposed by
Chad Smith
Status: | Merged | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Merged at revision: | 10ab9a7c231b6f7d01da3a3a4c08cf4db01f5a87 | ||||||||||||
Proposed branch: | ~chad.smith/cloud-init:ubuntu/devel | ||||||||||||
Merge into: | cloud-init:ubuntu/devel | ||||||||||||
Diff against target: |
574 lines (+317/-15) 16 files modified
ChangeLog (+58/-0) cloudinit/config/cc_puppet.py (+3/-0) cloudinit/config/cc_resizefs.py (+22/-0) cloudinit/settings.py (+1/-0) cloudinit/sources/DataSourceHetzner.py (+6/-0) cloudinit/util.py (+35/-9) cloudinit/version.py (+1/-1) debian/changelog (+11/-2) packages/debian/control.in (+2/-1) tests/data/mount_parse_ext.txt (+19/-0) tests/data/mount_parse_zfs.txt (+21/-0) tests/data/zpool_status_simple.txt (+10/-0) tests/unittests/test_datasource/test_common.py (+2/-0) tests/unittests/test_datasource/test_hetzner.py (+19/-1) tests/unittests/test_handler/test_handler_resizefs.py (+57/-1) tests/unittests/test_util.py (+50/-0) |
||||||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Server Team CI bot | continuous-integration | Approve | |
Scott Moser | Pending | ||
Review via email: mp+342239@code.launchpad.net |
Commit message
Sync upstream 18.2 release to Bionic for publish.
Description of the change
To post a comment you must log in.
Revision history for this message
Server Team CI bot (server-team-bot) wrote : | # |
review:
Approve
(continuous-integration)
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | diff --git a/ChangeLog b/ChangeLog | |||
2 | index be4c357..daa7ccf 100644 | |||
3 | --- a/ChangeLog | |||
4 | +++ b/ChangeLog | |||
5 | @@ -1,3 +1,61 @@ | |||
6 | 1 | 18.2: | ||
7 | 2 | - Hetzner: Exit early if dmi system-manufacturer is not Hetzner. | ||
8 | 3 | - Add missing dependency on isc-dhcp-client to trunk ubuntu packaging. | ||
9 | 4 | (LP: #1759307) | ||
10 | 5 | - FreeBSD: resizefs module now able to handle zfs/zpool. | ||
11 | 6 | [Dominic Schlegel] (LP: #1721243) | ||
12 | 7 | - cc_puppet: Revert regression of puppet creating ssl and ssl_cert dirs | ||
13 | 8 | - Enable IBMCloud datasource in settings.py. | ||
14 | 9 | - IBMCloud: Initial IBM Cloud datasource. | ||
15 | 10 | - tests: remove jsonschema from xenial tox environment. | ||
16 | 11 | - tests: Fix newly added schema unit tests to skip if no jsonschema. | ||
17 | 12 | - ec2: Adjust ec2 datasource after exception_cb change. | ||
18 | 13 | - Reduce AzurePreprovisioning HTTP timeouts. | ||
19 | 14 | [Douglas Jordan] (LP: #1752977) | ||
20 | 15 | - Revert the logic of exception_cb in read_url. | ||
21 | 16 | [Kurt Garloff] (LP: #1702160, #1298921) | ||
22 | 17 | - ubuntu-advantage: Add new config module to support | ||
23 | 18 | ubuntu-advantage-tools | ||
24 | 19 | - Handle global dns entries in netplan (LP: #1750884) | ||
25 | 20 | - Identify OpenTelekomCloud Xen as OpenStack DS. | ||
26 | 21 | [Kurt Garloff] (LP: #1756471) | ||
27 | 22 | - datasources: fix DataSource subclass get_hostname method signature | ||
28 | 23 | (LP: #1757176) | ||
29 | 24 | - OpenNebula: Update network to return v2 config rather than ENI. | ||
30 | 25 | [Akihiko Ota] | ||
31 | 26 | - Add Hetzner Cloud DataSource | ||
32 | 27 | - net: recognize iscsi root cases without ip= on kernel command line. | ||
33 | 28 | (LP: #1752391) | ||
34 | 29 | - tests: fix flakes warning for unused variable | ||
35 | 30 | - tests: patch leaked stderr messages from snap unit tests | ||
36 | 31 | - cc_snap: Add new module to install and configure snapd and snap | ||
37 | 32 | packages. | ||
38 | 33 | - tests: Make pylint happy and fix python2.6 uses of assertRaisesRegex. | ||
39 | 34 | - netplan: render bridge port-priority values (LP: #1735821) | ||
40 | 35 | - util: Fix subp regression. Allow specifying subp command as a string. | ||
41 | 36 | (LP: #1755965) | ||
42 | 37 | - doc: fix all warnings issued by 'tox -e doc' | ||
43 | 38 | - FreeBSD: Set hostname to FQDN. [Dominic Schlegel] (LP: #1753499) | ||
44 | 39 | - tests: fix run_tree and bddeb | ||
45 | 40 | - tests: Fix some warnings in tests that popped up with newer python. | ||
46 | 41 | - set_hostname: When present in metadata, set it before network bringup. | ||
47 | 42 | (LP: #1746455) | ||
48 | 43 | - tests: Centralize and re-use skipTest based on json schema presense. | ||
49 | 44 | - This commit fixes get_hostname on the AzureDataSource. | ||
50 | 45 | [Douglas Jordan] (LP: #1754495) | ||
51 | 46 | - shellify: raise TypeError on bad input. | ||
52 | 47 | - Make salt minion module work on FreeBSD. | ||
53 | 48 | [Dominic Schlegel] (LP: #1721503) | ||
54 | 49 | - Simplify some comparisions. [Rémy Léone] | ||
55 | 50 | - Change some list creation and population to literal. [Rémy Léone] | ||
56 | 51 | - GCE: fix reading of user-data that is not base64 encoded. (LP: #1752711) | ||
57 | 52 | - doc: fix chef install from apt packages example in RTD. | ||
58 | 53 | - Implement puppet 4 support [Romanos Skiadas] (LP: #1446804) | ||
59 | 54 | - subp: Fix subp usage with non-ascii characters when no system locale. | ||
60 | 55 | (LP: #1751051) | ||
61 | 56 | - salt: configure grains in grains file rather than in minion config. | ||
62 | 57 | [Daniel Wallace] | ||
63 | 58 | |||
64 | 1 | 18.1: | 59 | 18.1: |
65 | 2 | - OVF: Fix VMware support for 64-bit platforms. [Sankar Tanguturi] | 60 | - OVF: Fix VMware support for 64-bit platforms. [Sankar Tanguturi] |
66 | 3 | - ds-identify: Fix searching for iso9660 OVF cdroms. (LP: #1749980) | 61 | - ds-identify: Fix searching for iso9660 OVF cdroms. (LP: #1749980) |
67 | diff --git a/cloudinit/config/cc_puppet.py b/cloudinit/config/cc_puppet.py | |||
68 | index 297e072..4190a20 100644 | |||
69 | --- a/cloudinit/config/cc_puppet.py | |||
70 | +++ b/cloudinit/config/cc_puppet.py | |||
71 | @@ -140,6 +140,7 @@ def handle(name, cfg, cloud, log, _args): | |||
72 | 140 | # (TODO(harlowja) is this really needed??) | 140 | # (TODO(harlowja) is this really needed??) |
73 | 141 | cleaned_lines = [i.lstrip() for i in contents.splitlines()] | 141 | cleaned_lines = [i.lstrip() for i in contents.splitlines()] |
74 | 142 | cleaned_contents = '\n'.join(cleaned_lines) | 142 | cleaned_contents = '\n'.join(cleaned_lines) |
75 | 143 | # Move to puppet_config.read_file when dropping py2.7 | ||
76 | 143 | puppet_config.readfp( # pylint: disable=W1505 | 144 | puppet_config.readfp( # pylint: disable=W1505 |
77 | 144 | StringIO(cleaned_contents), | 145 | StringIO(cleaned_contents), |
78 | 145 | filename=p_constants.conf_path) | 146 | filename=p_constants.conf_path) |
79 | @@ -150,6 +151,8 @@ def handle(name, cfg, cloud, log, _args): | |||
80 | 150 | # Puppet ssl sub-directory isn't created yet | 151 | # Puppet ssl sub-directory isn't created yet |
81 | 151 | # Create it with the proper permissions and ownership | 152 | # Create it with the proper permissions and ownership |
82 | 152 | util.ensure_dir(p_constants.ssl_dir, 0o771) | 153 | util.ensure_dir(p_constants.ssl_dir, 0o771) |
83 | 154 | util.chownbyname(p_constants.ssl_dir, 'puppet', 'root') | ||
84 | 155 | util.ensure_dir(p_constants.ssl_cert_dir) | ||
85 | 153 | 156 | ||
86 | 154 | util.chownbyname(p_constants.ssl_cert_dir, 'puppet', 'root') | 157 | util.chownbyname(p_constants.ssl_cert_dir, 'puppet', 'root') |
87 | 155 | util.write_file(p_constants.ssl_cert_path, cfg) | 158 | util.write_file(p_constants.ssl_cert_path, cfg) |
88 | diff --git a/cloudinit/config/cc_resizefs.py b/cloudinit/config/cc_resizefs.py | |||
89 | index cec22bb..c8e1752 100644 | |||
90 | --- a/cloudinit/config/cc_resizefs.py | |||
91 | +++ b/cloudinit/config/cc_resizefs.py | |||
92 | @@ -84,6 +84,10 @@ def _resize_ufs(mount_point, devpth): | |||
93 | 84 | return ('growfs', devpth) | 84 | return ('growfs', devpth) |
94 | 85 | 85 | ||
95 | 86 | 86 | ||
96 | 87 | def _resize_zfs(mount_point, devpth): | ||
97 | 88 | return ('zpool', 'online', '-e', mount_point, devpth) | ||
98 | 89 | |||
99 | 90 | |||
100 | 87 | def _get_dumpfs_output(mount_point): | 91 | def _get_dumpfs_output(mount_point): |
101 | 88 | dumpfs_res, err = util.subp(['dumpfs', '-m', mount_point]) | 92 | dumpfs_res, err = util.subp(['dumpfs', '-m', mount_point]) |
102 | 89 | return dumpfs_res | 93 | return dumpfs_res |
103 | @@ -148,6 +152,7 @@ RESIZE_FS_PREFIXES_CMDS = [ | |||
104 | 148 | ('ext', _resize_ext), | 152 | ('ext', _resize_ext), |
105 | 149 | ('xfs', _resize_xfs), | 153 | ('xfs', _resize_xfs), |
106 | 150 | ('ufs', _resize_ufs), | 154 | ('ufs', _resize_ufs), |
107 | 155 | ('zfs', _resize_zfs), | ||
108 | 151 | ] | 156 | ] |
109 | 152 | 157 | ||
110 | 153 | RESIZE_FS_PRECHECK_CMDS = { | 158 | RESIZE_FS_PRECHECK_CMDS = { |
111 | @@ -188,6 +193,13 @@ def maybe_get_writable_device_path(devpath, info, log): | |||
112 | 188 | log.debug("Not attempting to resize devpath '%s': %s", devpath, info) | 193 | log.debug("Not attempting to resize devpath '%s': %s", devpath, info) |
113 | 189 | return None | 194 | return None |
114 | 190 | 195 | ||
115 | 196 | # FreeBSD zpool can also just use gpt/<label> | ||
116 | 197 | # with that in mind we can not do an os.stat on "gpt/whatever" | ||
117 | 198 | # therefore return the devpath already here. | ||
118 | 199 | if devpath.startswith('gpt/'): | ||
119 | 200 | log.debug('We have a gpt label - just go ahead') | ||
120 | 201 | return devpath | ||
121 | 202 | |||
122 | 191 | try: | 203 | try: |
123 | 192 | statret = os.stat(devpath) | 204 | statret = os.stat(devpath) |
124 | 193 | except OSError as exc: | 205 | except OSError as exc: |
125 | @@ -231,6 +243,16 @@ def handle(name, cfg, _cloud, log, args): | |||
126 | 231 | 243 | ||
127 | 232 | (devpth, fs_type, mount_point) = result | 244 | (devpth, fs_type, mount_point) = result |
128 | 233 | 245 | ||
129 | 246 | # if we have a zfs then our device path at this point | ||
130 | 247 | # is the zfs label. For example: vmzroot/ROOT/freebsd | ||
131 | 248 | # we will have to get the zpool name out of this | ||
132 | 249 | # and set the resize_what variable to the zpool | ||
133 | 250 | # so the _resize_zfs function gets the right attribute. | ||
134 | 251 | if fs_type == 'zfs': | ||
135 | 252 | zpool = devpth.split('/')[0] | ||
136 | 253 | devpth = util.get_device_info_from_zpool(zpool) | ||
137 | 254 | resize_what = zpool | ||
138 | 255 | |||
139 | 234 | info = "dev=%s mnt_point=%s path=%s" % (devpth, mount_point, resize_what) | 256 | info = "dev=%s mnt_point=%s path=%s" % (devpth, mount_point, resize_what) |
140 | 235 | log.debug("resize_info: %s" % info) | 257 | log.debug("resize_info: %s" % info) |
141 | 236 | 258 | ||
142 | diff --git a/cloudinit/settings.py b/cloudinit/settings.py | |||
143 | index 5fe749d..dde5749 100644 | |||
144 | --- a/cloudinit/settings.py | |||
145 | +++ b/cloudinit/settings.py | |||
146 | @@ -37,6 +37,7 @@ CFG_BUILTIN = { | |||
147 | 37 | 'Bigstep', | 37 | 'Bigstep', |
148 | 38 | 'Scaleway', | 38 | 'Scaleway', |
149 | 39 | 'Hetzner', | 39 | 'Hetzner', |
150 | 40 | 'IBMCloud', | ||
151 | 40 | # At the end to act as a 'catch' when none of the above work... | 41 | # At the end to act as a 'catch' when none of the above work... |
152 | 41 | 'None', | 42 | 'None', |
153 | 42 | ], | 43 | ], |
154 | diff --git a/cloudinit/sources/DataSourceHetzner.py b/cloudinit/sources/DataSourceHetzner.py | |||
155 | index 769fe13..5c75b65 100644 | |||
156 | --- a/cloudinit/sources/DataSourceHetzner.py | |||
157 | +++ b/cloudinit/sources/DataSourceHetzner.py | |||
158 | @@ -44,6 +44,8 @@ class DataSourceHetzner(sources.DataSource): | |||
159 | 44 | self.dsmode = sources.DSMODE_NETWORK | 44 | self.dsmode = sources.DSMODE_NETWORK |
160 | 45 | 45 | ||
161 | 46 | def get_data(self): | 46 | def get_data(self): |
162 | 47 | if not on_hetzner(): | ||
163 | 48 | return False | ||
164 | 47 | nic = cloudnet.find_fallback_nic() | 49 | nic = cloudnet.find_fallback_nic() |
165 | 48 | with cloudnet.EphemeralIPv4Network(nic, "169.254.0.1", 16, | 50 | with cloudnet.EphemeralIPv4Network(nic, "169.254.0.1", 16, |
166 | 49 | "169.254.255.255"): | 51 | "169.254.255.255"): |
167 | @@ -87,6 +89,10 @@ class DataSourceHetzner(sources.DataSource): | |||
168 | 87 | return self._network_config | 89 | return self._network_config |
169 | 88 | 90 | ||
170 | 89 | 91 | ||
171 | 92 | def on_hetzner(): | ||
172 | 93 | return util.read_dmi_data('system-manufacturer') == "Hetzner" | ||
173 | 94 | |||
174 | 95 | |||
175 | 90 | # Used to match classes to dependencies | 96 | # Used to match classes to dependencies |
176 | 91 | datasources = [ | 97 | datasources = [ |
177 | 92 | (DataSourceHetzner, (sources.DEP_FILESYSTEM, )), | 98 | (DataSourceHetzner, (sources.DEP_FILESYSTEM, )), |
178 | diff --git a/cloudinit/util.py b/cloudinit/util.py | |||
179 | index fb4ee5f..0ab2c48 100644 | |||
180 | --- a/cloudinit/util.py | |||
181 | +++ b/cloudinit/util.py | |||
182 | @@ -2234,7 +2234,7 @@ def get_path_dev_freebsd(path, mnt_list): | |||
183 | 2234 | return path_found | 2234 | return path_found |
184 | 2235 | 2235 | ||
185 | 2236 | 2236 | ||
187 | 2237 | def get_mount_info_freebsd(path, log=LOG): | 2237 | def get_mount_info_freebsd(path): |
188 | 2238 | (result, err) = subp(['mount', '-p', path], rcs=[0, 1]) | 2238 | (result, err) = subp(['mount', '-p', path], rcs=[0, 1]) |
189 | 2239 | if len(err): | 2239 | if len(err): |
190 | 2240 | # find a path if the input is not a mounting point | 2240 | # find a path if the input is not a mounting point |
191 | @@ -2248,23 +2248,49 @@ def get_mount_info_freebsd(path, log=LOG): | |||
192 | 2248 | return "/dev/" + label_part, ret[2], ret[1] | 2248 | return "/dev/" + label_part, ret[2], ret[1] |
193 | 2249 | 2249 | ||
194 | 2250 | 2250 | ||
195 | 2251 | def get_device_info_from_zpool(zpool): | ||
196 | 2252 | (zpoolstatus, err) = subp(['zpool', 'status', zpool]) | ||
197 | 2253 | if len(err): | ||
198 | 2254 | return None | ||
199 | 2255 | r = r'.*(ONLINE).*' | ||
200 | 2256 | for line in zpoolstatus.split("\n"): | ||
201 | 2257 | if re.search(r, line) and zpool not in line and "state" not in line: | ||
202 | 2258 | disk = line.split()[0] | ||
203 | 2259 | LOG.debug('found zpool "%s" on disk %s', zpool, disk) | ||
204 | 2260 | return disk | ||
205 | 2261 | |||
206 | 2262 | |||
207 | 2251 | def parse_mount(path): | 2263 | def parse_mount(path): |
209 | 2252 | (mountoutput, _err) = subp("mount") | 2264 | (mountoutput, _err) = subp(['mount']) |
210 | 2253 | mount_locs = mountoutput.splitlines() | 2265 | mount_locs = mountoutput.splitlines() |
211 | 2266 | # there are 2 types of mount outputs we have to parse therefore | ||
212 | 2267 | # the regex is a bit complex. to better understand this regex see: | ||
213 | 2268 | # https://regex101.com/r/2F6c1k/1 | ||
214 | 2269 | # https://regex101.com/r/T2en7a/1 | ||
215 | 2270 | regex = r'^(/dev/[\S]+|.*zroot\S*?) on (/[\S]*) ' + \ | ||
216 | 2271 | '(?=(?:type)[\s]+([\S]+)|\(([^,]*))' | ||
217 | 2254 | for line in mount_locs: | 2272 | for line in mount_locs: |
219 | 2255 | m = re.search(r'^(/dev/[\S]+) on (/.*) \((.+), .+, (.+)\)$', line) | 2273 | m = re.search(regex, line) |
220 | 2256 | if not m: | 2274 | if not m: |
221 | 2257 | continue | 2275 | continue |
222 | 2276 | devpth = m.group(1) | ||
223 | 2277 | mount_point = m.group(2) | ||
224 | 2278 | # above regex will either fill the fs_type in group(3) | ||
225 | 2279 | # or group(4) depending on the format we have. | ||
226 | 2280 | fs_type = m.group(3) | ||
227 | 2281 | if fs_type is None: | ||
228 | 2282 | fs_type = m.group(4) | ||
229 | 2283 | LOG.debug('found line in mount -> devpth: %s, mount_point: %s, ' | ||
230 | 2284 | 'fs_type: %s', devpth, mount_point, fs_type) | ||
231 | 2258 | # check whether the dev refers to a label on FreeBSD | 2285 | # check whether the dev refers to a label on FreeBSD |
232 | 2259 | # for example, if dev is '/dev/label/rootfs', we should | 2286 | # for example, if dev is '/dev/label/rootfs', we should |
233 | 2260 | # continue finding the real device like '/dev/da0'. | 2287 | # continue finding the real device like '/dev/da0'. |
236 | 2261 | devm = re.search('^(/dev/.+)p([0-9])$', m.group(1)) | 2288 | # this is only valid for non zfs file systems as a zpool |
237 | 2262 | if (not devm and is_FreeBSD()): | 2289 | # can have gpt labels as disk. |
238 | 2290 | devm = re.search('^(/dev/.+)p([0-9])$', devpth) | ||
239 | 2291 | if not devm and is_FreeBSD() and fs_type != 'zfs': | ||
240 | 2263 | return get_mount_info_freebsd(path) | 2292 | return get_mount_info_freebsd(path) |
245 | 2264 | devpth = m.group(1) | 2293 | elif mount_point == path: |
242 | 2265 | mount_point = m.group(2) | ||
243 | 2266 | fs_type = m.group(3) | ||
244 | 2267 | if mount_point == path: | ||
246 | 2268 | return devpth, fs_type, mount_point | 2294 | return devpth, fs_type, mount_point |
247 | 2269 | return None | 2295 | return None |
248 | 2270 | 2296 | ||
249 | diff --git a/cloudinit/version.py b/cloudinit/version.py | |||
250 | index 4a682ad..ccd0f84 100644 | |||
251 | --- a/cloudinit/version.py | |||
252 | +++ b/cloudinit/version.py | |||
253 | @@ -4,7 +4,7 @@ | |||
254 | 4 | # | 4 | # |
255 | 5 | # This file is part of cloud-init. See LICENSE file for license information. | 5 | # This file is part of cloud-init. See LICENSE file for license information. |
256 | 6 | 6 | ||
258 | 7 | __VERSION__ = "18.1" | 7 | __VERSION__ = "18.2" |
259 | 8 | 8 | ||
260 | 9 | FEATURES = [ | 9 | FEATURES = [ |
261 | 10 | # supports network config version 1 | 10 | # supports network config version 1 |
262 | diff --git a/debian/changelog b/debian/changelog | |||
263 | index 6f022bb..cdfbf6a 100644 | |||
264 | --- a/debian/changelog | |||
265 | +++ b/debian/changelog | |||
266 | @@ -1,8 +1,17 @@ | |||
268 | 1 | cloud-init (18.1-35-ge0f644b7-0ubuntu3) UNRELEASED; urgency=medium | 1 | cloud-init (18.2-0ubuntu1) bionic; urgency=medium |
269 | 2 | 2 | ||
270 | 3 | * debian/control: Add missing dependency on isc-dhcp-client (LP: #1759307). | 3 | * debian/control: Add missing dependency on isc-dhcp-client (LP: #1759307). |
271 | 4 | * New upstream snapshot. | ||
272 | 5 | - release 18.2 (LP: #1759318) | ||
273 | 6 | - Hetzner: Exit early if dmi system-manufacturer is not Hetzner. | ||
274 | 7 | - Add missing dependency on isc-dhcp-client to trunk ubuntu packaging. | ||
275 | 8 | (LP: #1759307) | ||
276 | 9 | - FreeBSD: resizefs module now able to handle zfs/zpool. | ||
277 | 10 | [Dominic Schlegel] (LP: #1721243) | ||
278 | 11 | - cc_puppet: Revert regression of puppet creating ssl and ssl_cert dirs | ||
279 | 12 | - Enable IBMCloud datasource in settings.py. | ||
280 | 4 | 13 | ||
282 | 5 | -- Scott Moser <smoser@ubuntu.com> Tue, 27 Mar 2018 12:19:44 -0400 | 14 | -- Chad Smith <chad.smith@canonical.com> Tue, 27 Mar 2018 14:59:58 -0600 |
283 | 6 | 15 | ||
284 | 7 | cloud-init (18.1-35-ge0f644b7-0ubuntu2) bionic; urgency=medium | 16 | cloud-init (18.1-35-ge0f644b7-0ubuntu2) bionic; urgency=medium |
285 | 8 | 17 | ||
286 | diff --git a/packages/debian/control.in b/packages/debian/control.in | |||
287 | index 265b261..46da6df 100644 | |||
288 | --- a/packages/debian/control.in | |||
289 | +++ b/packages/debian/control.in | |||
290 | @@ -10,7 +10,8 @@ Standards-Version: 3.9.6 | |||
291 | 10 | Package: cloud-init | 10 | Package: cloud-init |
292 | 11 | Architecture: all | 11 | Architecture: all |
293 | 12 | Depends: ${misc:Depends}, | 12 | Depends: ${misc:Depends}, |
295 | 13 | ${${python}:Depends} | 13 | ${${python}:Depends}, |
296 | 14 | isc-dhcp-client | ||
297 | 14 | Recommends: eatmydata, sudo, software-properties-common, gdisk | 15 | Recommends: eatmydata, sudo, software-properties-common, gdisk |
298 | 15 | XB-Python-Version: ${python:Versions} | 16 | XB-Python-Version: ${python:Versions} |
299 | 16 | Description: Init scripts for cloud instances | 17 | Description: Init scripts for cloud instances |
300 | diff --git a/tests/data/mount_parse_ext.txt b/tests/data/mount_parse_ext.txt | |||
301 | 17 | new file mode 100644 | 18 | new file mode 100644 |
302 | index 0000000..da0c870 | |||
303 | --- /dev/null | |||
304 | +++ b/tests/data/mount_parse_ext.txt | |||
305 | @@ -0,0 +1,19 @@ | |||
306 | 1 | /dev/mapper/vg00-lv_root on / type ext4 (rw,errors=remount-ro) | ||
307 | 2 | proc on /proc type proc (rw,noexec,nosuid,nodev) | ||
308 | 3 | sysfs on /sys type sysfs (rw,noexec,nosuid,nodev) | ||
309 | 4 | none on /sys/fs/cgroup type tmpfs (rw) | ||
310 | 5 | none on /sys/fs/fuse/connections type fusectl (rw) | ||
311 | 6 | none on /sys/kernel/debug type debugfs (rw) | ||
312 | 7 | none on /sys/kernel/security type securityfs (rw) | ||
313 | 8 | udev on /dev type devtmpfs (rw,mode=0755) | ||
314 | 9 | devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620) | ||
315 | 10 | none on /tmp type tmpfs (rw) | ||
316 | 11 | tmpfs on /run type tmpfs (rw,noexec,nosuid,size=10%,mode=0755) | ||
317 | 12 | none on /run/lock type tmpfs (rw,noexec,nosuid,nodev,size=5242880) | ||
318 | 13 | none on /run/shm type tmpfs (rw,nosuid,nodev) | ||
319 | 14 | none on /run/user type tmpfs (rw,noexec,nosuid,nodev,size=104857600,mode=0755) | ||
320 | 15 | none on /sys/fs/pstore type pstore (rw) | ||
321 | 16 | /dev/mapper/vg00-lv_var on /var type ext4 (rw) | ||
322 | 17 | rpc_pipefs on /run/rpc_pipefs type rpc_pipefs (rw) | ||
323 | 18 | systemd on /sys/fs/cgroup/systemd type cgroup (rw,noexec,nosuid,nodev,none,name=systemd) | ||
324 | 19 | 10.0.1.1:/backup on /backup type nfs (rw,noexec,nosuid,nodev,bg,nolock,tcp,nfsvers=3,hard,addr=10.0.1.1) | ||
325 | 0 | \ No newline at end of file | 20 | \ No newline at end of file |
326 | diff --git a/tests/data/mount_parse_zfs.txt b/tests/data/mount_parse_zfs.txt | |||
327 | 1 | new file mode 100644 | 21 | new file mode 100644 |
328 | index 0000000..08af04f | |||
329 | --- /dev/null | |||
330 | +++ b/tests/data/mount_parse_zfs.txt | |||
331 | @@ -0,0 +1,21 @@ | |||
332 | 1 | vmzroot/ROOT/freebsd on / (zfs, local, nfsv4acls) | ||
333 | 2 | devfs on /dev (devfs, local, multilabel) | ||
334 | 3 | fdescfs on /dev/fd (fdescfs) | ||
335 | 4 | vmzroot/root on /root (zfs, local, nfsv4acls) | ||
336 | 5 | vmzroot/tmp on /tmp (zfs, local, nosuid, nfsv4acls) | ||
337 | 6 | vmzroot/ROOT/freebsd/usr on /usr (zfs, local, nfsv4acls) | ||
338 | 7 | vmzroot/ROOT/freebsd/usr/local on /usr/local (zfs, local, nfsv4acls) | ||
339 | 8 | vmzroot/ROOT/freebsd/var on /var (zfs, local, nfsv4acls) | ||
340 | 9 | vmzroot/ROOT/freebsd/var/cache on /var/cache (zfs, local, noexec, nosuid, nfsv4acls) | ||
341 | 10 | vmzroot/ROOT/freebsd/var/crash on /var/crash (zfs, local, noexec, nosuid, nfsv4acls) | ||
342 | 11 | vmzroot/var/cron on /var/cron (zfs, local, nosuid, nfsv4acls) | ||
343 | 12 | vmzroot/ROOT/freebsd/var/db on /var/db (zfs, local, noatime, noexec, nosuid, nfsv4acls) | ||
344 | 13 | vmzroot/ROOT/freebsd/var/empty on /var/empty (zfs, local, noexec, nosuid, read-only, nfsv4acls) | ||
345 | 14 | vmzroot/var/log on /var/log (zfs, local, noexec, nosuid, nfsv4acls) | ||
346 | 15 | vmzroot/var/log/pf on /var/log/pf (zfs, local, noexec, nosuid, nfsv4acls) | ||
347 | 16 | vmzroot/var/mail on /var/mail (zfs, local, noexec, nosuid, nfsv4acls) | ||
348 | 17 | vmzroot/ROOT/freebsd/var/run on /var/run (zfs, local, noexec, nosuid, nfsv4acls) | ||
349 | 18 | vmzroot/var/spool on /var/spool (zfs, local, noexec, nosuid, nfsv4acls) | ||
350 | 19 | vmzroot/var/tmp on /var/tmp (zfs, local, nosuid, nfsv4acls) | ||
351 | 20 | 10.0.0.1:/vol/test on /mnt/test (nfs, read-only) | ||
352 | 21 | 10.0.0.2:/vol/tes2 on /mnt/test2 (nfs, nosuid) | ||
353 | 0 | \ No newline at end of file | 22 | \ No newline at end of file |
354 | diff --git a/tests/data/zpool_status_simple.txt b/tests/data/zpool_status_simple.txt | |||
355 | 1 | new file mode 100644 | 23 | new file mode 100644 |
356 | index 0000000..a2c573a | |||
357 | --- /dev/null | |||
358 | +++ b/tests/data/zpool_status_simple.txt | |||
359 | @@ -0,0 +1,10 @@ | |||
360 | 1 | pool: vmzroot | ||
361 | 2 | state: ONLINE | ||
362 | 3 | scan: none requested | ||
363 | 4 | config: | ||
364 | 5 | |||
365 | 6 | NAME STATE READ WRITE CKSUM | ||
366 | 7 | vmzroot ONLINE 0 0 0 | ||
367 | 8 | gpt/system ONLINE 0 0 0 | ||
368 | 9 | |||
369 | 10 | errors: No known data errors | ||
370 | 0 | \ No newline at end of file | 11 | \ No newline at end of file |
371 | diff --git a/tests/unittests/test_datasource/test_common.py b/tests/unittests/test_datasource/test_common.py | |||
372 | index 6d2dc5b..ec33388 100644 | |||
373 | --- a/tests/unittests/test_datasource/test_common.py | |||
374 | +++ b/tests/unittests/test_datasource/test_common.py | |||
375 | @@ -15,6 +15,7 @@ from cloudinit.sources import ( | |||
376 | 15 | DataSourceEc2 as Ec2, | 15 | DataSourceEc2 as Ec2, |
377 | 16 | DataSourceGCE as GCE, | 16 | DataSourceGCE as GCE, |
378 | 17 | DataSourceHetzner as Hetzner, | 17 | DataSourceHetzner as Hetzner, |
379 | 18 | DataSourceIBMCloud as IBMCloud, | ||
380 | 18 | DataSourceMAAS as MAAS, | 19 | DataSourceMAAS as MAAS, |
381 | 19 | DataSourceNoCloud as NoCloud, | 20 | DataSourceNoCloud as NoCloud, |
382 | 20 | DataSourceOpenNebula as OpenNebula, | 21 | DataSourceOpenNebula as OpenNebula, |
383 | @@ -33,6 +34,7 @@ DEFAULT_LOCAL = [ | |||
384 | 33 | ConfigDrive.DataSourceConfigDrive, | 34 | ConfigDrive.DataSourceConfigDrive, |
385 | 34 | DigitalOcean.DataSourceDigitalOcean, | 35 | DigitalOcean.DataSourceDigitalOcean, |
386 | 35 | Hetzner.DataSourceHetzner, | 36 | Hetzner.DataSourceHetzner, |
387 | 37 | IBMCloud.DataSourceIBMCloud, | ||
388 | 36 | NoCloud.DataSourceNoCloud, | 38 | NoCloud.DataSourceNoCloud, |
389 | 37 | OpenNebula.DataSourceOpenNebula, | 39 | OpenNebula.DataSourceOpenNebula, |
390 | 38 | OVF.DataSourceOVF, | 40 | OVF.DataSourceOVF, |
391 | diff --git a/tests/unittests/test_datasource/test_hetzner.py b/tests/unittests/test_datasource/test_hetzner.py | |||
392 | index f1d1525..a9c1259 100644 | |||
393 | --- a/tests/unittests/test_datasource/test_hetzner.py | |||
394 | +++ b/tests/unittests/test_datasource/test_hetzner.py | |||
395 | @@ -73,7 +73,10 @@ class TestDataSourceHetzner(CiTestCase): | |||
396 | 73 | @mock.patch('cloudinit.net.find_fallback_nic') | 73 | @mock.patch('cloudinit.net.find_fallback_nic') |
397 | 74 | @mock.patch('cloudinit.sources.helpers.hetzner.read_metadata') | 74 | @mock.patch('cloudinit.sources.helpers.hetzner.read_metadata') |
398 | 75 | @mock.patch('cloudinit.sources.helpers.hetzner.read_userdata') | 75 | @mock.patch('cloudinit.sources.helpers.hetzner.read_userdata') |
400 | 76 | def test_read_data(self, m_usermd, m_readmd, m_fallback_nic, m_net): | 76 | @mock.patch('cloudinit.sources.DataSourceHetzner.on_hetzner') |
401 | 77 | def test_read_data(self, m_on_hetzner, m_usermd, m_readmd, m_fallback_nic, | ||
402 | 78 | m_net): | ||
403 | 79 | m_on_hetzner.return_value = True | ||
404 | 77 | m_readmd.return_value = METADATA.copy() | 80 | m_readmd.return_value = METADATA.copy() |
405 | 78 | m_usermd.return_value = USERDATA | 81 | m_usermd.return_value = USERDATA |
406 | 79 | m_fallback_nic.return_value = 'eth0' | 82 | m_fallback_nic.return_value = 'eth0' |
407 | @@ -97,3 +100,18 @@ class TestDataSourceHetzner(CiTestCase): | |||
408 | 97 | self.assertIsInstance(ds.get_public_ssh_keys(), list) | 100 | self.assertIsInstance(ds.get_public_ssh_keys(), list) |
409 | 98 | self.assertEqual(ds.get_userdata_raw(), USERDATA) | 101 | self.assertEqual(ds.get_userdata_raw(), USERDATA) |
410 | 99 | self.assertEqual(ds.get_vendordata_raw(), METADATA.get('vendor_data')) | 102 | self.assertEqual(ds.get_vendordata_raw(), METADATA.get('vendor_data')) |
411 | 103 | |||
412 | 104 | @mock.patch('cloudinit.sources.helpers.hetzner.read_metadata') | ||
413 | 105 | @mock.patch('cloudinit.net.find_fallback_nic') | ||
414 | 106 | @mock.patch('cloudinit.sources.DataSourceHetzner.on_hetzner') | ||
415 | 107 | def test_not_on_hetzner_returns_false(self, m_on_hetzner, m_find_fallback, | ||
416 | 108 | m_read_md): | ||
417 | 109 | """If helper 'on_hetzner' returns False, return False from get_data.""" | ||
418 | 110 | m_on_hetzner.return_value = False | ||
419 | 111 | ds = self.get_ds() | ||
420 | 112 | ret = ds.get_data() | ||
421 | 113 | |||
422 | 114 | self.assertFalse(ret) | ||
423 | 115 | # These are a white box attempt to ensure it did not search. | ||
424 | 116 | m_find_fallback.assert_not_called() | ||
425 | 117 | m_read_md.assert_not_called() | ||
426 | diff --git a/tests/unittests/test_handler/test_handler_resizefs.py b/tests/unittests/test_handler/test_handler_resizefs.py | |||
427 | index c2a7f9f..7a7ba1f 100644 | |||
428 | --- a/tests/unittests/test_handler/test_handler_resizefs.py | |||
429 | +++ b/tests/unittests/test_handler/test_handler_resizefs.py | |||
430 | @@ -1,7 +1,8 @@ | |||
431 | 1 | # This file is part of cloud-init. See LICENSE file for license information. | 1 | # This file is part of cloud-init. See LICENSE file for license information. |
432 | 2 | 2 | ||
433 | 3 | from cloudinit.config.cc_resizefs import ( | 3 | from cloudinit.config.cc_resizefs import ( |
435 | 4 | can_skip_resize, handle, maybe_get_writable_device_path, _resize_btrfs) | 4 | can_skip_resize, handle, maybe_get_writable_device_path, _resize_btrfs, |
436 | 5 | _resize_zfs, _resize_xfs, _resize_ext, _resize_ufs) | ||
437 | 5 | 6 | ||
438 | 6 | from collections import namedtuple | 7 | from collections import namedtuple |
439 | 7 | import logging | 8 | import logging |
440 | @@ -60,6 +61,9 @@ class TestResizefs(CiTestCase): | |||
441 | 60 | res = can_skip_resize(fs_type, resize_what, devpth) | 61 | res = can_skip_resize(fs_type, resize_what, devpth) |
442 | 61 | self.assertTrue(res) | 62 | self.assertTrue(res) |
443 | 62 | 63 | ||
444 | 64 | def test_can_skip_resize_ext(self): | ||
445 | 65 | self.assertFalse(can_skip_resize('ext', '/', '/dev/sda1')) | ||
446 | 66 | |||
447 | 63 | def test_handle_noops_on_disabled(self): | 67 | def test_handle_noops_on_disabled(self): |
448 | 64 | """The handle function logs when the configuration disables resize.""" | 68 | """The handle function logs when the configuration disables resize.""" |
449 | 65 | cfg = {'resize_rootfs': False} | 69 | cfg = {'resize_rootfs': False} |
450 | @@ -122,6 +126,51 @@ class TestResizefs(CiTestCase): | |||
451 | 122 | logs = self.logs.getvalue() | 126 | logs = self.logs.getvalue() |
452 | 123 | self.assertIn("WARNING: Unable to find device '/dev/root'", logs) | 127 | self.assertIn("WARNING: Unable to find device '/dev/root'", logs) |
453 | 124 | 128 | ||
454 | 129 | def test_resize_zfs_cmd_return(self): | ||
455 | 130 | zpool = 'zroot' | ||
456 | 131 | devpth = 'gpt/system' | ||
457 | 132 | self.assertEqual(('zpool', 'online', '-e', zpool, devpth), | ||
458 | 133 | _resize_zfs(zpool, devpth)) | ||
459 | 134 | |||
460 | 135 | def test_resize_xfs_cmd_return(self): | ||
461 | 136 | mount_point = '/mnt/test' | ||
462 | 137 | devpth = '/dev/sda1' | ||
463 | 138 | self.assertEqual(('xfs_growfs', mount_point), | ||
464 | 139 | _resize_xfs(mount_point, devpth)) | ||
465 | 140 | |||
466 | 141 | def test_resize_ext_cmd_return(self): | ||
467 | 142 | mount_point = '/' | ||
468 | 143 | devpth = '/dev/sdb1' | ||
469 | 144 | self.assertEqual(('resize2fs', devpth), | ||
470 | 145 | _resize_ext(mount_point, devpth)) | ||
471 | 146 | |||
472 | 147 | def test_resize_ufs_cmd_return(self): | ||
473 | 148 | mount_point = '/' | ||
474 | 149 | devpth = '/dev/sda2' | ||
475 | 150 | self.assertEqual(('growfs', devpth), | ||
476 | 151 | _resize_ufs(mount_point, devpth)) | ||
477 | 152 | |||
478 | 153 | @mock.patch('cloudinit.util.get_mount_info') | ||
479 | 154 | @mock.patch('cloudinit.util.get_device_info_from_zpool') | ||
480 | 155 | @mock.patch('cloudinit.util.parse_mount') | ||
481 | 156 | def test_handle_zfs_root(self, mount_info, zpool_info, parse_mount): | ||
482 | 157 | devpth = 'vmzroot/ROOT/freebsd' | ||
483 | 158 | disk = 'gpt/system' | ||
484 | 159 | fs_type = 'zfs' | ||
485 | 160 | mount_point = '/' | ||
486 | 161 | |||
487 | 162 | mount_info.return_value = (devpth, fs_type, mount_point) | ||
488 | 163 | zpool_info.return_value = disk | ||
489 | 164 | parse_mount.return_value = (devpth, fs_type, mount_point) | ||
490 | 165 | |||
491 | 166 | cfg = {'resize_rootfs': True} | ||
492 | 167 | |||
493 | 168 | with mock.patch('cloudinit.config.cc_resizefs.do_resize') as dresize: | ||
494 | 169 | handle('cc_resizefs', cfg, _cloud=None, log=LOG, args=[]) | ||
495 | 170 | ret = dresize.call_args[0][0] | ||
496 | 171 | |||
497 | 172 | self.assertEqual(('zpool', 'online', '-e', 'vmzroot', disk), ret) | ||
498 | 173 | |||
499 | 125 | 174 | ||
500 | 126 | class TestRootDevFromCmdline(CiTestCase): | 175 | class TestRootDevFromCmdline(CiTestCase): |
501 | 127 | 176 | ||
502 | @@ -305,5 +354,12 @@ class TestMaybeGetDevicePathAsWritableBlock(CiTestCase): | |||
503 | 305 | ('btrfs', 'filesystem', 'resize', 'max', '/'), | 354 | ('btrfs', 'filesystem', 'resize', 'max', '/'), |
504 | 306 | _resize_btrfs("/", "/dev/sda1")) | 355 | _resize_btrfs("/", "/dev/sda1")) |
505 | 307 | 356 | ||
506 | 357 | @mock.patch('cloudinit.util.is_FreeBSD') | ||
507 | 358 | def test_maybe_get_writable_device_path_zfs_freebsd(self, freebsd): | ||
508 | 359 | freebsd.return_value = True | ||
509 | 360 | info = 'dev=gpt/system mnt_point=/ path=/' | ||
510 | 361 | devpth = maybe_get_writable_device_path('gpt/system', info, LOG) | ||
511 | 362 | self.assertEqual('gpt/system', devpth) | ||
512 | 363 | |||
513 | 308 | 364 | ||
514 | 309 | # vi: ts=4 expandtab | 365 | # vi: ts=4 expandtab |
515 | diff --git a/tests/unittests/test_util.py b/tests/unittests/test_util.py | |||
516 | index 67d9607..8685b8e 100644 | |||
517 | --- a/tests/unittests/test_util.py | |||
518 | +++ b/tests/unittests/test_util.py | |||
519 | @@ -366,6 +366,56 @@ class TestMountinfoParsing(helpers.ResourceUsingTestCase): | |||
520 | 366 | expected = ('none', 'tmpfs', '/run/lock') | 366 | expected = ('none', 'tmpfs', '/run/lock') |
521 | 367 | self.assertEqual(expected, util.parse_mount_info('/run/lock', lines)) | 367 | self.assertEqual(expected, util.parse_mount_info('/run/lock', lines)) |
522 | 368 | 368 | ||
523 | 369 | @mock.patch('cloudinit.util.subp') | ||
524 | 370 | def test_get_device_info_from_zpool(self, zpool_output): | ||
525 | 371 | # mock subp command from util.get_mount_info_fs_on_zpool | ||
526 | 372 | zpool_output.return_value = ( | ||
527 | 373 | self.readResource('zpool_status_simple.txt'), '' | ||
528 | 374 | ) | ||
529 | 375 | # save function return values and do asserts | ||
530 | 376 | ret = util.get_device_info_from_zpool('vmzroot') | ||
531 | 377 | self.assertEqual('gpt/system', ret) | ||
532 | 378 | self.assertIsNotNone(ret) | ||
533 | 379 | |||
534 | 380 | @mock.patch('cloudinit.util.subp') | ||
535 | 381 | def test_get_device_info_from_zpool_on_error(self, zpool_output): | ||
536 | 382 | # mock subp command from util.get_mount_info_fs_on_zpool | ||
537 | 383 | zpool_output.return_value = ( | ||
538 | 384 | self.readResource('zpool_status_simple.txt'), 'error' | ||
539 | 385 | ) | ||
540 | 386 | # save function return values and do asserts | ||
541 | 387 | ret = util.get_device_info_from_zpool('vmzroot') | ||
542 | 388 | self.assertIsNone(ret) | ||
543 | 389 | |||
544 | 390 | @mock.patch('cloudinit.util.subp') | ||
545 | 391 | def test_parse_mount_with_ext(self, mount_out): | ||
546 | 392 | mount_out.return_value = (self.readResource('mount_parse_ext.txt'), '') | ||
547 | 393 | # this one is valid and exists in mount_parse_ext.txt | ||
548 | 394 | ret = util.parse_mount('/var') | ||
549 | 395 | self.assertEqual(('/dev/mapper/vg00-lv_var', 'ext4', '/var'), ret) | ||
550 | 396 | # another one that is valid and exists | ||
551 | 397 | ret = util.parse_mount('/') | ||
552 | 398 | self.assertEqual(('/dev/mapper/vg00-lv_root', 'ext4', '/'), ret) | ||
553 | 399 | # this one exists in mount_parse_ext.txt | ||
554 | 400 | ret = util.parse_mount('/sys/kernel/debug') | ||
555 | 401 | self.assertIsNone(ret) | ||
556 | 402 | # this one does not even exist in mount_parse_ext.txt | ||
557 | 403 | ret = util.parse_mount('/not/existing/mount') | ||
558 | 404 | self.assertIsNone(ret) | ||
559 | 405 | |||
560 | 406 | @mock.patch('cloudinit.util.subp') | ||
561 | 407 | def test_parse_mount_with_zfs(self, mount_out): | ||
562 | 408 | mount_out.return_value = (self.readResource('mount_parse_zfs.txt'), '') | ||
563 | 409 | # this one is valid and exists in mount_parse_zfs.txt | ||
564 | 410 | ret = util.parse_mount('/var') | ||
565 | 411 | self.assertEqual(('vmzroot/ROOT/freebsd/var', 'zfs', '/var'), ret) | ||
566 | 412 | # this one is the root, valid and also exists in mount_parse_zfs.txt | ||
567 | 413 | ret = util.parse_mount('/') | ||
568 | 414 | self.assertEqual(('vmzroot/ROOT/freebsd', 'zfs', '/'), ret) | ||
569 | 415 | # this one does not even exist in mount_parse_ext.txt | ||
570 | 416 | ret = util.parse_mount('/not/existing/mount') | ||
571 | 417 | self.assertIsNone(ret) | ||
572 | 418 | |||
573 | 369 | 419 | ||
574 | 370 | class TestReadDMIData(helpers.FilesystemMockingTestCase): | 420 | class TestReadDMIData(helpers.FilesystemMockingTestCase): |
575 | 371 | 421 |
PASSED: Continuous integration, rev:10ab9a7c231 b6f7d01da3a3a4c 08cf4db01f5a87 /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 940/
https:/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
SUCCESS: MAAS Compatability Testing
IN_PROGRESS: Declarative: Post Actions
Click here to trigger a rebuild: /jenkins. ubuntu. com/server/ job/cloud- init-ci/ 940/rebuild
https:/