Merge lp:~nuclearbob/utah/image-handling into lp:utah
- image-handling
- Merge into dev
Status: | Merged |
---|---|
Merged at revision: | 937 |
Proposed branch: | lp:~nuclearbob/utah/image-handling |
Merge into: | lp:utah |
Diff against target: |
830 lines (+192/-186) 13 files modified
client.py (+2/-1) debian/changelog (+2/-0) examples/run_utah_tests.py (+43/-54) templates/utah-latecommand.jinja2 (+1/-0) utah/client/common.py (+28/-1) utah/iso.py (+3/-11) utah/provisioning/baremetal/__init__.py (+48/-0) utah/provisioning/baremetal/bamboofeeder.py (+0/-9) utah/provisioning/baremetal/cobbler.py (+13/-13) utah/provisioning/provisioning.py (+27/-79) utah/provisioning/ssh.py (+3/-9) utah/provisioning/vm.py (+21/-7) utah/run.py (+1/-2) |
To merge this branch: | bzr merge lp:~nuclearbob/utah/image-handling |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Max Brustkern (community) | Needs Resubmitting | ||
Review via email:
|
Commit message
Description of the change
This change moves image handling out of the Machine class. This is necessary for the upcoming LVM changes. This branch is based on:
https:/
I've tested it on a variety of VMs. I'm working on baremetal testing now, but I'd like to get some other eyes on it in the interim. Unlike my previous proposal, this doesn't drop Panda support.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andy Doan (doanac) wrote : | # |
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Max Brustkern (nuclearbob) wrote : | # |
I was actually planning to make similar changes to what you suggest in the LVM branch itself, since I'll be adding a new possibility (if skip provisioning AND image) that needs to be able to fall back on the provisioning functionality. I can try to do that separation and reorganization in this branch and just not add the LVM functionality yet.
- 937. By Max Brustkern
-
Removing extra newline from media info and install type
- 944. By Max Brustkern
-
Removed superfluous boottimeout member of the Machine class
- 945. By Max Brustkern
-
Simplified command line setup
- 946. By Max Brustkern
-
Initial _get_machine refactoring for testing purposes
- 947. By Max Brustkern
-
Created get_vm in vm.py
- 948. By Max Brustkern
-
Added dosctring for get_vm
- 949. By Max Brustkern
-
Added get_baremetal function
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Max Brustkern (nuclearbob) wrote : | # |
I've done some testing on this. VMs work correctly. Bare metal doesn't, but the issue I'm having right now is with getting the UUID, so I can't confirm that this actual change is causing any problems.
I've refactored the code to get a machine a bit. get_vm and get_baremetal now reside in different files, and we attempt to import them in run_utah_tests and call them when appropriate. I think that will make it easier to plug in other types in the future.
I still have the image fetch before determining whether we're going to skip provisioning. That's a prelude to the LVM changes; for LVM support, if skip provisioning was passed, and an image was passed, we need to inspect the image to compare it to what's on the machine. If it doesn't match, we need to provision. To avoid downloading the image twice, we get the image first, but only if the user actually asked for one. That way, a normal skip provisioning case where no image is provided won't download one.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Max Brustkern (nuclearbob) wrote : | # |
By setting all the steps in a config file I was able to get an install to complete on bare metal. Installing the client package is still failing, so I can look into that further, but I'm reasonably confident that these changes are not the source of that issue.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andy Doan (doanac) wrote : | # |
Looks pretty good. One thing I find a little awkward is the logic of this function:
def _get_machine(args):
image = args.image
if image and image.endswith(
image = ISO(image=
if args.skip_
# TBD: Inventory should be used to verify machine
# is not running other tests
return ProvisionedMach
return _get_unprovisio
How about something like:
def _get_machine(args):
if args.skip_
# TBD: Inventory should be used to verify machine
# is not running other tests
return ProvisionedMach
image = args.image
if image and image.endswith(
image = ISO(image=
return _get_unprovisio
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Max Brustkern (nuclearbob) wrote : | # |
I agree that getting an image before the ProvisionedMachine bit looks awkward, but for LVM, we need to have the image available so we can unpack it to get info from it, if and only if the user explicitly specified an image. If I don't do it now, I'm just going to do it in the next merge proposal.
![](/+icing/build/overlay/assets/skins/sam/images/close.gif)
Andy Doan (doanac) wrote : | # |
On 06/11/2013 08:08 PM, Max Brustkern wrote:
> I agree that getting an image before the ProvisionedMachine bit looks awkward, but for LVM, we need to have the image available so we can unpack it to get info from it, if and only if the user explicitly specified an image. If I don't do it now, I'm just going to do it in the next merge proposal.
ah - okay. LGTM then
Preview Diff
1 | === modified file 'client.py' |
2 | --- client.py 2013-06-03 16:27:13 +0000 |
3 | +++ client.py 2013-06-11 05:51:28 +0000 |
4 | @@ -32,6 +32,7 @@ |
5 | |
6 | from utah.client.common import ( |
7 | DEFAULT_STATE_FILE, |
8 | + get_install_type, |
9 | ReturnCodes, |
10 | ) |
11 | from utah.client.exceptions import UTAHClientError |
12 | @@ -61,7 +62,7 @@ |
13 | default='/var/lib/utah', help='Main test directory') |
14 | parser.add_argument('-i', '--install-type', |
15 | choices=['desktop', 'server', 'mini', 'alternate'], |
16 | - default='desktop', |
17 | + default=get_install_type(), |
18 | help=('Installation Variant ' |
19 | '(i.e. server, desktop, etc.)')) |
20 | parser.add_argument('-r', '--runlist', type=url_argument, |
21 | |
22 | === modified file 'debian/changelog' |
23 | --- debian/changelog 2013-06-10 18:43:43 +0000 |
24 | +++ debian/changelog 2013-06-11 05:51:28 +0000 |
25 | @@ -32,6 +32,8 @@ |
26 | [ Max Brustkern ] |
27 | * Added media-info to installation so it will work on non-desktop |
28 | installs |
29 | + * Updated installer and client to use /etc/utah/install-type |
30 | + * Moved image handling out of the Machine class |
31 | |
32 | -- Max Brustkern <max@canonical.com> Fri, 17 May 2013 10:17:05 -0400 |
33 | |
34 | |
35 | === modified file 'examples/run_utah_tests.py' |
36 | --- examples/run_utah_tests.py 2013-05-30 09:29:41 +0000 |
37 | +++ examples/run_utah_tests.py 2013-06-11 05:51:28 +0000 |
38 | @@ -18,7 +18,6 @@ |
39 | """Provision a machine a run a test.""" |
40 | |
41 | import logging |
42 | -import os |
43 | import sys |
44 | |
45 | from traceback import format_exception |
46 | @@ -29,9 +28,9 @@ |
47 | from utah.config import config |
48 | from utah.exceptions import UTAHException |
49 | from utah.group import check_user_group, print_group_error_message |
50 | +from utah.iso import ISO |
51 | from utah.parser import get_parser, parse_args # NOQA |
52 | from utah.provisioning.ssh import ProvisionedMachine |
53 | -from utah.provisioning.vm import TinySQLiteInventory |
54 | from utah.run import ( |
55 | configure_logging, |
56 | run_tests, |
57 | @@ -42,66 +41,56 @@ |
58 | |
59 | MISSING = [] |
60 | try: |
61 | - from utah.provisioning.baremetal.bamboofeeder import BambooFeederMachine |
62 | -except ImportError: |
63 | - MISSING.append('bamboofeeder') |
64 | -try: |
65 | - from utah.provisioning.baremetal.cobbler import CobblerMachine |
66 | -except ImportError: |
67 | - MISSING.append('cobbler') |
68 | -try: |
69 | - from utah.provisioning.baremetal.inventory import \ |
70 | - ManualBaremetalSQLiteInventory |
71 | + from utah.provisioning.baremetal import get_baremetal |
72 | except ImportError: |
73 | MISSING.append('baremetal') |
74 | +try: |
75 | + from utah.provisioning.vm import get_vm |
76 | +except ImportError: |
77 | + MISSING.append('vm') |
78 | + |
79 | + |
80 | +def _get_unprovisioned_machine(args, image=None): |
81 | + if image is None: |
82 | + image = ISO(arch=args.arch, installtype=config.installtype, |
83 | + series=config.series) |
84 | + kw = {'clean': (not args.no_destroy), |
85 | + 'boot': args.boot, |
86 | + 'debug': args.debug, |
87 | + 'image': image, |
88 | + 'initrd': args.initrd, |
89 | + 'kernel': args.kernel, |
90 | + 'name': args.name, |
91 | + 'new': True, |
92 | + 'preseed': args.preseed, |
93 | + 'rewrite': args.rewrite, |
94 | + 'xml': args.xml |
95 | + } |
96 | + if args.machinetype == 'physical': |
97 | + if 'baremetal' in MISSING: |
98 | + raise UTAHException( |
99 | + 'utah-baremetal package needed for physical machines') |
100 | + return get_baremetal(args.arch, **kw) |
101 | + else: |
102 | + if 'vm' in MISSING: |
103 | + raise UTAHException('Virtual Machine support is not available') |
104 | + # TODO: consider removing these as explicit command line arguments |
105 | + # and using config instead |
106 | + kw['diskbus'] = args.diskbus |
107 | + kw['emulator'] = args.emulator |
108 | + kw['disksizes'] = args.gigabytes |
109 | + return get_vm(**kw) |
110 | |
111 | |
112 | def _get_machine(args): |
113 | + image = args.image |
114 | + if image and image.endswith('.iso'): |
115 | + image = ISO(image=args.image) |
116 | if args.skip_provisioning: |
117 | # TBD: Inventory should be used to verify machine |
118 | # is not running other tests |
119 | - machine = ProvisionedMachine(name=args.name) |
120 | - else: |
121 | - kw = {'clean': (not args.no_destroy), |
122 | - 'new': True, |
123 | - } |
124 | - kw['arch'] = args.arch |
125 | - kw['boot'] = args.boot |
126 | - kw['debug'] = args.debug |
127 | - kw['image'] = args.image |
128 | - kw['initrd'] = args.initrd |
129 | - kw['kernel'] = args.kernel |
130 | - kw['name'] = args.name |
131 | - kw['preseed'] = args.preseed |
132 | - kw['rewrite'] = args.rewrite |
133 | - kw['series'] = args.series |
134 | - kw['xml'] = args.xml |
135 | - kw['installtype'] = args.type |
136 | - if args.machinetype == 'physical': |
137 | - if 'baremetal' in MISSING: |
138 | - raise UTAHException( |
139 | - 'utah-baremetal package needed for physical machines') |
140 | - if 'arm' in args.arch: |
141 | - if 'bamboofeeder' in MISSING: |
142 | - raise UTAHException( |
143 | - 'utah-bamboofeeder package needed for panda boards') |
144 | - inventory = ManualBaremetalSQLiteInventory( |
145 | - db=os.path.join('~', '.utah-bamboofeeder-inventory'), |
146 | - lockfile=os.path.join('~', '.utah-bamboofeeder-lock')) |
147 | - kw['machinetype'] = BambooFeederMachine |
148 | - else: |
149 | - if 'cobbler' in MISSING: |
150 | - raise UTAHException( |
151 | - 'utah-cobbler package needed for this machine type') |
152 | - inventory = ManualBaremetalSQLiteInventory() |
153 | - kw['machinetype'] = CobblerMachine |
154 | - else: |
155 | - inventory = TinySQLiteInventory() |
156 | - kw['diskbus'] = args.diskbus |
157 | - kw['emulator'] = args.emulator |
158 | - kw['disksizes'] = args.gigabytes |
159 | - machine = inventory.request(**kw) |
160 | - return machine |
161 | + return ProvisionedMachine(name=args.name) |
162 | + return _get_unprovisioned_machine(args, image) |
163 | |
164 | |
165 | def run_utah_tests(argv=None): |
166 | |
167 | === modified file 'templates/utah-latecommand.jinja2' |
168 | --- templates/utah-latecommand.jinja2 2013-06-06 03:09:49 +0000 |
169 | +++ templates/utah-latecommand.jinja2 2013-06-11 05:51:28 +0000 |
170 | @@ -28,6 +28,7 @@ |
171 | echo "{{uuid}}" > /target/etc/utah/uuid |
172 | echo "{{user}}" >> /target/etc/utah/users |
173 | echo "{{media_info}}" > /target/etc/utah/media-info |
174 | + echo "{{install_type}}" > /target/etc/utah/install-type |
175 | } |
176 | |
177 | copy_rsyslog_cfg() { |
178 | |
179 | === modified file 'utah/client/common.py' |
180 | --- utah/client/common.py 2013-06-10 18:43:43 +0000 |
181 | +++ utah/client/common.py 2013-06-11 05:51:28 +0000 |
182 | @@ -56,6 +56,7 @@ |
183 | DEFAULT_STATE_FILE = os.path.join(UTAH_DIR, "state.yaml") |
184 | |
185 | MEDIA_INFO = '/etc/utah/media-info' |
186 | +INSTALL_TYPE = '/etc/utah/install-type' |
187 | PRODUCT_UUID = '/sys/class/dmi/id/product_uuid' |
188 | |
189 | |
190 | @@ -392,7 +393,7 @@ |
191 | |
192 | try: |
193 | with open(MEDIA_INFO, 'r') as f: |
194 | - media_info = f.read() |
195 | + media_info = f.read().strip() |
196 | except IOError: |
197 | # If this fails, return the default |
198 | pass |
199 | @@ -400,6 +401,32 @@ |
200 | return media_info |
201 | |
202 | |
203 | +def get_install_type(): |
204 | + """Get the contents of the install-type file if available. |
205 | + |
206 | + :returns: |
207 | + The contents of the install-type file or ``'unknown'`` if not |
208 | + available. |
209 | + :rtype: str |
210 | + |
211 | + .. note:: This is only a best-effort approach. |
212 | + |
213 | + .. seealso:: :func:`get_media_info` |
214 | + |
215 | + """ |
216 | + |
217 | + install_type = 'unknown' |
218 | + |
219 | + try: |
220 | + with open(INSTALL_TYPE, 'r') as f: |
221 | + install_type = f.read().strip() |
222 | + except IOError: |
223 | + # If this fails, return the default |
224 | + pass |
225 | + |
226 | + return install_type |
227 | + |
228 | + |
229 | def get_product_uuid(): |
230 | """Get the product_uuid of the machine under test. |
231 | |
232 | |
233 | === modified file 'utah/iso.py' |
234 | --- utah/iso.py 2013-05-24 14:29:23 +0000 |
235 | +++ utah/iso.py 2013-06-11 05:51:28 +0000 |
236 | @@ -57,16 +57,8 @@ |
237 | |
238 | """Provide a simplified method of interfacing with images.""" |
239 | |
240 | - def __init__(self, arch=None, dlpercentincrement=None, dlretries=None, |
241 | - image=None, installtype=None, logger=None, series=None): |
242 | - if dlpercentincrement is None: |
243 | - self.dlpercentincrement = config.dlpercentincrement |
244 | - else: |
245 | - self.dlpercentincrement = dlpercentincrement |
246 | - if dlretries is None: |
247 | - self.dlretries = config.dlretries |
248 | - else: |
249 | - self.dlretries = dlretries |
250 | + def __init__(self, arch=None, image=None, installtype=None, logger=None, |
251 | + series=None): |
252 | if logger is None: |
253 | self._loggersetup() |
254 | else: |
255 | @@ -206,7 +198,7 @@ |
256 | percent = 100 * read / total |
257 | if percent >= self.percent: |
258 | self.logger.info('File %s%% downloaded', percent) |
259 | - self.percent += self.dlpercentincrement |
260 | + self.percent += config.dlpercentincrement |
261 | self.logger.debug('%s read, %s%% of %s total', read, percent, total) |
262 | |
263 | def listfiles(self, returnlist=False): |
264 | |
265 | === modified file 'utah/provisioning/baremetal/__init__.py' |
266 | --- utah/provisioning/baremetal/__init__.py 2013-04-04 13:42:40 +0000 |
267 | +++ utah/provisioning/baremetal/__init__.py 2013-06-11 05:51:28 +0000 |
268 | @@ -14,3 +14,51 @@ |
269 | # with this program. If not, see <http://www.gnu.org/licenses/>. |
270 | |
271 | """utah.provisioning.baremetal""" |
272 | + |
273 | + |
274 | +import os |
275 | + |
276 | +from utah.exceptions import UTAHException |
277 | +from utah.provisioning.baremetal.inventory import ( |
278 | + ManualBaremetalSQLiteInventory |
279 | +) |
280 | + |
281 | + |
282 | +MISSING = [] |
283 | +try: |
284 | + from utah.provisioning.baremetal.bamboofeeder import BambooFeederMachine |
285 | +except ImportError: |
286 | + MISSING.append('bamboofeeder') |
287 | +try: |
288 | + from utah.provisioning.baremetal.cobbler import CobblerMachine |
289 | +except ImportError: |
290 | + MISSING.append('cobbler') |
291 | + |
292 | + |
293 | +def get_baremetal(arch, **kw): |
294 | + """Return a Machine object with the passed in arch and arguments. |
295 | + |
296 | + :param arch: Architecture of machine (mainly used for ARM or not ARM) |
297 | + :type arch: str |
298 | + :param kw: All other parameters are passed to the Machine constructor |
299 | + :type kw: dict |
300 | + |
301 | + :returns: Appropriately constructed Machine object |
302 | + :rtype: object |
303 | + |
304 | + """ |
305 | + if 'arm' in arch: |
306 | + if 'bamboofeeder' in MISSING: |
307 | + raise UTAHException( |
308 | + 'utah-bamboofeeder package needed for panda boards') |
309 | + inventory = ManualBaremetalSQLiteInventory( |
310 | + db=os.path.join('~', '.utah-bamboofeeder-inventory'), |
311 | + lockfile=os.path.join('~', '.utah-bamboofeeder-lock')) |
312 | + kw['machinetype'] = BambooFeederMachine |
313 | + else: |
314 | + if 'cobbler' in MISSING: |
315 | + raise UTAHException( |
316 | + 'utah-cobbler package needed for this machine type') |
317 | + inventory = ManualBaremetalSQLiteInventory() |
318 | + kw['machinetype'] = CobblerMachine |
319 | + return inventory.request(**kw) |
320 | |
321 | === modified file 'utah/provisioning/baremetal/bamboofeeder.py' |
322 | --- utah/provisioning/baremetal/bamboofeeder.py 2013-05-24 14:29:23 +0000 |
323 | +++ utah/provisioning/baremetal/bamboofeeder.py 2013-06-11 05:51:28 +0000 |
324 | @@ -259,15 +259,6 @@ |
325 | provision_data.update_initrd(initrd_dir) |
326 | |
327 | self._setuplatecommand() |
328 | - # TODO: check if this is still needed |
329 | - if self.installtype == 'desktop': |
330 | - self.logger.info('Configuring latecommand for desktop') |
331 | - myfile = open(os.path.join(self.tmpdir, 'initrd.d', |
332 | - 'utah-latecommand'), 'a') |
333 | - myfile.write(""" |
334 | -chroot /target sh -c 'sed "/eth[0-9]/d" -i /etc/network/interfaces' |
335 | -""") |
336 | - myfile.close() |
337 | self._setuppreseed() |
338 | self.logger.debug('Copying preseed to download location') |
339 | preseedfile = os.path.join(config.wwwdir, '{}.cfg'.format(self.name)) |
340 | |
341 | === modified file 'utah/provisioning/baremetal/cobbler.py' |
342 | --- utah/provisioning/baremetal/cobbler.py 2013-05-24 14:29:23 +0000 |
343 | +++ utah/provisioning/baremetal/cobbler.py 2013-06-11 05:51:28 +0000 |
344 | @@ -57,21 +57,21 @@ |
345 | if self.image is None: |
346 | raise UTAHBMProvisioningException( |
347 | 'Image file required for cobbler installation') |
348 | - self._custominit() |
349 | + self._cmdlinesetup() |
350 | self._depcheck() |
351 | |
352 | self.isodir = None |
353 | |
354 | # TODO: Rework cinitrd to be less of a confusing collection of kludges |
355 | self.cinitrd = None |
356 | - if self.installtype in ['alternate', 'server']: |
357 | + if self.image.installtype in ['alternate', 'server']: |
358 | self.cinitrd = os.path.join('install', 'netboot', |
359 | - 'ubuntu-installer', self.arch, |
360 | + 'ubuntu-installer', self.image.arch, |
361 | 'initrd.gz') |
362 | - if self.arch == 'amd64': |
363 | + if self.image.arch == 'amd64': |
364 | self.carch = 'x86_64' |
365 | else: |
366 | - self.carch = self.arch |
367 | + self.carch = self.image.arch |
368 | self.cname = '-'.join([self.name, self.carch]) |
369 | self.logger.debug('Cobbler arch is %s', self.carch) |
370 | self.logger.debug('Cobbler machine init finished') |
371 | @@ -89,7 +89,7 @@ |
372 | self.cleanfile(self.tmpdir) |
373 | os.chmod(self.tmpdir, 0775) |
374 | |
375 | - if self.installtype in ['alternate', 'desktop', 'server']: |
376 | + if self.image.installtype in ['alternate', 'desktop', 'server']: |
377 | self.logger.info('Mounting image') |
378 | self.isodir = os.path.join(self.tmpdir, 'iso.d') |
379 | self.cleanfunction(self._removenfs) |
380 | @@ -118,7 +118,7 @@ |
381 | |
382 | self._setuplatecommand() |
383 | # TODO: see if this is needed for all quantal desktops |
384 | - if self.installtype == 'desktop': |
385 | + if self.image.installtype == 'desktop': |
386 | self.logger.info('Configuring latecommand for desktop') |
387 | myfile = open(os.path.join(self.tmpdir, 'initrd.d', |
388 | 'utah-latecommand'), 'a') |
389 | @@ -141,7 +141,7 @@ |
390 | |
391 | preseed = os.path.join(self.tmpdir, 'initrd.d', 'preseed.cfg') |
392 | |
393 | - if self.installtype in ['alternate', 'server']: |
394 | + if self.image.installtype in ['alternate', 'server']: |
395 | self.logger.info('Importing image') |
396 | self._cobble(['import', |
397 | '--name={}'.format(self.cname), |
398 | @@ -151,7 +151,7 @@ |
399 | '--name={}'.format(self.cname), |
400 | '--kernel={}'.format(kernel), |
401 | '--initrd={}'.format(initrd)]) |
402 | - elif self.installtype in ['mini', 'desktop']: |
403 | + elif self.image.installtype in ['mini', 'desktop']: |
404 | self.logger.info('Creating distro') |
405 | self._cobble(['distro', 'add', |
406 | '--name={}'.format(self.cname), |
407 | @@ -162,7 +162,7 @@ |
408 | '--name={}'.format(self.cname), |
409 | '--distro={}'.format(self.cname)]) |
410 | |
411 | - if self.installtype == 'desktop': |
412 | + if self.image.installtype == 'desktop': |
413 | self.logger.info('Setting up NFS for desktop install') |
414 | self.logger.debug('Adding export to NFS config file') |
415 | pipe = pipes.Template() |
416 | @@ -216,7 +216,7 @@ |
417 | retry(self.sshcheck, logmethod=self.logger.info, |
418 | retry_timeout=config.checktimeout) |
419 | |
420 | - if self.installtype == 'desktop': |
421 | + if self.image.installtype == 'desktop': |
422 | self._removenfs() |
423 | |
424 | self.active = True |
425 | @@ -318,11 +318,11 @@ |
426 | def _depcheck(self): |
427 | """Check for NFS if installtype requires it.""" |
428 | super(CobblerMachine, self)._depcheck() |
429 | - if self.installtype in ['alternate', 'desktop', 'server']: |
430 | + if self.image.installtype in ['alternate', 'desktop', 'server']: |
431 | cmd = config.nfscommand + ['status'] |
432 | if ProcessRunner(cmd).returncode != 0: |
433 | raise UTAHBMProvisioningException( |
434 | - 'NFS needed for {} install'.format(self.installtype)) |
435 | + 'NFS needed for {} install'.format(self.image.installtype)) |
436 | if not os.path.isfile(config.nfsconfigfile): |
437 | raise UTAHBMProvisioningException( |
438 | 'NFS config file: {nfsconfigfile} not available' |
439 | |
440 | === modified file 'utah/provisioning/provisioning.py' |
441 | --- utah/provisioning/provisioning.py 2013-06-06 03:09:49 +0000 |
442 | +++ utah/provisioning/provisioning.py 2013-06-11 05:51:28 +0000 |
443 | @@ -64,22 +64,17 @@ |
444 | |
445 | """ |
446 | |
447 | - def __init__(self, arch=config.arch, boot=config.boot, clean=True, |
448 | - debug=False, image=config.image, |
449 | - dlpercentincrement=config.dlpercentincrement, |
450 | - initrd=config.initrd, installtype=config.installtype, |
451 | - inventory=None, kernel=config.kernel, |
452 | - machineid=config.machineid, machineuuid=config.machineuuid, |
453 | - name=config.name, new=False, preseed=config.preseed, |
454 | - rewrite=config.rewrite, series=config.series, |
455 | + def __init__(self, boot=config.boot, clean=True, debug=False, |
456 | + image=None, initrd=config.initrd, inventory=None, |
457 | + kernel=config.kernel, machineid=config.machineid, |
458 | + machineuuid=config.machineuuid, name=config.name, new=False, |
459 | + preseed=config.preseed, rewrite=config.rewrite, |
460 | template=config.template, xml=config.xml): |
461 | """Initialize the object representing the machine. |
462 | |
463 | One of these groups of arguments should be included: |
464 | - series, installtype, arch: Download an ISO from the mirrors and use |
465 | - it to install the machine. |
466 | image, kernel, initrd, preseed: Install the machine from a |
467 | - specified image/ISO file, with an optional kernel, initrd, and |
468 | + specified image object, with an optional kernel, initrd, and |
469 | preseed. libvirt xml file can be passed in xml as well. |
470 | template: Clone the machine from a template or existing machine. |
471 | name: Request a specific machine. Combine with other groups to |
472 | @@ -88,8 +83,6 @@ |
473 | Other arguments: |
474 | clean: Enable cleanup functions. |
475 | debug: Enable debug logging. |
476 | - dlpercentincrement: How often to log download updates to INFO |
477 | - (All updates logged to DEBUG) |
478 | inventory: Inventory object managing the machine; used for cleanup |
479 | purposes. |
480 | machineid: Should be passed by the request method of an Inventory |
481 | @@ -116,24 +109,17 @@ |
482 | """ |
483 | # TODO: Make this work right with super at some point. |
484 | # TODO: Consider a global temp file creator, maybe as part of install. |
485 | - # TODO: Make everything use config.dlpercentincrement instead of |
486 | - # passing it around all the time |
487 | - self.arch = arch |
488 | self.boot = boot |
489 | self.clean = clean |
490 | self.debug = debug |
491 | - self.dlpercentincrement = dlpercentincrement |
492 | - self.installtype = installtype |
493 | + self.image = image |
494 | self.inventory = inventory |
495 | self.machineid = machineid |
496 | self.new = new |
497 | self.rewrite = rewrite |
498 | - self.series = series |
499 | self.template = template |
500 | self.uuid = uuid |
501 | |
502 | - self.boottimeout = config.boottimeout |
503 | - |
504 | # TODO: Move namesetup into vm |
505 | self._namesetup(name) |
506 | |
507 | @@ -148,19 +134,8 @@ |
508 | |
509 | fileargs = ['initrd', 'kernel', 'preseed', 'xml'] |
510 | |
511 | - if image is None: |
512 | - self.image = ISO(arch=arch, |
513 | - dlpercentincrement=dlpercentincrement, |
514 | - installtype=installtype, |
515 | - logger=self.logger, |
516 | - series=series) |
517 | - elif image.endswith('.iso'): |
518 | - self.image = ISO(dlpercentincrement=dlpercentincrement, |
519 | - image=image, |
520 | - logger=self.logger) |
521 | - else: |
522 | - fileargs.append('image') |
523 | - |
524 | + # TODO: make this a function that lives somewhere else: |
525 | + # TODO: maybe move this preparation out of this class |
526 | for item in fileargs: |
527 | # Ensure every file/url type argument is available locally |
528 | arg = locals()[item] |
529 | @@ -210,44 +185,32 @@ |
530 | else: |
531 | self.name = name |
532 | |
533 | - def _makename(self, arch=None, installtype=None, machineid=None, |
534 | - prefix=None, series=None): |
535 | + def _makename(self, machineid=None, prefix=None): |
536 | """Return a name for the machine based on how it will be installed. |
537 | |
538 | Generally used for VMs to comply with old vm-tools naming conventions. |
539 | Probably could be reduced or removed. |
540 | |
541 | - :param arch: Architecture of the machine |
542 | - :type arch: str |
543 | - :param installtype: Image type (desktop, server, mini, alternate) |
544 | - :type installtype: str |
545 | :param machineid: Unique numerical machine identifier |
546 | :type machineid: int |
547 | :param prefix: Prefix for machine name (defaults to utah) |
548 | :type prefix: str |
549 | - :param series: Release codename (i.e. precise, quantal, raring) |
550 | - :type series: str |
551 | :returns: Generated machine name, like utah-1-precise-i386 |
552 | :rtype: str |
553 | |
554 | """ |
555 | - if arch is None: |
556 | - arch = str(self.arch) |
557 | - if installtype is None: |
558 | - installtype = str(self.installtype) |
559 | if machineid is None: |
560 | machineid = str(self.machineid) |
561 | if prefix is None: |
562 | prefix = str(self.prefix) |
563 | - if series is None: |
564 | - series = str(self.series) |
565 | if machineid is not None: |
566 | prefix += ('-{}'.format(str(machineid))) |
567 | self.prefix = prefix |
568 | - if installtype == 'desktop': |
569 | - name = '-'.join([prefix, series, arch]) |
570 | + if self.image.installtype == 'desktop': |
571 | + name = '-'.join([prefix, self.image.series, self.image.arch]) |
572 | else: |
573 | - name = '-'.join([prefix, series, installtype, arch]) |
574 | + name = '-'.join([prefix, self.image.series, |
575 | + self.image.installtype, self.image.arch]) |
576 | return name |
577 | |
578 | def _loggersetup(self): |
579 | @@ -330,7 +293,7 @@ |
580 | |
581 | """ |
582 | if timeout is None: |
583 | - timeout = self.boottimeout |
584 | + timeout = config.boottimeout |
585 | if logmethod is None: |
586 | logmethod = self.logger.debug |
587 | utah.timeout.timeout(timeout, retry, self.pingcheck, |
588 | @@ -518,7 +481,7 @@ |
589 | percent = 100 * read / total |
590 | if percent >= self.percent: |
591 | self.logger.info('File %s%% downloaded', percent) |
592 | - self.percent += self.dlpercentincrement |
593 | + self.percent += config.dlpercentincrement |
594 | self.logger.debug('%s read, %s%% of %s total', read, percent, total) |
595 | |
596 | def _depcheck(self): |
597 | @@ -658,10 +621,10 @@ |
598 | if initrd is None: |
599 | self.logger.info('Unpacking initrd from image') |
600 | initrdpath = './install/initrd.gz' |
601 | - if self.installtype == 'mini': |
602 | + if self.image.installtype == 'mini': |
603 | self.logger.debug('Mini image detected') |
604 | initrdpath = 'initrd.gz' |
605 | - elif self.installtype == 'desktop': |
606 | + elif self.image.installtype == 'desktop': |
607 | self.logger.debug('Desktop image detected') |
608 | # TODO: scan for this like desktop |
609 | initrdpath = './casper/initrd.lz' |
610 | @@ -726,7 +689,8 @@ |
611 | user=config.user, |
612 | uuid=self.uuid, |
613 | log_file='/target/var/log/utah-install', |
614 | - media_info=self.image.media_info) |
615 | + media_info=self.image.media_info, |
616 | + install_type=self.image.installtype) |
617 | |
618 | filename = os.path.join(tmpdir, 'initrd.d', 'utah-setup') |
619 | template.write('utah-setup.jinja2', |
620 | @@ -758,7 +722,7 @@ |
621 | if 'passwd/username' in preseed: |
622 | self._rewrite_passwd_username(preseed) |
623 | |
624 | - if self.installtype == 'desktop': |
625 | + if self.image.installtype == 'desktop': |
626 | self._rewrite_failure_command(preseed) |
627 | |
628 | output_preseed_filename = os.path.join(tmpdir, |
629 | @@ -770,7 +734,7 @@ |
630 | self.logger.info('Not altering preseed because rewrite is %s', |
631 | self.rewrite) |
632 | |
633 | - if (self.installtype == 'desktop' and |
634 | + if (self.image.installtype == 'desktop' and |
635 | self.rewrite in ['all', 'minimal', 'casperonly']): |
636 | self._preseedcasper(tmpdir=tmpdir) |
637 | |
638 | @@ -782,7 +746,7 @@ |
639 | question = preseed['preseed/late_command'] |
640 | |
641 | log_file = '/var/log/utah-install' |
642 | - if self.installtype == 'desktop': |
643 | + if self.image.installtype == 'desktop': |
644 | self.logger.info('Changing d-i latecommand ' |
645 | 'to ubiquity success_command ' |
646 | 'and prepending ubiquity lines') |
647 | @@ -939,7 +903,7 @@ |
648 | pipe.append('cpio --quiet -o -H newc', '--') |
649 | # Desktop image loads initrd.gz, |
650 | # but for physical machines we should stick with lz |
651 | - if self.installtype == 'desktop': |
652 | + if self.image.installtype == 'desktop': |
653 | self.logger.debug('Using lzma because installtype is desktop') |
654 | pipe.append('lzma -9fc ', '--') |
655 | initrd = os.path.join(tmpdir, 'initrd.lz') |
656 | @@ -951,7 +915,7 @@ |
657 | raise UTAHProvisioningException('Failed to repack initrd') |
658 | return initrd |
659 | |
660 | - def _cmdlinesetup(self, boot=None): |
661 | + def _cmdlinesetup(self): |
662 | """Setup the command line for an unattended install. |
663 | |
664 | If any options known to be needed for an automated install are not |
665 | @@ -963,11 +927,7 @@ |
666 | """ |
667 | # TODO: update libvirtvm to work like the hardware provisioners |
668 | # or vice versa |
669 | - if boot is None: |
670 | - boot = self.boot |
671 | - if boot is None: |
672 | - boot = '' |
673 | - self.cmdline = boot |
674 | + self.cmdline = self.boot or '' |
675 | # TODO: Refactor this into lists like BambooFeederMachine |
676 | if self.rewrite == 'all': |
677 | self.logger.info('Adding needed command line options') |
678 | @@ -977,7 +937,7 @@ |
679 | ('log_host', self._ipaddr(config.bridge)), |
680 | ('log_port', str(self.rsyslog.port)), |
681 | ] |
682 | - if self.installtype == 'desktop': |
683 | + if self.image.installtype == 'desktop': |
684 | options.extend([ |
685 | 'automatic-ubiquity', |
686 | 'noprompt', |
687 | @@ -1008,18 +968,6 @@ |
688 | self.rewrite) |
689 | self.logger.info('Boot command line is: {}'.format(self.cmdline)) |
690 | |
691 | - def _custominit(self, arch=None, boot=None, installtype=None, series=None): |
692 | - """Setup installtype, arch, series and commandline.""" |
693 | - #TODO: Make this a proper __init__ method |
694 | - #by adding super call to Machine. |
695 | - if installtype is None: |
696 | - self.installtype = self.image.installtype |
697 | - if arch is None: |
698 | - self.arch = self.image.arch |
699 | - if series is None: |
700 | - self.series = self.image.series |
701 | - self._cmdlinesetup(boot=boot) |
702 | - |
703 | @staticmethod |
704 | def _ipaddr(ifname): |
705 | """Return the first IP address found for the given interface name. |
706 | |
707 | === modified file 'utah/provisioning/ssh.py' |
708 | --- utah/provisioning/ssh.py 2013-05-24 14:29:23 +0000 |
709 | +++ utah/provisioning/ssh.py 2013-06-11 05:51:28 +0000 |
710 | @@ -319,7 +319,7 @@ |
711 | finally: |
712 | self.ssh_client.close() |
713 | |
714 | - def sshpoll(self, timeout=None, |
715 | + def sshpoll(self, timeout=config.boottimeout, |
716 | checktimeout=config.checktimeout, logmethod=None): |
717 | """Run sshcheck over and over until timeout expires. |
718 | |
719 | @@ -331,8 +331,6 @@ |
720 | :type logmethod: function |
721 | |
722 | """ |
723 | - if timeout is None: |
724 | - timeout = self.boottimeout |
725 | if logmethod is None: |
726 | logmethod = self.ssh_logger.debug |
727 | utah.timeout.timeout(timeout, retry, self.sshcheck, |
728 | @@ -351,7 +349,7 @@ |
729 | |
730 | """A machine that is provisioned and can be accessed through ssh.""" |
731 | |
732 | - def __init__(self, name, installtype=None): |
733 | + def __init__(self, name): |
734 | SSHMixin.initialize(self) |
735 | self.name = name |
736 | self._loggersetup() |
737 | @@ -367,15 +365,11 @@ |
738 | self.check_timeout = 3 |
739 | self.connectivity_timeout = 60 |
740 | |
741 | - # TBD: Figure out install type by getting information through ssh |
742 | - if installtype is None: |
743 | - self.installtype = config.installtype |
744 | - |
745 | def activecheck(self): |
746 | """Check if machine is active. |
747 | |
748 | Given that the machine is already provisioned, it's considered to be |
749 | - active as long as it's reachable through ssh |
750 | + active as long as it's reachable through ssh. |
751 | |
752 | """ |
753 | if not self.active: |
754 | |
755 | === modified file 'utah/provisioning/vm.py' |
756 | --- utah/provisioning/vm.py 2013-05-24 14:29:23 +0000 |
757 | +++ utah/provisioning/vm.py 2013-06-11 05:51:28 +0000 |
758 | @@ -176,7 +176,7 @@ |
759 | if self.image is None: |
760 | raise UTAHVMProvisioningException( |
761 | 'Image file required for VM installation') |
762 | - self._custominit() |
763 | + self._cmdlinesetup() |
764 | if autoname: |
765 | self._namesetup() |
766 | self._loggerunsetup() |
767 | @@ -304,12 +304,12 @@ |
768 | 'Setting type to qemu in case no hardware virtualization present') |
769 | xmlt.getroot().set('type', self.emulator) |
770 | ose = xmlt.find('os') |
771 | - if self.arch == ('i386'): |
772 | + if self.image.arch == ('i386'): |
773 | ose.find('type').set('arch', 'i686') |
774 | - elif self.arch == ('amd64'): |
775 | + elif self.image.arch == ('amd64'): |
776 | ose.find('type').set('arch', 'x86_64') |
777 | else: |
778 | - ose.find('type').set('arch', self.arch) |
779 | + ose.find('type').set('arch', self.image.arch) |
780 | self.logger.debug('Setting up boot info') |
781 | for kernele in list(ose.iterfind('kernel')): |
782 | ose.remove(kernele) |
783 | @@ -553,13 +553,13 @@ |
784 | self.rsyslog.wait_for_booted(config.boot_steps, self.uuid) |
785 | try: |
786 | self.logger.info( |
787 | - 'Waiting %d seconds for ping response', self.boottimeout) |
788 | - self.pingpoll(timeout=self.boottimeout) |
789 | + 'Waiting %d seconds for ping response', config.boottimeout) |
790 | + self.pingpoll(timeout=config.boottimeout) |
791 | except UTAHTimeout: |
792 | # Ignore timeout for ping, since depending on the network |
793 | # configuration ssh might still work despite of the ping failure. |
794 | self.logger.warning('Network connectivity (ping) failure') |
795 | - self.sshpoll(timeout=self.boottimeout) |
796 | + self.sshpoll(timeout=config.boottimeout) |
797 | self.active = True |
798 | |
799 | |
800 | @@ -614,3 +614,17 @@ |
801 | self.execute( |
802 | "UPDATE machines SET state='destroyed' ""WHERE machineid=?", |
803 | [machineid]) |
804 | + |
805 | + |
806 | +def get_vm(**kw): |
807 | + """Return a Machine object for a VM with the passed in arguments. |
808 | + |
809 | + :param kw: All parameters are passed to the Machine constructor |
810 | + :type kw: dict |
811 | + |
812 | + :returns: Appropriately constructed Machine object |
813 | + :rtype: object |
814 | + |
815 | + """ |
816 | + inventory = TinySQLiteInventory() |
817 | + return inventory.request(**kw) |
818 | |
819 | === modified file 'utah/run.py' |
820 | --- utah/run.py 2013-05-24 14:29:23 +0000 |
821 | +++ utah/run.py 2013-06-11 05:51:28 +0000 |
822 | @@ -329,8 +329,7 @@ |
823 | # Write the machine name to standard out for log gathering |
824 | print('Running on machine: {}'.format(machine.name)) |
825 | |
826 | - extraopts = ('-f {} --install-type {}' |
827 | - .format(args.report_type, machine.installtype)) |
828 | + extraopts = '-f {}'.format(args.report_type) |
829 | |
830 | locallogs = [] |
831 |
nice start. I think _get_machine is still a bit too complex and can be broken up more. Here's a totally untested idea for inspiration:
http:// paste.ubuntu. com/5752180/
By doing that, you also open the door for being able to add some unit testing to each of the new functions. You might even be able to take this idea further and move the _get_physical_ machine type function to /storage/ doanac/ code/utah/ utah/provisioni ng/baremetal/ __init_ _.py