Merge lp:~nuclearbob/utah/baremetal-standardization into lp:utah
- baremetal-standardization
- Merge into dev
Status: | Merged | ||||
---|---|---|---|---|---|
Approved by: | Javier Collado | ||||
Approved revision: | 759 | ||||
Merged at revision: | 777 | ||||
Proposed branch: | lp:~nuclearbob/utah/baremetal-standardization | ||||
Merge into: | lp:utah | ||||
Diff against target: |
630 lines (+157/-167) 9 files modified
examples/run_test_bamboo_feeder.py (+2/-2) examples/run_test_cobbler.py (+2/-2) examples/run_utah_tests.py (+0/-1) utah/config.py (+2/-3) utah/provisioning/baremetal/bamboofeeder.py (+8/-26) utah/provisioning/baremetal/cobbler.py (+85/-110) utah/provisioning/baremetal/power.py (+18/-3) utah/provisioning/inventory/sqlite.py (+17/-17) utah/provisioning/provisioning.py (+23/-3) |
||||
To merge this branch: | bzr merge lp:~nuclearbob/utah/baremetal-standardization | ||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Javier Collado (community) | Approve | ||
Max Brustkern (community) | Needs Resubmitting | ||
Review via email: mp+136502@code.launchpad.net |
Commit message
Description of the change
This branch contains a number of changes I started working on at UDS to simplify and improve the existing bare metal provisioning methods. I'll detail them below.
Max Brustkern (nuclearbob) wrote : | # |
- 754. By Max Brustkern
-
Removed boot argument since the Machine class handles that now
Javier Collado (javier.collado) wrote : | # |
pyflakes returns the following problems:
utah/provisioni
utah/provisioni
First one should really be fixed.
Max Brustkern (nuclearbob) wrote : | # |
Definitely. That's a doozy. I'll go over both of these, and revisit integrating static analysis into my editor, or choosing a new editor that supports it.
- 755. By Max Brustkern
-
Merge default yaml work
Merged latest changes from dev branch - 756. By Max Brustkern
-
Resolved pep8 and pyflakes errors
- 757. By Max Brustkern
-
Moved inventory cleanup after cleanup initialization
- 758. By Max Brustkern
-
Removing direct interaction with nfsfile since we need sudo for that
Max Brustkern (nuclearbob) wrote : | # |
I've fixed static analysis errors and tested this with cobbler in magners, and it seems to be working well.
- 759. By Max Brustkern
-
Renamed Cobbler inventory to more genertic Baremetal
Javier Collado (javier.collado) wrote : | # |
The problems in my previous comment are fixed. Thanks.
(I still need to learn how to test bare metal provisioning at some point).
Preview Diff
1 | === modified file 'examples/run_test_bamboo_feeder.py' | |||
2 | --- examples/run_test_bamboo_feeder.py 2012-12-08 02:10:12 +0000 | |||
3 | +++ examples/run_test_bamboo_feeder.py 2012-12-10 20:39:20 +0000 | |||
4 | @@ -23,7 +23,7 @@ | |||
5 | 23 | from utah.exceptions import UTAHException | 23 | from utah.exceptions import UTAHException |
6 | 24 | from utah.group import check_user_group, print_group_error_message | 24 | from utah.group import check_user_group, print_group_error_message |
7 | 25 | from utah.provisioning.baremetal.bamboofeeder import BambooFeederMachine | 25 | from utah.provisioning.baremetal.bamboofeeder import BambooFeederMachine |
9 | 26 | from utah.provisioning.inventory.sqlite import ManualCobblerSQLiteInventory | 26 | from utah.provisioning.inventory.sqlite import ManualBaremetalSQLiteInventory |
10 | 27 | from utah.run import run_tests | 27 | from utah.run import run_tests |
11 | 28 | from utah.url import url_argument | 28 | from utah.url import url_argument |
12 | 29 | 29 | ||
13 | @@ -81,7 +81,7 @@ | |||
14 | 81 | machine = None | 81 | machine = None |
15 | 82 | 82 | ||
16 | 83 | try: | 83 | try: |
18 | 84 | inventory = ManualCobblerSQLiteInventory( | 84 | inventory = ManualBaremetalSQLiteInventory( |
19 | 85 | db=os.path.join('~', '.utah-bamboofeeder-inventory'), | 85 | db=os.path.join('~', '.utah-bamboofeeder-inventory'), |
20 | 86 | lockfile=os.path.join('~', '.utah-bamboofeeder-lock')) | 86 | lockfile=os.path.join('~', '.utah-bamboofeeder-lock')) |
21 | 87 | kw = {} | 87 | kw = {} |
22 | 88 | 88 | ||
23 | === modified file 'examples/run_test_cobbler.py' | |||
24 | --- examples/run_test_cobbler.py 2012-12-08 02:10:12 +0000 | |||
25 | +++ examples/run_test_cobbler.py 2012-12-10 20:39:20 +0000 | |||
26 | @@ -21,7 +21,7 @@ | |||
27 | 21 | from utah import config | 21 | from utah import config |
28 | 22 | from utah.exceptions import UTAHException | 22 | from utah.exceptions import UTAHException |
29 | 23 | from utah.group import check_user_group, print_group_error_message | 23 | from utah.group import check_user_group, print_group_error_message |
31 | 24 | from utah.provisioning.inventory.sqlite import ManualCobblerSQLiteInventory | 24 | from utah.provisioning.inventory.sqlite import ManualBaremetalSQLiteInventory |
32 | 25 | from utah.run import run_tests | 25 | from utah.run import run_tests |
33 | 26 | from utah.url import url_argument | 26 | from utah.url import url_argument |
34 | 27 | 27 | ||
35 | @@ -95,7 +95,7 @@ | |||
36 | 95 | sys.exit(4) | 95 | sys.exit(4) |
37 | 96 | 96 | ||
38 | 97 | try: | 97 | try: |
40 | 98 | inventory = ManualCobblerSQLiteInventory() | 98 | inventory = ManualBaremetalSQLiteInventory() |
41 | 99 | kw = {} | 99 | kw = {} |
42 | 100 | for arg in ['image', 'preseed', 'rewrite']: | 100 | for arg in ['image', 'preseed', 'rewrite']: |
43 | 101 | value = vars(args)[arg] | 101 | value = vars(args)[arg] |
44 | 102 | 102 | ||
45 | === modified file 'examples/run_utah_tests.py' | |||
46 | --- examples/run_utah_tests.py 2012-12-08 02:10:12 +0000 | |||
47 | +++ examples/run_utah_tests.py 2012-12-10 20:39:20 +0000 | |||
48 | @@ -22,7 +22,6 @@ | |||
49 | 22 | from utah.url import url_argument | 22 | from utah.url import url_argument |
50 | 23 | from utah.group import check_user_group, print_group_error_message | 23 | from utah.group import check_user_group, print_group_error_message |
51 | 24 | from utah.timeout import timeout, UTAHTimeout | 24 | from utah.timeout import timeout, UTAHTimeout |
52 | 25 | from utah import config | ||
53 | 26 | from run_install_test import run_install_test | 25 | from run_install_test import run_install_test |
54 | 27 | 26 | ||
55 | 28 | 27 | ||
56 | 29 | 28 | ||
57 | === modified file 'utah/config.py' | |||
58 | --- utah/config.py 2012-12-10 13:01:20 +0000 | |||
59 | +++ utah/config.py 2012-12-10 20:39:20 +0000 | |||
60 | @@ -99,10 +99,9 @@ | |||
61 | 99 | name=None, | 99 | name=None, |
62 | 100 | # Default setting of installing a new machine vs. using an existing one | 100 | # Default setting of installing a new machine vs. using an existing one |
63 | 101 | new=False, | 101 | new=False, |
64 | 102 | # Command to reload NFS configuration | ||
65 | 103 | # NFS options currently only used for cobbler-based desktop installs | 102 | # NFS options currently only used for cobbler-based desktop installs |
68 | 104 | nfscommand=['sudo', os.path.join('/', 'etc', 'init.d', | 103 | # NFS service command |
69 | 105 | 'nfs-kernel-server'), 'reload'], | 104 | nfscommand=['sudo', 'service', 'nfs-kernel-server'], |
70 | 106 | # Path to NFS config file | 105 | # Path to NFS config file |
71 | 107 | nfsconfigfile=os.path.join('/', 'etc', 'exports'), | 106 | nfsconfigfile=os.path.join('/', 'etc', 'exports'), |
72 | 108 | # Default options for NFS shares | 107 | # Default options for NFS shares |
73 | 109 | 108 | ||
74 | === modified file 'utah/provisioning/baremetal/bamboofeeder.py' | |||
75 | --- utah/provisioning/baremetal/bamboofeeder.py 2012-12-08 02:10:12 +0000 | |||
76 | +++ utah/provisioning/baremetal/bamboofeeder.py 2012-12-10 20:39:20 +0000 | |||
77 | @@ -39,30 +39,20 @@ | |||
78 | 39 | """ | 39 | """ |
79 | 40 | Provide a class to provision an ARM board from a bamboo-feeder setup. | 40 | Provide a class to provision an ARM board from a bamboo-feeder setup. |
80 | 41 | """ | 41 | """ |
82 | 42 | def __init__(self, cargs=None, inventory=None, name=None, powercmd=None, | 42 | def __init__(self, inventory=None, machineinfo=None, name=None, |
83 | 43 | preboot=None, *args, **kw): | 43 | preboot=None, *args, **kw): |
84 | 44 | # TODO: change the name of cargs here and elsewhere | 44 | # TODO: change the name of cargs here and elsewhere |
85 | 45 | # TODO: respect rewrite setting | 45 | # TODO: respect rewrite setting |
86 | 46 | if name is None: | 46 | if name is None: |
87 | 47 | raise UTAHBMProvisioningException( | 47 | raise UTAHBMProvisioningException( |
88 | 48 | 'Machine name reqired for bamboo-feeder machine') | 48 | 'Machine name reqired for bamboo-feeder machine') |
89 | 49 | if powercmd is not None: | ||
90 | 50 | self.powercmd = powercmd | ||
91 | 51 | elif cargs is not None: | ||
92 | 52 | self.power = {} | ||
93 | 53 | for item in cargs: | ||
94 | 54 | if 'power' in item: | ||
95 | 55 | self.power[item] = cargs[item] | ||
96 | 56 | else: | ||
97 | 57 | raise UTAHBMProvisioningException( | ||
98 | 58 | 'No power control information specified') | ||
99 | 59 | try: | 49 | try: |
101 | 60 | self.macaddress = cargs['mac-address'] | 50 | self.macaddress = machineinfo['mac-address'] |
102 | 61 | except AttributeError: | 51 | except AttributeError: |
105 | 62 | raise UTAHBMProvisioningException( | 52 | raise UTAHBMProvisioningException('No MAC address specified') |
104 | 63 | 'No MAC address specified') | ||
106 | 64 | self.inventory = inventory | 53 | self.inventory = inventory |
108 | 65 | super(BambooFeederMachine, self).__init__(*args, name=name, **kw) | 54 | super(BambooFeederMachine, self).__init__(*args, |
109 | 55 | machineinfo=machineinfo, name=name, **kw) | ||
110 | 66 | if self.inventory is not None: | 56 | if self.inventory is not None: |
111 | 67 | self.cleanfunction(self.inventory.release, machine=self) | 57 | self.cleanfunction(self.inventory.release, machine=self) |
112 | 68 | self._depcheck() | 58 | self._depcheck() |
113 | @@ -147,10 +137,10 @@ | |||
114 | 147 | headersize = filesize - datasize | 137 | headersize = filesize - datasize |
115 | 148 | self.logger.debug('uInitrd header size is ' + str(headersize)) | 138 | self.logger.debug('uInitrd header size is ' + str(headersize)) |
116 | 149 | pipe = pipes.Template() | 139 | pipe = pipes.Template() |
118 | 150 | pipe.prepend('dd if=$IN bs=1 skip=' + str(headersize), 'f-') | 140 | pipe.prepend('dd if=$IN bs=1 skip=' + str(headersize) + |
119 | 141 | ' 2>/dev/null', 'f-') | ||
120 | 151 | pipe.append('gunzip', '--') | 142 | pipe.append('gunzip', '--') |
121 | 152 | pipe.append('cpio -ivd 2>/dev/null', '-.') | 143 | pipe.append('cpio -ivd 2>/dev/null', '-.') |
122 | 153 | # TODO: suppress dd output | ||
123 | 154 | pipe.copy(self.initrd, '/dev/null') | 144 | pipe.copy(self.initrd, '/dev/null') |
124 | 155 | 145 | ||
125 | 156 | def _repackinitrd(self): | 146 | def _repackinitrd(self): |
126 | @@ -293,15 +283,7 @@ | |||
127 | 293 | 283 | ||
128 | 294 | self.provisioned = True | 284 | self.provisioned = True |
129 | 295 | self.active = True | 285 | self.active = True |
139 | 296 | # TODO: Make this a method that this and CobblerMachine can call | 286 | self._uuid_check() |
131 | 297 | uuid_check_command = ('[ "{uuid}" == "$(cat /etc/utah/uuid)" ]' | ||
132 | 298 | .format(uuid=self.uuid)) | ||
133 | 299 | if self.run(uuid_check_command)[0] != 0: | ||
134 | 300 | self.provisioned = False | ||
135 | 301 | raise UTAHBMProvisioningException( | ||
136 | 302 | 'Installed UUID differs from Machine UUID; ' | ||
137 | 303 | 'installation probably failed. ' | ||
138 | 304 | 'Try restarting cobbler or running cobbler sync') | ||
140 | 305 | self.logger.info('System installed') | 287 | self.logger.info('System installed') |
141 | 306 | self.cleanfunction(self.run, ( | 288 | self.cleanfunction(self.run, ( |
142 | 307 | 'dd', 'bs=512k', 'count=10', 'if=/dev/zero', 'of=/dev/mmcblk0'), | 289 | 'dd', 'bs=512k', 'count=10', 'if=/dev/zero', 'of=/dev/mmcblk0'), |
143 | 308 | 290 | ||
144 | === modified file 'utah/provisioning/baremetal/cobbler.py' | |||
145 | --- utah/provisioning/baremetal/cobbler.py 2012-12-08 02:10:12 +0000 | |||
146 | +++ utah/provisioning/baremetal/cobbler.py 2012-12-10 20:39:20 +0000 | |||
147 | @@ -21,6 +21,7 @@ | |||
148 | 21 | import os | 21 | import os |
149 | 22 | import pipes | 22 | import pipes |
150 | 23 | import shutil | 23 | import shutil |
151 | 24 | import subprocess | ||
152 | 24 | import tempfile | 25 | import tempfile |
153 | 25 | import time | 26 | import time |
154 | 26 | 27 | ||
155 | @@ -39,31 +40,28 @@ | |||
156 | 39 | """ | 40 | """ |
157 | 40 | Provide a class to provision a machine via cobbler. | 41 | Provide a class to provision a machine via cobbler. |
158 | 41 | """ | 42 | """ |
162 | 42 | # TODO: may need to fix DHCP/hostname issues, may just be magners | 43 | def __init__(self, inventory=None, machineinfo=None, |
163 | 43 | def __init__(self, cargs=None, inventory=None, name=None, preseed=None, | 44 | name=None, *args, **kw): |
164 | 44 | *args, **kw): | 45 | # TODO: support for reusing existing machines |
165 | 45 | if name is None: | 46 | if name is None: |
166 | 46 | raise UTAHBMProvisioningException( | 47 | raise UTAHBMProvisioningException( |
167 | 47 | 'Machine name reqired for cobbler machine') | 48 | 'Machine name reqired for cobbler machine') |
169 | 48 | if cargs is None: | 49 | if machineinfo is None: |
170 | 49 | raise UTAHBMProvisioningException( | 50 | raise UTAHBMProvisioningException( |
171 | 50 | 'No cobbler arguments given for machine creation') | 51 | 'No cobbler arguments given for machine creation') |
172 | 51 | else: | 52 | else: |
174 | 52 | self.cargs = cargs | 53 | self.machineinfo = machineinfo |
175 | 53 | self.power = {} | 54 | self.power = {} |
176 | 54 | for item in cargs: | ||
177 | 55 | if 'power' in item: | ||
178 | 56 | self.power[item] = cargs[item] | ||
179 | 57 | self.inventory = inventory | 55 | self.inventory = inventory |
184 | 58 | if preseed is None: | 56 | super(CobblerMachine, self).__init__(*args, machineinfo=machineinfo, |
185 | 59 | preseed = '/etc/utah/default-preseed.cfg' | 57 | name=name, **kw) |
186 | 60 | super(CobblerMachine, self).__init__(*args, name=name, | 58 | if self.inventory is not None: |
187 | 61 | preseed=preseed, **kw) | 59 | self.cleanfunction(self.inventory.release, machine=self) |
188 | 62 | if self.image is None: | 60 | if self.image is None: |
189 | 63 | raise UTAHBMProvisioningException( | 61 | raise UTAHBMProvisioningException( |
190 | 64 | 'Image file required for cobbler installation') | 62 | 'Image file required for cobbler installation') |
191 | 65 | self._custominit() | 63 | self._custominit() |
193 | 66 | # TODO: verify we have nfs support for desktop image | 64 | self._depcheck() |
194 | 67 | 65 | ||
195 | 68 | # TODO: Rework cinitrd to be less of a confusing collection of kludges | 66 | # TODO: Rework cinitrd to be less of a confusing collection of kludges |
196 | 69 | self.cinitrd = None | 67 | self.cinitrd = None |
197 | @@ -79,13 +77,27 @@ | |||
198 | 79 | self.logger.debug('Cobbler arch is ' + self.carch) | 77 | self.logger.debug('Cobbler arch is ' + self.carch) |
199 | 80 | self.logger.debug('Cobbler machine init finished') | 78 | self.logger.debug('Cobbler machine init finished') |
200 | 81 | 79 | ||
202 | 82 | def _provision(self, checktimeout=config.checktimeout, | 80 | def _load(self): |
203 | 81 | """ | ||
204 | 82 | Verify the machine is in cobbler. | ||
205 | 83 | """ | ||
206 | 84 | # TODO: consider reworking _cobble to provide this | ||
207 | 85 | # (only if we'll be using cobbler for a while longer) | ||
208 | 86 | machines = subprocess.check_output(['sudo', 'cobbler', 'system', | ||
209 | 87 | 'find']).splitlines() | ||
210 | 88 | if self.name not in machines: | ||
211 | 89 | raise UTAHBMProvisioningException('No machine named ' + self.name | ||
212 | 90 | + ' exists in cobbler') | ||
213 | 91 | else: | ||
214 | 92 | return True | ||
215 | 93 | |||
216 | 94 | def _create(self, checktimeout=config.checktimeout, | ||
217 | 83 | installtimeout=config.installtimeout): | 95 | installtimeout=config.installtimeout): |
218 | 84 | """ | 96 | """ |
219 | 85 | Install a machine. | 97 | Install a machine. |
220 | 86 | """ | 98 | """ |
221 | 87 | # TODO: support for reusing existing machines | ||
222 | 88 | self.tmpdir = tempfile.mkdtemp(prefix='/tmp/' + self.name + '_') | 99 | self.tmpdir = tempfile.mkdtemp(prefix='/tmp/' + self.name + '_') |
223 | 100 | self.cleanfile(self.tmpdir) | ||
224 | 89 | os.chdir(self.tmpdir) | 101 | os.chdir(self.tmpdir) |
225 | 90 | 102 | ||
226 | 91 | # TODO: try to remove this step, mount the iso and import that | 103 | # TODO: try to remove this step, mount the iso and import that |
227 | @@ -102,6 +114,7 @@ | |||
228 | 102 | initrd = self._prepareinitrd(initrd=cinitrd) | 114 | initrd = self._prepareinitrd(initrd=cinitrd) |
229 | 103 | self._unpackinitrd(initrd=initrd) | 115 | self._unpackinitrd(initrd=initrd) |
230 | 104 | self._setuplatecommand() | 116 | self._setuplatecommand() |
231 | 117 | # TODO: see if this is needed for all quantal desktops | ||
232 | 105 | if self.installtype == 'desktop': | 118 | if self.installtype == 'desktop': |
233 | 106 | self.logger.info('Configuring latecommand for desktop') | 119 | self.logger.info('Configuring latecommand for desktop') |
234 | 107 | myfile = open(os.path.join(self.tmpdir, 'initrd.d', | 120 | myfile = open(os.path.join(self.tmpdir, 'initrd.d', |
235 | @@ -118,18 +131,12 @@ | |||
236 | 118 | shutil.copyfile(initrd, cinitrd) | 131 | shutil.copyfile(initrd, cinitrd) |
237 | 119 | 132 | ||
238 | 120 | self.logger.info('Setting up system with cobbler') | 133 | self.logger.info('Setting up system with cobbler') |
245 | 121 | self.logger.debug('Removing old system') | 134 | self._cleanupcobbler() |
240 | 122 | self._cobble(['system', 'remove', '--name=' + self.name]) | ||
241 | 123 | self.logger.info('Removing old profile') | ||
242 | 124 | self._cobble(['profile', 'remove', '--name=' + self.cname]) | ||
243 | 125 | self.logger.info('Removing old distro') | ||
244 | 126 | self._cobble(['distro', 'remove', '--name=' + self.cname]) | ||
246 | 127 | 135 | ||
247 | 128 | preseed = os.path.join(self.tmpdir, 'initrd.d', 'preseed.cfg') | 136 | preseed = os.path.join(self.tmpdir, 'initrd.d', 'preseed.cfg') |
248 | 129 | 137 | ||
249 | 130 | if self.installtype in ['alternate', 'server']: | 138 | if self.installtype in ['alternate', 'server']: |
252 | 131 | # TODO: support more image types, | 139 | # TODO: maybe do this without unpacking ISO |
251 | 132 | # maybe do this without unpacking ISO | ||
253 | 133 | self.logger.info('Importing image') | 140 | self.logger.info('Importing image') |
254 | 134 | self._cobble(['import', '--name=' + self.cname, | 141 | self._cobble(['import', '--name=' + self.cname, |
255 | 135 | '--path=' + self.tmpdir, '--arch=' + self.carch]) | 142 | '--path=' + self.tmpdir, '--arch=' + self.carch]) |
256 | @@ -144,16 +151,14 @@ | |||
257 | 144 | if self.installtype == 'desktop': | 151 | if self.installtype == 'desktop': |
258 | 145 | self.logger.info('Setting up NFS for desktop install') | 152 | self.logger.info('Setting up NFS for desktop install') |
259 | 146 | self.logger.debug('Adding export to NFS config file') | 153 | self.logger.debug('Adding export to NFS config file') |
263 | 147 | # nfsfile = open(config.nfsconfigfile, 'a') | 154 | self.cleanfunction(self._removenfs) |
261 | 148 | # nfsfile.write("\n" + self.tmpdir + ' ' + config.nfsoptions + "\n") | ||
262 | 149 | # nfsfile.close() | ||
264 | 150 | pipe = pipes.Template() | 155 | pipe = pipes.Template() |
265 | 151 | pipe.append('sudo tee -a ' + str(config.nfsconfigfile) + | 156 | pipe.append('sudo tee -a ' + str(config.nfsconfigfile) + |
266 | 152 | ' >/dev/null', '-.') | 157 | ' >/dev/null', '-.') |
267 | 153 | pipe.open('/dev/null', 'w').write(self.tmpdir + ' ' + | 158 | pipe.open('/dev/null', 'w').write(self.tmpdir + ' ' + |
268 | 154 | config.nfsoptions + "\n") | 159 | config.nfsoptions + "\n") |
269 | 155 | self.logger.debug('Reloading NFS config') | 160 | self.logger.debug('Reloading NFS config') |
271 | 156 | self._runargs(config.nfscommand) | 161 | self._runargs(config.nfscommand + ['reload']) |
272 | 157 | self.logger.debug('Adding NFS boot options') | 162 | self.logger.debug('Adding NFS boot options') |
273 | 158 | try: | 163 | try: |
274 | 159 | iface = config.nfsiface | 164 | iface = config.nfsiface |
275 | @@ -164,6 +169,7 @@ | |||
276 | 164 | ip + ':' + self.tmpdir) | 169 | ip + ':' + self.tmpdir) |
277 | 165 | self.cmdline = self.cmdline.strip() | 170 | self.cmdline = self.cmdline.strip() |
278 | 166 | 171 | ||
279 | 172 | self.cleanfunction(self._cleanupcobbler) | ||
280 | 167 | self.logger.info('Adding kernel boot options to distro') | 173 | self.logger.info('Adding kernel boot options to distro') |
281 | 168 | self._cobble(['distro', 'edit', '--name=' + self.cname, | 174 | self._cobble(['distro', 'edit', '--name=' + self.cname, |
282 | 169 | '--kopts=' + self.cmdline]) | 175 | '--kopts=' + self.cmdline]) |
283 | @@ -175,17 +181,9 @@ | |||
284 | 175 | self.logger.info('Adding system') | 181 | self.logger.info('Adding system') |
285 | 176 | self._cobble(['system', 'add', '--name=' + self.name, | 182 | self._cobble(['system', 'add', '--name=' + self.name, |
286 | 177 | '--profile=' + self.cname, '--netboot-enabled=Y'] | 183 | '--profile=' + self.cname, '--netboot-enabled=Y'] |
289 | 178 | + ['--' + arg + '=' + self.cargs[arg] | 184 | + ['--' + arg + '=' + self.machineinfo[arg] |
290 | 179 | for arg in self.cargs]) | 185 | for arg in self.machineinfo]) |
291 | 180 | 186 | ||
292 | 181 | # Things seem to be working fine without this, | ||
293 | 182 | # but sometimes things still get broken | ||
294 | 183 | #self.logger.info('Syncing cobbler') | ||
295 | 184 | #self._cobble(['sync']) | ||
296 | 185 | try: | ||
297 | 186 | self._runargs(config.cobblerfix) | ||
298 | 187 | except AttributeError: | ||
299 | 188 | pass | ||
300 | 189 | self.restart() | 187 | self.restart() |
301 | 190 | 188 | ||
302 | 191 | self.logger.info('Waiting for installation to begin') | 189 | self.logger.info('Waiting for installation to begin') |
303 | @@ -204,45 +202,26 @@ | |||
304 | 204 | self.logger.info('Removing NFS share') | 202 | self.logger.info('Removing NFS share') |
305 | 205 | self._removenfs() | 203 | self._removenfs() |
306 | 206 | 204 | ||
307 | 207 | if self.debug: | ||
308 | 208 | self.logger.info('Leaving temp directory ' | ||
309 | 209 | 'because debug is enabled: ' + self.tmpdir) | ||
310 | 210 | else: | ||
311 | 211 | self.logger.info('Cleaning up temp directory') | ||
312 | 212 | self._cleanuptmpdir() | ||
313 | 213 | |||
314 | 214 | self.provisioned = True | 205 | self.provisioned = True |
315 | 215 | self.active = True | 206 | self.active = True |
324 | 216 | uuid_check_command = ('[ "{uuid}" == "$(cat /etc/utah/uuid)" ]' | 207 | self._uuid_check() |
317 | 217 | .format(uuid=self.uuid)) | ||
318 | 218 | if self.run(uuid_check_command)[0] != 0: | ||
319 | 219 | self.provisioned = False | ||
320 | 220 | raise UTAHBMProvisioningException( | ||
321 | 221 | 'Installed UUID differs from CobblerMachine UUID; ' | ||
322 | 222 | 'installation probably failed. ' | ||
323 | 223 | 'Try restarting cobbler or running cobbler sync') | ||
325 | 224 | self.logger.info('System installed') | 208 | self.logger.info('System installed') |
326 | 225 | return True | 209 | return True |
327 | 226 | 210 | ||
339 | 227 | def destroy(self, *args, **kw): | 211 | def _cobble(self, cmd, failok=False): |
329 | 228 | """ | ||
330 | 229 | Destroy the machine, and call super to release it in the inventory. | ||
331 | 230 | """ | ||
332 | 231 | # TODO: Remove the machine from cobbler | ||
333 | 232 | # TODO: Consider actually wrecking the install | ||
334 | 233 | super(CobblerMachine, self).destroy(*args, **kw) | ||
335 | 234 | self.__del__() | ||
336 | 235 | del self | ||
337 | 236 | |||
338 | 237 | def _cobble(self, cmd): | ||
340 | 238 | """ | 212 | """ |
341 | 239 | Pull out all cobbler commands so that we can later support changing | 213 | Pull out all cobbler commands so that we can later support changing |
342 | 240 | users, remote login, not sudo, etc. | 214 | users, remote login, not sudo, etc. |
343 | 241 | """ | 215 | """ |
345 | 242 | if self._runargs(['sudo', 'cobbler'] + cmd) != 0: | 216 | retcode = self._runargs(['sudo', 'cobbler'] + cmd) |
346 | 217 | if retcode != 0: | ||
347 | 243 | error_msg = 'Cobbler command failed: ' + ' '.join(cmd) | 218 | error_msg = 'Cobbler command failed: ' + ' '.join(cmd) |
350 | 244 | self.logger.error(error_msg) | 219 | if failok: |
351 | 245 | raise UTAHBMProvisioningException(error_msg) | 220 | self.logger.debug(error_msg) |
352 | 221 | return retcode | ||
353 | 222 | else: | ||
354 | 223 | self.logger.error(error_msg) | ||
355 | 224 | raise UTAHBMProvisioningException(error_msg) | ||
356 | 246 | 225 | ||
357 | 247 | def _start(self): | 226 | def _start(self): |
358 | 248 | """ | 227 | """ |
359 | @@ -250,9 +229,11 @@ | |||
360 | 250 | If that fails, use the powercommand generated by PowerMachine. | 229 | If that fails, use the powercommand generated by PowerMachine. |
361 | 251 | """ | 230 | """ |
362 | 252 | self.logger.debug('Starting system') | 231 | self.logger.debug('Starting system') |
366 | 253 | try: | 232 | retcode = self._cobble(['system', 'poweron', '--name=' + self.name], |
367 | 254 | self._cobble(['system', 'poweron', '--name=' + self.name]) | 233 | failok=True) |
368 | 255 | except UTAHBMProvisioningException: | 234 | if retcode != 0: |
369 | 235 | self.logger.info('Cobbler power command failed; falling back to ' | ||
370 | 236 | 'manual command') | ||
371 | 256 | self._runargs(self.powercommand() + ['on']) | 237 | self._runargs(self.powercommand() + ['on']) |
372 | 257 | self.active = True | 238 | self.active = True |
373 | 258 | 239 | ||
374 | @@ -262,52 +243,46 @@ | |||
375 | 262 | If that fails, use the powercommand generated by PowerMachine. | 243 | If that fails, use the powercommand generated by PowerMachine. |
376 | 263 | """ | 244 | """ |
377 | 264 | self.logger.debug('Stopping system') | 245 | self.logger.debug('Stopping system') |
381 | 265 | try: | 246 | retcode = self._cobble(['system', 'poweroff', '--name=' + self.name], |
382 | 266 | self._cobble(['system', 'poweroff', '--name=' + self.name]) | 247 | failok=True) |
383 | 267 | except UTAHBMProvisioningException: | 248 | if retcode != 0: |
384 | 249 | self.logger.info('Cobbler power command failed; falling back to ' | ||
385 | 250 | 'manual command') | ||
386 | 268 | self._runargs(self.powercommand() + ['off']) | 251 | self._runargs(self.powercommand() + ['off']) |
387 | 269 | self.active = False | 252 | self.active = False |
388 | 270 | 253 | ||
389 | 271 | def __del__(self): | ||
390 | 272 | """ | ||
391 | 273 | If the machine has an inventory, tell the inventory to release the | ||
392 | 274 | machine when the object is deleted. | ||
393 | 275 | If we exported a directory over NFS, remove it. | ||
394 | 276 | """ | ||
395 | 277 | if self.installtype == 'desktop': | ||
396 | 278 | self._removenfs() | ||
397 | 279 | if not self.debug: | ||
398 | 280 | self._cleanuptmpdir() | ||
399 | 281 | if self.inventory is not None: | ||
400 | 282 | self.inventory.release(machine=self) | ||
401 | 283 | |||
402 | 284 | def _removenfs(self): | 254 | def _removenfs(self): |
403 | 255 | """ | ||
404 | 256 | Remove our NFS configuration and reload NFS. | ||
405 | 257 | """ | ||
406 | 285 | self.logger.info('Removing NFS share') | 258 | self.logger.info('Removing NFS share') |
407 | 286 | self._runargs(['sudo', 'sed', '/' + self.tmpdir.replace('/', '\/') + | 259 | self._runargs(['sudo', 'sed', '/' + self.tmpdir.replace('/', '\/') + |
408 | 287 | '/d', '-i', config.nfsconfigfile]) | 260 | '/d', '-i', config.nfsconfigfile]) |
409 | 288 | self.logger.debug('Reloading NFS config') | 261 | self.logger.debug('Reloading NFS config') |
435 | 289 | self._runargs(config.nfscommand) | 262 | self._runargs(config.nfscommand + ['reload']) |
436 | 290 | 263 | ||
437 | 291 | def _cleanuptmpdir(self): | 264 | def _depcheck(self): |
438 | 292 | if os.path.isdir(self.tmpdir): | 265 | """ |
439 | 293 | # Cribbed from http://svn.python.org | 266 | Check for NFS if installtype is desktop. |
440 | 294 | # /projects/python/trunk/Mac/BuildScript/build-installer.py | 267 | """ |
441 | 295 | for dirpath, dirnames, filenames in os.walk(self.tmpdir): | 268 | super(CobblerMachine, self)._depcheck() |
442 | 296 | for name in (dirnames + filenames): | 269 | if self.installtype == 'desktop': |
443 | 297 | absolute_name = os.path.join(dirpath, name) | 270 | cmd = config.nfscommand + ['status'] |
444 | 298 | if not os.path.islink(absolute_name): | 271 | if self._runargs(cmd) != 0: |
445 | 299 | self.logger.debug('Changing permissions of ' | 272 | raise UTAHBMProvisioningException('NFS needed for desktop' |
446 | 300 | + absolute_name) | 273 | ' install') |
447 | 301 | os.chmod(absolute_name, 0775) | 274 | if not os.path.isfile(config.nfsconfigfile): |
448 | 302 | shutil.rmtree(self.tmpdir) | 275 | raise UTAHBMProvisioningException('NFS config file: ' + |
449 | 303 | 276 | config.nfsconfigfile + ' not available') | |
450 | 304 | def activecheck(self): | 277 | |
451 | 305 | """ | 278 | def _cleanupcobbler(self): |
452 | 306 | Start the machine if needed, and check for SSH login. | 279 | """ |
453 | 307 | """ | 280 | Remove our system, profile, and distro from Cobbler. |
454 | 308 | # TODO: Consider moving this to SSHMixin | 281 | """ |
455 | 309 | self.logger.debug('Checking if machine is active') | 282 | self.logger.debug('Removing old system') |
456 | 310 | self.provisioncheck() | 283 | self._cobble(['system', 'remove', '--name=' + self.name], failok=True) |
457 | 311 | if not self.active: | 284 | self.logger.info('Removing old profile') |
458 | 312 | self._start() | 285 | self._cobble(['profile', 'remove', '--name=' + self.cname], |
459 | 313 | self.sshcheck() | 286 | failok=True) |
460 | 287 | self.logger.info('Removing old distro') | ||
461 | 288 | self._cobble(['distro', 'remove', '--name=' + self.cname], failok=True) | ||
462 | 314 | 289 | ||
463 | === modified file 'utah/provisioning/baremetal/power.py' | |||
464 | --- utah/provisioning/baremetal/power.py 2012-12-08 02:10:12 +0000 | |||
465 | +++ utah/provisioning/baremetal/power.py 2012-12-10 20:39:20 +0000 | |||
466 | @@ -28,7 +28,23 @@ | |||
467 | 28 | """ | 28 | """ |
468 | 29 | Provide power cycle commands for the Sentry CDU. | 29 | Provide power cycle commands for the Sentry CDU. |
469 | 30 | """ | 30 | """ |
470 | 31 | def __init__(self, machineinfo=None, powercmd=None, *args, **kw): | ||
471 | 32 | """ | ||
472 | 33 | Store power control info for later use. | ||
473 | 34 | """ | ||
474 | 35 | if powercmd is not None: | ||
475 | 36 | self.powercmd = powercmd | ||
476 | 37 | elif machineinfo is not None: | ||
477 | 38 | self.power = {} | ||
478 | 39 | for item in machineinfo: | ||
479 | 40 | if 'power' in item: | ||
480 | 41 | self.power[item] = machineinfo[item] | ||
481 | 42 | super(PowerMixin, self).__init__(*args, **kw) | ||
482 | 43 | |||
483 | 31 | def powercommand(self): | 44 | def powercommand(self): |
484 | 45 | """ | ||
485 | 46 | Return the command used to control power for this machine. | ||
486 | 47 | """ | ||
487 | 32 | try: | 48 | try: |
488 | 33 | cmd = self.powercmd | 49 | cmd = self.powercmd |
489 | 34 | except AttributeError: | 50 | except AttributeError: |
490 | @@ -41,9 +57,8 @@ | |||
491 | 41 | '-p', self.power['power-pass'], | 57 | '-p', self.power['power-pass'], |
492 | 42 | '-o'] | 58 | '-o'] |
493 | 43 | else: | 59 | else: |
497 | 44 | raise UTAHProvisioningException('Power type ' + | 60 | raise UTAHBMProvisioningException('Power type ' + |
498 | 45 | self.power['power-type'] + | 61 | self.power['power-type'] + ' not supported') |
496 | 46 | ' not supported') | ||
499 | 47 | except AttributeError: | 62 | except AttributeError: |
500 | 48 | raise UTAHBMProvisioningException('Power commands not defined' | 63 | raise UTAHBMProvisioningException('Power commands not defined' |
501 | 49 | ' for this machine') | 64 | ' for this machine') |
502 | 50 | 65 | ||
503 | === modified file 'utah/provisioning/inventory/sqlite.py' | |||
504 | --- utah/provisioning/inventory/sqlite.py 2012-12-08 02:10:12 +0000 | |||
505 | +++ utah/provisioning/inventory/sqlite.py 2012-12-10 20:39:20 +0000 | |||
506 | @@ -93,7 +93,7 @@ | |||
507 | 93 | return False | 93 | return False |
508 | 94 | 94 | ||
509 | 95 | 95 | ||
511 | 96 | class ManualCobblerSQLiteInventory(SQLiteInventory): | 96 | class ManualBaremetalSQLiteInventory(SQLiteInventory): |
512 | 97 | """ | 97 | """ |
513 | 98 | Keep an inventory of manually entered machines for use with cobbler. | 98 | Keep an inventory of manually entered machines for use with cobbler. |
514 | 99 | All columns other than machineid, name, and state are assumed to be | 99 | All columns other than machineid, name, and state are assumed to be |
515 | @@ -107,7 +107,7 @@ | |||
516 | 107 | if not os.path.isfile(db): | 107 | if not os.path.isfile(db): |
517 | 108 | raise UTAHProvisioningInventoryException( | 108 | raise UTAHProvisioningInventoryException( |
518 | 109 | 'No machine database found at ' + db) | 109 | 'No machine database found at ' + db) |
520 | 110 | super(ManualCobblerSQLiteInventory, self).__init__(*args, db=db, | 110 | super(ManualBaremetalSQLiteInventory, self).__init__(*args, db=db, |
521 | 111 | lockfile=lockfile, | 111 | lockfile=lockfile, |
522 | 112 | **kw) | 112 | **kw) |
523 | 113 | machines_count = (self.connection | 113 | machines_count = (self.connection |
524 | @@ -128,37 +128,37 @@ | |||
525 | 128 | raise UTAHProvisioningInventoryException( | 128 | raise UTAHProvisioningInventoryException( |
526 | 129 | 'No machines meet criteria') | 129 | 'No machines meet criteria') |
527 | 130 | else: | 130 | else: |
535 | 131 | for machineinfo in result: | 131 | for minfo in result: |
536 | 132 | cargs = dict(machineinfo) | 132 | machineinfo = dict(minfo) |
537 | 133 | if cargs['state'] == 'available': | 133 | if machineinfo['state'] == 'available': |
538 | 134 | return self._take(cargs, *args, **kw) | 134 | return self._take(machineinfo, *args, **kw) |
539 | 135 | for machineinfo in result: | 135 | for minfo in result: |
540 | 136 | cargs = dict(machineinfo) | 136 | machineinfo = dict(minfo) |
541 | 137 | pid = cargs['pid'] | 137 | pid = machineinfo['pid'] |
542 | 138 | try: | 138 | try: |
543 | 139 | if not (psutil.pid_exists(pid) and ('utah' in | 139 | if not (psutil.pid_exists(pid) and ('utah' in |
544 | 140 | ' '.join(psutil.Process(pid).cmdline) | 140 | ' '.join(psutil.Process(pid).cmdline) |
545 | 141 | or 'run_test_cobbler.py' in | 141 | or 'run_test_cobbler.py' in |
546 | 142 | ' '.join(psutil.Process(pid).cmdline))): | 142 | ' '.join(psutil.Process(pid).cmdline))): |
548 | 143 | return self._take(cargs, *args, **kw) | 143 | return self._take(machineinfo, *args, **kw) |
549 | 144 | except ValueError: | 144 | except ValueError: |
550 | 145 | continue | 145 | continue |
551 | 146 | 146 | ||
552 | 147 | raise UTAHProvisioningInventoryException( | 147 | raise UTAHProvisioningInventoryException( |
553 | 148 | 'All machines meeting criteria are currently unavailable') | 148 | 'All machines meeting criteria are currently unavailable') |
554 | 149 | 149 | ||
560 | 150 | def _take(self, cargs, *args, **kw): | 150 | def _take(self, machineinfo, *args, **kw): |
561 | 151 | machineid = cargs.pop('machineid') | 151 | machineid = machineinfo.pop('machineid') |
562 | 152 | name = cargs.pop('name') | 152 | name = machineinfo.pop('name') |
563 | 153 | state = cargs.pop('state') | 153 | state = machineinfo.pop('state') |
564 | 154 | pid = cargs.pop('pid') | 154 | machineinfo.pop('pid') |
565 | 155 | update = self.connection.execute( | 155 | update = self.connection.execute( |
566 | 156 | "UPDATE machines SET pid=?, state='provisioned' WHERE machineid=?" | 156 | "UPDATE machines SET pid=?, state='provisioned' WHERE machineid=?" |
567 | 157 | "AND state=?", | 157 | "AND state=?", |
568 | 158 | [os.getpid(), machineid, state]).rowcount | 158 | [os.getpid(), machineid, state]).rowcount |
569 | 159 | if update == 1: | 159 | if update == 1: |
572 | 160 | machine = CobblerMachine(*args, cargs=cargs, | 160 | machine = CobblerMachine(*args, inventory=self, |
573 | 161 | inventory=self, name=name, **kw) | 161 | machineinfo=machineinfo, name=name, **kw) |
574 | 162 | self.machines.append(machine) | 162 | self.machines.append(machine) |
575 | 163 | return machine | 163 | return machine |
576 | 164 | elif update == 0: | 164 | elif update == 0: |
577 | 165 | 165 | ||
578 | === modified file 'utah/provisioning/provisioning.py' | |||
579 | --- utah/provisioning/provisioning.py 2012-12-06 16:30:34 +0000 | |||
580 | +++ utah/provisioning/provisioning.py 2012-12-10 20:39:20 +0000 | |||
581 | @@ -454,12 +454,12 @@ | |||
582 | 454 | from libvirt if it is registered there. | 454 | from libvirt if it is registered there. |
583 | 455 | For a physical machine, this should free it up to be used by another | 455 | For a physical machine, this should free it up to be used by another |
584 | 456 | process in the future, and destroy sensitive data if any exists. | 456 | process in the future, and destroy sensitive data if any exists. |
588 | 457 | Destroying the install on a physical machine is optional, but it | 457 | Destroying the install on a physical machine is optional. |
586 | 458 | should be powered off if remote power management is available. | ||
587 | 459 | Now returns False by default to work with multiple inheritance. | ||
589 | 460 | """ | 458 | """ |
590 | 459 | self.provisioned = False | ||
591 | 461 | self.__del__() | 460 | self.__del__() |
592 | 462 | del self | 461 | del self |
593 | 462 | return True | ||
594 | 463 | 463 | ||
595 | 464 | def _load(self): | 464 | def _load(self): |
596 | 465 | """ | 465 | """ |
597 | @@ -664,6 +664,16 @@ | |||
598 | 664 | def __del__(self): | 664 | def __del__(self): |
599 | 665 | self.cleanup() | 665 | self.cleanup() |
600 | 666 | 666 | ||
601 | 667 | def _uuid_check(self): | ||
602 | 668 | uuid_check_command = ('[ "{uuid}" == "$(cat /etc/utah/uuid)" ]' | ||
603 | 669 | .format(uuid=self.uuid)) | ||
604 | 670 | if self.run(uuid_check_command)[0] != 0: | ||
605 | 671 | self.provisioned = False | ||
606 | 672 | raise UTAHProvisioningException( | ||
607 | 673 | 'Installed UUID differs from Machine UUID; ' | ||
608 | 674 | 'installation probably failed. ' | ||
609 | 675 | 'Try rerunning install.') | ||
610 | 676 | |||
611 | 667 | 677 | ||
612 | 668 | class SSHMixin(object): | 678 | class SSHMixin(object): |
613 | 669 | """ | 679 | """ |
614 | @@ -885,6 +895,16 @@ | |||
615 | 885 | utah.timeout.timeout(timeout, retry, self.sshcheck, checktimeout, | 895 | utah.timeout.timeout(timeout, retry, self.sshcheck, checktimeout, |
616 | 886 | logmethod=logmethod) | 896 | logmethod=logmethod) |
617 | 887 | 897 | ||
618 | 898 | def activecheck(self): | ||
619 | 899 | """ | ||
620 | 900 | Start the machine if needed, and check for SSH login. | ||
621 | 901 | """ | ||
622 | 902 | self.logger.debug('Checking if machine is active') | ||
623 | 903 | self.provisioncheck() | ||
624 | 904 | if not self.active: | ||
625 | 905 | self._start() | ||
626 | 906 | self.sshcheck() | ||
627 | 907 | |||
628 | 888 | 908 | ||
629 | 889 | class CustomInstallMixin(object): | 909 | class CustomInstallMixin(object): |
630 | 890 | """ | 910 | """ |
CobblerMachine now uses the standardized cleanup functions, and removes the machine from cobbler as part of that. I improved some docstrings. I added an option to the _cobbler method of CobblerMachine to not raise an exception on failure for commands that may fail sometimes (i.e. power control and pre-emptively deleting the system info from cobbler.) I removed an old cobbler sync command that has been commented out for a while and the obsolete cobblerfix workaround for the lab. I moved processing of power-related arguments from the two separate machines into a new __init__ function for the PowerMixin. I cleaned up some indentation on exceptions. I changed the pipe involed with uInitrd manipulation to attempt to suppress the standard error of dd.