Merge lp:~barry/ubuntu-system-image/citrain-2.1 into lp:~ubuntu-managed-branches/ubuntu-system-image/system-image
- citrain-2.1
- Merge into system-image
Status: | Merged | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Merged at revision: | 229 | ||||||||||||||||
Proposed branch: | lp:~barry/ubuntu-system-image/citrain-2.1 | ||||||||||||||||
Merge into: | lp:~ubuntu-managed-branches/ubuntu-system-image/system-image | ||||||||||||||||
Diff against target: |
2123 lines (+901/-146) 65 files modified
MANIFEST.in (+1/-0) NEWS.rst (+48/-1) PKG-INFO (+1/-1) cli-manpage.rst (+1/-1) dbus-manpage.rst (+1/-1) debian/changelog (+32/-0) debian/control (+5/-4) debian/patches/01_send_ack_on_applyupdate.diff (+0/-16) debian/patches/lp1284217.patch (+106/-0) debian/patches/series (+1/-1) debian/rules (+1/-0) ini-manpage.rst (+1/-1) setup.cfg (+2/-2) setup.py (+1/-1) system_image.egg-info/PKG-INFO (+1/-1) system_image.egg-info/SOURCES.txt (+1/-1) systemimage/api.py (+5/-1) systemimage/bag.py (+1/-1) systemimage/bindings.py (+1/-1) systemimage/candidates.py (+1/-1) systemimage/channel.py (+1/-1) systemimage/config.py (+1/-1) systemimage/dbus.py (+51/-16) systemimage/device.py (+1/-1) systemimage/docs/conf.py (+1/-1) systemimage/download.py (+36/-2) systemimage/gpg.py (+95/-1) systemimage/helpers.py (+1/-1) systemimage/image.py (+1/-1) systemimage/index.py (+1/-1) systemimage/keyring.py (+3/-4) systemimage/logging.py (+29/-8) systemimage/main.py (+1/-1) systemimage/reactor.py (+1/-1) systemimage/reboot.py (+1/-1) systemimage/scores.py (+1/-1) systemimage/service.py (+14/-6) systemimage/settings.py (+1/-1) systemimage/state.py (+8/-10) systemimage/testing/controller.py (+17/-8) systemimage/testing/dbus.py (+8/-2) systemimage/testing/demo.py (+1/-1) systemimage/testing/helpers.py (+25/-7) systemimage/testing/nose.py (+21/-2) systemimage/tests/data/config_03.ini (+1/-1) systemimage/tests/data/index_24.json (+36/-0) systemimage/tests/test_api.py (+1/-1) systemimage/tests/test_bag.py (+1/-1) systemimage/tests/test_candidates.py (+1/-1) systemimage/tests/test_channel.py (+1/-1) systemimage/tests/test_config.py (+1/-1) systemimage/tests/test_dbus.py (+98/-8) systemimage/tests/test_download.py (+1/-1) systemimage/tests/test_gpg.py (+175/-2) systemimage/tests/test_helpers.py (+1/-1) systemimage/tests/test_image.py (+1/-1) systemimage/tests/test_index.py (+1/-1) systemimage/tests/test_keyring.py (+20/-5) systemimage/tests/test_main.py (+25/-1) systemimage/tests/test_scores.py (+1/-1) systemimage/tests/test_settings.py (+1/-1) systemimage/tests/test_state.py (+1/-1) systemimage/tests/test_winner.py (+1/-1) systemimage/version.txt (+1/-1) tox.ini (+1/-1) |
||||||||||||||||
To merge this branch: | bzr merge lp:~barry/ubuntu-system-image/citrain-2.1 | ||||||||||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Stéphane Graber | Pending | ||
Review via email: mp+207702@code.launchpad.net |
Commit message
New upstream release, along with some packaging changes. See changelog and NEWS.rst for details.
Description of the change
[ Stéphane Graber ]
* New upstream release.
* Set X-Auto-Uploader to no-rewrite-version
* Set Vcs-Bzr to the new target branch
[ Barry Warsaw ]
* New upstream release.
- LP: #1279056 - Internal improvements to SignatureError for
better debugging.
- LP: #1277589 - Better protection against race conditions.
- LP: #1260768 - Return empty string from ApplyUpdate D-Bus method.
- Request ubuntu-
with atomic rename.
- More detailed logging.
- Fixed D-Bus error logging.
- Added -L flag to nose2 tests for explicitly setting log file path.
- Added SYSTEMIMAGE_
which can be used to give virtualized buildds a fighting chance.
* d/patches/
* d/control: Bump Standards-Version to 3.9.5 with no other changes necessary.
- 234. By Barry Warsaw
-
* d/control:
- Bump Standards-Version to 3.9.5 with no other changes necessary.
- Add python3-psutil as Depends to system-image-dev. - 235. By Barry Warsaw
-
d/rules: Set SYSTEMIMAGE_
DBUS_DAEMON_ HUP_SLEEP_ SECONDS to 1 to deal with
buildd dbus-daemon SIGHUP timing issues. - 236. By Barry Warsaw
-
- LP: #1284217 - Send UpdateAvailable
Status during auto-downloading
from a previous CheckForUpdate, if cached status is available.
* d/patches/01_send_ ack_on_ applyupdate. diff: Removed; applied upstream.
* d/patches/lp1284217. patch: Added (see above). - 237. By Barry Warsaw
-
Update patch
Preview Diff
1 | === modified file 'MANIFEST.in' |
2 | --- MANIFEST.in 2013-12-13 13:55:51 +0000 |
3 | +++ MANIFEST.in 2014-02-25 17:45:25 +0000 |
4 | @@ -3,4 +3,5 @@ |
5 | prune build |
6 | prune dist |
7 | prune .tox |
8 | +prune .bzr |
9 | exclude .bzrignore |
10 | |
11 | === modified file 'NEWS.rst' |
12 | --- NEWS.rst 2013-12-13 13:55:51 +0000 |
13 | +++ NEWS.rst 2014-02-25 17:45:25 +0000 |
14 | @@ -2,7 +2,54 @@ |
15 | NEWS for system-image updater |
16 | ============================= |
17 | |
18 | -2.0.3 (2013-XX-XX) |
19 | +2.1 (2014-02-20) |
20 | +================ |
21 | + * Internal improvements to SignatureError for better debugging. (LP: #1279056) |
22 | + * Better protection against several possible race conditions during |
23 | + `CheckForUpdate()` (LP: #1277589) |
24 | + - Use a threading.Lock instance as the internal "checking for update" |
25 | + barrier instead of a boolean. This should eliminate the race window |
26 | + between testing and acquiring the checking lock. |
27 | + - Put an exclusive claim on the `com.canonical.SystemImage` system dbus |
28 | + name, and if we cannot get that claim, exit with an error code 2. This |
29 | + prevents multiple instances of the D-Bus system service from running at |
30 | + the same time. |
31 | + * Return the empty string from `ApplyUpdate()` D-Bus method. This restores |
32 | + the original API (patch merged from Ubuntu package, given by Didier |
33 | + Roche). (LP: #1260768) |
34 | + * Request ubuntu-download-manager to download all files to temporary |
35 | + destinations, then atomically rename them into place. This avoids |
36 | + clobbering by multiple processes and mimics changes coming in u-d-m. |
37 | + * Provide much more detailed logging. |
38 | + - `Mediator` instances have a helpful `repr` which also includes the id of |
39 | + the `State` object. |
40 | + - More logging during state transitions. |
41 | + - All emitted D-Bus signals are also logged (at debug level). |
42 | + * Added `-L` flag to nose test runner, which can be used to specify an |
43 | + explicit log file path for debugging. |
44 | + * Fixed D-Bus error logging. |
45 | + - Don't initialize the root logger, since this can interfere with |
46 | + python-dbus, which doesn't initialize its loggers correctly. |
47 | + - Only use `.format()` based interpolation for `systemimage` logs. |
48 | + * Give virtualized buildds a fighting chance against D-Bus by |
49 | + - using `org.freedesktop.DBus`s `ReloadConfig()` interface instead of |
50 | + SIGHUP. |
51 | + - add a configurable sleep call after the `ReloadConfig()`. This defaults |
52 | + to 0 since de-virtualized and local builds do not need them. Set the |
53 | + environment variable `SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS` to |
54 | + override. |
55 | + * Run the tox test suite for both Python 3.3 and 3.4. |
56 | + |
57 | +2.0.5 (2014-01-30) |
58 | +================== |
59 | + * MANIFEST.in: Make sure the .bzr directory doesn't end up in the |
60 | + sdist tarball. |
61 | + |
62 | +2.0.4 (2014-01-30) |
63 | +================== |
64 | + * No change release to test the new landing process. |
65 | + |
66 | +2.0.3 (2013-12-11) |
67 | ================== |
68 | * More attempted DEP-8 test failure fixes. |
69 | |
70 | |
71 | === modified file 'PKG-INFO' |
72 | --- PKG-INFO 2013-12-13 13:55:51 +0000 |
73 | +++ PKG-INFO 2014-02-25 17:45:25 +0000 |
74 | @@ -1,6 +1,6 @@ |
75 | Metadata-Version: 1.0 |
76 | Name: system-image |
77 | -Version: 2.0.3 |
78 | +Version: 2.1 |
79 | Summary: Ubuntu System Image Based Upgrades |
80 | Home-page: UNKNOWN |
81 | Author: Barry Warsaw |
82 | |
83 | === modified file 'cli-manpage.rst' |
84 | --- cli-manpage.rst 2013-12-13 13:55:51 +0000 |
85 | +++ cli-manpage.rst 2014-02-25 17:45:25 +0000 |
86 | @@ -8,7 +8,7 @@ |
87 | |
88 | :Author: Barry Warsaw <barry@ubuntu.com> |
89 | :Date: 2013-10-23 |
90 | -:Copyright: 2013 Canonical Ltd. |
91 | +:Copyright: 2013-2014 Canonical Ltd. |
92 | :Version: 2.0 |
93 | :Manual section: 1 |
94 | |
95 | |
96 | === modified file 'dbus-manpage.rst' |
97 | --- dbus-manpage.rst 2013-12-13 13:55:51 +0000 |
98 | +++ dbus-manpage.rst 2014-02-25 17:45:25 +0000 |
99 | @@ -8,7 +8,7 @@ |
100 | |
101 | :Author: Barry Warsaw <barry@ubuntu.com> |
102 | :Date: 2013-07-31 |
103 | -:Copyright: 2013 Canonical Ltd. |
104 | +:Copyright: 2013-2014 Canonical Ltd. |
105 | :Version: 1.0 |
106 | :Manual section: 8 |
107 | |
108 | |
109 | === modified file 'debian/changelog' |
110 | --- debian/changelog 2013-12-13 13:55:51 +0000 |
111 | +++ debian/changelog 2014-02-25 17:45:25 +0000 |
112 | @@ -1,3 +1,35 @@ |
113 | +system-image (2.1-0ubuntu4) UNRELEASED; urgency=medium |
114 | + |
115 | + [ Stéphane Graber ] |
116 | + * New upstream release. |
117 | + * Set X-Auto-Uploader to no-rewrite-version |
118 | + * Set Vcs-Bzr to the new target branch |
119 | + |
120 | + [ Barry Warsaw ] |
121 | + * New upstream release. |
122 | + - LP: #1279056 - Internal improvements to SignatureError for |
123 | + better debugging. |
124 | + - LP: #1277589 - Better protection against race conditions. |
125 | + - LP: #1260768 - Return empty string from ApplyUpdate D-Bus method. |
126 | + - LP: #1284217 - Send UpdateAvailableStatus during auto-downloading |
127 | + from a previous CheckForUpdate, if cached status is available. |
128 | + - Request ubuntu-download-manager to download to a temporary location, |
129 | + with atomic rename. |
130 | + - More detailed logging. |
131 | + - Fixed D-Bus error logging. |
132 | + - Added -L flag to nose2 tests for explicitly setting log file path. |
133 | + - Added SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS environment variable |
134 | + which can be used to give virtualized buildds a fighting chance. |
135 | + * d/patches/01_send_ack_on_applyupdate.diff: Removed; applied upstream. |
136 | + * d/patches/lp1284217.patch: Added (see above). |
137 | + * d/control: |
138 | + - Bump Standards-Version to 3.9.5 with no other changes necessary. |
139 | + - Add python3-psutil as Depends to system-image-dev. |
140 | + * d/rules: Set SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS to 1 to deal with |
141 | + buildd dbus-daemon SIGHUP timing issues. |
142 | + |
143 | + -- Barry Warsaw <barry@ubuntu.com> Thu, 20 Feb 2014 18:03:10 -0500 |
144 | + |
145 | system-image (2.0.3-0ubuntu2) trusty; urgency=low |
146 | |
147 | * Fix ApplyUpdate() to return an empty string as per spec if the update |
148 | |
149 | === modified file 'debian/control' |
150 | --- debian/control 2013-12-13 13:55:51 +0000 |
151 | +++ debian/control 2014-02-25 17:45:25 +0000 |
152 | @@ -18,10 +18,11 @@ |
153 | python3-psutil, |
154 | python3-setuptools, |
155 | ubuntu-download-manager |
156 | -Standards-Version: 3.9.4 |
157 | +Standards-Version: 3.9.5 |
158 | XS-Testsuite: autopkgtest |
159 | -Vcs-Bzr: https://code.launchpad.net/~ubuntu-system-image/ubuntu-system-image/client.pkg |
160 | -Vcs-Browser: http://bazaar.launchpad.net/~ubuntu-system-image/ubuntu-system-image/client.pkg/files |
161 | +Vcs-Bzr: https://code.launchpad.net/~ubuntu-managed-branches/ubuntu-system-image/system-image |
162 | +Vcs-Browser: http://bazaar.launchpad.net/~ubuntu-managed-branches/ubuntu-system-image/system-image/files |
163 | +X-Auto-Uploader: no-rewrite-version |
164 | |
165 | Package: system-image-cli |
166 | Architecture: all |
167 | @@ -53,7 +54,7 @@ |
168 | |
169 | Package: system-image-dev |
170 | Architecture: all |
171 | -Depends: python3-gnupg, ${misc:Depends}, ${python3:Depends} |
172 | +Depends: python3-gnupg, python3-psutil, ${misc:Depends}, ${python3:Depends} |
173 | Description: Ubuntu system image updater development |
174 | This is the development bits for the Ubuntu system image updater. |
175 | Install this package if you want to run the tests. |
176 | |
177 | === removed file 'debian/patches/01_send_ack_on_applyupdate.diff' |
178 | --- debian/patches/01_send_ack_on_applyupdate.diff 2013-12-13 13:55:51 +0000 |
179 | +++ debian/patches/01_send_ack_on_applyupdate.diff 1970-01-01 00:00:00 +0000 |
180 | @@ -1,16 +0,0 @@ |
181 | -Description: Send ack on apply diff |
182 | - Fix ApplyUpdate() to return an empty string as per spec if the update |
183 | - is successfull (LP: #1260712) |
184 | -Author: Didier Roche <didrocks@ubuntu.com> |
185 | -Bug-Ubuntu: https://bugs.launchpad.net/bugs/1260712 |
186 | - |
187 | ---- system-image-2.0.3.orig/systemimage/dbus.py |
188 | -+++ system-image-2.0.3/systemimage/dbus.py |
189 | -@@ -233,6 +233,7 @@ class Service(Object): |
190 | - def ApplyUpdate(self): |
191 | - """Apply the update, rebooting the device.""" |
192 | - GLib.timeout_add(50, self._apply_update) |
193 | -+ return "" |
194 | - |
195 | - @method('com.canonical.SystemImage', out_signature='isssa{ss}') |
196 | - def Info(self): |
197 | |
198 | === added file 'debian/patches/lp1284217.patch' |
199 | --- debian/patches/lp1284217.patch 1970-01-01 00:00:00 +0000 |
200 | +++ debian/patches/lp1284217.patch 2014-02-25 17:45:25 +0000 |
201 | @@ -0,0 +1,106 @@ |
202 | +Description: Backport of fix for LP: #1284217. This adds an additional |
203 | + UpdateAvailableStatus signal when a second CheckForUpdate is called while an |
204 | + auto-download is in progress, but only if we have cached status available. |
205 | +Origin: http://bazaar.launchpad.net/~ubuntu-system-image/ubuntu-system-image/client/revision/240?start_revid=240 |
206 | +Bug: http://pad.lv/1284217 |
207 | +Forwarded: not-needed |
208 | + |
209 | +=== modified file 'systemimage/dbus.py' |
210 | +--- old/systemimage/dbus.py 2014-02-18 22:31:55 +0000 |
211 | ++++ new/systemimage/dbus.py 2014-02-25 17:27:14 +0000 |
212 | +@@ -131,7 +131,20 @@ |
213 | + # Check-and-acquire the lock. |
214 | + log.info('test and acquire checking lock') |
215 | + if not self._checking.acquire(blocking=False): |
216 | +- # Check is already in progress, so there's nothing more to do. |
217 | ++ # Check is already in progress, so there's nothing more to do. If |
218 | ++ # there's status available (i.e. we are in the auto-downloading |
219 | ++ # phase of the last CFU), then send the status. |
220 | ++ if self._update is not None: |
221 | ++ self.UpdateAvailableStatus( |
222 | ++ self._update.is_available, |
223 | ++ self._downloading, |
224 | ++ self._update.version, |
225 | ++ self._update.size, |
226 | ++ self._update.last_update_date, |
227 | ++ # XXX 2013-08-22 - the u/i cannot currently currently |
228 | ++ # handle the array of dictionaries data type. LP: |
229 | ++ # #1215586 self._update.descriptions, |
230 | ++ "") |
231 | + log.info('checking lock not acquired') |
232 | + return |
233 | + log.info('checking lock acquired') |
234 | + |
235 | +=== modified file 'systemimage/tests/test_dbus.py' |
236 | +--- old/systemimage/tests/test_dbus.py 2014-02-18 20:02:59 +0000 |
237 | ++++ new/systemimage/tests/test_dbus.py 2014-02-25 17:27:14 +0000 |
238 | +@@ -23,13 +23,13 @@ |
239 | + 'TestDBusGetSet', |
240 | + 'TestDBusInfo', |
241 | + 'TestDBusInfoNoDetails', |
242 | +- 'TestDBusLP1277589', |
243 | + 'TestDBusMockFailApply', |
244 | + 'TestDBusMockFailPause', |
245 | + 'TestDBusMockFailResume', |
246 | + 'TestDBusMockNoUpdate', |
247 | + 'TestDBusMockUpdateAutoSuccess', |
248 | + 'TestDBusMockUpdateManualSuccess', |
249 | ++ 'TestDBusMultipleChecksInFlight', |
250 | + 'TestDBusPauseResume', |
251 | + 'TestDBusProgress', |
252 | + 'TestDBusRegressions', |
253 | +@@ -133,6 +133,7 @@ |
254 | + self.schedule(self.iface.CheckForUpdate) |
255 | + |
256 | + def _do_UpdateAvailableStatus(self, signal, path, *args, **kws): |
257 | ++ # We'll keep doing this until we get the UpdateDownloaded signal. |
258 | + self.uas_signals.append(args) |
259 | + self.schedule(self.iface.CheckForUpdate) |
260 | + |
261 | +@@ -1563,7 +1564,7 @@ |
262 | + """) |
263 | + |
264 | + |
265 | +-class TestDBusLP1277589(_LiveTesting): |
266 | ++class TestDBusMultipleChecksInFlight(_LiveTesting): |
267 | + def test_multiple_check_for_updates(self): |
268 | + # Log analysis of LP: #1277589 appears to show the following scenario, |
269 | + # reproduced in this test case: |
270 | +@@ -1588,18 +1589,23 @@ |
271 | + # signal, we'll immediately issue *another* CheckForUpdate, which |
272 | + # should run while the auto-download is working. |
273 | + # |
274 | +- # At the end, we should not get another UpdateAvailableStatus signal, |
275 | +- # but we should get the UpdateDownloaded signal. |
276 | ++ # As per LP: #1284217, we will get a second UpdateAvailableStatus |
277 | ++ # signal, since the status is available even while the original |
278 | ++ # request is being downloaded. |
279 | + reactor = DoubleCheckingReactor(self.iface) |
280 | + reactor.run() |
281 | +- self.assertEqual(len(reactor.uas_signals), 1) |
282 | +- (is_available, downloading, available_version, update_size, |
283 | +- last_update_date, |
284 | +- #descriptions, |
285 | +- error_reason) = reactor.uas_signals[0] |
286 | +- self.assertTrue(is_available) |
287 | +- self.assertTrue(downloading) |
288 | +- self.assertEqual(available_version, '1600') |
289 | +- self.assertEqual(update_size, 314572800) |
290 | +- self.assertEqual(last_update_date, 'Unknown') |
291 | +- self.assertEqual(error_reason, '') |
292 | ++ # We need to have received at least 2 signals, but due to timing |
293 | ++ # issues it could possibly be more. |
294 | ++ self.assertGreater(len(reactor.uas_signals), 1) |
295 | ++ # All received signals should have the same information. |
296 | ++ for signal in reactor.uas_signals: |
297 | ++ (is_available, downloading, available_version, update_size, |
298 | ++ last_update_date, |
299 | ++ #descriptions, |
300 | ++ error_reason) = signal |
301 | ++ self.assertTrue(is_available) |
302 | ++ self.assertTrue(downloading) |
303 | ++ self.assertEqual(available_version, '1600') |
304 | ++ self.assertEqual(update_size, 314572800) |
305 | ++ self.assertEqual(last_update_date, 'Unknown') |
306 | ++ self.assertEqual(error_reason, '') |
307 | + |
308 | |
309 | === modified file 'debian/patches/series' |
310 | --- debian/patches/series 2013-12-13 13:55:51 +0000 |
311 | +++ debian/patches/series 2014-02-25 17:45:25 +0000 |
312 | @@ -1,1 +1,1 @@ |
313 | -01_send_ack_on_applyupdate.diff |
314 | +lp1284217.patch |
315 | |
316 | === modified file 'debian/rules' |
317 | --- debian/rules 2013-12-13 13:55:51 +0000 |
318 | +++ debian/rules 2014-02-25 17:45:25 +0000 |
319 | @@ -16,6 +16,7 @@ |
320 | test-python%: |
321 | unset http_proxy; unset https_proxy; export HOME=/tmp; \ |
322 | export SYSTEMIMAGE_REACTOR_TIMEOUT=1200; \ |
323 | + export SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS=2; \ |
324 | nodot=$(shell echo $* | cut --complement -b 2); \ |
325 | tox -e py$${nodot} |
326 | |
327 | |
328 | === modified file 'ini-manpage.rst' |
329 | --- ini-manpage.rst 2013-12-13 13:55:51 +0000 |
330 | +++ ini-manpage.rst 2014-02-25 17:45:25 +0000 |
331 | @@ -9,7 +9,7 @@ |
332 | |
333 | :Author: Barry Warsaw <barry@ubuntu.com> |
334 | :Date: 2013-10-11 |
335 | -:Copyright: 2013 Canonical Ltd. |
336 | +:Copyright: 2013-2014 Canonical Ltd. |
337 | :Version: 1.9 |
338 | :Manual section: 5 |
339 | |
340 | |
341 | === modified file 'setup.cfg' |
342 | --- setup.cfg 2013-12-13 13:55:51 +0000 |
343 | +++ setup.cfg 2014-02-25 17:45:25 +0000 |
344 | @@ -4,7 +4,7 @@ |
345 | logging-filter = systemimage |
346 | |
347 | [egg_info] |
348 | +tag_svn_revision = 0 |
349 | +tag_date = 0 |
350 | tag_build = |
351 | -tag_date = 0 |
352 | -tag_svn_revision = 0 |
353 | |
354 | |
355 | === modified file 'setup.py' |
356 | --- setup.py 2013-12-13 13:55:51 +0000 |
357 | +++ setup.py 2014-02-25 17:45:25 +0000 |
358 | @@ -1,4 +1,4 @@ |
359 | -# Copyright (C) 2013 Canonical Ltd. |
360 | +# Copyright (C) 2013-2014 Canonical Ltd. |
361 | # Author: Barry Warsaw <barry@ubuntu.com> |
362 | |
363 | # This program is free software: you can redistribute it and/or modify |
364 | |
365 | === modified file 'system_image.egg-info/PKG-INFO' |
366 | --- system_image.egg-info/PKG-INFO 2013-12-13 13:55:51 +0000 |
367 | +++ system_image.egg-info/PKG-INFO 2014-02-25 17:45:25 +0000 |
368 | @@ -1,6 +1,6 @@ |
369 | Metadata-Version: 1.0 |
370 | Name: system-image |
371 | -Version: 2.0.3 |
372 | +Version: 2.1 |
373 | Summary: Ubuntu System Image Based Upgrades |
374 | Home-page: UNKNOWN |
375 | Author: Barry Warsaw |
376 | |
377 | === modified file 'system_image.egg-info/SOURCES.txt' |
378 | --- system_image.egg-info/SOURCES.txt 2013-12-13 13:55:51 +0000 |
379 | +++ system_image.egg-info/SOURCES.txt 2014-02-25 17:45:25 +0000 |
380 | @@ -7,7 +7,6 @@ |
381 | setup.py |
382 | tox.ini |
383 | unittest.cfg |
384 | -.bzr/branch/branch.conf |
385 | system_image.egg-info/PKG-INFO |
386 | system_image.egg-info/SOURCES.txt |
387 | system_image.egg-info/dependency_links.txt |
388 | @@ -126,6 +125,7 @@ |
389 | systemimage/tests/data/index_21.json |
390 | systemimage/tests/data/index_22.json |
391 | systemimage/tests/data/index_23.json |
392 | +systemimage/tests/data/index_24.json |
393 | systemimage/tests/data/key.pem |
394 | systemimage/tests/data/master-secring.gpg |
395 | systemimage/tests/data/nasty_cert.pem |
396 | |
397 | === modified file 'systemimage/api.py' |
398 | --- systemimage/api.py 2013-12-13 13:55:51 +0000 |
399 | +++ systemimage/api.py 2014-02-25 17:45:25 +0000 |
400 | @@ -1,4 +1,4 @@ |
401 | -# Copyright (C) 2013 Canonical Ltd. |
402 | +# Copyright (C) 2013-2014 Canonical Ltd. |
403 | # Author: Barry Warsaw <barry@ubuntu.com> |
404 | |
405 | # This program is free software: you can redistribute it and/or modify |
406 | @@ -73,6 +73,10 @@ |
407 | self._update = None |
408 | self._callback = callback |
409 | |
410 | + def __repr__(self): |
411 | + return '<Mediator at 0x{:x} | State at 0x{:x}>'.format( |
412 | + id(self), id(self._state)) |
413 | + |
414 | def cancel(self): |
415 | self._state.downloader.cancel() |
416 | |
417 | |
418 | === modified file 'systemimage/bag.py' |
419 | --- systemimage/bag.py 2013-12-13 13:55:51 +0000 |
420 | +++ systemimage/bag.py 2014-02-25 17:45:25 +0000 |
421 | @@ -1,4 +1,4 @@ |
422 | -# Copyright (C) 2013 Canonical Ltd. |
423 | +# Copyright (C) 2013-2014 Canonical Ltd. |
424 | # Author: Barry Warsaw <barry@ubuntu.com> |
425 | |
426 | # This program is free software: you can redistribute it and/or modify |
427 | |
428 | === modified file 'systemimage/bindings.py' |
429 | --- systemimage/bindings.py 2013-12-13 13:55:51 +0000 |
430 | +++ systemimage/bindings.py 2014-02-25 17:45:25 +0000 |
431 | @@ -1,4 +1,4 @@ |
432 | -# Copyright (C) 2013 Canonical Ltd. |
433 | +# Copyright (C) 2013-2014 Canonical Ltd. |
434 | # Author: Barry Warsaw <barry@ubuntu.com> |
435 | |
436 | # This program is free software: you can redistribute it and/or modify |
437 | |
438 | === modified file 'systemimage/candidates.py' |
439 | --- systemimage/candidates.py 2013-12-13 13:55:51 +0000 |
440 | +++ systemimage/candidates.py 2014-02-25 17:45:25 +0000 |
441 | @@ -1,4 +1,4 @@ |
442 | -# Copyright (C) 2013 Canonical Ltd. |
443 | +# Copyright (C) 2013-2014 Canonical Ltd. |
444 | # Author: Barry Warsaw <barry@ubuntu.com> |
445 | |
446 | # This program is free software: you can redistribute it and/or modify |
447 | |
448 | === modified file 'systemimage/channel.py' |
449 | --- systemimage/channel.py 2013-12-13 13:55:51 +0000 |
450 | +++ systemimage/channel.py 2014-02-25 17:45:25 +0000 |
451 | @@ -1,4 +1,4 @@ |
452 | -# Copyright (C) 2013 Canonical Ltd. |
453 | +# Copyright (C) 2013-2014 Canonical Ltd. |
454 | # Author: Barry Warsaw <barry@ubuntu.com> |
455 | |
456 | # This program is free software: you can redistribute it and/or modify |
457 | |
458 | === modified file 'systemimage/config.py' |
459 | --- systemimage/config.py 2013-12-13 13:55:51 +0000 |
460 | +++ systemimage/config.py 2014-02-25 17:45:25 +0000 |
461 | @@ -1,4 +1,4 @@ |
462 | -# Copyright (C) 2013 Canonical Ltd. |
463 | +# Copyright (C) 2013-2014 Canonical Ltd. |
464 | # Author: Barry Warsaw <barry@ubuntu.com> |
465 | |
466 | # This program is free software: you can redistribute it and/or modify |
467 | |
468 | === modified file 'systemimage/dbus.py' |
469 | --- systemimage/dbus.py 2013-12-13 13:55:51 +0000 |
470 | +++ systemimage/dbus.py 2014-02-25 17:45:25 +0000 |
471 | @@ -1,4 +1,4 @@ |
472 | -# Copyright (C) 2013 Canonical Ltd. |
473 | +# Copyright (C) 2013-2014 Canonical Ltd. |
474 | # Author: Barry Warsaw <barry@ubuntu.com> |
475 | |
476 | # This program is free software: you can redistribute it and/or modify |
477 | @@ -23,6 +23,7 @@ |
478 | |
479 | import os |
480 | import sys |
481 | +import logging |
482 | import traceback |
483 | |
484 | from dbus.service import Object, method, signal |
485 | @@ -31,9 +32,11 @@ |
486 | from systemimage.config import config |
487 | from systemimage.helpers import last_update_date, version_detail |
488 | from systemimage.settings import Settings |
489 | +from threading import Lock |
490 | |
491 | |
492 | EMPTYSTRING = '' |
493 | +log = logging.getLogger('systemimage') |
494 | |
495 | |
496 | class Loop: |
497 | @@ -68,7 +71,8 @@ |
498 | super().__init__(bus, object_path) |
499 | self._loop = loop |
500 | self._api = Mediator(self._progress_callback) |
501 | - self._checking = False |
502 | + log.info('Mediator created {}', self._api) |
503 | + self._checking = Lock() |
504 | self._update = None |
505 | self._downloading = False |
506 | self._paused = False |
507 | @@ -78,17 +82,22 @@ |
508 | |
509 | def _check_for_update(self): |
510 | # Asynchronous method call. |
511 | + log.info('Checking for update') |
512 | self._update = self._api.check_for_update() |
513 | # Do we have an update and can we auto-download it? |
514 | downloading = False |
515 | if self._update.is_available: |
516 | settings = Settings() |
517 | auto = settings.get('auto_download') |
518 | + log.info('Update available; auto-download: {}', auto) |
519 | if auto in ('1', '2'): |
520 | # XXX When we have access to the download service, we can |
521 | # check if we're on the wifi (auto == '1'). |
522 | - GLib.timeout_add(50, self._download) |
523 | + GLib.timeout_add(50, self._download, self._checking.release) |
524 | downloading = True |
525 | + else: |
526 | + log.info('release checking lock from _check_for_update()') |
527 | + self._checking.release() |
528 | self.UpdateAvailableStatus( |
529 | self._update.is_available, |
530 | downloading, |
531 | @@ -99,7 +108,6 @@ |
532 | # array of dictionaries data type. LP: #1215586 |
533 | #self._update.descriptions, |
534 | "") |
535 | - self._checking = False |
536 | # Stop GLib from calling this method again. |
537 | return False |
538 | |
539 | @@ -120,13 +128,17 @@ |
540 | whether the update is available or not. |
541 | """ |
542 | self._loop.keepalive() |
543 | - if self._checking: |
544 | + # Check-and-acquire the lock. |
545 | + log.info('test and acquire checking lock') |
546 | + if not self._checking.acquire(blocking=False): |
547 | # Check is already in progress, so there's nothing more to do. |
548 | + log.info('checking lock not acquired') |
549 | return |
550 | - self._checking = True |
551 | - # Reset any failure or in-progress state. Get a new mediator to reset |
552 | - # any of its state. |
553 | + log.info('checking lock acquired') |
554 | + # We've now acquired the lock. Reset any failure or in-progress |
555 | + # state. Get a new mediator to reset any of its state. |
556 | self._api = Mediator(self._progress_callback) |
557 | + log.info('Mediator recreated {}', self._api) |
558 | self._failure_count = 0 |
559 | self._last_error = '' |
560 | # Arrange for the actual check to happen in a little while, so that |
561 | @@ -141,24 +153,28 @@ |
562 | eta = 0 |
563 | self.UpdateProgress(percentage, eta) |
564 | |
565 | - def _download(self): |
566 | + def _download(self, release_checking=None): |
567 | if self._downloading and self._paused: |
568 | self._api.resume() |
569 | self._paused = False |
570 | + log.info('Download previously paused') |
571 | return |
572 | - if (self._downloading # Already in progress. |
573 | - or self._update is None # Not yet checked. |
574 | - or not self._update.is_available # No update available. |
575 | + if (self._downloading # Already in progress. |
576 | + or self._update is None # Not yet checked. |
577 | + or not self._update.is_available # No update available. |
578 | ): |
579 | + log.info('Download already in progress or not available') |
580 | return |
581 | if self._failure_count > 0: |
582 | self._failure_count += 1 |
583 | self.UpdateFailed(self._failure_count, self._last_error) |
584 | + log.info('Update failure count: {}', self._failure_count) |
585 | return |
586 | self._downloading = True |
587 | + log.info('Update is downloading') |
588 | try: |
589 | - # Always start by sending a UpdateProgress(0, 0). This is enough |
590 | - # to get the u/i's attention. |
591 | + # Always start by sending a UpdateProgress(0, 0). This is |
592 | + # enough to get the u/i's attention. |
593 | self.UpdateProgress(0, 0) |
594 | self._api.download() |
595 | except Exception: |
596 | @@ -167,13 +183,20 @@ |
597 | # value, but not the traceback. |
598 | self._last_error = EMPTYSTRING.join( |
599 | traceback.format_exception_only(*sys.exc_info()[:2])) |
600 | + log.info('Update failed: {}', self._last_error) |
601 | self.UpdateFailed(self._failure_count, self._last_error) |
602 | else: |
603 | + log.info('Update downloaded') |
604 | self.UpdateDownloaded() |
605 | self._failure_count = 0 |
606 | self._last_error = '' |
607 | self._rebootable = True |
608 | self._downloading = False |
609 | + log.info('release checking lock from _download()') |
610 | + if release_checking is not None: |
611 | + # We were auto-downloading, so we now have to release the checking |
612 | + # lock. If we were manually downloading, there would be no lock. |
613 | + release_checking() |
614 | # Stop GLib from calling this method again. |
615 | return False |
616 | |
617 | @@ -216,8 +239,6 @@ |
618 | return '' |
619 | |
620 | def _apply_update(self): |
621 | - # This signal may or may not get sent. We're racing against the |
622 | - # system reboot procedure. |
623 | self._loop.keepalive() |
624 | if not self._rebootable: |
625 | command_file = os.path.join( |
626 | @@ -227,12 +248,16 @@ |
627 | self.Rebooting(False) |
628 | return |
629 | self._api.reboot() |
630 | + # This code may or may not run. We're racing against the system |
631 | + # reboot procedure. |
632 | + self._rebootable = False |
633 | self.Rebooting(True) |
634 | |
635 | @method('com.canonical.SystemImage') |
636 | def ApplyUpdate(self): |
637 | """Apply the update, rebooting the device.""" |
638 | GLib.timeout_add(50, self._apply_update) |
639 | + return '' |
640 | |
641 | @method('com.canonical.SystemImage', out_signature='isssa{ss}') |
642 | def Info(self): |
643 | @@ -294,31 +319,40 @@ |
644 | #descriptions, |
645 | error_reason): |
646 | """Signal sent in response to a CheckForUpdate().""" |
647 | + log.debug('EMIT UpdateAvailableStatus({}, {}, {}, {}, {}, {})', |
648 | + is_available, downloading, available_version, update_size, |
649 | + last_update_date, repr(error_reason)) |
650 | self._loop.keepalive() |
651 | |
652 | @signal('com.canonical.SystemImage', signature='id') |
653 | def UpdateProgress(self, percentage, eta): |
654 | """Download progress.""" |
655 | + log.debug('EMIT UpdateProgress({}, {})', percentage, eta) |
656 | self._loop.keepalive() |
657 | |
658 | @signal('com.canonical.SystemImage') |
659 | def UpdateDownloaded(self): |
660 | """The update has been successfully downloaded.""" |
661 | + log.debug('EMIT UpdateDownloaded()') |
662 | self._loop.keepalive() |
663 | |
664 | @signal('com.canonical.SystemImage', signature='is') |
665 | def UpdateFailed(self, consecutive_failure_count, last_reason): |
666 | """The update failed for some reason.""" |
667 | + log.debug('EMIT UpdateFailed({}, {})', |
668 | + consecutive_failure_count, repr(last_reason)) |
669 | self._loop.keepalive() |
670 | |
671 | @signal('com.canonical.SystemImage', signature='i') |
672 | def UpdatePaused(self, percentage): |
673 | """The download got paused.""" |
674 | + log.debug('EMIT UpdatePaused({})', percentage) |
675 | self._loop.keepalive() |
676 | |
677 | @signal('com.canonical.SystemImage', signature='ss') |
678 | def SettingChanged(self, key, new_value): |
679 | """A setting value has change.""" |
680 | + log.debug('EMIT SettingChanged({}, {})', repr(key), repr(new_value)) |
681 | self._loop.keepalive() |
682 | |
683 | @signal('com.canonical.SystemImage', signature='b') |
684 | @@ -326,3 +360,4 @@ |
685 | """The system is rebooting.""" |
686 | # We don't need to keep the loop alive since we're probably just going |
687 | # to shutdown anyway. |
688 | + log.debug('EMIT Rebooting({})', status) |
689 | |
690 | === modified file 'systemimage/device.py' |
691 | --- systemimage/device.py 2013-12-13 13:55:51 +0000 |
692 | +++ systemimage/device.py 2014-02-25 17:45:25 +0000 |
693 | @@ -1,4 +1,4 @@ |
694 | -# Copyright (C) 2013 Canonical Ltd. |
695 | +# Copyright (C) 2013-2014 Canonical Ltd. |
696 | # Author: Barry Warsaw <barry@ubuntu.com> |
697 | |
698 | # This program is free software: you can redistribute it and/or modify |
699 | |
700 | === modified file 'systemimage/docs/conf.py' |
701 | --- systemimage/docs/conf.py 2013-12-13 13:55:51 +0000 |
702 | +++ systemimage/docs/conf.py 2014-02-25 17:45:25 +0000 |
703 | @@ -41,7 +41,7 @@ |
704 | |
705 | # General information about the project. |
706 | project = u'Image Update Resolver' |
707 | -copyright = u'2013, Canonical Ltd.' |
708 | +copyright = u'2013-2014, Canonical Ltd.' |
709 | |
710 | # The version info for the project you're documenting, acts as replacement for |
711 | # |version| and |release|, also used in various other places throughout the |
712 | |
713 | === modified file 'systemimage/download.py' |
714 | --- systemimage/download.py 2013-12-13 13:55:51 +0000 |
715 | +++ systemimage/download.py 2014-02-25 17:45:25 +0000 |
716 | @@ -1,4 +1,4 @@ |
717 | -# Copyright (C) 2013 Canonical Ltd. |
718 | +# Copyright (C) 2013-2014 Canonical Ltd. |
719 | # Author: Barry Warsaw <barry@ubuntu.com> |
720 | |
721 | # This program is free software: you can redistribute it and/or modify |
722 | @@ -22,12 +22,16 @@ |
723 | ] |
724 | |
725 | |
726 | +import os |
727 | import dbus |
728 | import logging |
729 | +import tempfile |
730 | |
731 | +from contextlib import ExitStack |
732 | from io import StringIO |
733 | from pprint import pformat |
734 | from systemimage.config import config |
735 | +from systemimage.helpers import safe_remove |
736 | from systemimage.reactor import Reactor |
737 | |
738 | |
739 | @@ -191,8 +195,25 @@ |
740 | for url, dst in downloads: |
741 | print('\t{} -> {}'.format(url, dst), file=fp) |
742 | log.info('{}'.format(fp.getvalue())) |
743 | + # As a workaround for LP: #1277589, ask u-d-m to download the files to |
744 | + # .tmp files, and if they succeed, then atomically move them into |
745 | + # their real location. |
746 | + renames = [] |
747 | + requests = [] |
748 | + for url, dst in downloads: |
749 | + head, tail = os.path.split(dst) |
750 | + fd, path = tempfile.mkstemp(suffix='.tmp', prefix='', dir=head) |
751 | + os.close(fd) |
752 | + renames.append((path, dst)) |
753 | + requests.append((url, path, '')) |
754 | + # mkstemp() creates the file system path, but if the files exist when |
755 | + # the group download is requested, ubuntu-download-manager will |
756 | + # complain and return an error. So, delete all temporary files now so |
757 | + # udm has a clear path to download to. |
758 | + for path, dst in renames: |
759 | + os.remove(path) |
760 | object_path = iface.createDownloadGroup( |
761 | - [(url, dst, '') for url, dst in downloads], |
762 | + requests, # The temporary requests. |
763 | '', # No hashes yet. |
764 | False, # Don't allow GSM yet. |
765 | # https://bugs.freedesktop.org/show_bug.cgi?id=55594 |
766 | @@ -220,6 +241,19 @@ |
767 | raise Canceled |
768 | if reactor.timed_out: |
769 | raise TimeoutError |
770 | + # Now that everything succeeded, rename the temporary files. Just to |
771 | + # be extra cautious, set up a context manager to safely remove all |
772 | + # temporary files in case of an error. If there are no errors, then |
773 | + # there will be nothing to remove. |
774 | + with ExitStack() as resources: |
775 | + for tmp, dst in renames: |
776 | + resources.callback(safe_remove, tmp) |
777 | + for tmp, dst in renames: |
778 | + os.rename(tmp, dst) |
779 | + # We only get here if all the renames succeeded, so there will be |
780 | + # no temporary files to remove, so we can throw away the new |
781 | + # ExitStack, which holds all the removals. |
782 | + resources.pop_all() |
783 | |
784 | def cancel(self): |
785 | """Cancel any current downloads.""" |
786 | |
787 | === modified file 'systemimage/gpg.py' |
788 | --- systemimage/gpg.py 2013-12-13 13:55:51 +0000 |
789 | +++ systemimage/gpg.py 2014-02-25 17:45:25 +0000 |
790 | @@ -1,4 +1,4 @@ |
791 | -# Copyright (C) 2013 Canonical Ltd. |
792 | +# Copyright (C) 2013-2014 Canonical Ltd. |
793 | # Author: Barry Warsaw <barry@ubuntu.com> |
794 | |
795 | # This program is free software: you can redistribute it and/or modify |
796 | @@ -23,6 +23,7 @@ |
797 | |
798 | import os |
799 | import gnupg |
800 | +import hashlib |
801 | import tarfile |
802 | |
803 | from contextlib import ExitStack |
804 | @@ -37,10 +38,70 @@ |
805 | always returns a boolean. This exception is used by other functions to |
806 | signal that a .asc file did not match. |
807 | """ |
808 | + def __init__(self, signature_path, data_path, |
809 | + keyrings=None, blacklist=None): |
810 | + super().__init__() |
811 | + self.signature_path = signature_path |
812 | + self.data_path = data_path |
813 | + self.keyrings = ([] if keyrings is None else keyrings) |
814 | + self.blacklist = blacklist |
815 | + # We have to calculate the checksums now, because it's possible that |
816 | + # the files will be temporary/atomic files, deleted when a context |
817 | + # manager exits. I.e. the files aren't guaranteed to exist after this |
818 | + # constructor runs. |
819 | + # |
820 | + # Also, md5 is fine; this is not a security critical context, we just |
821 | + # want to be able to quickly and easily compare the file on disk |
822 | + # against the file on the server. |
823 | + with open(self.signature_path, 'rb') as fp: |
824 | + self.signature_checksum = hashlib.md5(fp.read()).hexdigest() |
825 | + with open(self.data_path, 'rb') as fp: |
826 | + self.data_checksum = hashlib.md5(fp.read()).hexdigest() |
827 | + self.keyring_checksums = [] |
828 | + for path in self.keyrings: |
829 | + with open(path, 'rb') as fp: |
830 | + checksum = hashlib.md5(fp.read()).hexdigest() |
831 | + self.keyring_checksums.append(checksum) |
832 | + if self.blacklist is None: |
833 | + self.blacklist_checksum = None |
834 | + else: |
835 | + with open(self.blacklist, 'rb') as fp: |
836 | + self.blacklist_checksum = hashlib.md5(fp.read()).hexdigest() |
837 | + |
838 | + def __str__(self): |
839 | + if self.blacklist is None: |
840 | + checksum_str = 'no blacklist' |
841 | + path_str = '' |
842 | + else: |
843 | + checksum_str = self.blacklist_checksum |
844 | + path_str = self.blacklist |
845 | + return """ |
846 | + sig path : {0.signature_checksum} |
847 | + {0.signature_path} |
848 | + data path: {0.data_checksum} |
849 | + {0.data_path} |
850 | + keyrings : {0.keyring_checksums} |
851 | + {1} |
852 | + blacklist: {2} {3} |
853 | +""".format(self, list(self.keyrings), checksum_str, path_str) |
854 | + |
855 | |
856 | |
857 | class Context: |
858 | def __init__(self, *keyrings, blacklist=None): |
859 | + """Create a GPG signature verification context. |
860 | + |
861 | + :param keyrings: The list of keyrings to use for validating the |
862 | + signature on data files. |
863 | + :type keyrings: Sequence of .tar.xz keyring files, which will be |
864 | + unpacked to retrieve the actual .gpg keyring file. |
865 | + :param blacklist: The blacklist keyring, from which fingerprints to |
866 | + explicitly disallow are retrieved. |
867 | + :type blacklist: A .tar.xz keyring file, which will be unpacked to |
868 | + retrieve the actual .gpg keyring file. |
869 | + """ |
870 | + self.keyring_paths = keyrings |
871 | + self.blacklist_path = blacklist |
872 | self._ctx = None |
873 | self._stack = ExitStack() |
874 | self._keyrings = [] |
875 | @@ -112,6 +173,21 @@ |
876 | return set(info['keyid'] for info in self._ctx.list_keys()) |
877 | |
878 | def verify(self, signature_path, data_path): |
879 | + """Verify a GPG signature. |
880 | + |
881 | + This verifies that the data file signature is valid, given the |
882 | + keyrings and blacklist specified in the constructor. Specifically, we |
883 | + use GPG to extract the fingerprint in the signature path, and compare |
884 | + it against the fingerprints in the keyrings, subtracting any |
885 | + fingerprints in the blacklist. |
886 | + |
887 | + :param signature_path: The file system path to the detached signature |
888 | + file for the data file. |
889 | + :type signature_path: str |
890 | + :param data_path: The file system path to the data file. |
891 | + :type data_path: str |
892 | + :return: bool |
893 | + """ |
894 | with open(signature_path, 'rb') as sig_fp: |
895 | verified = self._ctx.verify_file(sig_fp, data_path) |
896 | # If the file is properly signed, we'll be able to get back a set of |
897 | @@ -120,3 +196,21 @@ |
898 | # loaded-up keyrings. If so, the signature succeeds. |
899 | return verified.fingerprint in (self.fingerprints - |
900 | self._blacklisted_fingerprints) |
901 | + |
902 | + def validate(self, signature_path, data_path): |
903 | + """Like .verify() but raises a SignatureError when invalid. |
904 | + |
905 | + :param signature_path: The file system path to the detached signature |
906 | + file for the data file. |
907 | + :type signature_path: str |
908 | + :param data_path: The file system path to the data file. |
909 | + :type data_path: str |
910 | + :return: None |
911 | + :raises SignatureError: when the signature cannot be verified. Note |
912 | + that the exception will contain extra information, namely the |
913 | + keyrings involved in the verification, as well as the blacklist |
914 | + file if there is one. |
915 | + """ |
916 | + if not self.verify(signature_path, data_path): |
917 | + raise SignatureError(signature_path, data_path, |
918 | + self.keyring_paths, self.blacklist_path) |
919 | |
920 | === modified file 'systemimage/helpers.py' |
921 | --- systemimage/helpers.py 2013-12-13 13:55:51 +0000 |
922 | +++ systemimage/helpers.py 2014-02-25 17:45:25 +0000 |
923 | @@ -1,4 +1,4 @@ |
924 | -# Copyright (C) 2013 Canonical Ltd. |
925 | +# Copyright (C) 2013-2014 Canonical Ltd. |
926 | # Author: Barry Warsaw <barry@ubuntu.com> |
927 | |
928 | # This program is free software: you can redistribute it and/or modify |
929 | |
930 | === modified file 'systemimage/image.py' |
931 | --- systemimage/image.py 2013-12-13 13:55:51 +0000 |
932 | +++ systemimage/image.py 2014-02-25 17:45:25 +0000 |
933 | @@ -1,4 +1,4 @@ |
934 | -# Copyright (C) 2013 Canonical Ltd. |
935 | +# Copyright (C) 2013-2014 Canonical Ltd. |
936 | # Author: Barry Warsaw <barry@ubuntu.com> |
937 | |
938 | # This program is free software: you can redistribute it and/or modify |
939 | |
940 | === modified file 'systemimage/index.py' |
941 | --- systemimage/index.py 2013-12-13 13:55:51 +0000 |
942 | +++ systemimage/index.py 2014-02-25 17:45:25 +0000 |
943 | @@ -1,4 +1,4 @@ |
944 | -# Copyright (C) 2013 Canonical Ltd. |
945 | +# Copyright (C) 2013-2014 Canonical Ltd. |
946 | # Author: Barry Warsaw <barry@ubuntu.com> |
947 | |
948 | # This program is free software: you can redistribute it and/or modify |
949 | |
950 | === modified file 'systemimage/keyring.py' |
951 | --- systemimage/keyring.py 2013-12-13 13:55:51 +0000 |
952 | +++ systemimage/keyring.py 2014-02-25 17:45:25 +0000 |
953 | @@ -1,4 +1,4 @@ |
954 | -# Copyright (C) 2013 Canonical Ltd. |
955 | +# Copyright (C) 2013-2014 Canonical Ltd. |
956 | # Author: Barry Warsaw <barry@ubuntu.com> |
957 | |
958 | # This program is free software: you can redistribute it and/or modify |
959 | @@ -31,7 +31,7 @@ |
960 | from datetime import datetime, timezone |
961 | from systemimage.config import config |
962 | from systemimage.download import DBusDownloadManager |
963 | -from systemimage.gpg import Context, SignatureError |
964 | +from systemimage.gpg import Context |
965 | from systemimage.helpers import makedirs, safe_remove |
966 | from urllib.parse import urljoin |
967 | |
968 | @@ -110,8 +110,7 @@ |
969 | stack.callback(os.remove, ascxz_dst) |
970 | signing_keyring = getattr(config.gpg, sigkr.replace('-', '_')) |
971 | with Context(signing_keyring, blacklist=blacklist) as ctx: |
972 | - if not ctx.verify(ascxz_dst, tarxz_dst): |
973 | - raise SignatureError |
974 | + ctx.validate(ascxz_dst, tarxz_dst) |
975 | # The signature is good, so now unpack the tarball, load the json file |
976 | # and verify its contents. |
977 | keyring_gpg = os.path.join(config.tempdir, 'keyring.gpg') |
978 | |
979 | === modified file 'systemimage/logging.py' |
980 | --- systemimage/logging.py 2013-12-13 13:55:51 +0000 |
981 | +++ systemimage/logging.py 2014-02-25 17:45:25 +0000 |
982 | @@ -1,4 +1,4 @@ |
983 | -# Copyright (C) 2013 Canonical Ltd. |
984 | +# Copyright (C) 2013-2014 Canonical Ltd. |
985 | # Author: Barry Warsaw <barry@ubuntu.com> |
986 | |
987 | # This program is free software: you can redistribute it and/or modify |
988 | @@ -36,12 +36,35 @@ |
989 | LOGFILE_PERMISSIONS = stat.S_IRUSR | stat.S_IWUSR |
990 | |
991 | |
992 | +# We want to support {}-style logging for all systemimage child loggers. One |
993 | +# way to do this is with a LogRecord factory, but to play nice with third |
994 | +# party loggers which might be using %-style, we have to make sure that we use |
995 | +# the default factory for everything else. |
996 | +# |
997 | +# This actually isn't the best way to do this because it still makes a global |
998 | +# change and we don't know how this will interact with other third party |
999 | +# loggers. A marginally better way to do this is to pass class instances to |
1000 | +# the logging calls. Those instances would have a __str__() method that does |
1001 | +# the .format() conversion. The problem with that is that it's a bit less |
1002 | +# convenient to make the logging calls because you can't pass strings |
1003 | +# directly. One such suggestion at <http://tinyurl.com/pjjwjxq> is to import |
1004 | +# the class as __ (i.e. double underscore) so your logging calls would look |
1005 | +# like: log.error(__('Message with {} {}'), foo, bar) |
1006 | + |
1007 | class FormattingLogRecord(logging.LogRecord): |
1008 | + def __init__(self, name, *args, **kws): |
1009 | + logger_path = name.split('.') |
1010 | + self._use_format = (logger_path[0] == 'systemimage') |
1011 | + super().__init__(name, *args, **kws) |
1012 | + |
1013 | def getMessage(self): |
1014 | - msg = str(self.msg) |
1015 | - if self.args: |
1016 | - msg = msg.format(*self.args) |
1017 | - return msg |
1018 | + if self._use_format: |
1019 | + msg = str(self.msg) |
1020 | + if self.args: |
1021 | + msg = msg.format(*self.args) |
1022 | + return msg |
1023 | + else: |
1024 | + return super().getMessage() |
1025 | |
1026 | |
1027 | def initialize(*, verbosity=0): |
1028 | @@ -53,9 +76,7 @@ |
1029 | 3: logging.CRITICAL, |
1030 | }.get(verbosity, logging.ERROR) |
1031 | level = min(level, config.system.loglevel) |
1032 | - # We're not going to propagate to the root logger anyway. |
1033 | - logging.basicConfig(style='{') |
1034 | - # Make sure logging uses {}-style messages. |
1035 | + # Make sure our library's logging uses {}-style messages. |
1036 | logging.setLogRecordFactory(FormattingLogRecord) |
1037 | # Now configure the application level logger based on the ini file. |
1038 | log = logging.getLogger('systemimage') |
1039 | |
1040 | === modified file 'systemimage/main.py' |
1041 | --- systemimage/main.py 2013-12-13 13:55:51 +0000 |
1042 | +++ systemimage/main.py 2014-02-25 17:45:25 +0000 |
1043 | @@ -1,4 +1,4 @@ |
1044 | -# Copyright (C) 2013 Canonical Ltd. |
1045 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1046 | # Author: Barry Warsaw <barry@ubuntu.com> |
1047 | |
1048 | # This program is free software: you can redistribute it and/or modify |
1049 | |
1050 | === modified file 'systemimage/reactor.py' |
1051 | --- systemimage/reactor.py 2013-12-13 13:55:51 +0000 |
1052 | +++ systemimage/reactor.py 2014-02-25 17:45:25 +0000 |
1053 | @@ -1,4 +1,4 @@ |
1054 | -# Copyright (C) 2013 Canonical Ltd. |
1055 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1056 | # Author: Barry Warsaw <barry@ubuntu.com> |
1057 | |
1058 | # This program is free software: you can redistribute it and/or modify |
1059 | |
1060 | === modified file 'systemimage/reboot.py' |
1061 | --- systemimage/reboot.py 2013-12-13 13:55:51 +0000 |
1062 | +++ systemimage/reboot.py 2014-02-25 17:45:25 +0000 |
1063 | @@ -1,4 +1,4 @@ |
1064 | -# Copyright (C) 2013 Canonical Ltd. |
1065 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1066 | # Author: Barry Warsaw <barry@ubuntu.com> |
1067 | |
1068 | # This program is free software: you can redistribute it and/or modify |
1069 | |
1070 | === modified file 'systemimage/scores.py' |
1071 | --- systemimage/scores.py 2013-12-13 13:55:51 +0000 |
1072 | +++ systemimage/scores.py 2014-02-25 17:45:25 +0000 |
1073 | @@ -1,4 +1,4 @@ |
1074 | -# Copyright (C) 2013 Canonical Ltd. |
1075 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1076 | # Author: Barry Warsaw <barry@ubuntu.com> |
1077 | |
1078 | # This program is free software: you can redistribute it and/or modify |
1079 | |
1080 | === modified file 'systemimage/service.py' |
1081 | --- systemimage/service.py 2013-12-13 13:55:51 +0000 |
1082 | +++ systemimage/service.py 2014-02-25 17:45:25 +0000 |
1083 | @@ -1,4 +1,4 @@ |
1084 | -# Copyright (C) 2013 Canonical Ltd. |
1085 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1086 | # Author: Barry Warsaw <barry@ubuntu.com> |
1087 | |
1088 | # This program is free software: you can redistribute it and/or modify |
1089 | @@ -28,10 +28,9 @@ |
1090 | |
1091 | from contextlib import ExitStack |
1092 | from dbus.mainloop.glib import DBusGMainLoop |
1093 | -from dbus.service import BusName |
1094 | from pkg_resources import resource_string as resource_bytes |
1095 | from systemimage.config import config |
1096 | -from systemimage.dbus import Loop, Service |
1097 | +from systemimage.dbus import Loop |
1098 | from systemimage.helpers import makedirs |
1099 | from systemimage.logging import initialize |
1100 | from systemimage.main import DEFAULT_CONFIG_FILE |
1101 | @@ -92,12 +91,20 @@ |
1102 | initialize(verbosity=args.verbose) |
1103 | log = logging.getLogger('systemimage') |
1104 | |
1105 | - log.info('SystemImage dbus main loop started [{}/{}]', |
1106 | - config.channel, config.device) |
1107 | DBusGMainLoop(set_as_default=True) |
1108 | |
1109 | system_bus = dbus.SystemBus() |
1110 | - bus_name = BusName('com.canonical.SystemImage', system_bus) |
1111 | + # Ensure we're the only owner of this bus name. |
1112 | + code = system_bus.request_name( |
1113 | + 'com.canonical.SystemImage', |
1114 | + dbus.bus.NAME_FLAG_DO_NOT_QUEUE) |
1115 | + if code == dbus.bus.REQUEST_NAME_REPLY_EXISTS: |
1116 | + # Another instance already owns this name. Exit. |
1117 | + log.error('Cannot get exclusive ownership of bus name.') |
1118 | + sys.exit(2) |
1119 | + |
1120 | + log.info('SystemImage dbus main loop starting [{}/{}]', |
1121 | + config.channel, config.device) |
1122 | |
1123 | with ExitStack() as stack: |
1124 | loop = Loop() |
1125 | @@ -107,6 +114,7 @@ |
1126 | config.dbus_service = get_service( |
1127 | testing_mode, system_bus, '/Service', loop) |
1128 | else: |
1129 | + from systemimage.dbus import Service |
1130 | config.dbus_service = Service(system_bus, '/Service', loop) |
1131 | try: |
1132 | loop.run() |
1133 | |
1134 | === modified file 'systemimage/settings.py' |
1135 | --- systemimage/settings.py 2013-12-13 13:55:51 +0000 |
1136 | +++ systemimage/settings.py 2014-02-25 17:45:25 +0000 |
1137 | @@ -1,4 +1,4 @@ |
1138 | -# Copyright (C) 2013 Canonical Ltd. |
1139 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1140 | # Author: Barry Warsaw <barry@ubuntu.com> |
1141 | |
1142 | # This program is free software: you can redistribute it and/or modify |
1143 | |
1144 | === modified file 'systemimage/state.py' |
1145 | --- systemimage/state.py 2013-12-13 13:55:51 +0000 |
1146 | +++ systemimage/state.py 2014-02-25 17:45:25 +0000 |
1147 | @@ -1,4 +1,4 @@ |
1148 | -# Copyright (C) 2013 Canonical Ltd. |
1149 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1150 | # Author: Barry Warsaw <barry@ubuntu.com> |
1151 | |
1152 | # This program is free software: you can redistribute it and/or modify |
1153 | @@ -217,7 +217,7 @@ |
1154 | urljoin(config.service.https_base, url))) |
1155 | get_keyring('blacklist', url, 'image-master') |
1156 | except SignatureError: |
1157 | - log.info('No signed blacklist found') |
1158 | + log.exception('No signed blacklist found') |
1159 | # The blacklist wasn't signed by the system image master. Maybe |
1160 | # there's a new system image master key? Let's find out. |
1161 | self._next.appendleft(self._get_master_key) |
1162 | @@ -293,14 +293,16 @@ |
1163 | # SIGNING key. There may or may not be a blacklist. |
1164 | ctx = stack.enter_context( |
1165 | Context(config.gpg.image_signing, blacklist=self.blacklist)) |
1166 | - if not ctx.verify(asc_path, channels_path): |
1167 | + try: |
1168 | + ctx.validate(asc_path, channels_path) |
1169 | + except SignatureError: |
1170 | # The signature on the channels.json file did not match. |
1171 | # Maybe there's a new image signing key on the server. If |
1172 | # we've already downloaded a new image signing key, then |
1173 | # there's nothing more to do but raise an exception. |
1174 | # Otherwise, if a new key *is* found, retry the current step. |
1175 | if count > 0: |
1176 | - raise SignatureError(channels_path) |
1177 | + raise |
1178 | self._next.appendleft(self._get_signing_key) |
1179 | log.info('channels.json not properly signed') |
1180 | return |
1181 | @@ -398,10 +400,7 @@ |
1182 | keyrings.append(config.gpg.device_signing) |
1183 | ctx = stack.enter_context( |
1184 | Context(*keyrings, blacklist=self.blacklist)) |
1185 | - if not ctx.verify(asc_path, index_path): |
1186 | - log.error('index.json signature failure: {} {}', |
1187 | - index_path, asc_path) |
1188 | - raise SignatureError(index_path) |
1189 | + ctx.validate(asc_path, index_path) |
1190 | # The signature was good. |
1191 | with open(index_path, encoding='utf-8') as fp: |
1192 | self.index = Index.from_json(fp.read()) |
1193 | @@ -512,8 +511,7 @@ |
1194 | # Verify the signatures on all the downloaded files. |
1195 | with Context(*keyrings, blacklist=self.blacklist) as ctx: |
1196 | for dst, asc in signatures: |
1197 | - if not ctx.verify(asc, dst): |
1198 | - raise SignatureError(dst) |
1199 | + ctx.validate(asc, dst) |
1200 | # Verify the checksums. |
1201 | for dst, checksum in checksums: |
1202 | with open(dst, 'rb') as fp: |
1203 | |
1204 | === modified file 'systemimage/testing/controller.py' |
1205 | --- systemimage/testing/controller.py 2013-12-13 13:55:51 +0000 |
1206 | +++ systemimage/testing/controller.py 2014-02-25 17:45:25 +0000 |
1207 | @@ -1,4 +1,4 @@ |
1208 | -# Copyright (C) 2013 Canonical Ltd. |
1209 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1210 | # Author: Barry Warsaw <barry@ubuntu.com> |
1211 | |
1212 | # This program is free software: you can redistribute it and/or modify |
1213 | @@ -25,8 +25,8 @@ |
1214 | import pwd |
1215 | import sys |
1216 | import dbus |
1217 | +import time |
1218 | import psutil |
1219 | -import signal |
1220 | import subprocess |
1221 | |
1222 | from contextlib import ExitStack |
1223 | @@ -38,6 +38,8 @@ |
1224 | |
1225 | |
1226 | SPACE = ' ' |
1227 | +OVERRIDE = os.environ.get('SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS') |
1228 | +HUP_SLEEP = (0 if OVERRIDE is None else int(OVERRIDE)) |
1229 | |
1230 | |
1231 | def start_system_image(controller): |
1232 | @@ -125,7 +127,7 @@ |
1233 | class Controller: |
1234 | """Start and stop D-Bus service under test.""" |
1235 | |
1236 | - def __init__(self): |
1237 | + def __init__(self, logfile=None): |
1238 | # Non-public. |
1239 | self._stack = ExitStack() |
1240 | self._stoppers = [] |
1241 | @@ -148,21 +150,22 @@ |
1242 | # We need a client.ini file for the subprocess. |
1243 | ini_tmpdir = self._stack.enter_context(temporary_directory()) |
1244 | ini_vardir = self._stack.enter_context(temporary_directory()) |
1245 | + ini_logfile = (os.path.join(ini_tmpdir, 'client.log') |
1246 | + if logfile is None |
1247 | + else logfile) |
1248 | self.ini_path = os.path.join(self.tmpdir, 'client.ini') |
1249 | template = resource_bytes( |
1250 | 'systemimage.tests.data', 'config_03.ini').decode('utf-8') |
1251 | with open(self.ini_path, 'w', encoding='utf-8') as fp: |
1252 | - print(template.format(tmpdir=ini_tmpdir, vardir=ini_vardir), |
1253 | + print(template.format(tmpdir=ini_tmpdir, vardir=ini_vardir, |
1254 | + logfile=ini_logfile), |
1255 | file=fp) |
1256 | |
1257 | def _configure_services(self): |
1258 | - # If the daemon is already running, kill all the children and HUP the |
1259 | - # daemon to reset dbus activation. |
1260 | + # If the dbus-daemon is already running, kill all the children. |
1261 | if self.daemon_pid is not None: |
1262 | for stopper in self._stoppers: |
1263 | stopper(self) |
1264 | - process = psutil.Process(self.daemon_pid) |
1265 | - process.send_signal(signal.SIGHUP) |
1266 | del self._stoppers[:] |
1267 | # Now we have to set up the .service files. We use the Python |
1268 | # executable used to run the tests, executing the entry point as would |
1269 | @@ -178,6 +181,12 @@ |
1270 | with open(service_path, 'w', encoding='utf-8') as fp: |
1271 | fp.write(config) |
1272 | self._stoppers.append(stopper) |
1273 | + # If the dbus-daemon is running, reload its configuration files. |
1274 | + if self.daemon_pid is not None: |
1275 | + service = dbus.SystemBus().get_object('org.freedesktop.DBus', '/') |
1276 | + iface = dbus.Interface(service, 'org.freedesktop.DBus') |
1277 | + iface.ReloadConfig() |
1278 | + time.sleep(HUP_SLEEP) |
1279 | |
1280 | def set_mode(self, *, cert_pem=None, service_mode=''): |
1281 | self.mode = service_mode |
1282 | |
1283 | === modified file 'systemimage/testing/dbus.py' |
1284 | --- systemimage/testing/dbus.py 2013-12-13 13:55:51 +0000 |
1285 | +++ systemimage/testing/dbus.py 2014-02-25 17:45:25 +0000 |
1286 | @@ -1,4 +1,4 @@ |
1287 | -# Copyright (C) 2013 Canonical Ltd. |
1288 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1289 | # Author: Barry Warsaw <barry@ubuntu.com> |
1290 | |
1291 | # This program is free software: you can redistribute it and/or modify |
1292 | @@ -31,6 +31,8 @@ |
1293 | from systemimage.helpers import makedirs, safe_remove |
1294 | from unittest.mock import patch |
1295 | |
1296 | +from systemimage.testing.helpers import debug |
1297 | + |
1298 | |
1299 | SPACE = ' ' |
1300 | SIGNAL_DELAY_SECS = 5 |
1301 | @@ -65,7 +67,11 @@ |
1302 | @method('com.canonical.SystemImage') |
1303 | def Reset(self): |
1304 | self._api = Mediator() |
1305 | - self._checking = False |
1306 | + try: |
1307 | + self._checking.release() |
1308 | + except RuntimeError: |
1309 | + # Lock is already released. |
1310 | + pass |
1311 | self._update = None |
1312 | self._downloading = False |
1313 | self._rebootable = False |
1314 | |
1315 | === modified file 'systemimage/testing/demo.py' |
1316 | --- systemimage/testing/demo.py 2013-12-13 13:55:51 +0000 |
1317 | +++ systemimage/testing/demo.py 2014-02-25 17:45:25 +0000 |
1318 | @@ -1,4 +1,4 @@ |
1319 | -# Copyright (C) 2013 Canonical Ltd. |
1320 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1321 | # Author: Barry Warsaw <barry@ubuntu.com> |
1322 | |
1323 | # This program is free software: you can redistribute it and/or modify |
1324 | |
1325 | === modified file 'systemimage/testing/helpers.py' |
1326 | --- systemimage/testing/helpers.py 2013-12-13 13:55:51 +0000 |
1327 | +++ systemimage/testing/helpers.py 2014-02-25 17:45:25 +0000 |
1328 | @@ -1,4 +1,4 @@ |
1329 | -# Copyright (C) 2013 Canonical Ltd. |
1330 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1331 | # Author: Barry Warsaw <barry@ubuntu.com> |
1332 | |
1333 | # This program is free software: you can redistribute it and/or modify |
1334 | @@ -107,6 +107,12 @@ |
1335 | # Please shut up. |
1336 | pass |
1337 | |
1338 | + def handle_one_request(self): |
1339 | + try: |
1340 | + super().handle_one_request() |
1341 | + except ConnectionResetError: |
1342 | + super().handle_one_request() |
1343 | + |
1344 | def do_GET(self): |
1345 | # If we requested the magic 'user-agent.txt' file, send back the |
1346 | # value of the User-Agent header. Otherwise, vend as normal. |
1347 | @@ -166,7 +172,16 @@ |
1348 | for conn in connections: |
1349 | if conn.fileno() != -1: |
1350 | # Disallow sends and receives. |
1351 | - conn.shutdown(SHUT_RDWR) |
1352 | + try: |
1353 | + conn.shutdown(SHUT_RDWR) |
1354 | + except OSError: |
1355 | + # I'm ignoring all OSErrors here, although the only |
1356 | + # one I've seen semi-consistency is ENOTCONN [107] |
1357 | + # "Transport endpoint is not connected". I don't know |
1358 | + # why this happens, but it tells me that the client |
1359 | + # has already exited. We're shutting down, so who |
1360 | + # cares? (Or am I masking a real error?) |
1361 | + pass |
1362 | conn.close() |
1363 | server.shutdown() |
1364 | thread.join() |
1365 | @@ -332,7 +347,7 @@ |
1366 | setup_keyring_txz(keyring + '.gpg', signing_kr, json_data, dst) |
1367 | |
1368 | |
1369 | -def setup_index(index, todir, keyring): |
1370 | +def setup_index(index, todir, keyring, write_callback=None): |
1371 | for image in get_index(index).images: |
1372 | for filerec in image.files: |
1373 | path = (filerec.path[1:] |
1374 | @@ -340,10 +355,13 @@ |
1375 | else filerec.path) |
1376 | dst = os.path.join(todir, path) |
1377 | makedirs(os.path.dirname(dst)) |
1378 | - contents = EMPTYSTRING.join( |
1379 | - os.path.splitext(filerec.path)[0].split('/')) |
1380 | - with open(dst, 'w', encoding='utf-8') as fp: |
1381 | - fp.write(contents) |
1382 | + if write_callback is None: |
1383 | + contents = EMPTYSTRING.join( |
1384 | + os.path.splitext(filerec.path)[0].split('/')) |
1385 | + with open(dst, 'w', encoding='utf-8') as fp: |
1386 | + fp.write(contents) |
1387 | + else: |
1388 | + write_callback(dst) |
1389 | # Sign with the specified signing key. |
1390 | sign(dst, keyring) |
1391 | |
1392 | |
1393 | === modified file 'systemimage/testing/nose.py' |
1394 | --- systemimage/testing/nose.py 2013-12-13 13:55:51 +0000 |
1395 | +++ systemimage/testing/nose.py 2014-02-25 17:45:25 +0000 |
1396 | @@ -1,4 +1,4 @@ |
1397 | -# Copyright (C) 2013 Canonical Ltd. |
1398 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1399 | # Author: Barry Warsaw <barry@ubuntu.com> |
1400 | |
1401 | # This program is free software: you can redistribute it and/or modify |
1402 | @@ -75,15 +75,24 @@ |
1403 | super().__init__() |
1404 | self.patterns = [] |
1405 | self.verbosity = 0 |
1406 | + self.log_file = None |
1407 | self.addArgument(self.patterns, 'P', 'pattern', |
1408 | 'Add a test matching pattern') |
1409 | def bump(ignore): |
1410 | self.verbosity += 1 |
1411 | self.addFlag(bump, 'V', 'Verbosity', |
1412 | 'Increase system-image verbosity') |
1413 | + def set_log_file(path): |
1414 | + self.log_file = path[0] |
1415 | + self.addOption(set_log_file, 'L', 'logfile', |
1416 | + 'Set the log file for the test run', |
1417 | + nargs=1) |
1418 | |
1419 | @configuration |
1420 | def startTestRun(self, event): |
1421 | + from systemimage.config import config |
1422 | + if self.log_file is not None: |
1423 | + config.system.logfile = self.log_file |
1424 | DBusGMainLoop(set_as_default=True) |
1425 | initialize(verbosity=self.verbosity) |
1426 | # We need to set up the dbus service controller, since all the tests |
1427 | @@ -92,7 +101,7 @@ |
1428 | # individual services, and we can write new dbus configuration files |
1429 | # and HUP the dbus-launch to re-read them, but we cannot change bus |
1430 | # addresses after the initial one is set. |
1431 | - SystemImagePlugin.controller = Controller() |
1432 | + SystemImagePlugin.controller = Controller(self.log_file) |
1433 | SystemImagePlugin.controller.start() |
1434 | atexit.register(SystemImagePlugin.controller.stop) |
1435 | |
1436 | @@ -123,3 +132,13 @@ |
1437 | SystemImagePlugin.controller.stop() |
1438 | # Let other plugins continue printing. |
1439 | return None |
1440 | + |
1441 | + ## def startTest(self, event): |
1442 | + ## from systemimage.testing.helpers import debug |
1443 | + ## with debug() as dlog: |
1444 | + ## dlog('vvvvv', event.test) |
1445 | + |
1446 | + ## def stopTest(self, event): |
1447 | + ## from systemimage.testing.helpers import debug |
1448 | + ## with debug() as dlog: |
1449 | + ## dlog('^^^^^', event.test) |
1450 | |
1451 | === modified file 'systemimage/tests/data/config_03.ini' |
1452 | --- systemimage/tests/data/config_03.ini 2013-12-13 13:55:51 +0000 |
1453 | +++ systemimage/tests/data/config_03.ini 2014-02-25 17:45:25 +0000 |
1454 | @@ -14,7 +14,7 @@ |
1455 | timeout: 1s |
1456 | build_file: {tmpdir}/ubuntu-build |
1457 | tempdir: {tmpdir}/tmp |
1458 | -logfile: {tmpdir}/client.log |
1459 | +logfile: {logfile} |
1460 | loglevel: info |
1461 | settings_db: {vardir}/settings.db |
1462 | |
1463 | |
1464 | === added file 'systemimage/tests/data/index_24.json' |
1465 | --- systemimage/tests/data/index_24.json 1970-01-01 00:00:00 +0000 |
1466 | +++ systemimage/tests/data/index_24.json 2014-02-25 17:45:25 +0000 |
1467 | @@ -0,0 +1,36 @@ |
1468 | +{ |
1469 | + "global": { |
1470 | + "generated_at": "Thu Aug 01 08:01:00 UTC 2013" |
1471 | + }, |
1472 | + "images": [ |
1473 | + { |
1474 | + "description": "Full", |
1475 | + "files": [ |
1476 | + { |
1477 | + "checksum": "5b05b298e974f3b9e40f0a1a8188f50984a4f18fb329e050324296632d3d9dfc", |
1478 | + "order": 3, |
1479 | + "path": "/3/4/5.txt", |
1480 | + "signature": "/3/4/5.txt.asc", |
1481 | + "size": 104857600 |
1482 | + }, |
1483 | + { |
1484 | + "checksum": "5b05b298e974f3b9e40f0a1a8188f50984a4f18fb329e050324296632d3d9dfc", |
1485 | + "order": 1, |
1486 | + "path": "/4/5/6.txt", |
1487 | + "signature": "/4/5/6.txt.asc", |
1488 | + "size": 104857600 |
1489 | + }, |
1490 | + { |
1491 | + "checksum": "5b05b298e974f3b9e40f0a1a8188f50984a4f18fb329e050324296632d3d9dfc", |
1492 | + "order": 2, |
1493 | + "path": "/5/6/7.txt", |
1494 | + "signature": "/5/6/7.txt.asc", |
1495 | + "size": 104857600 |
1496 | + } |
1497 | + ], |
1498 | + "type": "full", |
1499 | + "version": 1600, |
1500 | + "bootme": true |
1501 | + } |
1502 | + ] |
1503 | +} |
1504 | |
1505 | === modified file 'systemimage/tests/test_api.py' |
1506 | --- systemimage/tests/test_api.py 2013-12-13 13:55:51 +0000 |
1507 | +++ systemimage/tests/test_api.py 2014-02-25 17:45:25 +0000 |
1508 | @@ -1,4 +1,4 @@ |
1509 | -# Copyright (C) 2013 Canonical Ltd. |
1510 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1511 | # Author: Barry Warsaw <barry@ubuntu.com> |
1512 | |
1513 | # This program is free software: you can redistribute it and/or modify |
1514 | |
1515 | === modified file 'systemimage/tests/test_bag.py' |
1516 | --- systemimage/tests/test_bag.py 2013-12-13 13:55:51 +0000 |
1517 | +++ systemimage/tests/test_bag.py 2014-02-25 17:45:25 +0000 |
1518 | @@ -1,4 +1,4 @@ |
1519 | -# Copyright (C) 2013 Canonical Ltd. |
1520 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1521 | # Author: Barry Warsaw <barry@ubuntu.com> |
1522 | |
1523 | # This program is free software: you can redistribute it and/or modify |
1524 | |
1525 | === modified file 'systemimage/tests/test_candidates.py' |
1526 | --- systemimage/tests/test_candidates.py 2013-12-13 13:55:51 +0000 |
1527 | +++ systemimage/tests/test_candidates.py 2014-02-25 17:45:25 +0000 |
1528 | @@ -1,4 +1,4 @@ |
1529 | -# Copyright (C) 2013 Canonical Ltd. |
1530 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1531 | # Author: Barry Warsaw <barry@ubuntu.com> |
1532 | |
1533 | # This program is free software: you can redistribute it and/or modify |
1534 | |
1535 | === modified file 'systemimage/tests/test_channel.py' |
1536 | --- systemimage/tests/test_channel.py 2013-12-13 13:55:51 +0000 |
1537 | +++ systemimage/tests/test_channel.py 2014-02-25 17:45:25 +0000 |
1538 | @@ -1,4 +1,4 @@ |
1539 | -# Copyright (C) 2013 Canonical Ltd. |
1540 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1541 | # Author: Barry Warsaw <barry@ubuntu.com> |
1542 | |
1543 | # This program is free software: you can redistribute it and/or modify |
1544 | |
1545 | === modified file 'systemimage/tests/test_config.py' |
1546 | --- systemimage/tests/test_config.py 2013-12-13 13:55:51 +0000 |
1547 | +++ systemimage/tests/test_config.py 2014-02-25 17:45:25 +0000 |
1548 | @@ -1,4 +1,4 @@ |
1549 | -# Copyright (C) 2013 Canonical Ltd. |
1550 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1551 | # Author: Barry Warsaw <barry@ubuntu.com> |
1552 | |
1553 | # This program is free software: you can redistribute it and/or modify |
1554 | |
1555 | === modified file 'systemimage/tests/test_dbus.py' |
1556 | --- systemimage/tests/test_dbus.py 2013-12-13 13:55:51 +0000 |
1557 | +++ systemimage/tests/test_dbus.py 2014-02-25 17:45:25 +0000 |
1558 | @@ -1,4 +1,4 @@ |
1559 | -# Copyright (C) 2013 Canonical Ltd. |
1560 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1561 | # Author: Barry Warsaw <barry@ubuntu.com> |
1562 | |
1563 | # This program is free software: you can redistribute it and/or modify |
1564 | @@ -23,6 +23,7 @@ |
1565 | 'TestDBusGetSet', |
1566 | 'TestDBusInfo', |
1567 | 'TestDBusInfoNoDetails', |
1568 | + 'TestDBusLP1277589', |
1569 | 'TestDBusMockFailApply', |
1570 | 'TestDBusMockFailPause', |
1571 | 'TestDBusMockFailResume', |
1572 | @@ -122,6 +123,23 @@ |
1573 | self.quit() |
1574 | |
1575 | |
1576 | +class DoubleCheckingReactor(Reactor): |
1577 | + def __init__(self, iface): |
1578 | + super().__init__(dbus.SystemBus()) |
1579 | + self.iface = iface |
1580 | + self.uas_signals = [] |
1581 | + self.react_to('UpdateAvailableStatus') |
1582 | + self.react_to('UpdateDownloaded') |
1583 | + self.schedule(self.iface.CheckForUpdate) |
1584 | + |
1585 | + def _do_UpdateAvailableStatus(self, signal, path, *args, **kws): |
1586 | + self.uas_signals.append(args) |
1587 | + self.schedule(self.iface.CheckForUpdate) |
1588 | + |
1589 | + def _do_UpdateDownloaded(self, *args, **kws): |
1590 | + self.quit() |
1591 | + |
1592 | + |
1593 | class _TestBase(unittest.TestCase): |
1594 | """Base class for all DBus testing.""" |
1595 | |
1596 | @@ -262,13 +280,14 @@ |
1597 | safe_remove(self.reboot_log) |
1598 | super().tearDown() |
1599 | |
1600 | - def _prepare_index(self, index_file): |
1601 | + def _prepare_index(self, index_file, write_callback=None): |
1602 | serverdir = SystemImagePlugin.controller.serverdir |
1603 | index_path = os.path.join(serverdir, 'stable', 'nexus7', 'index.json') |
1604 | head, tail = os.path.split(index_path) |
1605 | copy(index_file, head, tail) |
1606 | sign(index_path, 'device-signing.gpg') |
1607 | - setup_index(index_file, serverdir, 'device-signing.gpg') |
1608 | + setup_index(index_file, serverdir, 'device-signing.gpg', |
1609 | + write_callback) |
1610 | |
1611 | def _touch_build(self, version): |
1612 | # Unlike the touch_build() helper, this one uses our own config object |
1613 | @@ -374,6 +393,23 @@ |
1614 | self.assertEqual(last_update_date, '2013-01-20 12:01:45') |
1615 | # All other values are undefined. |
1616 | |
1617 | + def test_check_for_update_twice(self): |
1618 | + # Issue two CheckForUpdate calls immediate after each other. |
1619 | + self.download_always() |
1620 | + reactor = SignalCapturingReactor('UpdateAvailableStatus') |
1621 | + def two_calls(): |
1622 | + self.iface.CheckForUpdate() |
1623 | + self.iface.CheckForUpdate() |
1624 | + reactor.run(two_calls) |
1625 | + self.assertEqual(len(reactor.signals), 1) |
1626 | + # There's one boolean argument to the result. |
1627 | + (is_available, downloading, available_version, update_size, |
1628 | + last_update_date, |
1629 | + # descriptions, |
1630 | + error_reason) = reactor.signals[0] |
1631 | + self.assertTrue(is_available) |
1632 | + self.assertTrue(downloading) |
1633 | + |
1634 | @unittest.skip('LP: #1215586') |
1635 | def test_get_multilingual_descriptions(self): |
1636 | # The descriptions are multilingual. |
1637 | @@ -1368,7 +1404,9 @@ |
1638 | def setUp(self): |
1639 | super().setUp() |
1640 | # We have to hack the files to be rather large so that the download |
1641 | - # doesn't complete before we get a chance to pause it. |
1642 | + # doesn't complete before we get a chance to pause it. Of course, |
1643 | + # this breaks the signatures because we're changing the file contents |
1644 | + # after the .asc files have been written. |
1645 | for path in ('3/4/5.txt', '4/5/6.txt', '5/6/7.txt'): |
1646 | full_path = os.path.join( |
1647 | SystemImagePlugin.controller.serverdir, path) |
1648 | @@ -1391,15 +1429,25 @@ |
1649 | self.assertTrue(reactor.paused) |
1650 | # Now let's resume the download. Because we intentionally corrupted |
1651 | # the downloaded files, we'll get an UpdateFailed signal instead of |
1652 | - # the successful UpdateDownloaded signal. We can ignore that. |
1653 | + # the successful UpdateDownloaded signal. |
1654 | reactor = SignalCapturingReactor('UpdateFailed') |
1655 | reactor.run(self.iface.DownloadUpdate, timeout=60) |
1656 | self.assertEqual(len(reactor.signals), 1) |
1657 | - # We've gotten one error and the first file that failed is 5.txt. |
1658 | + # The error message will include lots of details on the SignatureError |
1659 | + # that results. The key thing is that it's 5.txt that is the first |
1660 | + # file to fail its signature check. |
1661 | failure_count, last_error = reactor.signals[0] |
1662 | self.assertEqual(failure_count, 1) |
1663 | - # Watch out for the trailing newline. |
1664 | - self.assertEqual(os.path.basename(last_error[:-1]), '5.txt') |
1665 | + check_next = False |
1666 | + for line in last_error.splitlines(): |
1667 | + line = line.strip() |
1668 | + if check_next: |
1669 | + self.assertEqual(os.path.basename(line), '5.txt') |
1670 | + break |
1671 | + if line.startswith('data path:'): |
1672 | + check_next = True |
1673 | + else: |
1674 | + raise AssertionError('Did not find expected error output') |
1675 | |
1676 | |
1677 | class TestDBusUseCache(_LiveTesting): |
1678 | @@ -1513,3 +1561,45 @@ |
1679 | update 5.txt 5.txt.asc |
1680 | unmount system |
1681 | """) |
1682 | + |
1683 | + |
1684 | +class TestDBusLP1277589(_LiveTesting): |
1685 | + def test_multiple_check_for_updates(self): |
1686 | + # Log analysis of LP: #1277589 appears to show the following scenario, |
1687 | + # reproduced in this test case: |
1688 | + # |
1689 | + # * Automatic updates are enabled. |
1690 | + # * No image signing or image master keys are present. |
1691 | + # * A full update is checked. |
1692 | + # - A new image master key and image signing key is downloaded. |
1693 | + # - Update is available |
1694 | + # |
1695 | + # Start by creating some big files which will take a while to |
1696 | + # download. |
1697 | + def write_callback(dst): |
1698 | + # Write a 100 MiB sized file. |
1699 | + with open(dst, 'wb') as fp: |
1700 | + for i in range(25600): |
1701 | + fp.write(b'x' * 4096) |
1702 | + self._prepare_index('index_24.json', write_callback) |
1703 | + # Create a reactor that will exit when the UpdateDownloaded signal is |
1704 | + # received. We're going to issue a CheckForUpdate with automatic |
1705 | + # updates enabled. As soon as we receive the UpdateAvailableStatus |
1706 | + # signal, we'll immediately issue *another* CheckForUpdate, which |
1707 | + # should run while the auto-download is working. |
1708 | + # |
1709 | + # At the end, we should not get another UpdateAvailableStatus signal, |
1710 | + # but we should get the UpdateDownloaded signal. |
1711 | + reactor = DoubleCheckingReactor(self.iface) |
1712 | + reactor.run() |
1713 | + self.assertEqual(len(reactor.uas_signals), 1) |
1714 | + (is_available, downloading, available_version, update_size, |
1715 | + last_update_date, |
1716 | + #descriptions, |
1717 | + error_reason) = reactor.uas_signals[0] |
1718 | + self.assertTrue(is_available) |
1719 | + self.assertTrue(downloading) |
1720 | + self.assertEqual(available_version, '1600') |
1721 | + self.assertEqual(update_size, 314572800) |
1722 | + self.assertEqual(last_update_date, 'Unknown') |
1723 | + self.assertEqual(error_reason, '') |
1724 | |
1725 | === modified file 'systemimage/tests/test_download.py' |
1726 | --- systemimage/tests/test_download.py 2013-12-13 13:55:51 +0000 |
1727 | +++ systemimage/tests/test_download.py 2014-02-25 17:45:25 +0000 |
1728 | @@ -1,4 +1,4 @@ |
1729 | -# Copyright (C) 2013 Canonical Ltd. |
1730 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1731 | # Author: Barry Warsaw <barry@ubuntu.com> |
1732 | |
1733 | # This program is free software: you can redistribute it and/or modify |
1734 | |
1735 | === modified file 'systemimage/tests/test_gpg.py' |
1736 | --- systemimage/tests/test_gpg.py 2013-12-13 13:55:51 +0000 |
1737 | +++ systemimage/tests/test_gpg.py 2014-02-25 17:45:25 +0000 |
1738 | @@ -1,4 +1,4 @@ |
1739 | -# Copyright (C) 2013 Canonical Ltd. |
1740 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1741 | # Author: Barry Warsaw <barry@ubuntu.com> |
1742 | |
1743 | # This program is free software: you can redistribute it and/or modify |
1744 | @@ -18,15 +18,20 @@ |
1745 | __all__ = [ |
1746 | 'TestKeyrings', |
1747 | 'TestSignature', |
1748 | + 'TestSignatureError', |
1749 | ] |
1750 | |
1751 | |
1752 | import os |
1753 | +import sys |
1754 | +import hashlib |
1755 | import unittest |
1756 | +import traceback |
1757 | |
1758 | from contextlib import ExitStack |
1759 | +from io import StringIO |
1760 | from systemimage.config import config |
1761 | -from systemimage.gpg import Context |
1762 | +from systemimage.gpg import Context, SignatureError |
1763 | from systemimage.helpers import temporary_directory |
1764 | from systemimage.testing.helpers import ( |
1765 | configuration, copy, setup_keyring_txz, setup_keyrings, sign) |
1766 | @@ -330,3 +335,171 @@ |
1767 | with Context(keyring_1, keyring_2, blacklist=blacklist) as ctx: |
1768 | self.assertFalse( |
1769 | ctx.verify(channels_json + '.asc', channels_json)) |
1770 | + |
1771 | + @configuration |
1772 | + def test_good_validation(self): |
1773 | + # The .validate() method does nothing if the signature is good. |
1774 | + channels_json = os.path.join(self._tmpdir, 'channels.json') |
1775 | + copy('channels_01.json', self._tmpdir, dst=channels_json) |
1776 | + sign(channels_json, 'image-signing.gpg') |
1777 | + with temporary_directory() as tmpdir: |
1778 | + keyring = os.path.join(tmpdir, 'image-signing.tar.xz') |
1779 | + setup_keyring_txz('image-signing.gpg', 'image-master.gpg', |
1780 | + dict(type='image-signing'), keyring) |
1781 | + with Context(keyring) as ctx: |
1782 | + self.assertIsNone( |
1783 | + ctx.validate(channels_json + '.asc', channels_json)) |
1784 | + |
1785 | + |
1786 | +class TestSignatureError(unittest.TestCase): |
1787 | + def setUp(self): |
1788 | + self._stack = ExitStack() |
1789 | + self._tmpdir = self._stack.enter_context(temporary_directory()) |
1790 | + |
1791 | + def tearDown(self): |
1792 | + self._stack.close() |
1793 | + |
1794 | + def test_extra_data(self): |
1795 | + # A SignatureError includes extra information about the path to the |
1796 | + # signature file, and the path to the data file. You also get the md5 |
1797 | + # checksums of those two paths. |
1798 | + signature_path = os.path.join(self._tmpdir, 'signature') |
1799 | + data_path = os.path.join(self._tmpdir, 'data') |
1800 | + with open(signature_path, 'wb') as fp: |
1801 | + fp.write(b'012345') |
1802 | + with open(data_path, 'wb') as fp: |
1803 | + fp.write(b'67890a') |
1804 | + error = SignatureError(signature_path, data_path) |
1805 | + self.assertEqual(error.signature_path, signature_path) |
1806 | + self.assertEqual(error.data_path, data_path) |
1807 | + self.assertEqual( |
1808 | + error.signature_checksum, 'd6a9a933c8aafc51e55ac0662b6e4d4a') |
1809 | + self.assertEqual( |
1810 | + error.data_checksum, 'e82780258de250078f7ad3f595d71f6d') |
1811 | + |
1812 | + @configuration |
1813 | + def test_signature_invalid(self): |
1814 | + # The .validate() method raises a SignatureError exception with extra |
1815 | + # information when the signature is invalid. |
1816 | + channels_json = os.path.join(self._tmpdir, 'channels.json') |
1817 | + copy('channels_01.json', self._tmpdir, dst=channels_json) |
1818 | + sign(channels_json, 'device-signing.gpg') |
1819 | + # Verify the signature with the pubkey. |
1820 | + with temporary_directory() as tmpdir: |
1821 | + dst = os.path.join(tmpdir, 'image-signing.tar.xz') |
1822 | + setup_keyring_txz('image-signing.gpg', 'image-master.gpg', |
1823 | + dict(type='image-signing'), dst) |
1824 | + # Get the dst's checksum now, because the file will get deleted |
1825 | + # when the tmpdir context manager exits. |
1826 | + with open(dst, 'rb') as fp: |
1827 | + dst_checksum = hashlib.md5(fp.read()).hexdigest() |
1828 | + with Context(dst) as ctx: |
1829 | + with self.assertRaises(SignatureError) as cm: |
1830 | + ctx.validate(channels_json + '.asc', channels_json) |
1831 | + error = cm.exception |
1832 | + basename = os.path.basename |
1833 | + self.assertEqual(basename(error.signature_path), 'channels.json.asc') |
1834 | + self.assertEqual(basename(error.data_path), 'channels.json') |
1835 | + # The contents of the signature file are not predictable. |
1836 | + with open(channels_json + '.asc', 'rb') as fp: |
1837 | + checksum = hashlib.md5(fp.read()).hexdigest() |
1838 | + self.assertEqual(error.signature_checksum, checksum) |
1839 | + self.assertEqual( |
1840 | + error.data_checksum, '715c63fecbf44b62f9fa04a82dfa7d29') |
1841 | + basenames = [basename(path) for path in error.keyrings] |
1842 | + self.assertEqual(basenames, ['image-signing.tar.xz']) |
1843 | + self.assertIsNone(error.blacklist) |
1844 | + self.assertEqual(error.keyring_checksums, [dst_checksum]) |
1845 | + self.assertIsNone(error.blacklist_checksum) |
1846 | + |
1847 | + @configuration |
1848 | + def test_signature_invalid_due_to_blacklist(self): |
1849 | + # Like above, but we put the device signing key id in the blacklist. |
1850 | + channels_json = os.path.join(self._tmpdir, 'channels.json') |
1851 | + copy('channels_01.json', self._tmpdir, dst=channels_json) |
1852 | + sign(channels_json, 'device-signing.gpg') |
1853 | + # Verify the signature with the pubkey. |
1854 | + with temporary_directory() as tmpdir: |
1855 | + keyring_1 = os.path.join(tmpdir, 'image-signing.tar.xz') |
1856 | + keyring_2 = os.path.join(tmpdir, 'device-signing.tar.xz') |
1857 | + blacklist = os.path.join(tmpdir, 'blacklist.tar.xz') |
1858 | + setup_keyring_txz('image-signing.gpg', 'image-master.gpg', |
1859 | + dict(type='image-signing'), keyring_1) |
1860 | + setup_keyring_txz('device-signing.gpg', 'image-signing.gpg', |
1861 | + dict(type='device-signing'), keyring_2) |
1862 | + # We're letting the device signing pubkey stand in for a blacklist. |
1863 | + setup_keyring_txz('device-signing.gpg', 'image-master.gpg', |
1864 | + dict(type='blacklist'), blacklist) |
1865 | + # Get the keyring checksums now, because the files will get |
1866 | + # deleted when the tmpdir context manager exits. |
1867 | + keyring_checksums = [] |
1868 | + for path in (keyring_1, keyring_2): |
1869 | + with open(path, 'rb') as fp: |
1870 | + checksum = hashlib.md5(fp.read()).hexdigest() |
1871 | + keyring_checksums.append(checksum) |
1872 | + with open(blacklist, 'rb') as fp: |
1873 | + blacklist_checksum = hashlib.md5(fp.read()).hexdigest() |
1874 | + with Context(keyring_1, keyring_2, blacklist=blacklist) as ctx: |
1875 | + with self.assertRaises(SignatureError) as cm: |
1876 | + ctx.validate(channels_json + '.asc', channels_json) |
1877 | + error = cm.exception |
1878 | + basename = os.path.basename |
1879 | + self.assertEqual(basename(error.signature_path), 'channels.json.asc') |
1880 | + self.assertEqual(basename(error.data_path), 'channels.json') |
1881 | + # The contents of the signature file are not predictable. |
1882 | + with open(channels_json + '.asc', 'rb') as fp: |
1883 | + checksum = hashlib.md5(fp.read()).hexdigest() |
1884 | + self.assertEqual(error.signature_checksum, checksum) |
1885 | + self.assertEqual( |
1886 | + error.data_checksum, '715c63fecbf44b62f9fa04a82dfa7d29') |
1887 | + basenames = [basename(path) for path in error.keyrings] |
1888 | + self.assertEqual(basenames, ['image-signing.tar.xz', |
1889 | + 'device-signing.tar.xz']) |
1890 | + self.assertEqual(basename(error.blacklist), 'blacklist.tar.xz') |
1891 | + self.assertEqual(error.keyring_checksums, keyring_checksums) |
1892 | + self.assertEqual(error.blacklist_checksum, blacklist_checksum) |
1893 | + |
1894 | + @configuration |
1895 | + def test_signature_error_logging(self): |
1896 | + # The repr/str of the SignatureError should contain lots of useful |
1897 | + # information that will make debugging easier. |
1898 | + channels_json = os.path.join(self._tmpdir, 'channels.json') |
1899 | + copy('channels_01.json', self._tmpdir, dst=channels_json) |
1900 | + sign(channels_json, 'device-signing.gpg') |
1901 | + # Verify the signature with the pubkey. |
1902 | + tmpdir = self._stack.enter_context(temporary_directory()) |
1903 | + dst = os.path.join(tmpdir, 'image-signing.tar.xz') |
1904 | + setup_keyring_txz('image-signing.gpg', 'image-master.gpg', |
1905 | + dict(type='image-signing'), dst) |
1906 | + output = StringIO() |
1907 | + with Context(dst) as ctx: |
1908 | + try: |
1909 | + ctx.validate(channels_json + '.asc', channels_json) |
1910 | + except SignatureError: |
1911 | + # For our purposes, log.exception() is essentially a wrapper |
1912 | + # around this traceback call. We don't really care about the |
1913 | + # full stack trace though. |
1914 | + e = sys.exc_info() |
1915 | + traceback.print_exception(e[0], e[1], e[2], |
1916 | + limit=0, file=output) |
1917 | + # 2014-02-12 BAW: Yuck, but I can't get assertRegex() to work properly. |
1918 | + for i, line in enumerate(output.getvalue().splitlines()): |
1919 | + if i == 0: |
1920 | + self.assertEqual(line, 'Traceback (most recent call last):') |
1921 | + elif i == 1: |
1922 | + self.assertEqual(line, 'systemimage.gpg.SignatureError: ') |
1923 | + elif i == 2: |
1924 | + self.assertTrue(line.startswith(' sig path :')) |
1925 | + elif i == 3: |
1926 | + self.assertTrue(line.endswith('/channels.json.asc')) |
1927 | + elif i == 4: |
1928 | + self.assertEqual( |
1929 | + line, ' data path: 715c63fecbf44b62f9fa04a82dfa7d29') |
1930 | + elif i == 5: |
1931 | + self.assertTrue(line.endswith('/channels.json')) |
1932 | + elif i == 6: |
1933 | + self.assertTrue(line.startswith(' keyrings :')) |
1934 | + elif i == 7: |
1935 | + self.assertTrue(line.endswith("/image-signing.tar.xz']")) |
1936 | + elif i == 8: |
1937 | + self.assertEqual(line, ' blacklist: no blacklist ') |
1938 | |
1939 | === modified file 'systemimage/tests/test_helpers.py' |
1940 | --- systemimage/tests/test_helpers.py 2013-12-13 13:55:51 +0000 |
1941 | +++ systemimage/tests/test_helpers.py 2014-02-25 17:45:25 +0000 |
1942 | @@ -1,4 +1,4 @@ |
1943 | -# Copyright (C) 2013 Canonical Ltd. |
1944 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1945 | # Author: Barry Warsaw <barry@ubuntu.com> |
1946 | |
1947 | # This program is free software: you can redistribute it and/or modify |
1948 | |
1949 | === modified file 'systemimage/tests/test_image.py' |
1950 | --- systemimage/tests/test_image.py 2013-12-13 13:55:51 +0000 |
1951 | +++ systemimage/tests/test_image.py 2014-02-25 17:45:25 +0000 |
1952 | @@ -1,4 +1,4 @@ |
1953 | -# Copyright (C) 2013 Canonical Ltd. |
1954 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1955 | # Author: Barry Warsaw <barry@ubuntu.com> |
1956 | |
1957 | # This program is free software: you can redistribute it and/or modify |
1958 | |
1959 | === modified file 'systemimage/tests/test_index.py' |
1960 | --- systemimage/tests/test_index.py 2013-12-13 13:55:51 +0000 |
1961 | +++ systemimage/tests/test_index.py 2014-02-25 17:45:25 +0000 |
1962 | @@ -1,4 +1,4 @@ |
1963 | -# Copyright (C) 2013 Canonical Ltd. |
1964 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1965 | # Author: Barry Warsaw <barry@ubuntu.com> |
1966 | |
1967 | # This program is free software: you can redistribute it and/or modify |
1968 | |
1969 | === modified file 'systemimage/tests/test_keyring.py' |
1970 | --- systemimage/tests/test_keyring.py 2013-12-13 13:55:51 +0000 |
1971 | +++ systemimage/tests/test_keyring.py 2014-02-25 17:45:25 +0000 |
1972 | @@ -1,4 +1,4 @@ |
1973 | -# Copyright (C) 2013 Canonical Ltd. |
1974 | +# Copyright (C) 2013-2014 Canonical Ltd. |
1975 | # Author: Barry Warsaw <barry@ubuntu.com> |
1976 | |
1977 | # This program is free software: you can redistribute it and/or modify |
1978 | @@ -22,6 +22,7 @@ |
1979 | |
1980 | |
1981 | import os |
1982 | +import hashlib |
1983 | import unittest |
1984 | |
1985 | from contextlib import ExitStack |
1986 | @@ -174,11 +175,25 @@ |
1987 | setup_keyrings() |
1988 | # Use the spare key as the blacklist, signed by itself. Since this |
1989 | # won't match the image-signing key, the check will fail. |
1990 | + server_path = os.path.join(self._serverdir, 'gpg', 'blacklist.tar.xz') |
1991 | setup_keyring_txz( |
1992 | - 'spare.gpg', 'spare.gpg', dict(type='blacklist'), |
1993 | - os.path.join(self._serverdir, 'gpg', 'blacklist.tar.xz')) |
1994 | - self.assertRaises(SignatureError, get_keyring, |
1995 | - 'blacklist', 'gpg/blacklist.tar.xz', 'image-master') |
1996 | + 'spare.gpg', 'spare.gpg', dict(type='blacklist'), server_path) |
1997 | + with self.assertRaises(SignatureError) as cm: |
1998 | + get_keyring('blacklist', 'gpg/blacklist.tar.xz', 'image-master') |
1999 | + error = cm.exception |
2000 | + # The local file name will be keyring.tar.xz in the cache directory. |
2001 | + basename = os.path.basename |
2002 | + self.assertEqual(basename(error.data_path), 'keyring.tar.xz') |
2003 | + self.assertEqual(basename(error.signature_path), 'keyring.tar.xz.asc') |
2004 | + # The crafted blacklist.tar.xz file will have an unpredictable |
2005 | + # checksum due to tarfile variablility. |
2006 | + with open(server_path, 'rb') as fp: |
2007 | + checksum = hashlib.md5(fp.read()).hexdigest() |
2008 | + self.assertEqual(error.data_checksum, checksum) |
2009 | + # The signature file's checksum is also unpredictable. |
2010 | + with open(server_path + '.asc', 'rb') as fp: |
2011 | + checksum = hashlib.md5(fp.read()).hexdigest() |
2012 | + self.assertEqual(error.signature_checksum, checksum) |
2013 | |
2014 | @configuration |
2015 | def test_blacklisted_signature(self): |
2016 | |
2017 | === modified file 'systemimage/tests/test_main.py' |
2018 | --- systemimage/tests/test_main.py 2013-12-13 13:55:51 +0000 |
2019 | +++ systemimage/tests/test_main.py 2014-02-25 17:45:25 +0000 |
2020 | @@ -1,4 +1,4 @@ |
2021 | -# Copyright (C) 2013 Canonical Ltd. |
2022 | +# Copyright (C) 2013-2014 Canonical Ltd. |
2023 | # Author: Barry Warsaw <barry@ubuntu.com> |
2024 | |
2025 | # This program is free software: you can redistribute it and/or modify |
2026 | @@ -26,11 +26,13 @@ |
2027 | |
2028 | |
2029 | import os |
2030 | +import sys |
2031 | import dbus |
2032 | import stat |
2033 | import time |
2034 | import shutil |
2035 | import unittest |
2036 | +import subprocess |
2037 | |
2038 | from contextlib import ExitStack, contextmanager |
2039 | from datetime import datetime |
2040 | @@ -781,3 +783,25 @@ |
2041 | self.assertEqual(stat.filemode(mode), 'drwx--S---') |
2042 | mode = os.stat(config.system.logfile).st_mode |
2043 | self.assertEqual(stat.filemode(mode), '-rw-------') |
2044 | + |
2045 | + def test_single_instance(self): |
2046 | + # Only one instance of the system-image-dbus service is allowed to |
2047 | + # remain active on a single system bus. |
2048 | + self.assertIsNone(find_dbus_process(self.ini_path)) |
2049 | + self._activate() |
2050 | + proc = find_dbus_process(self.ini_path) |
2051 | + # Attempt to start a second process on the same system bus. |
2052 | + env = dict( |
2053 | + DBUS_SYSTEM_BUS_ADDRESS=os.environ['DBUS_SYSTEM_BUS_ADDRESS']) |
2054 | + args = (sys.executable, '-m', 'systemimage.service', |
2055 | + '-C', self.ini_path) |
2056 | + second = subprocess.Popen(args, universal_newlines=True, env=env) |
2057 | + # Allow a TimeoutExpired exception to fail the test. |
2058 | + try: |
2059 | + code = second.wait(timeout=10) |
2060 | + except subprocess.TimeoutExpired: |
2061 | + second.kill() |
2062 | + second.communicate() |
2063 | + raise |
2064 | + self.assertNotEqual(second.pid, proc) |
2065 | + self.assertEqual(code, 2) |
2066 | |
2067 | === modified file 'systemimage/tests/test_scores.py' |
2068 | --- systemimage/tests/test_scores.py 2013-12-13 13:55:51 +0000 |
2069 | +++ systemimage/tests/test_scores.py 2014-02-25 17:45:25 +0000 |
2070 | @@ -1,4 +1,4 @@ |
2071 | -# Copyright (C) 2013 Canonical Ltd. |
2072 | +# Copyright (C) 2013-2014 Canonical Ltd. |
2073 | # Author: Barry Warsaw <barry@ubuntu.com> |
2074 | |
2075 | # This program is free software: you can redistribute it and/or modify |
2076 | |
2077 | === modified file 'systemimage/tests/test_settings.py' |
2078 | --- systemimage/tests/test_settings.py 2013-12-13 13:55:51 +0000 |
2079 | +++ systemimage/tests/test_settings.py 2014-02-25 17:45:25 +0000 |
2080 | @@ -1,4 +1,4 @@ |
2081 | -# Copyright (C) 2013 Canonical Ltd. |
2082 | +# Copyright (C) 2013-2014 Canonical Ltd. |
2083 | # Author: Barry Warsaw <barry@ubuntu.com> |
2084 | |
2085 | # This program is free software: you can redistribute it and/or modify |
2086 | |
2087 | === modified file 'systemimage/tests/test_state.py' |
2088 | --- systemimage/tests/test_state.py 2013-12-13 13:55:51 +0000 |
2089 | +++ systemimage/tests/test_state.py 2014-02-25 17:45:25 +0000 |
2090 | @@ -1,4 +1,4 @@ |
2091 | -# Copyright (C) 2013 Canonical Ltd. |
2092 | +# Copyright (C) 2013-2014 Canonical Ltd. |
2093 | # Author: Barry Warsaw <barry@ubuntu.com> |
2094 | |
2095 | # This program is free software: you can redistribute it and/or modify |
2096 | |
2097 | === modified file 'systemimage/tests/test_winner.py' |
2098 | --- systemimage/tests/test_winner.py 2013-12-13 13:55:51 +0000 |
2099 | +++ systemimage/tests/test_winner.py 2014-02-25 17:45:25 +0000 |
2100 | @@ -1,4 +1,4 @@ |
2101 | -# Copyright (C) 2013 Canonical Ltd. |
2102 | +# Copyright (C) 2013-2014 Canonical Ltd. |
2103 | # Author: Barry Warsaw <barry@ubuntu.com> |
2104 | |
2105 | # This program is free software: you can redistribute it and/or modify |
2106 | |
2107 | === modified file 'systemimage/version.txt' |
2108 | --- systemimage/version.txt 2013-12-13 13:55:51 +0000 |
2109 | +++ systemimage/version.txt 2014-02-25 17:45:25 +0000 |
2110 | @@ -1,1 +1,1 @@ |
2111 | -2.0.3 |
2112 | +2.1 |
2113 | |
2114 | === modified file 'tox.ini' |
2115 | --- tox.ini 2013-12-13 13:55:51 +0000 |
2116 | +++ tox.ini 2014-02-25 17:45:25 +0000 |
2117 | @@ -1,5 +1,5 @@ |
2118 | [tox] |
2119 | -envlist = py33 |
2120 | +envlist = py33, py34 |
2121 | |
2122 | [testenv] |
2123 | commands = python -m nose2 -v |