Merge lp:~barry/ubuntu-system-image/citrain-2.1 into lp:~ubuntu-managed-branches/ubuntu-system-image/system-image

Proposed by Barry Warsaw
Status: Merged
Merged at revision: 229
Proposed branch: lp:~barry/ubuntu-system-image/citrain-2.1
Merge into: lp:~ubuntu-managed-branches/ubuntu-system-image/system-image
Diff against target: 2123 lines (+901/-146)
65 files modified
MANIFEST.in (+1/-0)
NEWS.rst (+48/-1)
PKG-INFO (+1/-1)
cli-manpage.rst (+1/-1)
dbus-manpage.rst (+1/-1)
debian/changelog (+32/-0)
debian/control (+5/-4)
debian/patches/01_send_ack_on_applyupdate.diff (+0/-16)
debian/patches/lp1284217.patch (+106/-0)
debian/patches/series (+1/-1)
debian/rules (+1/-0)
ini-manpage.rst (+1/-1)
setup.cfg (+2/-2)
setup.py (+1/-1)
system_image.egg-info/PKG-INFO (+1/-1)
system_image.egg-info/SOURCES.txt (+1/-1)
systemimage/api.py (+5/-1)
systemimage/bag.py (+1/-1)
systemimage/bindings.py (+1/-1)
systemimage/candidates.py (+1/-1)
systemimage/channel.py (+1/-1)
systemimage/config.py (+1/-1)
systemimage/dbus.py (+51/-16)
systemimage/device.py (+1/-1)
systemimage/docs/conf.py (+1/-1)
systemimage/download.py (+36/-2)
systemimage/gpg.py (+95/-1)
systemimage/helpers.py (+1/-1)
systemimage/image.py (+1/-1)
systemimage/index.py (+1/-1)
systemimage/keyring.py (+3/-4)
systemimage/logging.py (+29/-8)
systemimage/main.py (+1/-1)
systemimage/reactor.py (+1/-1)
systemimage/reboot.py (+1/-1)
systemimage/scores.py (+1/-1)
systemimage/service.py (+14/-6)
systemimage/settings.py (+1/-1)
systemimage/state.py (+8/-10)
systemimage/testing/controller.py (+17/-8)
systemimage/testing/dbus.py (+8/-2)
systemimage/testing/demo.py (+1/-1)
systemimage/testing/helpers.py (+25/-7)
systemimage/testing/nose.py (+21/-2)
systemimage/tests/data/config_03.ini (+1/-1)
systemimage/tests/data/index_24.json (+36/-0)
systemimage/tests/test_api.py (+1/-1)
systemimage/tests/test_bag.py (+1/-1)
systemimage/tests/test_candidates.py (+1/-1)
systemimage/tests/test_channel.py (+1/-1)
systemimage/tests/test_config.py (+1/-1)
systemimage/tests/test_dbus.py (+98/-8)
systemimage/tests/test_download.py (+1/-1)
systemimage/tests/test_gpg.py (+175/-2)
systemimage/tests/test_helpers.py (+1/-1)
systemimage/tests/test_image.py (+1/-1)
systemimage/tests/test_index.py (+1/-1)
systemimage/tests/test_keyring.py (+20/-5)
systemimage/tests/test_main.py (+25/-1)
systemimage/tests/test_scores.py (+1/-1)
systemimage/tests/test_settings.py (+1/-1)
systemimage/tests/test_state.py (+1/-1)
systemimage/tests/test_winner.py (+1/-1)
systemimage/version.txt (+1/-1)
tox.ini (+1/-1)
To merge this branch: bzr merge lp:~barry/ubuntu-system-image/citrain-2.1
Reviewer Review Type Date Requested Status
Stéphane Graber Pending
Review via email: mp+207702@code.launchpad.net

Commit message

New upstream release, along with some packaging changes. See changelog and NEWS.rst for details.

Description of the change

  [ Stéphane Graber ]
  * New upstream release.
  * Set X-Auto-Uploader to no-rewrite-version
  * Set Vcs-Bzr to the new target branch

  [ Barry Warsaw ]
  * New upstream release.
    - LP: #1279056 - Internal improvements to SignatureError for
      better debugging.
    - LP: #1277589 - Better protection against race conditions.
    - LP: #1260768 - Return empty string from ApplyUpdate D-Bus method.
    - Request ubuntu-download-manager to download to a temporary location,
      with atomic rename.
    - More detailed logging.
    - Fixed D-Bus error logging.
    - Added -L flag to nose2 tests for explicitly setting log file path.
    - Added SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS environment variable
      which can be used to give virtualized buildds a fighting chance.
  * d/patches/01_send_ack_on_applyupdate.diff - Removed; applied upstream.
  * d/control: Bump Standards-Version to 3.9.5 with no other changes necessary.

To post a comment you must log in.
234. By Barry Warsaw

* d/control:
  - Bump Standards-Version to 3.9.5 with no other changes necessary.
  - Add python3-psutil as Depends to system-image-dev.

235. By Barry Warsaw

d/rules: Set SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS to 1 to deal with
buildd dbus-daemon SIGHUP timing issues.

236. By Barry Warsaw

  - LP: #1284217 - Send UpdateAvailableStatus during auto-downloading
    from a previous CheckForUpdate, if cached status is available.
* d/patches/01_send_ack_on_applyupdate.diff: Removed; applied upstream.
* d/patches/lp1284217.patch: Added (see above).

237. By Barry Warsaw

Update patch

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'MANIFEST.in'
--- MANIFEST.in 2013-12-13 13:55:51 +0000
+++ MANIFEST.in 2014-02-25 17:45:25 +0000
@@ -3,4 +3,5 @@
3prune build3prune build
4prune dist4prune dist
5prune .tox5prune .tox
6prune .bzr
6exclude .bzrignore7exclude .bzrignore
78
=== modified file 'NEWS.rst'
--- NEWS.rst 2013-12-13 13:55:51 +0000
+++ NEWS.rst 2014-02-25 17:45:25 +0000
@@ -2,7 +2,54 @@
2NEWS for system-image updater2NEWS for system-image updater
3=============================3=============================
44
52.0.3 (2013-XX-XX)52.1 (2014-02-20)
6================
7 * Internal improvements to SignatureError for better debugging. (LP: #1279056)
8 * Better protection against several possible race conditions during
9 `CheckForUpdate()` (LP: #1277589)
10 - Use a threading.Lock instance as the internal "checking for update"
11 barrier instead of a boolean. This should eliminate the race window
12 between testing and acquiring the checking lock.
13 - Put an exclusive claim on the `com.canonical.SystemImage` system dbus
14 name, and if we cannot get that claim, exit with an error code 2. This
15 prevents multiple instances of the D-Bus system service from running at
16 the same time.
17 * Return the empty string from `ApplyUpdate()` D-Bus method. This restores
18 the original API (patch merged from Ubuntu package, given by Didier
19 Roche). (LP: #1260768)
20 * Request ubuntu-download-manager to download all files to temporary
21 destinations, then atomically rename them into place. This avoids
22 clobbering by multiple processes and mimics changes coming in u-d-m.
23 * Provide much more detailed logging.
24 - `Mediator` instances have a helpful `repr` which also includes the id of
25 the `State` object.
26 - More logging during state transitions.
27 - All emitted D-Bus signals are also logged (at debug level).
28 * Added `-L` flag to nose test runner, which can be used to specify an
29 explicit log file path for debugging.
30 * Fixed D-Bus error logging.
31 - Don't initialize the root logger, since this can interfere with
32 python-dbus, which doesn't initialize its loggers correctly.
33 - Only use `.format()` based interpolation for `systemimage` logs.
34 * Give virtualized buildds a fighting chance against D-Bus by
35 - using `org.freedesktop.DBus`s `ReloadConfig()` interface instead of
36 SIGHUP.
37 - add a configurable sleep call after the `ReloadConfig()`. This defaults
38 to 0 since de-virtualized and local builds do not need them. Set the
39 environment variable `SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS` to
40 override.
41 * Run the tox test suite for both Python 3.3 and 3.4.
42
432.0.5 (2014-01-30)
44==================
45 * MANIFEST.in: Make sure the .bzr directory doesn't end up in the
46 sdist tarball.
47
482.0.4 (2014-01-30)
49==================
50 * No change release to test the new landing process.
51
522.0.3 (2013-12-11)
6==================53==================
7 * More attempted DEP-8 test failure fixes.54 * More attempted DEP-8 test failure fixes.
855
956
=== modified file 'PKG-INFO'
--- PKG-INFO 2013-12-13 13:55:51 +0000
+++ PKG-INFO 2014-02-25 17:45:25 +0000
@@ -1,6 +1,6 @@
1Metadata-Version: 1.01Metadata-Version: 1.0
2Name: system-image2Name: system-image
3Version: 2.0.33Version: 2.1
4Summary: Ubuntu System Image Based Upgrades4Summary: Ubuntu System Image Based Upgrades
5Home-page: UNKNOWN5Home-page: UNKNOWN
6Author: Barry Warsaw6Author: Barry Warsaw
77
=== modified file 'cli-manpage.rst'
--- cli-manpage.rst 2013-12-13 13:55:51 +0000
+++ cli-manpage.rst 2014-02-25 17:45:25 +0000
@@ -8,7 +8,7 @@
88
9:Author: Barry Warsaw <barry@ubuntu.com>9:Author: Barry Warsaw <barry@ubuntu.com>
10:Date: 2013-10-2310:Date: 2013-10-23
11:Copyright: 2013 Canonical Ltd.11:Copyright: 2013-2014 Canonical Ltd.
12:Version: 2.012:Version: 2.0
13:Manual section: 113:Manual section: 1
1414
1515
=== modified file 'dbus-manpage.rst'
--- dbus-manpage.rst 2013-12-13 13:55:51 +0000
+++ dbus-manpage.rst 2014-02-25 17:45:25 +0000
@@ -8,7 +8,7 @@
88
9:Author: Barry Warsaw <barry@ubuntu.com>9:Author: Barry Warsaw <barry@ubuntu.com>
10:Date: 2013-07-3110:Date: 2013-07-31
11:Copyright: 2013 Canonical Ltd.11:Copyright: 2013-2014 Canonical Ltd.
12:Version: 1.012:Version: 1.0
13:Manual section: 813:Manual section: 8
1414
1515
=== modified file 'debian/changelog'
--- debian/changelog 2013-12-13 13:55:51 +0000
+++ debian/changelog 2014-02-25 17:45:25 +0000
@@ -1,3 +1,35 @@
1system-image (2.1-0ubuntu4) UNRELEASED; urgency=medium
2
3 [ Stéphane Graber ]
4 * New upstream release.
5 * Set X-Auto-Uploader to no-rewrite-version
6 * Set Vcs-Bzr to the new target branch
7
8 [ Barry Warsaw ]
9 * New upstream release.
10 - LP: #1279056 - Internal improvements to SignatureError for
11 better debugging.
12 - LP: #1277589 - Better protection against race conditions.
13 - LP: #1260768 - Return empty string from ApplyUpdate D-Bus method.
14 - LP: #1284217 - Send UpdateAvailableStatus during auto-downloading
15 from a previous CheckForUpdate, if cached status is available.
16 - Request ubuntu-download-manager to download to a temporary location,
17 with atomic rename.
18 - More detailed logging.
19 - Fixed D-Bus error logging.
20 - Added -L flag to nose2 tests for explicitly setting log file path.
21 - Added SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS environment variable
22 which can be used to give virtualized buildds a fighting chance.
23 * d/patches/01_send_ack_on_applyupdate.diff: Removed; applied upstream.
24 * d/patches/lp1284217.patch: Added (see above).
25 * d/control:
26 - Bump Standards-Version to 3.9.5 with no other changes necessary.
27 - Add python3-psutil as Depends to system-image-dev.
28 * d/rules: Set SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS to 1 to deal with
29 buildd dbus-daemon SIGHUP timing issues.
30
31 -- Barry Warsaw <barry@ubuntu.com> Thu, 20 Feb 2014 18:03:10 -0500
32
1system-image (2.0.3-0ubuntu2) trusty; urgency=low33system-image (2.0.3-0ubuntu2) trusty; urgency=low
234
3 * Fix ApplyUpdate() to return an empty string as per spec if the update35 * Fix ApplyUpdate() to return an empty string as per spec if the update
436
=== modified file 'debian/control'
--- debian/control 2013-12-13 13:55:51 +0000
+++ debian/control 2014-02-25 17:45:25 +0000
@@ -18,10 +18,11 @@
18 python3-psutil,18 python3-psutil,
19 python3-setuptools,19 python3-setuptools,
20 ubuntu-download-manager20 ubuntu-download-manager
21Standards-Version: 3.9.421Standards-Version: 3.9.5
22XS-Testsuite: autopkgtest22XS-Testsuite: autopkgtest
23Vcs-Bzr: https://code.launchpad.net/~ubuntu-system-image/ubuntu-system-image/client.pkg23Vcs-Bzr: https://code.launchpad.net/~ubuntu-managed-branches/ubuntu-system-image/system-image
24Vcs-Browser: http://bazaar.launchpad.net/~ubuntu-system-image/ubuntu-system-image/client.pkg/files24Vcs-Browser: http://bazaar.launchpad.net/~ubuntu-managed-branches/ubuntu-system-image/system-image/files
25X-Auto-Uploader: no-rewrite-version
2526
26Package: system-image-cli27Package: system-image-cli
27Architecture: all28Architecture: all
@@ -53,7 +54,7 @@
5354
54Package: system-image-dev55Package: system-image-dev
55Architecture: all56Architecture: all
56Depends: python3-gnupg, ${misc:Depends}, ${python3:Depends}57Depends: python3-gnupg, python3-psutil, ${misc:Depends}, ${python3:Depends}
57Description: Ubuntu system image updater development58Description: Ubuntu system image updater development
58 This is the development bits for the Ubuntu system image updater.59 This is the development bits for the Ubuntu system image updater.
59 Install this package if you want to run the tests.60 Install this package if you want to run the tests.
6061
=== removed file 'debian/patches/01_send_ack_on_applyupdate.diff'
--- debian/patches/01_send_ack_on_applyupdate.diff 2013-12-13 13:55:51 +0000
+++ debian/patches/01_send_ack_on_applyupdate.diff 1970-01-01 00:00:00 +0000
@@ -1,16 +0,0 @@
1Description: Send ack on apply diff
2 Fix ApplyUpdate() to return an empty string as per spec if the update
3 is successfull (LP: #1260712)
4Author: Didier Roche <didrocks@ubuntu.com>
5Bug-Ubuntu: https://bugs.launchpad.net/bugs/1260712
6
7--- system-image-2.0.3.orig/systemimage/dbus.py
8+++ system-image-2.0.3/systemimage/dbus.py
9@@ -233,6 +233,7 @@ class Service(Object):
10 def ApplyUpdate(self):
11 """Apply the update, rebooting the device."""
12 GLib.timeout_add(50, self._apply_update)
13+ return ""
14
15 @method('com.canonical.SystemImage', out_signature='isssa{ss}')
16 def Info(self):
170
=== added file 'debian/patches/lp1284217.patch'
--- debian/patches/lp1284217.patch 1970-01-01 00:00:00 +0000
+++ debian/patches/lp1284217.patch 2014-02-25 17:45:25 +0000
@@ -0,0 +1,106 @@
1Description: Backport of fix for LP: #1284217. This adds an additional
2 UpdateAvailableStatus signal when a second CheckForUpdate is called while an
3 auto-download is in progress, but only if we have cached status available.
4Origin: http://bazaar.launchpad.net/~ubuntu-system-image/ubuntu-system-image/client/revision/240?start_revid=240
5Bug: http://pad.lv/1284217
6Forwarded: not-needed
7
8=== modified file 'systemimage/dbus.py'
9--- old/systemimage/dbus.py 2014-02-18 22:31:55 +0000
10+++ new/systemimage/dbus.py 2014-02-25 17:27:14 +0000
11@@ -131,7 +131,20 @@
12 # Check-and-acquire the lock.
13 log.info('test and acquire checking lock')
14 if not self._checking.acquire(blocking=False):
15- # Check is already in progress, so there's nothing more to do.
16+ # Check is already in progress, so there's nothing more to do. If
17+ # there's status available (i.e. we are in the auto-downloading
18+ # phase of the last CFU), then send the status.
19+ if self._update is not None:
20+ self.UpdateAvailableStatus(
21+ self._update.is_available,
22+ self._downloading,
23+ self._update.version,
24+ self._update.size,
25+ self._update.last_update_date,
26+ # XXX 2013-08-22 - the u/i cannot currently currently
27+ # handle the array of dictionaries data type. LP:
28+ # #1215586 self._update.descriptions,
29+ "")
30 log.info('checking lock not acquired')
31 return
32 log.info('checking lock acquired')
33
34=== modified file 'systemimage/tests/test_dbus.py'
35--- old/systemimage/tests/test_dbus.py 2014-02-18 20:02:59 +0000
36+++ new/systemimage/tests/test_dbus.py 2014-02-25 17:27:14 +0000
37@@ -23,13 +23,13 @@
38 'TestDBusGetSet',
39 'TestDBusInfo',
40 'TestDBusInfoNoDetails',
41- 'TestDBusLP1277589',
42 'TestDBusMockFailApply',
43 'TestDBusMockFailPause',
44 'TestDBusMockFailResume',
45 'TestDBusMockNoUpdate',
46 'TestDBusMockUpdateAutoSuccess',
47 'TestDBusMockUpdateManualSuccess',
48+ 'TestDBusMultipleChecksInFlight',
49 'TestDBusPauseResume',
50 'TestDBusProgress',
51 'TestDBusRegressions',
52@@ -133,6 +133,7 @@
53 self.schedule(self.iface.CheckForUpdate)
54
55 def _do_UpdateAvailableStatus(self, signal, path, *args, **kws):
56+ # We'll keep doing this until we get the UpdateDownloaded signal.
57 self.uas_signals.append(args)
58 self.schedule(self.iface.CheckForUpdate)
59
60@@ -1563,7 +1564,7 @@
61 """)
62
63
64-class TestDBusLP1277589(_LiveTesting):
65+class TestDBusMultipleChecksInFlight(_LiveTesting):
66 def test_multiple_check_for_updates(self):
67 # Log analysis of LP: #1277589 appears to show the following scenario,
68 # reproduced in this test case:
69@@ -1588,18 +1589,23 @@
70 # signal, we'll immediately issue *another* CheckForUpdate, which
71 # should run while the auto-download is working.
72 #
73- # At the end, we should not get another UpdateAvailableStatus signal,
74- # but we should get the UpdateDownloaded signal.
75+ # As per LP: #1284217, we will get a second UpdateAvailableStatus
76+ # signal, since the status is available even while the original
77+ # request is being downloaded.
78 reactor = DoubleCheckingReactor(self.iface)
79 reactor.run()
80- self.assertEqual(len(reactor.uas_signals), 1)
81- (is_available, downloading, available_version, update_size,
82- last_update_date,
83- #descriptions,
84- error_reason) = reactor.uas_signals[0]
85- self.assertTrue(is_available)
86- self.assertTrue(downloading)
87- self.assertEqual(available_version, '1600')
88- self.assertEqual(update_size, 314572800)
89- self.assertEqual(last_update_date, 'Unknown')
90- self.assertEqual(error_reason, '')
91+ # We need to have received at least 2 signals, but due to timing
92+ # issues it could possibly be more.
93+ self.assertGreater(len(reactor.uas_signals), 1)
94+ # All received signals should have the same information.
95+ for signal in reactor.uas_signals:
96+ (is_available, downloading, available_version, update_size,
97+ last_update_date,
98+ #descriptions,
99+ error_reason) = signal
100+ self.assertTrue(is_available)
101+ self.assertTrue(downloading)
102+ self.assertEqual(available_version, '1600')
103+ self.assertEqual(update_size, 314572800)
104+ self.assertEqual(last_update_date, 'Unknown')
105+ self.assertEqual(error_reason, '')
106
0107
=== modified file 'debian/patches/series'
--- debian/patches/series 2013-12-13 13:55:51 +0000
+++ debian/patches/series 2014-02-25 17:45:25 +0000
@@ -1,1 +1,1 @@
101_send_ack_on_applyupdate.diff1lp1284217.patch
22
=== modified file 'debian/rules'
--- debian/rules 2013-12-13 13:55:51 +0000
+++ debian/rules 2014-02-25 17:45:25 +0000
@@ -16,6 +16,7 @@
16test-python%:16test-python%:
17 unset http_proxy; unset https_proxy; export HOME=/tmp; \17 unset http_proxy; unset https_proxy; export HOME=/tmp; \
18 export SYSTEMIMAGE_REACTOR_TIMEOUT=1200; \18 export SYSTEMIMAGE_REACTOR_TIMEOUT=1200; \
19 export SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS=2; \
19 nodot=$(shell echo $* | cut --complement -b 2); \20 nodot=$(shell echo $* | cut --complement -b 2); \
20 tox -e py$${nodot}21 tox -e py$${nodot}
2122
2223
=== modified file 'ini-manpage.rst'
--- ini-manpage.rst 2013-12-13 13:55:51 +0000
+++ ini-manpage.rst 2014-02-25 17:45:25 +0000
@@ -9,7 +9,7 @@
99
10:Author: Barry Warsaw <barry@ubuntu.com>10:Author: Barry Warsaw <barry@ubuntu.com>
11:Date: 2013-10-1111:Date: 2013-10-11
12:Copyright: 2013 Canonical Ltd.12:Copyright: 2013-2014 Canonical Ltd.
13:Version: 1.913:Version: 1.9
14:Manual section: 514:Manual section: 5
1515
1616
=== modified file 'setup.cfg'
--- setup.cfg 2013-12-13 13:55:51 +0000
+++ setup.cfg 2014-02-25 17:45:25 +0000
@@ -4,7 +4,7 @@
4logging-filter = systemimage4logging-filter = systemimage
55
6[egg_info]6[egg_info]
7tag_svn_revision = 0
8tag_date = 0
7tag_build = 9tag_build =
8tag_date = 0
9tag_svn_revision = 0
1010
1111
=== modified file 'setup.py'
--- setup.py 2013-12-13 13:55:51 +0000
+++ setup.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'system_image.egg-info/PKG-INFO'
--- system_image.egg-info/PKG-INFO 2013-12-13 13:55:51 +0000
+++ system_image.egg-info/PKG-INFO 2014-02-25 17:45:25 +0000
@@ -1,6 +1,6 @@
1Metadata-Version: 1.01Metadata-Version: 1.0
2Name: system-image2Name: system-image
3Version: 2.0.33Version: 2.1
4Summary: Ubuntu System Image Based Upgrades4Summary: Ubuntu System Image Based Upgrades
5Home-page: UNKNOWN5Home-page: UNKNOWN
6Author: Barry Warsaw6Author: Barry Warsaw
77
=== modified file 'system_image.egg-info/SOURCES.txt'
--- system_image.egg-info/SOURCES.txt 2013-12-13 13:55:51 +0000
+++ system_image.egg-info/SOURCES.txt 2014-02-25 17:45:25 +0000
@@ -7,7 +7,6 @@
7setup.py7setup.py
8tox.ini8tox.ini
9unittest.cfg9unittest.cfg
10.bzr/branch/branch.conf
11system_image.egg-info/PKG-INFO10system_image.egg-info/PKG-INFO
12system_image.egg-info/SOURCES.txt11system_image.egg-info/SOURCES.txt
13system_image.egg-info/dependency_links.txt12system_image.egg-info/dependency_links.txt
@@ -126,6 +125,7 @@
126systemimage/tests/data/index_21.json125systemimage/tests/data/index_21.json
127systemimage/tests/data/index_22.json126systemimage/tests/data/index_22.json
128systemimage/tests/data/index_23.json127systemimage/tests/data/index_23.json
128systemimage/tests/data/index_24.json
129systemimage/tests/data/key.pem129systemimage/tests/data/key.pem
130systemimage/tests/data/master-secring.gpg130systemimage/tests/data/master-secring.gpg
131systemimage/tests/data/nasty_cert.pem131systemimage/tests/data/nasty_cert.pem
132132
=== modified file 'systemimage/api.py'
--- systemimage/api.py 2013-12-13 13:55:51 +0000
+++ systemimage/api.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -73,6 +73,10 @@
73 self._update = None73 self._update = None
74 self._callback = callback74 self._callback = callback
7575
76 def __repr__(self):
77 return '<Mediator at 0x{:x} | State at 0x{:x}>'.format(
78 id(self), id(self._state))
79
76 def cancel(self):80 def cancel(self):
77 self._state.downloader.cancel()81 self._state.downloader.cancel()
7882
7983
=== modified file 'systemimage/bag.py'
--- systemimage/bag.py 2013-12-13 13:55:51 +0000
+++ systemimage/bag.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/bindings.py'
--- systemimage/bindings.py 2013-12-13 13:55:51 +0000
+++ systemimage/bindings.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/candidates.py'
--- systemimage/candidates.py 2013-12-13 13:55:51 +0000
+++ systemimage/candidates.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/channel.py'
--- systemimage/channel.py 2013-12-13 13:55:51 +0000
+++ systemimage/channel.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/config.py'
--- systemimage/config.py 2013-12-13 13:55:51 +0000
+++ systemimage/config.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/dbus.py'
--- systemimage/dbus.py 2013-12-13 13:55:51 +0000
+++ systemimage/dbus.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -23,6 +23,7 @@
2323
24import os24import os
25import sys25import sys
26import logging
26import traceback27import traceback
2728
28from dbus.service import Object, method, signal29from dbus.service import Object, method, signal
@@ -31,9 +32,11 @@
31from systemimage.config import config32from systemimage.config import config
32from systemimage.helpers import last_update_date, version_detail33from systemimage.helpers import last_update_date, version_detail
33from systemimage.settings import Settings34from systemimage.settings import Settings
35from threading import Lock
3436
3537
36EMPTYSTRING = ''38EMPTYSTRING = ''
39log = logging.getLogger('systemimage')
3740
3841
39class Loop:42class Loop:
@@ -68,7 +71,8 @@
68 super().__init__(bus, object_path)71 super().__init__(bus, object_path)
69 self._loop = loop72 self._loop = loop
70 self._api = Mediator(self._progress_callback)73 self._api = Mediator(self._progress_callback)
71 self._checking = False74 log.info('Mediator created {}', self._api)
75 self._checking = Lock()
72 self._update = None76 self._update = None
73 self._downloading = False77 self._downloading = False
74 self._paused = False78 self._paused = False
@@ -78,17 +82,22 @@
7882
79 def _check_for_update(self):83 def _check_for_update(self):
80 # Asynchronous method call.84 # Asynchronous method call.
85 log.info('Checking for update')
81 self._update = self._api.check_for_update()86 self._update = self._api.check_for_update()
82 # Do we have an update and can we auto-download it?87 # Do we have an update and can we auto-download it?
83 downloading = False88 downloading = False
84 if self._update.is_available:89 if self._update.is_available:
85 settings = Settings()90 settings = Settings()
86 auto = settings.get('auto_download')91 auto = settings.get('auto_download')
92 log.info('Update available; auto-download: {}', auto)
87 if auto in ('1', '2'):93 if auto in ('1', '2'):
88 # XXX When we have access to the download service, we can94 # XXX When we have access to the download service, we can
89 # check if we're on the wifi (auto == '1').95 # check if we're on the wifi (auto == '1').
90 GLib.timeout_add(50, self._download)96 GLib.timeout_add(50, self._download, self._checking.release)
91 downloading = True97 downloading = True
98 else:
99 log.info('release checking lock from _check_for_update()')
100 self._checking.release()
92 self.UpdateAvailableStatus(101 self.UpdateAvailableStatus(
93 self._update.is_available,102 self._update.is_available,
94 downloading,103 downloading,
@@ -99,7 +108,6 @@
99 # array of dictionaries data type. LP: #1215586108 # array of dictionaries data type. LP: #1215586
100 #self._update.descriptions,109 #self._update.descriptions,
101 "")110 "")
102 self._checking = False
103 # Stop GLib from calling this method again.111 # Stop GLib from calling this method again.
104 return False112 return False
105113
@@ -120,13 +128,17 @@
120 whether the update is available or not.128 whether the update is available or not.
121 """129 """
122 self._loop.keepalive()130 self._loop.keepalive()
123 if self._checking:131 # Check-and-acquire the lock.
132 log.info('test and acquire checking lock')
133 if not self._checking.acquire(blocking=False):
124 # Check is already in progress, so there's nothing more to do.134 # Check is already in progress, so there's nothing more to do.
135 log.info('checking lock not acquired')
125 return136 return
126 self._checking = True137 log.info('checking lock acquired')
127 # Reset any failure or in-progress state. Get a new mediator to reset138 # We've now acquired the lock. Reset any failure or in-progress
128 # any of its state.139 # state. Get a new mediator to reset any of its state.
129 self._api = Mediator(self._progress_callback)140 self._api = Mediator(self._progress_callback)
141 log.info('Mediator recreated {}', self._api)
130 self._failure_count = 0142 self._failure_count = 0
131 self._last_error = ''143 self._last_error = ''
132 # Arrange for the actual check to happen in a little while, so that144 # Arrange for the actual check to happen in a little while, so that
@@ -141,24 +153,28 @@
141 eta = 0153 eta = 0
142 self.UpdateProgress(percentage, eta)154 self.UpdateProgress(percentage, eta)
143155
144 def _download(self):156 def _download(self, release_checking=None):
145 if self._downloading and self._paused:157 if self._downloading and self._paused:
146 self._api.resume()158 self._api.resume()
147 self._paused = False159 self._paused = False
160 log.info('Download previously paused')
148 return161 return
149 if (self._downloading # Already in progress.162 if (self._downloading # Already in progress.
150 or self._update is None # Not yet checked.163 or self._update is None # Not yet checked.
151 or not self._update.is_available # No update available.164 or not self._update.is_available # No update available.
152 ):165 ):
166 log.info('Download already in progress or not available')
153 return167 return
154 if self._failure_count > 0:168 if self._failure_count > 0:
155 self._failure_count += 1169 self._failure_count += 1
156 self.UpdateFailed(self._failure_count, self._last_error)170 self.UpdateFailed(self._failure_count, self._last_error)
171 log.info('Update failure count: {}', self._failure_count)
157 return172 return
158 self._downloading = True173 self._downloading = True
174 log.info('Update is downloading')
159 try:175 try:
160 # Always start by sending a UpdateProgress(0, 0). This is enough176 # Always start by sending a UpdateProgress(0, 0). This is
161 # to get the u/i's attention.177 # enough to get the u/i's attention.
162 self.UpdateProgress(0, 0)178 self.UpdateProgress(0, 0)
163 self._api.download()179 self._api.download()
164 except Exception:180 except Exception:
@@ -167,13 +183,20 @@
167 # value, but not the traceback.183 # value, but not the traceback.
168 self._last_error = EMPTYSTRING.join(184 self._last_error = EMPTYSTRING.join(
169 traceback.format_exception_only(*sys.exc_info()[:2]))185 traceback.format_exception_only(*sys.exc_info()[:2]))
186 log.info('Update failed: {}', self._last_error)
170 self.UpdateFailed(self._failure_count, self._last_error)187 self.UpdateFailed(self._failure_count, self._last_error)
171 else:188 else:
189 log.info('Update downloaded')
172 self.UpdateDownloaded()190 self.UpdateDownloaded()
173 self._failure_count = 0191 self._failure_count = 0
174 self._last_error = ''192 self._last_error = ''
175 self._rebootable = True193 self._rebootable = True
176 self._downloading = False194 self._downloading = False
195 log.info('release checking lock from _download()')
196 if release_checking is not None:
197 # We were auto-downloading, so we now have to release the checking
198 # lock. If we were manually downloading, there would be no lock.
199 release_checking()
177 # Stop GLib from calling this method again.200 # Stop GLib from calling this method again.
178 return False201 return False
179202
@@ -216,8 +239,6 @@
216 return ''239 return ''
217240
218 def _apply_update(self):241 def _apply_update(self):
219 # This signal may or may not get sent. We're racing against the
220 # system reboot procedure.
221 self._loop.keepalive()242 self._loop.keepalive()
222 if not self._rebootable:243 if not self._rebootable:
223 command_file = os.path.join(244 command_file = os.path.join(
@@ -227,12 +248,16 @@
227 self.Rebooting(False)248 self.Rebooting(False)
228 return249 return
229 self._api.reboot()250 self._api.reboot()
251 # This code may or may not run. We're racing against the system
252 # reboot procedure.
253 self._rebootable = False
230 self.Rebooting(True)254 self.Rebooting(True)
231255
232 @method('com.canonical.SystemImage')256 @method('com.canonical.SystemImage')
233 def ApplyUpdate(self):257 def ApplyUpdate(self):
234 """Apply the update, rebooting the device."""258 """Apply the update, rebooting the device."""
235 GLib.timeout_add(50, self._apply_update)259 GLib.timeout_add(50, self._apply_update)
260 return ''
236261
237 @method('com.canonical.SystemImage', out_signature='isssa{ss}')262 @method('com.canonical.SystemImage', out_signature='isssa{ss}')
238 def Info(self):263 def Info(self):
@@ -294,31 +319,40 @@
294 #descriptions,319 #descriptions,
295 error_reason):320 error_reason):
296 """Signal sent in response to a CheckForUpdate()."""321 """Signal sent in response to a CheckForUpdate()."""
322 log.debug('EMIT UpdateAvailableStatus({}, {}, {}, {}, {}, {})',
323 is_available, downloading, available_version, update_size,
324 last_update_date, repr(error_reason))
297 self._loop.keepalive()325 self._loop.keepalive()
298326
299 @signal('com.canonical.SystemImage', signature='id')327 @signal('com.canonical.SystemImage', signature='id')
300 def UpdateProgress(self, percentage, eta):328 def UpdateProgress(self, percentage, eta):
301 """Download progress."""329 """Download progress."""
330 log.debug('EMIT UpdateProgress({}, {})', percentage, eta)
302 self._loop.keepalive()331 self._loop.keepalive()
303332
304 @signal('com.canonical.SystemImage')333 @signal('com.canonical.SystemImage')
305 def UpdateDownloaded(self):334 def UpdateDownloaded(self):
306 """The update has been successfully downloaded."""335 """The update has been successfully downloaded."""
336 log.debug('EMIT UpdateDownloaded()')
307 self._loop.keepalive()337 self._loop.keepalive()
308338
309 @signal('com.canonical.SystemImage', signature='is')339 @signal('com.canonical.SystemImage', signature='is')
310 def UpdateFailed(self, consecutive_failure_count, last_reason):340 def UpdateFailed(self, consecutive_failure_count, last_reason):
311 """The update failed for some reason."""341 """The update failed for some reason."""
342 log.debug('EMIT UpdateFailed({}, {})',
343 consecutive_failure_count, repr(last_reason))
312 self._loop.keepalive()344 self._loop.keepalive()
313345
314 @signal('com.canonical.SystemImage', signature='i')346 @signal('com.canonical.SystemImage', signature='i')
315 def UpdatePaused(self, percentage):347 def UpdatePaused(self, percentage):
316 """The download got paused."""348 """The download got paused."""
349 log.debug('EMIT UpdatePaused({})', percentage)
317 self._loop.keepalive()350 self._loop.keepalive()
318351
319 @signal('com.canonical.SystemImage', signature='ss')352 @signal('com.canonical.SystemImage', signature='ss')
320 def SettingChanged(self, key, new_value):353 def SettingChanged(self, key, new_value):
321 """A setting value has change."""354 """A setting value has change."""
355 log.debug('EMIT SettingChanged({}, {})', repr(key), repr(new_value))
322 self._loop.keepalive()356 self._loop.keepalive()
323357
324 @signal('com.canonical.SystemImage', signature='b')358 @signal('com.canonical.SystemImage', signature='b')
@@ -326,3 +360,4 @@
326 """The system is rebooting."""360 """The system is rebooting."""
327 # We don't need to keep the loop alive since we're probably just going361 # We don't need to keep the loop alive since we're probably just going
328 # to shutdown anyway.362 # to shutdown anyway.
363 log.debug('EMIT Rebooting({})', status)
329364
=== modified file 'systemimage/device.py'
--- systemimage/device.py 2013-12-13 13:55:51 +0000
+++ systemimage/device.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/docs/conf.py'
--- systemimage/docs/conf.py 2013-12-13 13:55:51 +0000
+++ systemimage/docs/conf.py 2014-02-25 17:45:25 +0000
@@ -41,7 +41,7 @@
4141
42# General information about the project.42# General information about the project.
43project = u'Image Update Resolver'43project = u'Image Update Resolver'
44copyright = u'2013, Canonical Ltd.'44copyright = u'2013-2014, Canonical Ltd.'
4545
46# The version info for the project you're documenting, acts as replacement for46# The version info for the project you're documenting, acts as replacement for
47# |version| and |release|, also used in various other places throughout the47# |version| and |release|, also used in various other places throughout the
4848
=== modified file 'systemimage/download.py'
--- systemimage/download.py 2013-12-13 13:55:51 +0000
+++ systemimage/download.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -22,12 +22,16 @@
22 ]22 ]
2323
2424
25import os
25import dbus26import dbus
26import logging27import logging
28import tempfile
2729
30from contextlib import ExitStack
28from io import StringIO31from io import StringIO
29from pprint import pformat32from pprint import pformat
30from systemimage.config import config33from systemimage.config import config
34from systemimage.helpers import safe_remove
31from systemimage.reactor import Reactor35from systemimage.reactor import Reactor
3236
3337
@@ -191,8 +195,25 @@
191 for url, dst in downloads:195 for url, dst in downloads:
192 print('\t{} -> {}'.format(url, dst), file=fp)196 print('\t{} -> {}'.format(url, dst), file=fp)
193 log.info('{}'.format(fp.getvalue()))197 log.info('{}'.format(fp.getvalue()))
198 # As a workaround for LP: #1277589, ask u-d-m to download the files to
199 # .tmp files, and if they succeed, then atomically move them into
200 # their real location.
201 renames = []
202 requests = []
203 for url, dst in downloads:
204 head, tail = os.path.split(dst)
205 fd, path = tempfile.mkstemp(suffix='.tmp', prefix='', dir=head)
206 os.close(fd)
207 renames.append((path, dst))
208 requests.append((url, path, ''))
209 # mkstemp() creates the file system path, but if the files exist when
210 # the group download is requested, ubuntu-download-manager will
211 # complain and return an error. So, delete all temporary files now so
212 # udm has a clear path to download to.
213 for path, dst in renames:
214 os.remove(path)
194 object_path = iface.createDownloadGroup(215 object_path = iface.createDownloadGroup(
195 [(url, dst, '') for url, dst in downloads],216 requests, # The temporary requests.
196 '', # No hashes yet.217 '', # No hashes yet.
197 False, # Don't allow GSM yet.218 False, # Don't allow GSM yet.
198 # https://bugs.freedesktop.org/show_bug.cgi?id=55594219 # https://bugs.freedesktop.org/show_bug.cgi?id=55594
@@ -220,6 +241,19 @@
220 raise Canceled241 raise Canceled
221 if reactor.timed_out:242 if reactor.timed_out:
222 raise TimeoutError243 raise TimeoutError
244 # Now that everything succeeded, rename the temporary files. Just to
245 # be extra cautious, set up a context manager to safely remove all
246 # temporary files in case of an error. If there are no errors, then
247 # there will be nothing to remove.
248 with ExitStack() as resources:
249 for tmp, dst in renames:
250 resources.callback(safe_remove, tmp)
251 for tmp, dst in renames:
252 os.rename(tmp, dst)
253 # We only get here if all the renames succeeded, so there will be
254 # no temporary files to remove, so we can throw away the new
255 # ExitStack, which holds all the removals.
256 resources.pop_all()
223257
224 def cancel(self):258 def cancel(self):
225 """Cancel any current downloads."""259 """Cancel any current downloads."""
226260
=== modified file 'systemimage/gpg.py'
--- systemimage/gpg.py 2013-12-13 13:55:51 +0000
+++ systemimage/gpg.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -23,6 +23,7 @@
2323
24import os24import os
25import gnupg25import gnupg
26import hashlib
26import tarfile27import tarfile
2728
28from contextlib import ExitStack29from contextlib import ExitStack
@@ -37,10 +38,70 @@
37 always returns a boolean. This exception is used by other functions to38 always returns a boolean. This exception is used by other functions to
38 signal that a .asc file did not match.39 signal that a .asc file did not match.
39 """40 """
41 def __init__(self, signature_path, data_path,
42 keyrings=None, blacklist=None):
43 super().__init__()
44 self.signature_path = signature_path
45 self.data_path = data_path
46 self.keyrings = ([] if keyrings is None else keyrings)
47 self.blacklist = blacklist
48 # We have to calculate the checksums now, because it's possible that
49 # the files will be temporary/atomic files, deleted when a context
50 # manager exits. I.e. the files aren't guaranteed to exist after this
51 # constructor runs.
52 #
53 # Also, md5 is fine; this is not a security critical context, we just
54 # want to be able to quickly and easily compare the file on disk
55 # against the file on the server.
56 with open(self.signature_path, 'rb') as fp:
57 self.signature_checksum = hashlib.md5(fp.read()).hexdigest()
58 with open(self.data_path, 'rb') as fp:
59 self.data_checksum = hashlib.md5(fp.read()).hexdigest()
60 self.keyring_checksums = []
61 for path in self.keyrings:
62 with open(path, 'rb') as fp:
63 checksum = hashlib.md5(fp.read()).hexdigest()
64 self.keyring_checksums.append(checksum)
65 if self.blacklist is None:
66 self.blacklist_checksum = None
67 else:
68 with open(self.blacklist, 'rb') as fp:
69 self.blacklist_checksum = hashlib.md5(fp.read()).hexdigest()
70
71 def __str__(self):
72 if self.blacklist is None:
73 checksum_str = 'no blacklist'
74 path_str = ''
75 else:
76 checksum_str = self.blacklist_checksum
77 path_str = self.blacklist
78 return """
79 sig path : {0.signature_checksum}
80 {0.signature_path}
81 data path: {0.data_checksum}
82 {0.data_path}
83 keyrings : {0.keyring_checksums}
84 {1}
85 blacklist: {2} {3}
86""".format(self, list(self.keyrings), checksum_str, path_str)
87
4088
4189
42class Context:90class Context:
43 def __init__(self, *keyrings, blacklist=None):91 def __init__(self, *keyrings, blacklist=None):
92 """Create a GPG signature verification context.
93
94 :param keyrings: The list of keyrings to use for validating the
95 signature on data files.
96 :type keyrings: Sequence of .tar.xz keyring files, which will be
97 unpacked to retrieve the actual .gpg keyring file.
98 :param blacklist: The blacklist keyring, from which fingerprints to
99 explicitly disallow are retrieved.
100 :type blacklist: A .tar.xz keyring file, which will be unpacked to
101 retrieve the actual .gpg keyring file.
102 """
103 self.keyring_paths = keyrings
104 self.blacklist_path = blacklist
44 self._ctx = None105 self._ctx = None
45 self._stack = ExitStack()106 self._stack = ExitStack()
46 self._keyrings = []107 self._keyrings = []
@@ -112,6 +173,21 @@
112 return set(info['keyid'] for info in self._ctx.list_keys())173 return set(info['keyid'] for info in self._ctx.list_keys())
113174
114 def verify(self, signature_path, data_path):175 def verify(self, signature_path, data_path):
176 """Verify a GPG signature.
177
178 This verifies that the data file signature is valid, given the
179 keyrings and blacklist specified in the constructor. Specifically, we
180 use GPG to extract the fingerprint in the signature path, and compare
181 it against the fingerprints in the keyrings, subtracting any
182 fingerprints in the blacklist.
183
184 :param signature_path: The file system path to the detached signature
185 file for the data file.
186 :type signature_path: str
187 :param data_path: The file system path to the data file.
188 :type data_path: str
189 :return: bool
190 """
115 with open(signature_path, 'rb') as sig_fp:191 with open(signature_path, 'rb') as sig_fp:
116 verified = self._ctx.verify_file(sig_fp, data_path)192 verified = self._ctx.verify_file(sig_fp, data_path)
117 # If the file is properly signed, we'll be able to get back a set of193 # If the file is properly signed, we'll be able to get back a set of
@@ -120,3 +196,21 @@
120 # loaded-up keyrings. If so, the signature succeeds.196 # loaded-up keyrings. If so, the signature succeeds.
121 return verified.fingerprint in (self.fingerprints -197 return verified.fingerprint in (self.fingerprints -
122 self._blacklisted_fingerprints)198 self._blacklisted_fingerprints)
199
200 def validate(self, signature_path, data_path):
201 """Like .verify() but raises a SignatureError when invalid.
202
203 :param signature_path: The file system path to the detached signature
204 file for the data file.
205 :type signature_path: str
206 :param data_path: The file system path to the data file.
207 :type data_path: str
208 :return: None
209 :raises SignatureError: when the signature cannot be verified. Note
210 that the exception will contain extra information, namely the
211 keyrings involved in the verification, as well as the blacklist
212 file if there is one.
213 """
214 if not self.verify(signature_path, data_path):
215 raise SignatureError(signature_path, data_path,
216 self.keyring_paths, self.blacklist_path)
123217
=== modified file 'systemimage/helpers.py'
--- systemimage/helpers.py 2013-12-13 13:55:51 +0000
+++ systemimage/helpers.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/image.py'
--- systemimage/image.py 2013-12-13 13:55:51 +0000
+++ systemimage/image.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/index.py'
--- systemimage/index.py 2013-12-13 13:55:51 +0000
+++ systemimage/index.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/keyring.py'
--- systemimage/keyring.py 2013-12-13 13:55:51 +0000
+++ systemimage/keyring.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -31,7 +31,7 @@
31from datetime import datetime, timezone31from datetime import datetime, timezone
32from systemimage.config import config32from systemimage.config import config
33from systemimage.download import DBusDownloadManager33from systemimage.download import DBusDownloadManager
34from systemimage.gpg import Context, SignatureError34from systemimage.gpg import Context
35from systemimage.helpers import makedirs, safe_remove35from systemimage.helpers import makedirs, safe_remove
36from urllib.parse import urljoin36from urllib.parse import urljoin
3737
@@ -110,8 +110,7 @@
110 stack.callback(os.remove, ascxz_dst)110 stack.callback(os.remove, ascxz_dst)
111 signing_keyring = getattr(config.gpg, sigkr.replace('-', '_'))111 signing_keyring = getattr(config.gpg, sigkr.replace('-', '_'))
112 with Context(signing_keyring, blacklist=blacklist) as ctx:112 with Context(signing_keyring, blacklist=blacklist) as ctx:
113 if not ctx.verify(ascxz_dst, tarxz_dst):113 ctx.validate(ascxz_dst, tarxz_dst)
114 raise SignatureError
115 # The signature is good, so now unpack the tarball, load the json file114 # The signature is good, so now unpack the tarball, load the json file
116 # and verify its contents.115 # and verify its contents.
117 keyring_gpg = os.path.join(config.tempdir, 'keyring.gpg')116 keyring_gpg = os.path.join(config.tempdir, 'keyring.gpg')
118117
=== modified file 'systemimage/logging.py'
--- systemimage/logging.py 2013-12-13 13:55:51 +0000
+++ systemimage/logging.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -36,12 +36,35 @@
36LOGFILE_PERMISSIONS = stat.S_IRUSR | stat.S_IWUSR36LOGFILE_PERMISSIONS = stat.S_IRUSR | stat.S_IWUSR
3737
3838
39# We want to support {}-style logging for all systemimage child loggers. One
40# way to do this is with a LogRecord factory, but to play nice with third
41# party loggers which might be using %-style, we have to make sure that we use
42# the default factory for everything else.
43#
44# This actually isn't the best way to do this because it still makes a global
45# change and we don't know how this will interact with other third party
46# loggers. A marginally better way to do this is to pass class instances to
47# the logging calls. Those instances would have a __str__() method that does
48# the .format() conversion. The problem with that is that it's a bit less
49# convenient to make the logging calls because you can't pass strings
50# directly. One such suggestion at <http://tinyurl.com/pjjwjxq> is to import
51# the class as __ (i.e. double underscore) so your logging calls would look
52# like: log.error(__('Message with {} {}'), foo, bar)
53
39class FormattingLogRecord(logging.LogRecord):54class FormattingLogRecord(logging.LogRecord):
55 def __init__(self, name, *args, **kws):
56 logger_path = name.split('.')
57 self._use_format = (logger_path[0] == 'systemimage')
58 super().__init__(name, *args, **kws)
59
40 def getMessage(self):60 def getMessage(self):
41 msg = str(self.msg)61 if self._use_format:
42 if self.args:62 msg = str(self.msg)
43 msg = msg.format(*self.args)63 if self.args:
44 return msg64 msg = msg.format(*self.args)
65 return msg
66 else:
67 return super().getMessage()
4568
4669
47def initialize(*, verbosity=0):70def initialize(*, verbosity=0):
@@ -53,9 +76,7 @@
53 3: logging.CRITICAL,76 3: logging.CRITICAL,
54 }.get(verbosity, logging.ERROR)77 }.get(verbosity, logging.ERROR)
55 level = min(level, config.system.loglevel)78 level = min(level, config.system.loglevel)
56 # We're not going to propagate to the root logger anyway.79 # Make sure our library's logging uses {}-style messages.
57 logging.basicConfig(style='{')
58 # Make sure logging uses {}-style messages.
59 logging.setLogRecordFactory(FormattingLogRecord)80 logging.setLogRecordFactory(FormattingLogRecord)
60 # Now configure the application level logger based on the ini file.81 # Now configure the application level logger based on the ini file.
61 log = logging.getLogger('systemimage')82 log = logging.getLogger('systemimage')
6283
=== modified file 'systemimage/main.py'
--- systemimage/main.py 2013-12-13 13:55:51 +0000
+++ systemimage/main.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/reactor.py'
--- systemimage/reactor.py 2013-12-13 13:55:51 +0000
+++ systemimage/reactor.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/reboot.py'
--- systemimage/reboot.py 2013-12-13 13:55:51 +0000
+++ systemimage/reboot.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/scores.py'
--- systemimage/scores.py 2013-12-13 13:55:51 +0000
+++ systemimage/scores.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/service.py'
--- systemimage/service.py 2013-12-13 13:55:51 +0000
+++ systemimage/service.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -28,10 +28,9 @@
2828
29from contextlib import ExitStack29from contextlib import ExitStack
30from dbus.mainloop.glib import DBusGMainLoop30from dbus.mainloop.glib import DBusGMainLoop
31from dbus.service import BusName
32from pkg_resources import resource_string as resource_bytes31from pkg_resources import resource_string as resource_bytes
33from systemimage.config import config32from systemimage.config import config
34from systemimage.dbus import Loop, Service33from systemimage.dbus import Loop
35from systemimage.helpers import makedirs34from systemimage.helpers import makedirs
36from systemimage.logging import initialize35from systemimage.logging import initialize
37from systemimage.main import DEFAULT_CONFIG_FILE36from systemimage.main import DEFAULT_CONFIG_FILE
@@ -92,12 +91,20 @@
92 initialize(verbosity=args.verbose)91 initialize(verbosity=args.verbose)
93 log = logging.getLogger('systemimage')92 log = logging.getLogger('systemimage')
9493
95 log.info('SystemImage dbus main loop started [{}/{}]',
96 config.channel, config.device)
97 DBusGMainLoop(set_as_default=True)94 DBusGMainLoop(set_as_default=True)
9895
99 system_bus = dbus.SystemBus()96 system_bus = dbus.SystemBus()
100 bus_name = BusName('com.canonical.SystemImage', system_bus)97 # Ensure we're the only owner of this bus name.
98 code = system_bus.request_name(
99 'com.canonical.SystemImage',
100 dbus.bus.NAME_FLAG_DO_NOT_QUEUE)
101 if code == dbus.bus.REQUEST_NAME_REPLY_EXISTS:
102 # Another instance already owns this name. Exit.
103 log.error('Cannot get exclusive ownership of bus name.')
104 sys.exit(2)
105
106 log.info('SystemImage dbus main loop starting [{}/{}]',
107 config.channel, config.device)
101108
102 with ExitStack() as stack:109 with ExitStack() as stack:
103 loop = Loop()110 loop = Loop()
@@ -107,6 +114,7 @@
107 config.dbus_service = get_service(114 config.dbus_service = get_service(
108 testing_mode, system_bus, '/Service', loop)115 testing_mode, system_bus, '/Service', loop)
109 else:116 else:
117 from systemimage.dbus import Service
110 config.dbus_service = Service(system_bus, '/Service', loop)118 config.dbus_service = Service(system_bus, '/Service', loop)
111 try:119 try:
112 loop.run()120 loop.run()
113121
=== modified file 'systemimage/settings.py'
--- systemimage/settings.py 2013-12-13 13:55:51 +0000
+++ systemimage/settings.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/state.py'
--- systemimage/state.py 2013-12-13 13:55:51 +0000
+++ systemimage/state.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -217,7 +217,7 @@
217 urljoin(config.service.https_base, url)))217 urljoin(config.service.https_base, url)))
218 get_keyring('blacklist', url, 'image-master')218 get_keyring('blacklist', url, 'image-master')
219 except SignatureError:219 except SignatureError:
220 log.info('No signed blacklist found')220 log.exception('No signed blacklist found')
221 # The blacklist wasn't signed by the system image master. Maybe221 # The blacklist wasn't signed by the system image master. Maybe
222 # there's a new system image master key? Let's find out.222 # there's a new system image master key? Let's find out.
223 self._next.appendleft(self._get_master_key)223 self._next.appendleft(self._get_master_key)
@@ -293,14 +293,16 @@
293 # SIGNING key. There may or may not be a blacklist.293 # SIGNING key. There may or may not be a blacklist.
294 ctx = stack.enter_context(294 ctx = stack.enter_context(
295 Context(config.gpg.image_signing, blacklist=self.blacklist))295 Context(config.gpg.image_signing, blacklist=self.blacklist))
296 if not ctx.verify(asc_path, channels_path):296 try:
297 ctx.validate(asc_path, channels_path)
298 except SignatureError:
297 # The signature on the channels.json file did not match.299 # The signature on the channels.json file did not match.
298 # Maybe there's a new image signing key on the server. If300 # Maybe there's a new image signing key on the server. If
299 # we've already downloaded a new image signing key, then301 # we've already downloaded a new image signing key, then
300 # there's nothing more to do but raise an exception.302 # there's nothing more to do but raise an exception.
301 # Otherwise, if a new key *is* found, retry the current step.303 # Otherwise, if a new key *is* found, retry the current step.
302 if count > 0:304 if count > 0:
303 raise SignatureError(channels_path)305 raise
304 self._next.appendleft(self._get_signing_key)306 self._next.appendleft(self._get_signing_key)
305 log.info('channels.json not properly signed')307 log.info('channels.json not properly signed')
306 return308 return
@@ -398,10 +400,7 @@
398 keyrings.append(config.gpg.device_signing)400 keyrings.append(config.gpg.device_signing)
399 ctx = stack.enter_context(401 ctx = stack.enter_context(
400 Context(*keyrings, blacklist=self.blacklist))402 Context(*keyrings, blacklist=self.blacklist))
401 if not ctx.verify(asc_path, index_path):403 ctx.validate(asc_path, index_path)
402 log.error('index.json signature failure: {} {}',
403 index_path, asc_path)
404 raise SignatureError(index_path)
405 # The signature was good.404 # The signature was good.
406 with open(index_path, encoding='utf-8') as fp:405 with open(index_path, encoding='utf-8') as fp:
407 self.index = Index.from_json(fp.read())406 self.index = Index.from_json(fp.read())
@@ -512,8 +511,7 @@
512 # Verify the signatures on all the downloaded files.511 # Verify the signatures on all the downloaded files.
513 with Context(*keyrings, blacklist=self.blacklist) as ctx:512 with Context(*keyrings, blacklist=self.blacklist) as ctx:
514 for dst, asc in signatures:513 for dst, asc in signatures:
515 if not ctx.verify(asc, dst):514 ctx.validate(asc, dst)
516 raise SignatureError(dst)
517 # Verify the checksums.515 # Verify the checksums.
518 for dst, checksum in checksums:516 for dst, checksum in checksums:
519 with open(dst, 'rb') as fp:517 with open(dst, 'rb') as fp:
520518
=== modified file 'systemimage/testing/controller.py'
--- systemimage/testing/controller.py 2013-12-13 13:55:51 +0000
+++ systemimage/testing/controller.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -25,8 +25,8 @@
25import pwd25import pwd
26import sys26import sys
27import dbus27import dbus
28import time
28import psutil29import psutil
29import signal
30import subprocess30import subprocess
3131
32from contextlib import ExitStack32from contextlib import ExitStack
@@ -38,6 +38,8 @@
3838
3939
40SPACE = ' '40SPACE = ' '
41OVERRIDE = os.environ.get('SYSTEMIMAGE_DBUS_DAEMON_HUP_SLEEP_SECONDS')
42HUP_SLEEP = (0 if OVERRIDE is None else int(OVERRIDE))
4143
4244
43def start_system_image(controller):45def start_system_image(controller):
@@ -125,7 +127,7 @@
125class Controller:127class Controller:
126 """Start and stop D-Bus service under test."""128 """Start and stop D-Bus service under test."""
127129
128 def __init__(self):130 def __init__(self, logfile=None):
129 # Non-public.131 # Non-public.
130 self._stack = ExitStack()132 self._stack = ExitStack()
131 self._stoppers = []133 self._stoppers = []
@@ -148,21 +150,22 @@
148 # We need a client.ini file for the subprocess.150 # We need a client.ini file for the subprocess.
149 ini_tmpdir = self._stack.enter_context(temporary_directory())151 ini_tmpdir = self._stack.enter_context(temporary_directory())
150 ini_vardir = self._stack.enter_context(temporary_directory())152 ini_vardir = self._stack.enter_context(temporary_directory())
153 ini_logfile = (os.path.join(ini_tmpdir, 'client.log')
154 if logfile is None
155 else logfile)
151 self.ini_path = os.path.join(self.tmpdir, 'client.ini')156 self.ini_path = os.path.join(self.tmpdir, 'client.ini')
152 template = resource_bytes(157 template = resource_bytes(
153 'systemimage.tests.data', 'config_03.ini').decode('utf-8')158 'systemimage.tests.data', 'config_03.ini').decode('utf-8')
154 with open(self.ini_path, 'w', encoding='utf-8') as fp:159 with open(self.ini_path, 'w', encoding='utf-8') as fp:
155 print(template.format(tmpdir=ini_tmpdir, vardir=ini_vardir),160 print(template.format(tmpdir=ini_tmpdir, vardir=ini_vardir,
161 logfile=ini_logfile),
156 file=fp)162 file=fp)
157163
158 def _configure_services(self):164 def _configure_services(self):
159 # If the daemon is already running, kill all the children and HUP the165 # If the dbus-daemon is already running, kill all the children.
160 # daemon to reset dbus activation.
161 if self.daemon_pid is not None:166 if self.daemon_pid is not None:
162 for stopper in self._stoppers:167 for stopper in self._stoppers:
163 stopper(self)168 stopper(self)
164 process = psutil.Process(self.daemon_pid)
165 process.send_signal(signal.SIGHUP)
166 del self._stoppers[:]169 del self._stoppers[:]
167 # Now we have to set up the .service files. We use the Python170 # Now we have to set up the .service files. We use the Python
168 # executable used to run the tests, executing the entry point as would171 # executable used to run the tests, executing the entry point as would
@@ -178,6 +181,12 @@
178 with open(service_path, 'w', encoding='utf-8') as fp:181 with open(service_path, 'w', encoding='utf-8') as fp:
179 fp.write(config)182 fp.write(config)
180 self._stoppers.append(stopper)183 self._stoppers.append(stopper)
184 # If the dbus-daemon is running, reload its configuration files.
185 if self.daemon_pid is not None:
186 service = dbus.SystemBus().get_object('org.freedesktop.DBus', '/')
187 iface = dbus.Interface(service, 'org.freedesktop.DBus')
188 iface.ReloadConfig()
189 time.sleep(HUP_SLEEP)
181190
182 def set_mode(self, *, cert_pem=None, service_mode=''):191 def set_mode(self, *, cert_pem=None, service_mode=''):
183 self.mode = service_mode192 self.mode = service_mode
184193
=== modified file 'systemimage/testing/dbus.py'
--- systemimage/testing/dbus.py 2013-12-13 13:55:51 +0000
+++ systemimage/testing/dbus.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -31,6 +31,8 @@
31from systemimage.helpers import makedirs, safe_remove31from systemimage.helpers import makedirs, safe_remove
32from unittest.mock import patch32from unittest.mock import patch
3333
34from systemimage.testing.helpers import debug
35
3436
35SPACE = ' '37SPACE = ' '
36SIGNAL_DELAY_SECS = 538SIGNAL_DELAY_SECS = 5
@@ -65,7 +67,11 @@
65 @method('com.canonical.SystemImage')67 @method('com.canonical.SystemImage')
66 def Reset(self):68 def Reset(self):
67 self._api = Mediator()69 self._api = Mediator()
68 self._checking = False70 try:
71 self._checking.release()
72 except RuntimeError:
73 # Lock is already released.
74 pass
69 self._update = None75 self._update = None
70 self._downloading = False76 self._downloading = False
71 self._rebootable = False77 self._rebootable = False
7278
=== modified file 'systemimage/testing/demo.py'
--- systemimage/testing/demo.py 2013-12-13 13:55:51 +0000
+++ systemimage/testing/demo.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/testing/helpers.py'
--- systemimage/testing/helpers.py 2013-12-13 13:55:51 +0000
+++ systemimage/testing/helpers.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -107,6 +107,12 @@
107 # Please shut up.107 # Please shut up.
108 pass108 pass
109109
110 def handle_one_request(self):
111 try:
112 super().handle_one_request()
113 except ConnectionResetError:
114 super().handle_one_request()
115
110 def do_GET(self):116 def do_GET(self):
111 # If we requested the magic 'user-agent.txt' file, send back the117 # If we requested the magic 'user-agent.txt' file, send back the
112 # value of the User-Agent header. Otherwise, vend as normal.118 # value of the User-Agent header. Otherwise, vend as normal.
@@ -166,7 +172,16 @@
166 for conn in connections:172 for conn in connections:
167 if conn.fileno() != -1:173 if conn.fileno() != -1:
168 # Disallow sends and receives.174 # Disallow sends and receives.
169 conn.shutdown(SHUT_RDWR)175 try:
176 conn.shutdown(SHUT_RDWR)
177 except OSError:
178 # I'm ignoring all OSErrors here, although the only
179 # one I've seen semi-consistency is ENOTCONN [107]
180 # "Transport endpoint is not connected". I don't know
181 # why this happens, but it tells me that the client
182 # has already exited. We're shutting down, so who
183 # cares? (Or am I masking a real error?)
184 pass
170 conn.close()185 conn.close()
171 server.shutdown()186 server.shutdown()
172 thread.join()187 thread.join()
@@ -332,7 +347,7 @@
332 setup_keyring_txz(keyring + '.gpg', signing_kr, json_data, dst)347 setup_keyring_txz(keyring + '.gpg', signing_kr, json_data, dst)
333348
334349
335def setup_index(index, todir, keyring):350def setup_index(index, todir, keyring, write_callback=None):
336 for image in get_index(index).images:351 for image in get_index(index).images:
337 for filerec in image.files:352 for filerec in image.files:
338 path = (filerec.path[1:]353 path = (filerec.path[1:]
@@ -340,10 +355,13 @@
340 else filerec.path)355 else filerec.path)
341 dst = os.path.join(todir, path)356 dst = os.path.join(todir, path)
342 makedirs(os.path.dirname(dst))357 makedirs(os.path.dirname(dst))
343 contents = EMPTYSTRING.join(358 if write_callback is None:
344 os.path.splitext(filerec.path)[0].split('/'))359 contents = EMPTYSTRING.join(
345 with open(dst, 'w', encoding='utf-8') as fp:360 os.path.splitext(filerec.path)[0].split('/'))
346 fp.write(contents)361 with open(dst, 'w', encoding='utf-8') as fp:
362 fp.write(contents)
363 else:
364 write_callback(dst)
347 # Sign with the specified signing key.365 # Sign with the specified signing key.
348 sign(dst, keyring)366 sign(dst, keyring)
349367
350368
=== modified file 'systemimage/testing/nose.py'
--- systemimage/testing/nose.py 2013-12-13 13:55:51 +0000
+++ systemimage/testing/nose.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -75,15 +75,24 @@
75 super().__init__()75 super().__init__()
76 self.patterns = []76 self.patterns = []
77 self.verbosity = 077 self.verbosity = 0
78 self.log_file = None
78 self.addArgument(self.patterns, 'P', 'pattern',79 self.addArgument(self.patterns, 'P', 'pattern',
79 'Add a test matching pattern')80 'Add a test matching pattern')
80 def bump(ignore):81 def bump(ignore):
81 self.verbosity += 182 self.verbosity += 1
82 self.addFlag(bump, 'V', 'Verbosity',83 self.addFlag(bump, 'V', 'Verbosity',
83 'Increase system-image verbosity')84 'Increase system-image verbosity')
85 def set_log_file(path):
86 self.log_file = path[0]
87 self.addOption(set_log_file, 'L', 'logfile',
88 'Set the log file for the test run',
89 nargs=1)
8490
85 @configuration91 @configuration
86 def startTestRun(self, event):92 def startTestRun(self, event):
93 from systemimage.config import config
94 if self.log_file is not None:
95 config.system.logfile = self.log_file
87 DBusGMainLoop(set_as_default=True)96 DBusGMainLoop(set_as_default=True)
88 initialize(verbosity=self.verbosity)97 initialize(verbosity=self.verbosity)
89 # We need to set up the dbus service controller, since all the tests98 # We need to set up the dbus service controller, since all the tests
@@ -92,7 +101,7 @@
92 # individual services, and we can write new dbus configuration files101 # individual services, and we can write new dbus configuration files
93 # and HUP the dbus-launch to re-read them, but we cannot change bus102 # and HUP the dbus-launch to re-read them, but we cannot change bus
94 # addresses after the initial one is set.103 # addresses after the initial one is set.
95 SystemImagePlugin.controller = Controller()104 SystemImagePlugin.controller = Controller(self.log_file)
96 SystemImagePlugin.controller.start()105 SystemImagePlugin.controller.start()
97 atexit.register(SystemImagePlugin.controller.stop)106 atexit.register(SystemImagePlugin.controller.stop)
98107
@@ -123,3 +132,13 @@
123 SystemImagePlugin.controller.stop()132 SystemImagePlugin.controller.stop()
124 # Let other plugins continue printing.133 # Let other plugins continue printing.
125 return None134 return None
135
136 ## def startTest(self, event):
137 ## from systemimage.testing.helpers import debug
138 ## with debug() as dlog:
139 ## dlog('vvvvv', event.test)
140
141 ## def stopTest(self, event):
142 ## from systemimage.testing.helpers import debug
143 ## with debug() as dlog:
144 ## dlog('^^^^^', event.test)
126145
=== modified file 'systemimage/tests/data/config_03.ini'
--- systemimage/tests/data/config_03.ini 2013-12-13 13:55:51 +0000
+++ systemimage/tests/data/config_03.ini 2014-02-25 17:45:25 +0000
@@ -14,7 +14,7 @@
14timeout: 1s14timeout: 1s
15build_file: {tmpdir}/ubuntu-build15build_file: {tmpdir}/ubuntu-build
16tempdir: {tmpdir}/tmp16tempdir: {tmpdir}/tmp
17logfile: {tmpdir}/client.log17logfile: {logfile}
18loglevel: info18loglevel: info
19settings_db: {vardir}/settings.db19settings_db: {vardir}/settings.db
2020
2121
=== added file 'systemimage/tests/data/index_24.json'
--- systemimage/tests/data/index_24.json 1970-01-01 00:00:00 +0000
+++ systemimage/tests/data/index_24.json 2014-02-25 17:45:25 +0000
@@ -0,0 +1,36 @@
1{
2 "global": {
3 "generated_at": "Thu Aug 01 08:01:00 UTC 2013"
4 },
5 "images": [
6 {
7 "description": "Full",
8 "files": [
9 {
10 "checksum": "5b05b298e974f3b9e40f0a1a8188f50984a4f18fb329e050324296632d3d9dfc",
11 "order": 3,
12 "path": "/3/4/5.txt",
13 "signature": "/3/4/5.txt.asc",
14 "size": 104857600
15 },
16 {
17 "checksum": "5b05b298e974f3b9e40f0a1a8188f50984a4f18fb329e050324296632d3d9dfc",
18 "order": 1,
19 "path": "/4/5/6.txt",
20 "signature": "/4/5/6.txt.asc",
21 "size": 104857600
22 },
23 {
24 "checksum": "5b05b298e974f3b9e40f0a1a8188f50984a4f18fb329e050324296632d3d9dfc",
25 "order": 2,
26 "path": "/5/6/7.txt",
27 "signature": "/5/6/7.txt.asc",
28 "size": 104857600
29 }
30 ],
31 "type": "full",
32 "version": 1600,
33 "bootme": true
34 }
35 ]
36}
037
=== modified file 'systemimage/tests/test_api.py'
--- systemimage/tests/test_api.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_api.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_bag.py'
--- systemimage/tests/test_bag.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_bag.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_candidates.py'
--- systemimage/tests/test_candidates.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_candidates.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_channel.py'
--- systemimage/tests/test_channel.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_channel.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_config.py'
--- systemimage/tests/test_config.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_config.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_dbus.py'
--- systemimage/tests/test_dbus.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_dbus.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -23,6 +23,7 @@
23 'TestDBusGetSet',23 'TestDBusGetSet',
24 'TestDBusInfo',24 'TestDBusInfo',
25 'TestDBusInfoNoDetails',25 'TestDBusInfoNoDetails',
26 'TestDBusLP1277589',
26 'TestDBusMockFailApply',27 'TestDBusMockFailApply',
27 'TestDBusMockFailPause',28 'TestDBusMockFailPause',
28 'TestDBusMockFailResume',29 'TestDBusMockFailResume',
@@ -122,6 +123,23 @@
122 self.quit()123 self.quit()
123124
124125
126class DoubleCheckingReactor(Reactor):
127 def __init__(self, iface):
128 super().__init__(dbus.SystemBus())
129 self.iface = iface
130 self.uas_signals = []
131 self.react_to('UpdateAvailableStatus')
132 self.react_to('UpdateDownloaded')
133 self.schedule(self.iface.CheckForUpdate)
134
135 def _do_UpdateAvailableStatus(self, signal, path, *args, **kws):
136 self.uas_signals.append(args)
137 self.schedule(self.iface.CheckForUpdate)
138
139 def _do_UpdateDownloaded(self, *args, **kws):
140 self.quit()
141
142
125class _TestBase(unittest.TestCase):143class _TestBase(unittest.TestCase):
126 """Base class for all DBus testing."""144 """Base class for all DBus testing."""
127145
@@ -262,13 +280,14 @@
262 safe_remove(self.reboot_log)280 safe_remove(self.reboot_log)
263 super().tearDown()281 super().tearDown()
264282
265 def _prepare_index(self, index_file):283 def _prepare_index(self, index_file, write_callback=None):
266 serverdir = SystemImagePlugin.controller.serverdir284 serverdir = SystemImagePlugin.controller.serverdir
267 index_path = os.path.join(serverdir, 'stable', 'nexus7', 'index.json')285 index_path = os.path.join(serverdir, 'stable', 'nexus7', 'index.json')
268 head, tail = os.path.split(index_path)286 head, tail = os.path.split(index_path)
269 copy(index_file, head, tail)287 copy(index_file, head, tail)
270 sign(index_path, 'device-signing.gpg')288 sign(index_path, 'device-signing.gpg')
271 setup_index(index_file, serverdir, 'device-signing.gpg')289 setup_index(index_file, serverdir, 'device-signing.gpg',
290 write_callback)
272291
273 def _touch_build(self, version):292 def _touch_build(self, version):
274 # Unlike the touch_build() helper, this one uses our own config object293 # Unlike the touch_build() helper, this one uses our own config object
@@ -374,6 +393,23 @@
374 self.assertEqual(last_update_date, '2013-01-20 12:01:45')393 self.assertEqual(last_update_date, '2013-01-20 12:01:45')
375 # All other values are undefined.394 # All other values are undefined.
376395
396 def test_check_for_update_twice(self):
397 # Issue two CheckForUpdate calls immediate after each other.
398 self.download_always()
399 reactor = SignalCapturingReactor('UpdateAvailableStatus')
400 def two_calls():
401 self.iface.CheckForUpdate()
402 self.iface.CheckForUpdate()
403 reactor.run(two_calls)
404 self.assertEqual(len(reactor.signals), 1)
405 # There's one boolean argument to the result.
406 (is_available, downloading, available_version, update_size,
407 last_update_date,
408 # descriptions,
409 error_reason) = reactor.signals[0]
410 self.assertTrue(is_available)
411 self.assertTrue(downloading)
412
377 @unittest.skip('LP: #1215586')413 @unittest.skip('LP: #1215586')
378 def test_get_multilingual_descriptions(self):414 def test_get_multilingual_descriptions(self):
379 # The descriptions are multilingual.415 # The descriptions are multilingual.
@@ -1368,7 +1404,9 @@
1368 def setUp(self):1404 def setUp(self):
1369 super().setUp()1405 super().setUp()
1370 # We have to hack the files to be rather large so that the download1406 # We have to hack the files to be rather large so that the download
1371 # doesn't complete before we get a chance to pause it.1407 # doesn't complete before we get a chance to pause it. Of course,
1408 # this breaks the signatures because we're changing the file contents
1409 # after the .asc files have been written.
1372 for path in ('3/4/5.txt', '4/5/6.txt', '5/6/7.txt'):1410 for path in ('3/4/5.txt', '4/5/6.txt', '5/6/7.txt'):
1373 full_path = os.path.join(1411 full_path = os.path.join(
1374 SystemImagePlugin.controller.serverdir, path)1412 SystemImagePlugin.controller.serverdir, path)
@@ -1391,15 +1429,25 @@
1391 self.assertTrue(reactor.paused)1429 self.assertTrue(reactor.paused)
1392 # Now let's resume the download. Because we intentionally corrupted1430 # Now let's resume the download. Because we intentionally corrupted
1393 # the downloaded files, we'll get an UpdateFailed signal instead of1431 # the downloaded files, we'll get an UpdateFailed signal instead of
1394 # the successful UpdateDownloaded signal. We can ignore that.1432 # the successful UpdateDownloaded signal.
1395 reactor = SignalCapturingReactor('UpdateFailed')1433 reactor = SignalCapturingReactor('UpdateFailed')
1396 reactor.run(self.iface.DownloadUpdate, timeout=60)1434 reactor.run(self.iface.DownloadUpdate, timeout=60)
1397 self.assertEqual(len(reactor.signals), 1)1435 self.assertEqual(len(reactor.signals), 1)
1398 # We've gotten one error and the first file that failed is 5.txt.1436 # The error message will include lots of details on the SignatureError
1437 # that results. The key thing is that it's 5.txt that is the first
1438 # file to fail its signature check.
1399 failure_count, last_error = reactor.signals[0]1439 failure_count, last_error = reactor.signals[0]
1400 self.assertEqual(failure_count, 1)1440 self.assertEqual(failure_count, 1)
1401 # Watch out for the trailing newline.1441 check_next = False
1402 self.assertEqual(os.path.basename(last_error[:-1]), '5.txt')1442 for line in last_error.splitlines():
1443 line = line.strip()
1444 if check_next:
1445 self.assertEqual(os.path.basename(line), '5.txt')
1446 break
1447 if line.startswith('data path:'):
1448 check_next = True
1449 else:
1450 raise AssertionError('Did not find expected error output')
14031451
14041452
1405class TestDBusUseCache(_LiveTesting):1453class TestDBusUseCache(_LiveTesting):
@@ -1513,3 +1561,45 @@
1513update 5.txt 5.txt.asc1561update 5.txt 5.txt.asc
1514unmount system1562unmount system
1515""")1563""")
1564
1565
1566class TestDBusLP1277589(_LiveTesting):
1567 def test_multiple_check_for_updates(self):
1568 # Log analysis of LP: #1277589 appears to show the following scenario,
1569 # reproduced in this test case:
1570 #
1571 # * Automatic updates are enabled.
1572 # * No image signing or image master keys are present.
1573 # * A full update is checked.
1574 # - A new image master key and image signing key is downloaded.
1575 # - Update is available
1576 #
1577 # Start by creating some big files which will take a while to
1578 # download.
1579 def write_callback(dst):
1580 # Write a 100 MiB sized file.
1581 with open(dst, 'wb') as fp:
1582 for i in range(25600):
1583 fp.write(b'x' * 4096)
1584 self._prepare_index('index_24.json', write_callback)
1585 # Create a reactor that will exit when the UpdateDownloaded signal is
1586 # received. We're going to issue a CheckForUpdate with automatic
1587 # updates enabled. As soon as we receive the UpdateAvailableStatus
1588 # signal, we'll immediately issue *another* CheckForUpdate, which
1589 # should run while the auto-download is working.
1590 #
1591 # At the end, we should not get another UpdateAvailableStatus signal,
1592 # but we should get the UpdateDownloaded signal.
1593 reactor = DoubleCheckingReactor(self.iface)
1594 reactor.run()
1595 self.assertEqual(len(reactor.uas_signals), 1)
1596 (is_available, downloading, available_version, update_size,
1597 last_update_date,
1598 #descriptions,
1599 error_reason) = reactor.uas_signals[0]
1600 self.assertTrue(is_available)
1601 self.assertTrue(downloading)
1602 self.assertEqual(available_version, '1600')
1603 self.assertEqual(update_size, 314572800)
1604 self.assertEqual(last_update_date, 'Unknown')
1605 self.assertEqual(error_reason, '')
15161606
=== modified file 'systemimage/tests/test_download.py'
--- systemimage/tests/test_download.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_download.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_gpg.py'
--- systemimage/tests/test_gpg.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_gpg.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -18,15 +18,20 @@
18__all__ = [18__all__ = [
19 'TestKeyrings',19 'TestKeyrings',
20 'TestSignature',20 'TestSignature',
21 'TestSignatureError',
21 ]22 ]
2223
2324
24import os25import os
26import sys
27import hashlib
25import unittest28import unittest
29import traceback
2630
27from contextlib import ExitStack31from contextlib import ExitStack
32from io import StringIO
28from systemimage.config import config33from systemimage.config import config
29from systemimage.gpg import Context34from systemimage.gpg import Context, SignatureError
30from systemimage.helpers import temporary_directory35from systemimage.helpers import temporary_directory
31from systemimage.testing.helpers import (36from systemimage.testing.helpers import (
32 configuration, copy, setup_keyring_txz, setup_keyrings, sign)37 configuration, copy, setup_keyring_txz, setup_keyrings, sign)
@@ -330,3 +335,171 @@
330 with Context(keyring_1, keyring_2, blacklist=blacklist) as ctx:335 with Context(keyring_1, keyring_2, blacklist=blacklist) as ctx:
331 self.assertFalse(336 self.assertFalse(
332 ctx.verify(channels_json + '.asc', channels_json))337 ctx.verify(channels_json + '.asc', channels_json))
338
339 @configuration
340 def test_good_validation(self):
341 # The .validate() method does nothing if the signature is good.
342 channels_json = os.path.join(self._tmpdir, 'channels.json')
343 copy('channels_01.json', self._tmpdir, dst=channels_json)
344 sign(channels_json, 'image-signing.gpg')
345 with temporary_directory() as tmpdir:
346 keyring = os.path.join(tmpdir, 'image-signing.tar.xz')
347 setup_keyring_txz('image-signing.gpg', 'image-master.gpg',
348 dict(type='image-signing'), keyring)
349 with Context(keyring) as ctx:
350 self.assertIsNone(
351 ctx.validate(channels_json + '.asc', channels_json))
352
353
354class TestSignatureError(unittest.TestCase):
355 def setUp(self):
356 self._stack = ExitStack()
357 self._tmpdir = self._stack.enter_context(temporary_directory())
358
359 def tearDown(self):
360 self._stack.close()
361
362 def test_extra_data(self):
363 # A SignatureError includes extra information about the path to the
364 # signature file, and the path to the data file. You also get the md5
365 # checksums of those two paths.
366 signature_path = os.path.join(self._tmpdir, 'signature')
367 data_path = os.path.join(self._tmpdir, 'data')
368 with open(signature_path, 'wb') as fp:
369 fp.write(b'012345')
370 with open(data_path, 'wb') as fp:
371 fp.write(b'67890a')
372 error = SignatureError(signature_path, data_path)
373 self.assertEqual(error.signature_path, signature_path)
374 self.assertEqual(error.data_path, data_path)
375 self.assertEqual(
376 error.signature_checksum, 'd6a9a933c8aafc51e55ac0662b6e4d4a')
377 self.assertEqual(
378 error.data_checksum, 'e82780258de250078f7ad3f595d71f6d')
379
380 @configuration
381 def test_signature_invalid(self):
382 # The .validate() method raises a SignatureError exception with extra
383 # information when the signature is invalid.
384 channels_json = os.path.join(self._tmpdir, 'channels.json')
385 copy('channels_01.json', self._tmpdir, dst=channels_json)
386 sign(channels_json, 'device-signing.gpg')
387 # Verify the signature with the pubkey.
388 with temporary_directory() as tmpdir:
389 dst = os.path.join(tmpdir, 'image-signing.tar.xz')
390 setup_keyring_txz('image-signing.gpg', 'image-master.gpg',
391 dict(type='image-signing'), dst)
392 # Get the dst's checksum now, because the file will get deleted
393 # when the tmpdir context manager exits.
394 with open(dst, 'rb') as fp:
395 dst_checksum = hashlib.md5(fp.read()).hexdigest()
396 with Context(dst) as ctx:
397 with self.assertRaises(SignatureError) as cm:
398 ctx.validate(channels_json + '.asc', channels_json)
399 error = cm.exception
400 basename = os.path.basename
401 self.assertEqual(basename(error.signature_path), 'channels.json.asc')
402 self.assertEqual(basename(error.data_path), 'channels.json')
403 # The contents of the signature file are not predictable.
404 with open(channels_json + '.asc', 'rb') as fp:
405 checksum = hashlib.md5(fp.read()).hexdigest()
406 self.assertEqual(error.signature_checksum, checksum)
407 self.assertEqual(
408 error.data_checksum, '715c63fecbf44b62f9fa04a82dfa7d29')
409 basenames = [basename(path) for path in error.keyrings]
410 self.assertEqual(basenames, ['image-signing.tar.xz'])
411 self.assertIsNone(error.blacklist)
412 self.assertEqual(error.keyring_checksums, [dst_checksum])
413 self.assertIsNone(error.blacklist_checksum)
414
415 @configuration
416 def test_signature_invalid_due_to_blacklist(self):
417 # Like above, but we put the device signing key id in the blacklist.
418 channels_json = os.path.join(self._tmpdir, 'channels.json')
419 copy('channels_01.json', self._tmpdir, dst=channels_json)
420 sign(channels_json, 'device-signing.gpg')
421 # Verify the signature with the pubkey.
422 with temporary_directory() as tmpdir:
423 keyring_1 = os.path.join(tmpdir, 'image-signing.tar.xz')
424 keyring_2 = os.path.join(tmpdir, 'device-signing.tar.xz')
425 blacklist = os.path.join(tmpdir, 'blacklist.tar.xz')
426 setup_keyring_txz('image-signing.gpg', 'image-master.gpg',
427 dict(type='image-signing'), keyring_1)
428 setup_keyring_txz('device-signing.gpg', 'image-signing.gpg',
429 dict(type='device-signing'), keyring_2)
430 # We're letting the device signing pubkey stand in for a blacklist.
431 setup_keyring_txz('device-signing.gpg', 'image-master.gpg',
432 dict(type='blacklist'), blacklist)
433 # Get the keyring checksums now, because the files will get
434 # deleted when the tmpdir context manager exits.
435 keyring_checksums = []
436 for path in (keyring_1, keyring_2):
437 with open(path, 'rb') as fp:
438 checksum = hashlib.md5(fp.read()).hexdigest()
439 keyring_checksums.append(checksum)
440 with open(blacklist, 'rb') as fp:
441 blacklist_checksum = hashlib.md5(fp.read()).hexdigest()
442 with Context(keyring_1, keyring_2, blacklist=blacklist) as ctx:
443 with self.assertRaises(SignatureError) as cm:
444 ctx.validate(channels_json + '.asc', channels_json)
445 error = cm.exception
446 basename = os.path.basename
447 self.assertEqual(basename(error.signature_path), 'channels.json.asc')
448 self.assertEqual(basename(error.data_path), 'channels.json')
449 # The contents of the signature file are not predictable.
450 with open(channels_json + '.asc', 'rb') as fp:
451 checksum = hashlib.md5(fp.read()).hexdigest()
452 self.assertEqual(error.signature_checksum, checksum)
453 self.assertEqual(
454 error.data_checksum, '715c63fecbf44b62f9fa04a82dfa7d29')
455 basenames = [basename(path) for path in error.keyrings]
456 self.assertEqual(basenames, ['image-signing.tar.xz',
457 'device-signing.tar.xz'])
458 self.assertEqual(basename(error.blacklist), 'blacklist.tar.xz')
459 self.assertEqual(error.keyring_checksums, keyring_checksums)
460 self.assertEqual(error.blacklist_checksum, blacklist_checksum)
461
462 @configuration
463 def test_signature_error_logging(self):
464 # The repr/str of the SignatureError should contain lots of useful
465 # information that will make debugging easier.
466 channels_json = os.path.join(self._tmpdir, 'channels.json')
467 copy('channels_01.json', self._tmpdir, dst=channels_json)
468 sign(channels_json, 'device-signing.gpg')
469 # Verify the signature with the pubkey.
470 tmpdir = self._stack.enter_context(temporary_directory())
471 dst = os.path.join(tmpdir, 'image-signing.tar.xz')
472 setup_keyring_txz('image-signing.gpg', 'image-master.gpg',
473 dict(type='image-signing'), dst)
474 output = StringIO()
475 with Context(dst) as ctx:
476 try:
477 ctx.validate(channels_json + '.asc', channels_json)
478 except SignatureError:
479 # For our purposes, log.exception() is essentially a wrapper
480 # around this traceback call. We don't really care about the
481 # full stack trace though.
482 e = sys.exc_info()
483 traceback.print_exception(e[0], e[1], e[2],
484 limit=0, file=output)
485 # 2014-02-12 BAW: Yuck, but I can't get assertRegex() to work properly.
486 for i, line in enumerate(output.getvalue().splitlines()):
487 if i == 0:
488 self.assertEqual(line, 'Traceback (most recent call last):')
489 elif i == 1:
490 self.assertEqual(line, 'systemimage.gpg.SignatureError: ')
491 elif i == 2:
492 self.assertTrue(line.startswith(' sig path :'))
493 elif i == 3:
494 self.assertTrue(line.endswith('/channels.json.asc'))
495 elif i == 4:
496 self.assertEqual(
497 line, ' data path: 715c63fecbf44b62f9fa04a82dfa7d29')
498 elif i == 5:
499 self.assertTrue(line.endswith('/channels.json'))
500 elif i == 6:
501 self.assertTrue(line.startswith(' keyrings :'))
502 elif i == 7:
503 self.assertTrue(line.endswith("/image-signing.tar.xz']"))
504 elif i == 8:
505 self.assertEqual(line, ' blacklist: no blacklist ')
333506
=== modified file 'systemimage/tests/test_helpers.py'
--- systemimage/tests/test_helpers.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_helpers.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_image.py'
--- systemimage/tests/test_image.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_image.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_index.py'
--- systemimage/tests/test_index.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_index.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_keyring.py'
--- systemimage/tests/test_keyring.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_keyring.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -22,6 +22,7 @@
2222
2323
24import os24import os
25import hashlib
25import unittest26import unittest
2627
27from contextlib import ExitStack28from contextlib import ExitStack
@@ -174,11 +175,25 @@
174 setup_keyrings()175 setup_keyrings()
175 # Use the spare key as the blacklist, signed by itself. Since this176 # Use the spare key as the blacklist, signed by itself. Since this
176 # won't match the image-signing key, the check will fail.177 # won't match the image-signing key, the check will fail.
178 server_path = os.path.join(self._serverdir, 'gpg', 'blacklist.tar.xz')
177 setup_keyring_txz(179 setup_keyring_txz(
178 'spare.gpg', 'spare.gpg', dict(type='blacklist'),180 'spare.gpg', 'spare.gpg', dict(type='blacklist'), server_path)
179 os.path.join(self._serverdir, 'gpg', 'blacklist.tar.xz'))181 with self.assertRaises(SignatureError) as cm:
180 self.assertRaises(SignatureError, get_keyring,182 get_keyring('blacklist', 'gpg/blacklist.tar.xz', 'image-master')
181 'blacklist', 'gpg/blacklist.tar.xz', 'image-master')183 error = cm.exception
184 # The local file name will be keyring.tar.xz in the cache directory.
185 basename = os.path.basename
186 self.assertEqual(basename(error.data_path), 'keyring.tar.xz')
187 self.assertEqual(basename(error.signature_path), 'keyring.tar.xz.asc')
188 # The crafted blacklist.tar.xz file will have an unpredictable
189 # checksum due to tarfile variablility.
190 with open(server_path, 'rb') as fp:
191 checksum = hashlib.md5(fp.read()).hexdigest()
192 self.assertEqual(error.data_checksum, checksum)
193 # The signature file's checksum is also unpredictable.
194 with open(server_path + '.asc', 'rb') as fp:
195 checksum = hashlib.md5(fp.read()).hexdigest()
196 self.assertEqual(error.signature_checksum, checksum)
182197
183 @configuration198 @configuration
184 def test_blacklisted_signature(self):199 def test_blacklisted_signature(self):
185200
=== modified file 'systemimage/tests/test_main.py'
--- systemimage/tests/test_main.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_main.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
@@ -26,11 +26,13 @@
2626
2727
28import os28import os
29import sys
29import dbus30import dbus
30import stat31import stat
31import time32import time
32import shutil33import shutil
33import unittest34import unittest
35import subprocess
3436
35from contextlib import ExitStack, contextmanager37from contextlib import ExitStack, contextmanager
36from datetime import datetime38from datetime import datetime
@@ -781,3 +783,25 @@
781 self.assertEqual(stat.filemode(mode), 'drwx--S---')783 self.assertEqual(stat.filemode(mode), 'drwx--S---')
782 mode = os.stat(config.system.logfile).st_mode784 mode = os.stat(config.system.logfile).st_mode
783 self.assertEqual(stat.filemode(mode), '-rw-------')785 self.assertEqual(stat.filemode(mode), '-rw-------')
786
787 def test_single_instance(self):
788 # Only one instance of the system-image-dbus service is allowed to
789 # remain active on a single system bus.
790 self.assertIsNone(find_dbus_process(self.ini_path))
791 self._activate()
792 proc = find_dbus_process(self.ini_path)
793 # Attempt to start a second process on the same system bus.
794 env = dict(
795 DBUS_SYSTEM_BUS_ADDRESS=os.environ['DBUS_SYSTEM_BUS_ADDRESS'])
796 args = (sys.executable, '-m', 'systemimage.service',
797 '-C', self.ini_path)
798 second = subprocess.Popen(args, universal_newlines=True, env=env)
799 # Allow a TimeoutExpired exception to fail the test.
800 try:
801 code = second.wait(timeout=10)
802 except subprocess.TimeoutExpired:
803 second.kill()
804 second.communicate()
805 raise
806 self.assertNotEqual(second.pid, proc)
807 self.assertEqual(code, 2)
784808
=== modified file 'systemimage/tests/test_scores.py'
--- systemimage/tests/test_scores.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_scores.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_settings.py'
--- systemimage/tests/test_settings.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_settings.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_state.py'
--- systemimage/tests/test_state.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_state.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/tests/test_winner.py'
--- systemimage/tests/test_winner.py 2013-12-13 13:55:51 +0000
+++ systemimage/tests/test_winner.py 2014-02-25 17:45:25 +0000
@@ -1,4 +1,4 @@
1# Copyright (C) 2013 Canonical Ltd.1# Copyright (C) 2013-2014 Canonical Ltd.
2# Author: Barry Warsaw <barry@ubuntu.com>2# Author: Barry Warsaw <barry@ubuntu.com>
33
4# This program is free software: you can redistribute it and/or modify4# This program is free software: you can redistribute it and/or modify
55
=== modified file 'systemimage/version.txt'
--- systemimage/version.txt 2013-12-13 13:55:51 +0000
+++ systemimage/version.txt 2014-02-25 17:45:25 +0000
@@ -1,1 +1,1 @@
12.0.312.1
22
=== modified file 'tox.ini'
--- tox.ini 2013-12-13 13:55:51 +0000
+++ tox.ini 2014-02-25 17:45:25 +0000
@@ -1,5 +1,5 @@
1[tox]1[tox]
2envlist = py332envlist = py33, py34
33
4[testenv]4[testenv]
5commands = python -m nose2 -v5commands = python -m nose2 -v

Subscribers

People subscribed via source and target branches