Merge lp:~barry/ubuntu-system-image/citrain302 into lp:~ubuntu-managed-branches/ubuntu-system-image/system-image

Proposed by Barry Warsaw on 2015-09-25
Status: Merged
Approved by: Barry Warsaw on 2015-09-30
Approved revision: 245
Merged at revision: 244
Proposed branch: lp:~barry/ubuntu-system-image/citrain302
Merge into: lp:~ubuntu-managed-branches/ubuntu-system-image/system-image
Diff against target: 395 lines (+147/-18)
14 files modified
NEWS.rst (+5/-0)
PKG-INFO (+1/-1)
debian/changelog (+11/-0)
debian/rules (+3/-0)
setup.cfg (+1/-1)
system_image.egg-info/PKG-INFO (+1/-1)
system_image.egg-info/SOURCES.txt (+1/-0)
system_image.egg-info/pbr.json (+1/-0)
systemimage/helpers.py (+12/-6)
systemimage/logging.py (+10/-6)
systemimage/testing/dbus.py (+28/-0)
systemimage/tests/test_dbus.py (+26/-2)
systemimage/tests/test_helpers.py (+46/-0)
systemimage/version.txt (+1/-1)
To merge this branch: bzr merge lp:~barry/ubuntu-system-image/citrain302
Reviewer Review Type Date Requested Status
Barry Warsaw Pending
Review via email: mp+272481@code.launchpad.net

Commit message

Don't crash when one of the .ini files is a dangling symlink. (LP: #1495688)

Description of the change

3.0.2 (2015-09-22)
==================
 * Don't crash when one of the .ini files is a dangling symlink.
   (LP: #1495688)

To post a comment you must log in.
245. By Barry Warsaw on 2015-09-28

d/rules: override_dh_auto_clean because otherwise, pybuild will
remove the .egg-info files and that causes the Jenkins job in the CI
train to fail.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'NEWS.rst'
2--- NEWS.rst 2015-06-17 15:18:22 +0000
3+++ NEWS.rst 2015-09-28 21:36:51 +0000
4@@ -2,6 +2,11 @@
5 NEWS for system-image updater
6 =============================
7
8+3.0.2 (2015-09-22)
9+==================
10+ * Don't crash when one of the .ini files is a dangling symlink.
11+ (LP: #1495688)
12+
13 3.0.1 (2015-06-16)
14 ==================
15 * When `--progress=json` is used, print an error record to stdout if the
16
17=== modified file 'PKG-INFO'
18--- PKG-INFO 2015-06-17 15:18:22 +0000
19+++ PKG-INFO 2015-09-28 21:36:51 +0000
20@@ -1,6 +1,6 @@
21 Metadata-Version: 1.0
22 Name: system-image
23-Version: 3.0.1
24+Version: 3.0.2
25 Summary: Ubuntu System Image Based Upgrades
26 Home-page: UNKNOWN
27 Author: Barry Warsaw
28
29=== modified file 'debian/changelog'
30--- debian/changelog 2015-06-18 16:23:37 +0000
31+++ debian/changelog 2015-09-28 21:36:51 +0000
32@@ -1,3 +1,14 @@
33+system-image (3.0.2-0ubuntu1) wily; urgency=medium
34+
35+ * New upstream release.
36+ - LP: #1495688 - Don't crash when one of the .ini files is a dangling
37+ symlink.
38+ - d/rules: override_dh_auto_clean because otherwise, pybuild will
39+ remove the .egg-info files and that causes the Jenkins job in the CI
40+ train to fail.
41+
42+ -- Barry Warsaw <barry@ubuntu.com> Fri, 25 Sep 2015 15:28:37 -0400
43+
44 system-image (3.0.1-0ubuntu1) wily; urgency=medium
45
46 * New rebuild forced.
47
48=== modified file 'debian/rules'
49--- debian/rules 2015-05-09 15:50:39 +0000
50+++ debian/rules 2015-09-28 21:36:51 +0000
51@@ -56,3 +56,6 @@
52 rst2man dbus-manpage.rst > debian/tmp/system-image-dbus.man
53 rst2man ini-manpage.rst > debian/tmp/client-ini.man
54 dh_installman
55+
56+# We don't want the buildds to remove the .egg-info files.
57+override_dh_auto_clean:
58
59=== modified file 'setup.cfg'
60--- setup.cfg 2015-06-17 15:18:22 +0000
61+++ setup.cfg 2015-09-28 21:36:51 +0000
62@@ -4,7 +4,7 @@
63 logging-filter = systemimage
64
65 [egg_info]
66-tag_date = 0
67 tag_svn_revision = 0
68 tag_build =
69+tag_date = 0
70
71
72=== modified file 'system_image.egg-info/PKG-INFO'
73--- system_image.egg-info/PKG-INFO 2015-06-17 15:18:22 +0000
74+++ system_image.egg-info/PKG-INFO 2015-09-28 21:36:51 +0000
75@@ -1,6 +1,6 @@
76 Metadata-Version: 1.0
77 Name: system-image
78-Version: 3.0.1
79+Version: 3.0.2
80 Summary: Ubuntu System Image Based Upgrades
81 Home-page: UNKNOWN
82 Author: Barry Warsaw
83
84=== modified file 'system_image.egg-info/SOURCES.txt'
85--- system_image.egg-info/SOURCES.txt 2015-05-08 21:41:15 +0000
86+++ system_image.egg-info/SOURCES.txt 2015-09-28 21:36:51 +0000
87@@ -14,6 +14,7 @@
88 system_image.egg-info/SOURCES.txt
89 system_image.egg-info/dependency_links.txt
90 system_image.egg-info/entry_points.txt
91+system_image.egg-info/pbr.json
92 system_image.egg-info/requires.txt
93 system_image.egg-info/top_level.txt
94 systemimage/__init__.py
95
96=== added file 'system_image.egg-info/pbr.json'
97--- system_image.egg-info/pbr.json 1970-01-01 00:00:00 +0000
98+++ system_image.egg-info/pbr.json 2015-09-28 21:36:51 +0000
99@@ -0,0 +1,1 @@
100+{"is_release": true, "git_version": "4ecc87c"}
101\ No newline at end of file
102
103=== modified file 'systemimage/helpers.py'
104--- systemimage/helpers.py 2015-05-08 21:41:15 +0000
105+++ systemimage/helpers.py 2015-09-28 21:36:51 +0000
106@@ -41,7 +41,7 @@
107 import logging
108 import tempfile
109
110-from contextlib import ExitStack, contextmanager
111+from contextlib import ExitStack, contextmanager, suppress
112 from datetime import datetime, timedelta
113 from hashlib import sha256
114 from importlib import import_module
115@@ -252,13 +252,19 @@
116 try:
117 timestamp = datetime.fromtimestamp(os.stat(LAST_UPDATE_FILE).st_mtime)
118 except (FileNotFoundError, PermissionError):
119- # We fall back to the latest mtime of the config.d/*.ini files.
120- timestamps = sorted(
121- datetime.fromtimestamp(path.stat().st_mtime)
122- for path in config.ini_files)
123+ # We fall back to the latest mtime of the config.d/*.ini files. For
124+ # robustness, watch out for two possibilities: the config file could
125+ # have been deleted after the system started up (thus making
126+ # config.ini_files include nonexistent files), and the ini file could
127+ # be a dangling symlink. For the latter, use lstat().
128+ timestamps = []
129+ for path in config.ini_files:
130+ with suppress(FileNotFoundError):
131+ timestamps.append(
132+ datetime.fromtimestamp(path.lstat().st_mtime))
133 if len(timestamps) == 0:
134 return 'Unknown'
135- timestamp = timestamps[-1]
136+ timestamp = sorted(timestamps)[-1]
137 return str(timestamp.replace(microsecond=0))
138
139
140
141=== modified file 'systemimage/logging.py'
142--- systemimage/logging.py 2015-05-08 21:41:15 +0000
143+++ systemimage/logging.py 2015-09-28 21:36:51 +0000
144@@ -18,6 +18,7 @@
145 __all__ = [
146 'debug_logging',
147 'initialize',
148+ 'make_handler',
149 ]
150
151
152@@ -68,13 +69,16 @@
153 return super().getMessage()
154
155
156-def _make_handler(path):
157+def make_handler(path):
158 # issue21539 - mkdir(..., exist_ok=True)
159 with suppress(FileExistsError):
160 path.parent.mkdir(DEFAULT_DIRMODE, parents=True)
161 path.touch(LOGFILE_PERMISSIONS)
162 # Our handler will output in UTF-8 using {} style logging.
163- return logging.FileHandler(bytes(path), encoding='utf-8')
164+ formatter = logging.Formatter(style='{', fmt=MSG_FMT, datefmt=DATE_FMT)
165+ handler = logging.FileHandler(bytes(path), encoding='utf-8')
166+ handler.setFormatter(formatter)
167+ return handler
168
169
170 def initialize(*, verbosity=0):
171@@ -95,13 +99,11 @@
172 # Now configure the application level logger based on the ini file.
173 log = logging.getLogger(name)
174 try:
175- handler = _make_handler(Path(config.system.logfile))
176+ handler = make_handler(Path(config.system.logfile))
177 except PermissionError:
178- handler = _make_handler(
179+ handler = make_handler(
180 Path(xdg_cache_home) / 'system-image' / 'client.log')
181 handler.setLevel(level)
182- formatter = logging.Formatter(style='{', fmt=MSG_FMT, datefmt=DATE_FMT)
183- handler.setFormatter(formatter)
184 log.addHandler(handler)
185 log.propagate = False
186 # If we want more verbosity, add a stream handler.
187@@ -111,6 +113,8 @@
188 else: # pragma: no cover
189 handler = logging.StreamHandler(stream=sys.stderr)
190 handler.setLevel(level)
191+ formatter = logging.Formatter(
192+ style='{', fmt=MSG_FMT, datefmt=DATE_FMT)
193 handler.setFormatter(formatter)
194 log.addHandler(handler)
195 # Set the overall level on the log object to the minimum level.
196
197=== modified file 'systemimage/testing/dbus.py'
198--- systemimage/testing/dbus.py 2015-05-08 21:41:15 +0000
199+++ systemimage/testing/dbus.py 2015-09-28 21:36:51 +0000
200@@ -22,6 +22,7 @@
201
202
203 import os
204+import logging
205
206 try:
207 import pycurl
208@@ -30,10 +31,12 @@
209
210 from dbus.service import method, signal
211 from gi.repository import GLib
212+from pathlib import Path
213 from systemimage.api import Mediator
214 from systemimage.config import config
215 from systemimage.dbus import Service, log_and_exit
216 from systemimage.helpers import MiB, makedirs, safe_remove, version_detail
217+from systemimage.logging import make_handler
218 from unittest.mock import patch
219
220
221@@ -74,6 +77,10 @@
222 class _LiveTestableService(Service):
223 """For testing purposes only."""
224
225+ def __init__(self, bus, object_path, loop):
226+ super().__init__(bus, object_path, loop)
227+ self._debug_handler = None
228+
229 @log_and_exit
230 @method('com.canonical.SystemImage')
231 def Reset(self):
232@@ -103,6 +110,27 @@
233 def TornDown(self):
234 pass
235
236+ @log_and_exit
237+ @method('com.canonical.SystemImage',
238+ in_signature='ss',
239+ out_signature='ss')
240+ def DebugDBusTo(self, filename, level_name):
241+ # Get the existing logging level and logging file name.
242+ dbus_log = logging.getLogger('systemimage.dbus')
243+ old_level = logging.getLevelName(dbus_log.getEffectiveLevel())
244+ old_filename = config.system.logfile
245+ # Remove any previous D-Bus debugging handler.
246+ if self._debug_handler is not None:
247+ dbus_log.removeHandler(self._debug_handler)
248+ self._debug_handler = None
249+ new_level = getattr(logging, level_name.upper())
250+ dbus_log.setLevel(new_level)
251+ if filename != '':
252+ self._debug_handler = make_handler(Path(filename))
253+ self._debug_handler.setLevel(new_level)
254+ dbus_log.addHandler(self._debug_handler)
255+ return old_filename, old_level
256+
257
258 class _UpdateAutoSuccess(Service):
259 """Normal update in auto-download mode."""
260
261=== modified file 'systemimage/tests/test_dbus.py'
262--- systemimage/tests/test_dbus.py 2015-05-08 21:41:15 +0000
263+++ systemimage/tests/test_dbus.py 2015-09-28 21:36:51 +0000
264@@ -44,10 +44,12 @@
265
266
267 import os
268+import sys
269 import dbus
270 import json
271 import time
272 import shutil
273+import tempfile
274 import unittest
275
276 from contextlib import ExitStack, suppress
277@@ -61,6 +63,7 @@
278 from systemimage.helpers import MiB, safe_remove
279 from systemimage.reactor import Reactor
280 from systemimage.settings import Settings
281+from systemimage.testing.controller import USING_PYCURL
282 from systemimage.testing.helpers import (
283 copy, data_path, find_dbus_process, make_http_server, setup_index,
284 setup_keyring_txz, setup_keyrings, sign, terminate_service, touch_build,
285@@ -78,6 +81,27 @@
286 'last_update_date error_reason')
287
288
289+def capture_dbus_calls(function):
290+ def inner(self, *args, **kws):
291+ with ExitStack() as resources:
292+ fd, filename = tempfile.mkstemp('.log')
293+ os.close(fd)
294+ resources.callback(os.remove, filename)
295+ old_filename, old_level = self.iface.DebugDBusTo(filename, 'debug')
296+ try:
297+ result = function(self, *args, **kws)
298+ finally:
299+ self.iface.DebugDBusTo('', old_level)
300+ with open(filename, 'r', encoding='utf-8') as fp:
301+ print('\nvvvvv', function.__name__,
302+ 'dbus calls vvvvv', file=sys.stderr)
303+ sys.stderr.write(fp.read())
304+ print('^^^^^', function.__name__,
305+ 'dbus calls ^^^^^', file=sys.stderr)
306+ return result
307+ return inner
308+
309+
310 def tweak_checksums(checksum):
311 index_path = os.path.join(
312 SystemImagePlugin.controller.serverdir,
313@@ -1717,7 +1741,9 @@
314 write_bytes(full_path, 750)
315 tweak_checksums('')
316
317+ @capture_dbus_calls
318 def test_pause(self):
319+ # Set up some extra D-Bus debugging.
320 self.download_manually()
321 touch_build(0, use_config=self.config)
322 reactor = SignalCapturingReactor('UpdateAvailableStatus')
323@@ -1966,8 +1992,6 @@
324 self.assertEqual(reactor.uas_signals[0], reactor.uas_signals[1])
325
326
327-from systemimage.testing.controller import USING_PYCURL
328-
329 @unittest.skipIf(os.getuid() == 0, 'Test cannot succeed when run as root')
330 @unittest.skipUnless(USING_PYCURL, 'LP: #1411866')
331 class TestDBusCheckForUpdateToUnwritablePartition(_LiveTesting):
332
333=== modified file 'systemimage/tests/test_helpers.py'
334--- systemimage/tests/test_helpers.py 2015-05-08 21:41:15 +0000
335+++ systemimage/tests/test_helpers.py 2015-09-28 21:36:51 +0000
336@@ -255,6 +255,52 @@
337 # The last update date will be the date of the 99_build.ini file.
338 self.assertEqual(last_update_date(), '2022-01-02 03:04:05')
339
340+ @configuration
341+ def test_dangling_symlink(self, config):
342+ # LP: #1495688 reports a problem where /userdata/.last_update doesn't
343+ # exist, and the files in the config.d directory are dangling
344+ # symlinks. In this case, there's really little that can be done to
345+ # find a reliable last update date, but at least we don't crash.
346+ #
347+ # Start by deleting any existing .ini files in config.d.
348+ for path in Path(config.config_d).iterdir():
349+ if path.suffix == '.ini':
350+ path.unlink()
351+ with ExitStack() as stack:
352+ tmpdir = stack.enter_context(temporary_directory())
353+ userdata_path = Path(tmpdir) / '.last_update'
354+ stack.enter_context(patch('systemimage.helpers.LAST_UPDATE_FILE',
355+ str(userdata_path)))
356+ # Do not create the .last_update file.
357+ missing_ini = Path(tmpdir) / 'missing.ini'
358+ config.ini_files = [missing_ini]
359+ # Do not create the missing.ini file, but do create a symlink from
360+ # a config.d file to this missing file.
361+ default_ini = Path(config.config_d) / '00_default.ini'
362+ default_ini.symlink_to(missing_ini)
363+ last_update_date()
364+ self.assertEqual(last_update_date(), 'Unknown')
365+
366+ @configuration
367+ def test_post_startup_delete(self, config):
368+ # Like test_dangling_symlink() except that an existing ini file gets
369+ # deleted after system startup, so some of the files that
370+ # last_update_date() looks at will throw an exception.
371+ #
372+ # Start by deleting any existing .ini files in config.d. This time
373+ # however we don't update config.ini_files.
374+ for path in Path(config.config_d).iterdir():
375+ if path.suffix == '.ini':
376+ path.unlink()
377+ with ExitStack() as stack:
378+ tmpdir = stack.enter_context(temporary_directory())
379+ userdata_path = Path(tmpdir) / '.last_update'
380+ stack.enter_context(patch('systemimage.helpers.LAST_UPDATE_FILE',
381+ str(userdata_path)))
382+ # Do not create the .last_update file.
383+ last_update_date()
384+ self.assertEqual(last_update_date(), 'Unknown')
385+
386
387 class TestPhasedPercentage(unittest.TestCase):
388 def setUp(self):
389
390=== modified file 'systemimage/version.txt'
391--- systemimage/version.txt 2015-06-17 15:18:22 +0000
392+++ systemimage/version.txt 2015-09-28 21:36:51 +0000
393@@ -1,1 +1,1 @@
394-3.0.1
395+3.0.2

Subscribers

People subscribed via source and target branches