Merge lp:~mvo/unattended-upgrades/minimal-step-paranonia-lp1020680 into lp:~mvo/unattended-upgrades/debian-sid-mirror
- minimal-step-paranonia-lp1020680
- Merge into debian-sid-mirror
Proposed by
Michael Vogt
Status: | Needs review |
---|---|
Proposed branch: | lp:~mvo/unattended-upgrades/minimal-step-paranonia-lp1020680 |
Merge into: | lp:~mvo/unattended-upgrades/debian-sid-mirror |
Diff against target: |
1929 lines (+450/-271) 23 files modified
.bzrignore (+2/-0) debian/changelog (+23/-2) debian/control (+5/-4) debian/rules (+13/-11) debian/tests/control (+2/-0) debian/tests/run-tests (+4/-0) debian/unattended-upgrades.init (+1/-1) pm/sleep.d/10_unattended-upgrades-hibernate (+1/-1) setup.py (+4/-3) test/Makefile (+9/-2) test/create_debug_lock.py (+2/-1) test/test_against_real_archive.py (+14/-4) test/test_conffile.py (+4/-4) test/test_in_chroot.py (+41/-34) test/test_logdir.py (+3/-2) test/test_mail.py (+33/-16) test/test_minimal_partitions.py (+10/-10) test/test_origin_pattern.py (+31/-15) test/test_pep8.py (+17/-0) test/test_pyflakes.py (+4/-2) test/test_substitute.py (+2/-7) unattended-upgrade (+222/-149) unattended-upgrade-shutdown (+3/-3) |
To merge this branch: | bzr merge lp:~mvo/unattended-upgrades/minimal-step-paranonia-lp1020680 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Michael Vogt | Pending | ||
Review via email: mp+128536@code.launchpad.net |
Commit message
Description of the change
Add extra paranoia in the minimal upgade mode to ensure we are not hit by #1020680
To post a comment you must log in.
Unmerged revisions
- 304. By Michael Vogt
-
pass the blacklist to upgrade_
in_minimal_ steps() as the conffile checking code may expand it - 303. By Michael Vogt
-
add a additional check_changes_
for_sanity( ) into upgrade_ in_minimal_ steps() code - 302. By Michael Vogt
-
test/Makefile: fix cleanup
- 301. By Michael Vogt
-
debian/
tests/run- tests: add run-test script - 300. By Michael Vogt
-
add dep8 tests
- 299. By Michael Vogt
-
Fixed debug output when a package has no candidates (LP: #1046438)
- 298. By Michael Vogt
-
releasing version 0.79.3ubuntu1
- 297. By Michael Vogt
-
fully pep8 clean (yeah!)
- 296. By Michael Vogt
-
add pyflakes/pep8 tests
- 295. By Michael Vogt
-
add py2 compatiblity back for easier backporting and run the tests in both py2/py3 and also run python-coverage
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === added file '.bzrignore' |
2 | --- .bzrignore 1970-01-01 00:00:00 +0000 |
3 | +++ .bzrignore 2012-10-08 16:08:26 +0000 |
4 | @@ -0,0 +1,2 @@ |
5 | +__pycache__ |
6 | +build |
7 | |
8 | === modified file 'debian/changelog' |
9 | --- debian/changelog 2012-08-14 13:04:45 +0000 |
10 | +++ debian/changelog 2012-10-08 16:08:26 +0000 |
11 | @@ -1,3 +1,24 @@ |
12 | +unattended-upgrades (0.79.3ubuntu2) quantal; urgency=low |
13 | + |
14 | + [ Marc Tardif ] |
15 | + * Fixed debug output when a package has no candidates (LP: #1046438) |
16 | + |
17 | + [ Michael Vogt ] |
18 | + * debian/test/: |
19 | + - add dep8 tests |
20 | + |
21 | + -- Marc Tardif <marc@ubuntu.com> Thu, 13 Sep 2012 16:16:04 -0400 |
22 | + |
23 | +unattended-upgrades (0.79.3ubuntu1) quantal; urgency=low |
24 | + |
25 | + [ Thomas Kluyver ] |
26 | + * Port to Python 3 |
27 | + |
28 | + [ Brian Murray ] |
29 | + * unattended-upgrade: fix typo in debugging output |
30 | + |
31 | + -- Michael Vogt <michael.vogt@ubuntu.com> Mon, 10 Sep 2012 09:55:19 +0200 |
32 | + |
33 | unattended-upgrades (0.79.4) UNRELEASED; urgency=low |
34 | |
35 | * data/50unattended-upgrades.{Debian,Ubuntu}: |
36 | @@ -63,7 +84,7 @@ |
37 | - added, thanks to Pedro Ribeiro, closes: #678738 |
38 | * po/sk.po: |
39 | - added, thanks to helix84, closes: #677471 |
40 | - |
41 | + |
42 | [ Teodor MICU ] |
43 | * debian/unattended-upgrades.init: |
44 | - fixes new style lsb-init output, closes: #678030 |
45 | @@ -93,7 +114,7 @@ |
46 | - updated, thanks to Joe Dalton, closes: #677804 |
47 | * po/de.po: |
48 | - updated, thanks to Chris Leick (closes: #678204) |
49 | - |
50 | + |
51 | [ Teodor MICU ] |
52 | * debian/unattended-upgrades.init: |
53 | - use new style lsb-init output, closes: #678030 |
54 | |
55 | === modified file 'debian/control' |
56 | --- debian/control 2012-07-13 18:51:00 +0000 |
57 | +++ debian/control 2012-10-08 16:08:26 +0000 |
58 | @@ -3,14 +3,15 @@ |
59 | Priority: optional |
60 | Maintainer: Michael Vogt <michael.vogt@ubuntu.com> |
61 | Build-Depends: debhelper (>= 7.0.50~), po-debconf, lsb-release |
62 | -Build-Depends-Indep: python (>= 2.6.6-3~), python-distutils-extra |
63 | -Standards-Version: 3.8.3 |
64 | +Build-Depends-Indep: python3, python3-distutils-extra |
65 | +Standards-Version: 3.9.3 |
66 | Vcs-Bzr: http://code.launchpad.net/~ubuntu-core-dev/unattended-upgrades/ubuntu/ |
67 | +XS-Testsuite: autopkgtest |
68 | |
69 | Package: unattended-upgrades |
70 | Architecture: all |
71 | -Depends: ${shlibs:Depends}, ${misc:Depends}, debconf, python, |
72 | - python-apt (>= 0.7.90), apt-utils, apt, ucf, lsb-release, |
73 | +Depends: ${shlibs:Depends}, ${misc:Depends}, debconf, python3, |
74 | + python3-apt (>= 0.7.90), apt-utils, apt, ucf, lsb-release, |
75 | lsb-base (>= 3.2-14) |
76 | Suggests: bsd-mailx, mail-transport-agent |
77 | Description: automatic installation of security upgrades |
78 | |
79 | === modified file 'debian/rules' |
80 | --- debian/rules 2011-11-09 19:20:50 +0000 |
81 | +++ debian/rules 2012-10-08 16:08:26 +0000 |
82 | @@ -3,24 +3,26 @@ |
83 | DIST=$(shell /usr/bin/lsb_release -i -s) |
84 | |
85 | %: |
86 | - dh $@ --with python2 |
87 | + dh $@ --with python3 |
88 | |
89 | override_dh_auto_build: |
90 | # copy the right template into place |
91 | cp data/50unattended-upgrades.$(DIST) data/50unattended-upgrades |
92 | - dh_auto_build |
93 | + python3 setup.py build |
94 | + |
95 | +override_dh_auto_install: |
96 | + python3 setup.py install \ |
97 | + --root=$(CURDIR)/debian/unattended-upgrades \ |
98 | + --install-layout=deb |
99 | |
100 | override_dh_auto_clean: |
101 | # Sanity-check before upload. |
102 | - set -e; if [ -e /usr/lib/python$(PYVER)/py_compile.py ]; then \ |
103 | - for f in unattended-upgrade unattended-upgrade-shutdown; do \ |
104 | - ln -nsf $$f $$f.py; \ |
105 | - python$(PYVER) /usr/lib/python$(PYVER)/py_compile.py \ |
106 | - $$f.py; \ |
107 | - rm -f $$f.py; \ |
108 | - done; \ |
109 | - fi |
110 | - dh_auto_clean |
111 | + set -e; for f in unattended-upgrade unattended-upgrade-shutdown; do \ |
112 | + ln -nsf $$f $$f.py; \ |
113 | + py3compile $$f.py; \ |
114 | + rm -f $$f.py; \ |
115 | + done |
116 | + python3 setup.py clean -a |
117 | |
118 | override_dh_installinit: |
119 | # we do not want to run the init script in the postinst/prerm, its |
120 | |
121 | === added directory 'debian/tests' |
122 | === added file 'debian/tests/control' |
123 | --- debian/tests/control 1970-01-01 00:00:00 +0000 |
124 | +++ debian/tests/control 2012-10-08 16:08:26 +0000 |
125 | @@ -0,0 +1,2 @@ |
126 | +Tests: run-tests |
127 | +Depends: @, make, python-dev, python3-dev, python-coverage |
128 | |
129 | === added file 'debian/tests/run-tests' |
130 | --- debian/tests/run-tests 1970-01-01 00:00:00 +0000 |
131 | +++ debian/tests/run-tests 2012-10-08 16:08:26 +0000 |
132 | @@ -0,0 +1,4 @@ |
133 | +#!/bin/sh |
134 | + |
135 | +cd test |
136 | +make |
137 | |
138 | === modified file 'debian/unattended-upgrades.init' |
139 | --- debian/unattended-upgrades.init 2012-06-29 08:09:25 +0000 |
140 | +++ debian/unattended-upgrades.init 2012-10-08 16:08:26 +0000 |
141 | @@ -33,7 +33,7 @@ |
142 | stop) |
143 | if [ -e $SHUTDOWN_HELPER ]; then |
144 | [ "$VERBOSE" != "no" ] && log_action_begin_msg "Checking for running $DESC" |
145 | - python $SHUTDOWN_HELPER |
146 | + python3 $SHUTDOWN_HELPER |
147 | [ "$VERBOSE" != "no" ] && log_action_end_msg $? "$NAME" |
148 | fi |
149 | ;; |
150 | |
151 | === modified file 'pm/sleep.d/10_unattended-upgrades-hibernate' |
152 | --- pm/sleep.d/10_unattended-upgrades-hibernate 2011-11-08 16:56:58 +0000 |
153 | +++ pm/sleep.d/10_unattended-upgrades-hibernate 2012-10-08 16:08:26 +0000 |
154 | @@ -17,7 +17,7 @@ |
155 | case "${1}" in |
156 | hibernate) |
157 | if [ -e $SHUTDOWN_HELPER ]; then |
158 | - python $SHUTDOWN_HELPER |
159 | + python3 $SHUTDOWN_HELPER |
160 | fi |
161 | ;; |
162 | resume|thaw) |
163 | |
164 | === modified file 'setup.py' |
165 | --- setup.py 2011-10-24 10:20:18 +0000 |
166 | +++ setup.py 2012-10-08 16:08:26 +0000 |
167 | @@ -1,9 +1,10 @@ |
168 | #!/usr/bin/env python |
169 | |
170 | from distutils.core import setup |
171 | -from DistUtilsExtra.command import * |
172 | -import glob |
173 | -import os |
174 | +from DistUtilsExtra.command import ( |
175 | + build_extra, |
176 | + build_i18n, |
177 | + ) |
178 | |
179 | |
180 | setup(name='unattended-upgrades', version='0.1', |
181 | |
182 | === modified file 'test/Makefile' |
183 | --- test/Makefile 2011-10-07 09:22:19 +0000 |
184 | +++ test/Makefile 2012-10-08 16:08:26 +0000 |
185 | @@ -3,12 +3,15 @@ |
186 | all: check |
187 | |
188 | check: |
189 | + # test with both py2 and py3 for now and also use coverage for the py2 |
190 | set -e; \ |
191 | find . -name 'test_*.py' | \ |
192 | while read file; do \ |
193 | - echo "Running $$file"; \ |
194 | if [ -x $$file ]; then \ |
195 | - python $$file ; \ |
196 | + echo "Running $$file with python3"; \ |
197 | + python3 $$file ; \ |
198 | + echo "Running $$file with python"; \ |
199 | + python-coverage run -a $$file; \ |
200 | fi \ |
201 | done |
202 | |
203 | @@ -16,4 +19,8 @@ |
204 | rm -rf ./aptroot/var/cache/ |
205 | rm -rf ./aptroot/var/lib/apt |
206 | rm -rf ./aptroot/var/run |
207 | + find .. -type d -name __pycache__ | xargs rm -rf |
208 | |
209 | +coverage-html: |
210 | + echo "output in htmlcov/ |
211 | + python-coverage html |
212 | \ No newline at end of file |
213 | |
214 | === modified file 'test/create_debug_lock.py' |
215 | --- test/create_debug_lock.py 2011-06-16 07:10:07 +0000 |
216 | +++ test/create_debug_lock.py 2012-10-08 16:08:26 +0000 |
217 | @@ -1,4 +1,4 @@ |
218 | -#!/usr/bin/python |
219 | +#!/usr/bin/python3 |
220 | # |
221 | # create a lock file so that unattended-upgrades-shutdown pauses |
222 | # on shutdown -- useful for testing |
223 | @@ -7,6 +7,7 @@ |
224 | import os |
225 | import time |
226 | |
227 | + |
228 | pid = os.fork() |
229 | if pid == 0: |
230 | os.setsid() |
231 | |
232 | === modified file 'test/test_against_real_archive.py' (properties changed: +x to -x) |
233 | --- test/test_against_real_archive.py 2012-02-28 10:48:18 +0000 |
234 | +++ test/test_against_real_archive.py 2012-10-08 16:08:26 +0000 |
235 | @@ -1,22 +1,31 @@ |
236 | -#!/usr/bin/python |
237 | +#!/usr/bin/python3 |
238 | +"""Test unattended_upgrades against the real archive in a chroot. |
239 | + |
240 | +Note that this test is not run by the makefile in this folder, as it requires |
241 | +network access, and it fails in some situations (unclear which). |
242 | +""" |
243 | |
244 | import apt |
245 | import apt_pkg |
246 | import glob |
247 | +import logging |
248 | import os |
249 | import re |
250 | import unittest |
251 | |
252 | import unattended_upgrade |
253 | |
254 | + |
255 | apt_pkg.config.set("APT::Architecture", "amd64") |
256 | |
257 | + |
258 | class MockOptions(): |
259 | def __init__(self, debug=True, dry_run=True): |
260 | self.debug = debug |
261 | self.dry_run = dry_run |
262 | self.minimal_upgrade_steps = False |
263 | |
264 | + |
265 | class TestAgainstRealArchive(unittest.TestCase): |
266 | |
267 | def setUp(self): |
268 | @@ -40,11 +49,13 @@ |
269 | apt_pkg.config.set("APT::UnattendedUpgrades::LogDir", logdir) |
270 | unattended_upgrade.DISTRO_CODENAME = "lucid" |
271 | res = unattended_upgrade.main(options, os.path.abspath("./aptroot")) |
272 | + logging.debug(res) |
273 | # check if the log file exists |
274 | self.assertTrue(os.path.exists(logfile)) |
275 | - log = open(logfile).read() |
276 | + with open(logfile) as fp: |
277 | + log = fp.read() |
278 | # check that stuff worked |
279 | - self.assertFalse(" ERROR " in log) |
280 | + self.assertFalse(" ERROR " in log, log) |
281 | # check if we actually have the expected ugprade in it |
282 | self.assertTrue( |
283 | re.search("INFO Packages that are upgraded:.*awstats", log)) |
284 | @@ -59,4 +70,3 @@ |
285 | |
286 | if __name__ == "__main__": |
287 | unittest.main() |
288 | - |
289 | |
290 | === modified file 'test/test_conffile.py' |
291 | --- test/test_conffile.py 2012-06-28 20:06:37 +0000 |
292 | +++ test/test_conffile.py 2012-10-08 16:08:26 +0000 |
293 | @@ -1,12 +1,13 @@ |
294 | -#!/usr/bin/python |
295 | +#!/usr/bin/python3 |
296 | |
297 | import apt_pkg |
298 | import logging |
299 | import unittest |
300 | -import sys |
301 | + |
302 | |
303 | from unattended_upgrade import conffile_prompt |
304 | |
305 | + |
306 | class ConffilePromptTestCase(unittest.TestCase): |
307 | |
308 | def setUp(self): |
309 | @@ -19,7 +20,7 @@ |
310 | test_pkg = "./packages/conf-test-package_1.1.deb" |
311 | self.assertTrue(conffile_prompt(test_pkg, prefix="./root.conffile"), |
312 | "conffile prompt detection incorrect") |
313 | - |
314 | + |
315 | def test_will_not_prompt(self): |
316 | # conf-test 0.9 is installed, 1.0 gets installed |
317 | # they both have the same config files |
318 | @@ -57,4 +58,3 @@ |
319 | if __name__ == "__main__": |
320 | logging.basicConfig(level=logging.DEBUG) |
321 | unittest.main() |
322 | - |
323 | |
324 | === modified file 'test/test_in_chroot.py' |
325 | --- test/test_in_chroot.py 2012-06-28 20:06:37 +0000 |
326 | +++ test/test_in_chroot.py 2012-10-08 16:08:26 +0000 |
327 | @@ -1,4 +1,4 @@ |
328 | -#!/usr/bin/python |
329 | +#!/usr/bin/python3 |
330 | |
331 | import apt |
332 | import logging |
333 | @@ -15,7 +15,8 @@ |
334 | #SOURCES_LIST=""" |
335 | #deb http://ftp.de.debian.org/debian squeeze main contrib non-free |
336 | #deb http://ftp.de.debian.org/debian squeeze-updates main contrib non-free |
337 | -#deb http://ftp.de.debian.org/debian squeeze-proposed-updates main contrib non-f#ree |
338 | +#deb http://ftp.de.debian.org/debian squeeze-proposed-updates main contrib \ |
339 | +# non-free |
340 | #deb http://security.debian.org squeeze/updates main contrib non-free |
341 | #""" |
342 | #DISTRO="squeeze" |
343 | @@ -27,7 +28,7 @@ |
344 | |
345 | |
346 | # ubuntu |
347 | -SOURCES_LIST=""" |
348 | +SOURCES_LIST = """ |
349 | deb http://archive.ubuntu.com/ubuntu/ lucid main restricted |
350 | deb-src http://archive.ubuntu.com/ubuntu/ lucid main restricted |
351 | |
352 | @@ -37,35 +38,39 @@ |
353 | deb http://security.ubuntu.com/ubuntu/ lucid-security main restricted |
354 | deb-src http://security.ubuntu.com/ubuntu/ lucid-security main restricted |
355 | """ |
356 | -DISTRO="lucid" |
357 | -ARCH="i386" |
358 | -TARBALL="%s-%s.tgz" % (DISTRO, ARCH) |
359 | -MIRROR="http://archive.ubuntu.com/ubuntu" |
360 | -APT_CONF="""APT::Architecture "%s";""" % ARCH |
361 | -ORIGINS_PATTERN="origin=Ubuntu,archive=lucid-security" |
362 | +DISTRO = "lucid" |
363 | +ARCH = "i386" |
364 | +TARBALL = "%s-%s.tgz" % (DISTRO, ARCH) |
365 | +MIRROR = "http://archive.ubuntu.com/ubuntu" |
366 | +APT_CONF = """APT::Architecture "%s";""" % ARCH |
367 | +ORIGINS_PATTERN = "origin=Ubuntu,archive=lucid-security" |
368 | |
369 | apt.apt_pkg.config.set("APT::Architecture", ARCH) |
370 | sys.path.insert(0, "..") |
371 | import unattended_upgrade |
372 | |
373 | + |
374 | class MockOptions(object): |
375 | debug = True |
376 | dry_run = False |
377 | minimal_upgrade_steps = False |
378 | |
379 | + |
380 | class TestUnattendedUpgrade(unittest.TestCase): |
381 | |
382 | def _create_new_debootstrap_tarball(self, tarball, target): |
383 | - print "creating initial test tarball, this is needed only once" |
384 | + print("creating initial test tarball, this is needed only once") |
385 | # force i386 |
386 | - subprocess.call(["debootstrap", |
387 | - "--arch=%s" % ARCH, |
388 | - # smaller version of the minimal system |
389 | - "--variant=minbase", |
390 | - "--include=python-apt,apt-utils,gpgv,ubuntu-keyring,ca-certificates", |
391 | - DISTRO, |
392 | - target, |
393 | - MIRROR]) |
394 | + subprocess.call( |
395 | + ["debootstrap", |
396 | + "--arch=%s" % ARCH, |
397 | + # smaller version of the minimal system |
398 | + "--variant=minbase", |
399 | + "--include=python-apt,apt-utils,gpgv,ubuntu-keyring," |
400 | + "ca-certificates", |
401 | + DISTRO, |
402 | + target, |
403 | + MIRROR]) |
404 | subprocess.call(["chroot", target, "apt-get", "clean"]) |
405 | subprocess.call(["tar", "czf", tarball, target]) |
406 | |
407 | @@ -73,7 +78,7 @@ |
408 | subprocess.call(["tar", "xzf", tarball]) |
409 | |
410 | def test_normal_upgrade(self): |
411 | - print "Running normal unattended upgrade in chroot" |
412 | + print("Running normal unattended upgrade in chroot") |
413 | options = MockOptions() |
414 | options.minimal_upgrade_steps = False |
415 | # run it |
416 | @@ -82,7 +87,7 @@ |
417 | self.assertTrue(self._verify_install_log_in_real_chroot(target)) |
418 | |
419 | def test_minimal_steps_upgrade(self): |
420 | - print "Running minimal steps unattended upgrade in chroot" |
421 | + print("Running minimal steps unattended upgrade in chroot") |
422 | options = MockOptions() |
423 | options.minimal_upgrade_steps = True |
424 | # run it |
425 | @@ -91,14 +96,15 @@ |
426 | self.assertTrue(self._verify_install_log_in_real_chroot(target)) |
427 | |
428 | def test_upgrade_on_shutdown_upgrade(self): |
429 | - print "Running unattended upgrade on shutdown (download and install) in chroot" |
430 | + print("Running unattended upgrade on shutdown (download and install) " |
431 | + "in chroot") |
432 | # ensure that it actually installs in shutdown env mode |
433 | options = MockOptions() |
434 | os.environ["UNATTENDED_UPGRADES_FORCE_INSTALL_ON_SHUTDOWN"] = "1" |
435 | apt.apt_pkg.config.set("Unattended-Upgrade::InstallOnShutdown", "1") |
436 | target = self._run_upgrade_test_in_real_chroot(options) |
437 | self.assertTrue(self._verify_install_log_in_real_chroot(target)) |
438 | - |
439 | + |
440 | def _get_lockfile_location(self, target): |
441 | return os.path.join( |
442 | target, "var/log/unattended-upgrades/unattended-upgrades.log") |
443 | @@ -114,14 +120,13 @@ |
444 | open(os.path.join(target, "etc/apt/apt.conf"), "w").write(APT_CONF) |
445 | open(os.path.join(target, "etc/apt/sources.list"), "w").write( |
446 | SOURCES_LIST) |
447 | - |
448 | |
449 | def _run_upgrade_test_in_real_chroot(self, options, clean_chroot=True): |
450 | """ helper that runs the unattended-upgrade in a chroot |
451 | and does some basic verifications |
452 | """ |
453 | if os.getuid() != 0: |
454 | - print "Skipping because uid != 0" |
455 | + print("Skipping because uid != 0") |
456 | return |
457 | |
458 | # clear to avoid pollution in the chroot |
459 | @@ -130,7 +135,7 @@ |
460 | # create chroot |
461 | target = "./test-chroot.%s" % DISTRO |
462 | |
463 | - # setup chroot if needed |
464 | + # setup chroot if needed |
465 | if clean_chroot: |
466 | self._setup_chroot(target) |
467 | |
468 | @@ -143,7 +148,7 @@ |
469 | if not os.path.exists("/var/log/unattended-upgrades/"): |
470 | os.makedirs("/var/log/unattended-upgrades/") |
471 | # make sure we are up-to-date |
472 | - subprocess.call(["apt-get","update", "-q", "-q"]) |
473 | + subprocess.call(["apt-get", "update", "-q", "-q"]) |
474 | # run it |
475 | apt.apt_pkg.config.clear("Unattended-Upgrade::Allowed-Origins") |
476 | apt.apt_pkg.config.clear("Unattended-Upgrade::Origins-Pattern") |
477 | @@ -153,7 +158,7 @@ |
478 | unattended_upgrade.main(options) |
479 | os._exit(0) |
480 | else: |
481 | - has_progress=False |
482 | + has_progress = False |
483 | all_progress = "" |
484 | last_progress = "" |
485 | progress_log = os.path.join( |
486 | @@ -171,9 +176,9 @@ |
487 | if pid == apid: |
488 | ret = os.WEXITSTATUS(status) |
489 | break |
490 | - #print "*******************", all_progress |
491 | + #print("*******************", all_progress) |
492 | self.assertEqual(ret, 0) |
493 | - # this number is a bit random, we just want to be sure we have |
494 | + # this number is a bit random, we just want to be sure we have |
495 | # progress data |
496 | self.assertTrue(has_progress, True) |
497 | self.assertTrue(len(all_progress) > 5) |
498 | @@ -183,11 +188,12 @@ |
499 | # examine log |
500 | log = self._get_lockfile_location(target) |
501 | logfile = open(log).read() |
502 | - #print logfile |
503 | - NEEDLE_PKG="ca-certificates" |
504 | + #print(logfile) |
505 | + NEEDLE_PKG = "ca-certificates" |
506 | if not re.search( |
507 | "Packages that are upgraded:.*%s" % NEEDLE_PKG, logfile): |
508 | - logging.warn("Can not find expected %s upgrade in log" % NEEDLE_PKG) |
509 | + logging.warn("Can not find expected %s upgrade in log" % |
510 | + NEEDLE_PKG) |
511 | return False |
512 | if "ERROR Installing the upgrades failed" in logfile: |
513 | logging.warn("Got a ERROR in the logfile") |
514 | @@ -196,9 +202,10 @@ |
515 | target, "var/log/unattended-upgrades/*-dpkg*.log") |
516 | dpkg_logfile = open(glob.glob(dpkg_log)[0]).read() |
517 | if not "Preparing to replace %s" % NEEDLE_PKG in dpkg_logfile: |
518 | - logging.warn("Did not find %s upgrade in the dpkg.log" % NEEDLE_PKG) |
519 | + logging.warn("Did not find %s upgrade in the dpkg.log" % |
520 | + NEEDLE_PKG) |
521 | return False |
522 | - #print dpkg_logfile |
523 | + #print(dpkg_logfile) |
524 | return True |
525 | |
526 | |
527 | |
528 | === modified file 'test/test_logdir.py' |
529 | --- test/test_logdir.py 2012-06-29 07:08:35 +0000 |
530 | +++ test/test_logdir.py 2012-10-08 16:08:26 +0000 |
531 | @@ -1,9 +1,8 @@ |
532 | -#!/usr/bin/python |
533 | +#!/usr/bin/python3 |
534 | |
535 | import apt_pkg |
536 | import logging |
537 | import os |
538 | -import mock |
539 | import sys |
540 | import tempfile |
541 | import unittest |
542 | @@ -11,10 +10,12 @@ |
543 | sys.path.insert(0, "..") |
544 | from unattended_upgrade import _setup_logging |
545 | |
546 | + |
547 | class MockOptions: |
548 | dry_run = False |
549 | debug = False |
550 | |
551 | + |
552 | class TestLogdir(unittest.TestCase): |
553 | |
554 | def setUp(self): |
555 | |
556 | === modified file 'test/test_mail.py' |
557 | --- test/test_mail.py 2012-07-13 12:20:03 +0000 |
558 | +++ test/test_mail.py 2012-10-08 16:08:26 +0000 |
559 | @@ -1,11 +1,14 @@ |
560 | -#!/usr/bin/python |
561 | +#!/usr/bin/python3 |
562 | # -*- coding: utf-8 -*- |
563 | +from __future__ import unicode_literals |
564 | + |
565 | import apt_pkg |
566 | import os |
567 | +import sys |
568 | import unittest |
569 | |
570 | +from io import StringIO |
571 | from email.parser import Parser |
572 | -from StringIO import StringIO |
573 | |
574 | import unattended_upgrade |
575 | from unattended_upgrade import send_summary_mail, setup_apt_listchanges |
576 | @@ -40,9 +43,10 @@ |
577 | res = successful |
578 | pkgs_kept_back = [] |
579 | # include some unicode chars here for good measure |
580 | - mem_log = StringIO(u"mem_log text üöä") |
581 | + mem_log = StringIO("mem_log text üöä") |
582 | logfile_dpkg = "./apt-term.log" |
583 | - open("./apt-term.log", "w").write("logfile_dpkg text") |
584 | + with open("./apt-term.log", "w") as fp: |
585 | + fp.write("logfile_dpkg text") |
586 | return (pkgs, res, pkgs_kept_back, mem_log, logfile_dpkg) |
587 | |
588 | def _verify_common_mail_content(self, mail_txt): |
589 | @@ -50,47 +54,55 @@ |
590 | self.assertTrue(expected_string in mail_txt) |
591 | |
592 | def test_summary_mail_reboot(self): |
593 | - open("./reboot-required","w").write("") |
594 | + with open("./reboot-required", "w") as fp: |
595 | + fp.write("") |
596 | send_summary_mail(*self._return_mock_data()) |
597 | os.unlink("./reboot-required") |
598 | - mail_txt = open("mail.txt").read() |
599 | + # this is used for py2 compat for py3 only we can do |
600 | + # remove the "rb" and the subsequent '.decode("utf-8")' |
601 | + with open("mail.txt", "rb") as fp: |
602 | + mail_txt = fp.read().decode("utf-8") |
603 | self.assertTrue("[reboot required]" in mail_txt) |
604 | self._verify_common_mail_content(mail_txt) |
605 | - |
606 | + |
607 | def test_summary_mail_no_reboot(self): |
608 | send_summary_mail(*self._return_mock_data()) |
609 | - mail_txt = open("mail.txt").read() |
610 | + with open("mail.txt", "rb") as fp: |
611 | + mail_txt = fp.read().decode("utf-8") |
612 | self.assertFalse("[reboot required]" in mail_txt) |
613 | self._verify_common_mail_content(mail_txt) |
614 | - |
615 | + |
616 | def test_summary_mail_only_on_error(self): |
617 | # default is to always send mail, ensure this is correct |
618 | # for both success and failure |
619 | apt_pkg.config.set("Unattended-Upgrade::MailOnlyOnError", "false") |
620 | send_summary_mail(*self._return_mock_data(successful=True)) |
621 | - self._verify_common_mail_content(open("mail.txt").read()) |
622 | + with open("mail.txt", "rb") as fp: |
623 | + self._verify_common_mail_content(fp.read().decode("utf-8")) |
624 | os.remove("mail.txt") |
625 | # now with a simulated failure |
626 | send_summary_mail(*self._return_mock_data(successful=False)) |
627 | - self._verify_common_mail_content(open("mail.txt").read()) |
628 | + with open("mail.txt", "rb") as fp: |
629 | + self._verify_common_mail_content(fp.read().decode("utf-8")) |
630 | os.remove("mail.txt") |
631 | # now test with "MailOnlyOnError" |
632 | apt_pkg.config.set("Unattended-Upgrade::MailOnlyOnError", "true") |
633 | send_summary_mail(*self._return_mock_data(successful=True)) |
634 | self.assertFalse(os.path.exists("mail.txt")) |
635 | send_summary_mail(*self._return_mock_data(successful=False)) |
636 | - mail_txt = open("mail.txt").read() |
637 | + with open("mail.txt", "rb") as fp: |
638 | + mail_txt = fp.read().decode("utf-8") |
639 | self._verify_common_mail_content(mail_txt) |
640 | self.assertTrue("Unattended upgrade returned: False" in mail_txt) |
641 | self.assertTrue(os.path.exists("mail.txt")) |
642 | |
643 | def test_apt_listchanges(self): |
644 | # test with sendmail available |
645 | - unattended_upgrade.SENDMAIL_BINARY="/bin/true" |
646 | + unattended_upgrade.SENDMAIL_BINARY = "/bin/true" |
647 | setup_apt_listchanges("./data/listchanges.conf.mail") |
648 | self.assertEqual(os.environ["APT_LISTCHANGES_FRONTEND"], "mail") |
649 | # test without sendmail |
650 | - unattended_upgrade.SENDMAIL_BINARY="/bin/not-here-xxxxxxxxx" |
651 | + unattended_upgrade.SENDMAIL_BINARY = "/bin/not-here-xxxxxxxxx" |
652 | setup_apt_listchanges("./data/listchanges.conf.pager") |
653 | self.assertEqual(os.environ["APT_LISTCHANGES_FRONTEND"], "none") |
654 | |
655 | @@ -108,6 +120,7 @@ |
656 | # we don't accidently try |
657 | self.assertFalse('text/plain; charset="utf-8"' in mail_txt) |
658 | |
659 | + |
660 | class SendmailTestCase(CommonTestsForMailxAndSendmail, unittest.TestCase): |
661 | |
662 | def setUp(self): |
663 | @@ -117,10 +130,15 @@ |
664 | def _verify_common_mail_content(self, mail_txt): |
665 | CommonTestsForMailxAndSendmail._verify_common_mail_content( |
666 | self, mail_txt) |
667 | + |
668 | + # python2 needs this as utf8 encoded string (not unicode) |
669 | + if sys.version < '3': |
670 | + mail_txt = mail_txt.encode("utf-8") |
671 | + |
672 | msg = Parser().parsestr(mail_txt) |
673 | content_type = msg["Content-Type"] |
674 | self.assertEqual(content_type, 'text/plain; charset="utf-8"') |
675 | - |
676 | + |
677 | |
678 | class SendmailAndMailxTestCase(SendmailTestCase): |
679 | |
680 | @@ -132,4 +150,3 @@ |
681 | if __name__ == "__main__": |
682 | #logging.basicConfig(level=logging.DEBUG) |
683 | unittest.main() |
684 | - |
685 | |
686 | === modified file 'test/test_minimal_partitions.py' |
687 | --- test/test_minimal_partitions.py 2011-11-18 10:46:15 +0000 |
688 | +++ test/test_minimal_partitions.py 2012-10-08 16:08:26 +0000 |
689 | @@ -1,33 +1,32 @@ |
690 | -#!/usr/bin/python |
691 | +#!/usr/bin/python3 |
692 | |
693 | import apt |
694 | import apt_pkg |
695 | import os |
696 | -import logging |
697 | import unittest |
698 | -import sys |
699 | -import time |
700 | |
701 | import unattended_upgrade |
702 | |
703 | + |
704 | class LogInstallProgressMock(unattended_upgrade.LogInstallProgress): |
705 | |
706 | # klass data so that we can veriy in the test as the actual |
707 | # object is destroyed |
708 | DATA = [] |
709 | - |
710 | + |
711 | # overwrite to log the data |
712 | def status_change(self, pkg, percent, status): |
713 | - print pkg, percent |
714 | + print(pkg, percent) |
715 | self.DATA.append([pkg, percent]) |
716 | |
717 | + |
718 | class TestMinimalPartitions(unittest.TestCase): |
719 | |
720 | def setUp(self): |
721 | # setup dry-run mode for apt |
722 | apt_pkg.config.set("Dir::Cache", "/tmp") |
723 | - apt_pkg.config.set("Debug::NoLocking","1") |
724 | - apt_pkg.config.set("Debug::pkgDPkgPM","1") |
725 | + apt_pkg.config.set("Debug::NoLocking", "1") |
726 | + apt_pkg.config.set("Debug::pkgDPkgPM", "1") |
727 | apt_pkg.config.set("Dir::State::extended_states", "./extended_states") |
728 | apt_pkg.config.clear("Dpkg::Post-Invoke") |
729 | apt_pkg.config.clear("Dpkg::Pre-Install-Pkgs") |
730 | @@ -40,10 +39,11 @@ |
731 | def test_upgrade_in_minimal_steps(self): |
732 | self.cache.upgrade(True) |
733 | pkgs_to_upgrade = [pkg.name for pkg in self.cache.get_changes()] |
734 | - unattended_upgrade.PROGRESS_LOG="./aptroot/var/run/unatteded-upgrades.progress" |
735 | + unattended_upgrade.PROGRESS_LOG = \ |
736 | + "./aptroot/var/run/unatteded-upgrades.progress" |
737 | unattended_upgrade.LogInstallProgress = LogInstallProgressMock |
738 | unattended_upgrade.upgrade_in_minimal_steps( |
739 | - self.cache, pkgs_to_upgrade) |
740 | + self.cache, pkgs_to_upgrade, "", []) |
741 | # ensure we count upwarts |
742 | last_percent = -1 |
743 | for (pkg, percent) in LogInstallProgressMock.DATA: |
744 | |
745 | === modified file 'test/test_origin_pattern.py' |
746 | --- test/test_origin_pattern.py 2012-08-14 11:55:21 +0000 |
747 | +++ test/test_origin_pattern.py 2012-10-08 16:08:26 +0000 |
748 | @@ -1,4 +1,4 @@ |
749 | -#!/usr/bin/python |
750 | +#!/usr/bin/python3 |
751 | |
752 | import apt_pkg |
753 | import logging |
754 | @@ -12,49 +12,63 @@ |
755 | UnknownMatcherError, |
756 | ) |
757 | |
758 | + |
759 | class MockOrigin(): |
760 | pass |
761 | + |
762 | + |
763 | class MockCandidate(): |
764 | pass |
765 | + |
766 | + |
767 | class MockPackage(): |
768 | pass |
769 | + |
770 | + |
771 | class MockCache(list): |
772 | pass |
773 | + |
774 | + |
775 | class MockDepCache(): |
776 | pass |
777 | |
778 | + |
779 | class TestOriginPatern(unittest.TestCase): |
780 | |
781 | def setUp(self): |
782 | pass |
783 | + |
784 | def tearDown(self): |
785 | pass |
786 | + |
787 | def test_match_whitelist_string(self): |
788 | origin = self._get_mock_origin( |
789 | "OriginUbuntu", "LabelUbuntu", "ArchiveUbuntu", |
790 | "archive.ubuntu.com", "main") |
791 | # good |
792 | - s="o=OriginUbuntu" |
793 | + s = "o=OriginUbuntu" |
794 | self.assertTrue(match_whitelist_string(s, origin)) |
795 | - s="o=OriginUbuntu,l=LabelUbuntu,a=ArchiveUbuntu,site=archive.ubuntu.com" |
796 | + s = "o=OriginUbuntu,l=LabelUbuntu,a=ArchiveUbuntu," \ |
797 | + "site=archive.ubuntu.com" |
798 | self.assertTrue(match_whitelist_string(s, origin)) |
799 | # bad |
800 | - s="" |
801 | - self.assertFalse(match_whitelist_string(s, origin)) |
802 | - s="o=something" |
803 | - self.assertFalse(match_whitelist_string(s, origin)) |
804 | - s="o=LabelUbuntu,a=no-match" |
805 | + s = "" |
806 | + self.assertFalse(match_whitelist_string(s, origin)) |
807 | + s = "o=something" |
808 | + self.assertFalse(match_whitelist_string(s, origin)) |
809 | + s = "o=LabelUbuntu,a=no-match" |
810 | self.assertFalse(match_whitelist_string(s, origin)) |
811 | # with escaping |
812 | origin = self._get_mock_origin("Google, Inc.", archive="stable") |
813 | # good |
814 | - s="o=Google\, Inc.,a=stable" |
815 | + s = "o=Google\, Inc.,a=stable" |
816 | self.assertTrue(match_whitelist_string(s, origin)) |
817 | |
818 | def test_match_whitelist_from_conffile(self): |
819 | # read some |
820 | apt_pkg.config.clear("Unattended-Upgrade") |
821 | - apt_pkg.read_config_file(apt_pkg.config, "./data/50unattended-upgrades.Test") |
822 | + apt_pkg.read_config_file( |
823 | + apt_pkg.config, "./data/50unattended-upgrades.Test") |
824 | allowed_origins = unattended_upgrade.get_allowed_origins() |
825 | #print allowed_origins |
826 | self.assertTrue("o=aOrigin,a=aArchive" in allowed_origins) |
827 | @@ -63,7 +77,8 @@ |
828 | |
829 | def test_compatiblity(self): |
830 | apt_pkg.config.clear("Unattended-Upgrade") |
831 | - apt_pkg.read_config_file(apt_pkg.config, "./data/50unattended-upgrades.compat") |
832 | + apt_pkg.read_config_file( |
833 | + apt_pkg.config, "./data/50unattended-upgrades.compat") |
834 | allowed_origins = unattended_upgrade.get_allowed_origins() |
835 | #print allowed_origins |
836 | self.assertTrue("o=Google\, Inc.,a=stable" in allowed_origins) |
837 | @@ -74,7 +89,7 @@ |
838 | |
839 | def test_unkown_matcher(self): |
840 | apt_pkg.config.clear("Unattended-Upgrade") |
841 | - s="xxx=OriginUbuntu" |
842 | + s = "xxx=OriginUbuntu" |
843 | with self.assertRaises(UnknownMatcherError): |
844 | self.assertTrue(match_whitelist_string(s, None)) |
845 | |
846 | @@ -90,10 +105,12 @@ |
847 | allowed_origins = ["o=Ubuntu"] |
848 | blacklist = ["linux-.*"] |
849 | # with blacklist pkg |
850 | - self.assertFalse(check_changes_for_sanity(cache, allowed_origins, blacklist)) |
851 | + self.assertFalse( |
852 | + check_changes_for_sanity(cache, allowed_origins, blacklist)) |
853 | # with "normal" pkg |
854 | pkg.name = "apt" |
855 | - self.assertTrue(check_changes_for_sanity(cache, allowed_origins, blacklist)) |
856 | + self.assertTrue( |
857 | + check_changes_for_sanity(cache, allowed_origins, blacklist)) |
858 | |
859 | def _get_mock_origin(self, aorigin="", label="", archive="", |
860 | site="", component=""): |
861 | @@ -125,4 +142,3 @@ |
862 | if __name__ == "__main__": |
863 | logging.basicConfig(level=logging.DEBUG) |
864 | unittest.main() |
865 | - |
866 | |
867 | === added file 'test/test_pep8.py' |
868 | --- test/test_pep8.py 1970-01-01 00:00:00 +0000 |
869 | +++ test/test_pep8.py 2012-10-08 16:08:26 +0000 |
870 | @@ -0,0 +1,17 @@ |
871 | +import glob |
872 | +import os |
873 | +import subprocess |
874 | +import unittest |
875 | + |
876 | + |
877 | +class PackagePep8TestCase(unittest.TestCase): |
878 | + |
879 | + def test_all_code(self): |
880 | + res = 0 |
881 | + py_files = glob.glob(os.path.join(os.path.dirname(__file__), "*.py")) |
882 | + res += subprocess.call(["pep8", "--repeat", ] + py_files) |
883 | + self.assertEqual(res, 0) |
884 | + |
885 | + |
886 | +if __name__ == "__main__": |
887 | + unittest.main() |
888 | |
889 | === modified file 'test/test_pyflakes.py' (properties changed: -x to +x) |
890 | --- test/test_pyflakes.py 2012-07-13 16:40:10 +0000 |
891 | +++ test/test_pyflakes.py 2012-10-08 16:08:26 +0000 |
892 | @@ -2,12 +2,14 @@ |
893 | import subprocess |
894 | import unittest |
895 | |
896 | + |
897 | class TestPyflakesClean(unittest.TestCase): |
898 | """ ensure that the tree is pyflakes clean """ |
899 | |
900 | def test_pyflakes_clean(self): |
901 | - target = os.path.join(os.path.dirname(__file__), "..", "unattended-upgrade") |
902 | - self.assertEqual(subprocess.call(["pyflakes", target]), 0) |
903 | + path = os.path.join( |
904 | + os.path.dirname(__file__), "unattended_upgrade.py") |
905 | + self.assertEqual(subprocess.call(["pyflakes", path]), 0) |
906 | |
907 | |
908 | if __name__ == "__main__": |
909 | |
910 | === modified file 'test/test_substitute.py' |
911 | --- test/test_substitute.py 2011-02-04 11:05:12 +0000 |
912 | +++ test/test_substitute.py 2012-10-08 16:08:26 +0000 |
913 | @@ -1,17 +1,13 @@ |
914 | -#!/usr/bin/python |
915 | +#!/usr/bin/python3 |
916 | |
917 | -import apt |
918 | import apt_pkg |
919 | -import os |
920 | import logging |
921 | import unittest |
922 | -import sys |
923 | - |
924 | -from StringIO import StringIO |
925 | |
926 | import unattended_upgrade |
927 | from unattended_upgrade import substitute, get_allowed_origins |
928 | |
929 | + |
930 | class TestSubstitude(unittest.TestCase): |
931 | |
932 | def setUp(self): |
933 | @@ -36,4 +32,3 @@ |
934 | if __name__ == "__main__": |
935 | logging.basicConfig(level=logging.DEBUG) |
936 | unittest.main() |
937 | - |
938 | |
939 | === modified file 'unattended-upgrade' |
940 | --- unattended-upgrade 2012-08-14 11:55:21 +0000 |
941 | +++ unattended-upgrade 2012-10-08 16:08:26 +0000 |
942 | @@ -1,4 +1,4 @@ |
943 | -#!/usr/bin/python |
944 | +#!/usr/bin/python3 |
945 | # Copyright (c) 2005-2012 Canonical Ltd |
946 | # |
947 | # AUTHOR: |
948 | @@ -20,26 +20,27 @@ |
949 | # along with unattended-upgrades; if not, write to the Free Software |
950 | # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA |
951 | # |
952 | - |
953 | import apt_inst |
954 | import apt_pkg |
955 | |
956 | import copy |
957 | import datetime |
958 | -import email.Charset |
959 | +import email.charset |
960 | import fcntl |
961 | import re |
962 | import os |
963 | import string |
964 | import sys |
965 | |
966 | -from email.Message import Message |
967 | +from io import StringIO |
968 | +from email.message import Message |
969 | from optparse import OptionParser |
970 | -from StringIO import StringIO |
971 | -from subprocess import Popen, PIPE |
972 | - |
973 | -import warnings |
974 | -warnings.filterwarnings("ignore", "apt API not stable yet", FutureWarning) |
975 | + |
976 | +from subprocess import ( |
977 | + Popen, |
978 | + PIPE, |
979 | + ) |
980 | + |
981 | import apt |
982 | import logging |
983 | import lsb_release |
984 | @@ -57,29 +58,32 @@ |
985 | DISTRO_ID = lsb_release.get_distro_information()['ID'] |
986 | |
987 | # progress information is written here |
988 | -PROGRESS_LOG="/var/run/unattended-upgrades.progress" |
989 | +PROGRESS_LOG = "/var/run/unattended-upgrades.progress" |
990 | |
991 | # set from the sigint signal handler |
992 | -SIGNAL_STOP_REQUEST=False |
993 | +SIGNAL_STOP_REQUEST = False |
994 | + |
995 | |
996 | class UnknownMatcherError(ValueError): |
997 | pass |
998 | |
999 | |
1000 | class UnattendedUpgradesCache(apt.Cache): |
1001 | - |
1002 | + |
1003 | def __init__(self, rootdir, allowed_origins): |
1004 | apt.Cache.__init__(self, rootdir=rootdir) |
1005 | self.allowed_origins = allowed_origins |
1006 | - # ensure we update the candidate versions |
1007 | + # ensure we update the candidate versions |
1008 | self.adjust_candidate_versions() |
1009 | + |
1010 | def clear(self): |
1011 | apt.Cache.clear(self) |
1012 | - # ensure we update the candidate versions |
1013 | + # ensure we update the candidate versions |
1014 | self.adjust_candidate_versions() |
1015 | + |
1016 | def adjust_candidate_versions(self): |
1017 | """ Adjust candidate versions to match highest allowed origin |
1018 | - |
1019 | + |
1020 | This adjusts the origin even if the candidate has a higher |
1021 | version |
1022 | """ |
1023 | @@ -114,20 +118,19 @@ |
1024 | (/var/run/unattended-upgrades.progress by default) |
1025 | """ |
1026 | |
1027 | - LOG = PROGRESS_LOG |
1028 | + LOG = PROGRESS_LOG |
1029 | |
1030 | def status_change(self, pkg, percent, status): |
1031 | - f=open(self.LOG, "w") |
1032 | - f.write(_("Progress: %s %% (%s)") % (percent, pkg)) |
1033 | - f.close() |
1034 | + with open(self.LOG, "w") as f: |
1035 | + f.write(_("Progress: %s %% (%s)") % (percent, pkg)) |
1036 | |
1037 | def _fixup_fds(self): |
1038 | - required_fds = [ 0, 1, 2, # stdin, stdout, stderr |
1039 | - self.writefd, |
1040 | - self.write_stream.fileno(), |
1041 | - self.statusfd, |
1042 | - self.status_stream.fileno() |
1043 | - ] |
1044 | + required_fds = [0, 1, 2, # stdin, stdout, stderr |
1045 | + self.writefd, |
1046 | + self.write_stream.fileno(), |
1047 | + self.statusfd, |
1048 | + self.status_stream.fileno() |
1049 | + ] |
1050 | # ensure that our required fds close on exec |
1051 | for fd in required_fds[3:]: |
1052 | old_flags = fcntl.fcntl(fd, fcntl.F_GETFD) |
1053 | @@ -140,19 +143,19 @@ |
1054 | try: |
1055 | fd = int(fdname) |
1056 | except Exception as e: |
1057 | - print "ERROR: can not get fd for '%s'" % fdname |
1058 | + print("ERROR: can not get fd for '%s'" % fdname) |
1059 | if fd in required_fds: |
1060 | continue |
1061 | try: |
1062 | os.close(fd) |
1063 | - #print "closed: ", fd |
1064 | + #print("closed: ", fd) |
1065 | except OSError as e: |
1066 | # there will be one fd that can not be closed |
1067 | # as its the fd from pythons internal diropen() |
1068 | # so its ok to ignore one close error |
1069 | error_count += 1 |
1070 | if error_count > 1: |
1071 | - print "ERROR: os.close(%s): %s" % (fd, e) |
1072 | + print("ERROR: os.close(%s): %s" % (fd, e)) |
1073 | |
1074 | def fork(self): |
1075 | pid = os.fork() |
1076 | @@ -162,53 +165,62 @@ |
1077 | |
1078 | |
1079 | class Unlocked: |
1080 | - """ context manager for unlocking the apt lock while cache.commit() |
1081 | - is run |
1082 | - """ |
1083 | + """ |
1084 | + Context manager for unlocking the apt lock while cache.commit() is run |
1085 | + """ |
1086 | + |
1087 | def __enter__(self): |
1088 | try: |
1089 | apt_pkg.pkgsystem_unlock() |
1090 | except: |
1091 | pass |
1092 | + |
1093 | def __exit__(self, exc_type, exc_value, exc_tb): |
1094 | try: |
1095 | apt_pkg.pkgsystem_unlock() |
1096 | except: |
1097 | pass |
1098 | |
1099 | + |
1100 | def is_dpkg_journal_dirty(): |
1101 | """ |
1102 | - test if the dpkg journal is dirty |
1103 | + Return True if the dpkg journal is dirty |
1104 | (similar to debSystem::CheckUpdates) |
1105 | """ |
1106 | - d = os.path.dirname( |
1107 | - apt_pkg.config.find_file("Dir::State::status"))+"/updates" |
1108 | + d = os.path.join( |
1109 | + os.path.dirname(apt_pkg.config.find_file("Dir::State::status")), |
1110 | + "updates") |
1111 | for f in os.listdir(d): |
1112 | if re.match("[0-9]+", f): |
1113 | return True |
1114 | return False |
1115 | |
1116 | + |
1117 | def signal_handler(signal, frame): |
1118 | logging.warn("SIGUSR1 recieved, will stop") |
1119 | global SIGNAL_STOP_REQUEST |
1120 | - SIGNAL_STOP_REQUEST=True |
1121 | + SIGNAL_STOP_REQUEST = True |
1122 | + |
1123 | |
1124 | def substitute(line): |
1125 | - """ substitude known mappings and return a new string |
1126 | + """ substitude known mappings and return a new string |
1127 | |
1128 | Currently supported ${distro-release} |
1129 | """ |
1130 | - mapping = {"distro_codename" : get_distro_codename(), |
1131 | - "distro_id" : get_distro_id(), |
1132 | + mapping = {"distro_codename": get_distro_codename(), |
1133 | + "distro_id": get_distro_id(), |
1134 | } |
1135 | return string.Template(line).substitute(mapping) |
1136 | - |
1137 | + |
1138 | + |
1139 | def get_distro_codename(): |
1140 | return DISTRO_CODENAME |
1141 | |
1142 | + |
1143 | def get_distro_id(): |
1144 | return DISTRO_ID |
1145 | |
1146 | + |
1147 | def get_allowed_origins_legacy(): |
1148 | """ legacy support for old Allowed-Origins var """ |
1149 | allowed_origins = [] |
1150 | @@ -222,10 +234,11 @@ |
1151 | distro_id = re.sub(r'([^\\]),', r'\1\\,', distro_id) |
1152 | distro_codename = re.sub(r'([^\\]),', r'\1\\,', distro_codename) |
1153 | # convert to new format |
1154 | - allowed_origins.append("o=%s,a=%s" % (substitute(distro_id), |
1155 | + allowed_origins.append("o=%s,a=%s" % (substitute(distro_id), |
1156 | substitute(distro_codename))) |
1157 | return allowed_origins |
1158 | |
1159 | + |
1160 | def get_allowed_origins(): |
1161 | """ return a list of allowed origins from apt.conf |
1162 | |
1163 | @@ -236,6 +249,7 @@ |
1164 | allowed_origins.append(substitute(s)) |
1165 | return allowed_origins |
1166 | |
1167 | + |
1168 | def match_whitelist_string(whitelist, origin): |
1169 | """ |
1170 | take a whitelist string in the form "origin=Debian,label=Debian-Security" |
1171 | @@ -251,9 +265,10 @@ |
1172 | whitelist = whitelist.replace("\,", "%2C") |
1173 | for token in whitelist.split(","): |
1174 | # strip and unquote the "," back |
1175 | - (what, value) = [s.strip().replace("%2C",",") |
1176 | + (what, value) = [s.strip().replace("%2C", ",") |
1177 | for s in token.split("=")] |
1178 | - #logging.debug("matching '%s'='%s' against '%s'" % (what, value, origin)) |
1179 | + #logging.debug("matching '%s'='%s' against '%s'" % ( |
1180 | + # what, value, origin)) |
1181 | # first char is apt-cache policy output, send is the name |
1182 | # in the Release file |
1183 | if what in ("o", "origin"): |
1184 | @@ -272,6 +287,7 @@ |
1185 | what, token)) |
1186 | return res |
1187 | |
1188 | + |
1189 | def upgrade_normal(cache, pkgs_to_upgrade, logfile_dpkg): |
1190 | error = None |
1191 | res = False |
1192 | @@ -279,18 +295,19 @@ |
1193 | try: |
1194 | with Unlocked(): |
1195 | res = cache.commit(install_progress=iprogress) |
1196 | - except SystemError,e: |
1197 | + except SystemError as e: |
1198 | error = e |
1199 | if res: |
1200 | logging.info(_("All upgrades installed")) |
1201 | else: |
1202 | logging.error(_("Installing the upgrades failed!")) |
1203 | logging.error(_("error message: '%s'") % error) |
1204 | - logging.error(_("dpkg returned a error! See '%s' for details") % \ |
1205 | + logging.error(_("dpkg returned a error! See '%s' for details") % |
1206 | logfile_dpkg) |
1207 | return res |
1208 | |
1209 | -def upgrade_in_minimal_steps(cache, pkgs_to_upgrade, logfile_dpkg=""): |
1210 | + |
1211 | +def upgrade_in_minimal_steps(cache, pkgs_to_upgrade, logfile_dpkg, blacklist): |
1212 | |
1213 | install_log = LogInstallProgress() |
1214 | install_log.LOG += ".minimal-steps" |
1215 | @@ -298,6 +315,9 @@ |
1216 | # setup signal handler |
1217 | signal.signal(signal.SIGUSR1, signal_handler) |
1218 | |
1219 | + # double check any changes we do |
1220 | + allowed_origins = get_allowed_origins() |
1221 | + |
1222 | # to upgrade contains the package names |
1223 | to_upgrade = set(pkgs_to_upgrade) |
1224 | while True: |
1225 | @@ -314,19 +334,28 @@ |
1226 | pkg.mark_install() |
1227 | else: |
1228 | continue |
1229 | + # double check that we are not running into side effects like |
1230 | + # what could have been caused LP: #1020680 |
1231 | + if not check_changes_for_sanity(cache, allowed_origins, blacklist): |
1232 | + logging.error( |
1233 | + "Internal error while building a minimal partition." |
1234 | + "Cache has not allowed changes") |
1235 | + return False |
1236 | changes = [pkg.name for pkg in cache.get_changes()] |
1237 | if len(changes) == 1: |
1238 | logging.debug("found leaf package %s" % pkg.name) |
1239 | smallest_partition = changes |
1240 | break |
1241 | if len(changes) < len(smallest_partition): |
1242 | - logging.debug("found partition of size %s (%s)" % (len(changes), changes)) |
1243 | + logging.debug("found partition of size %s (%s)" % ( |
1244 | + len(changes), changes)) |
1245 | smallest_partition = changes |
1246 | cache.clear() |
1247 | |
1248 | # write progress log information |
1249 | if len(pkgs_to_upgrade) > 0: |
1250 | - percent = (len(pkgs_to_upgrade)-len(to_upgrade)) / float(len(pkgs_to_upgrade))*100.0 |
1251 | + percent = ((len(pkgs_to_upgrade) - len(to_upgrade)) / |
1252 | + float(len(pkgs_to_upgrade)) * 100.0) |
1253 | else: |
1254 | percent = 100.0 |
1255 | install_log.status_change(pkg=",".join(smallest_partition), |
1256 | @@ -343,19 +372,20 @@ |
1257 | if not res: |
1258 | raise Exception("cache.commit() returned false") |
1259 | cache.open() |
1260 | - except Exception, e: |
1261 | + except Exception as e: |
1262 | logging.error(_("Installing the upgrades failed!")) |
1263 | logging.error(_("error message: '%s'") % e) |
1264 | - logging.error(_("dpkg returned a error! See '%s' for details") % \ |
1265 | + logging.error(_("dpkg returned a error! See '%s' for details") % |
1266 | logfile_dpkg) |
1267 | return False |
1268 | - to_upgrade = to_upgrade-set(smallest_partition) |
1269 | + to_upgrade = to_upgrade - set(smallest_partition) |
1270 | logging.debug("left to upgrade %s" % to_upgrade) |
1271 | if len(to_upgrade) == 0: |
1272 | logging.info(_("All upgrades installed")) |
1273 | break |
1274 | return True |
1275 | |
1276 | + |
1277 | def is_allowed_origin(ver, allowed_origins): |
1278 | if not ver: |
1279 | return False |
1280 | @@ -365,6 +395,7 @@ |
1281 | return True |
1282 | return False |
1283 | |
1284 | + |
1285 | def is_pkgname_in_blacklist(pkgname, blacklist): |
1286 | for blacklist_regexp in blacklist: |
1287 | if re.match(blacklist_regexp, pkgname): |
1288 | @@ -372,6 +403,7 @@ |
1289 | return True |
1290 | return False |
1291 | |
1292 | + |
1293 | def check_changes_for_sanity(cache, allowed_origins, blacklist): |
1294 | if cache._depcache.broken_count != 0: |
1295 | return False |
1296 | @@ -392,33 +424,39 @@ |
1297 | ignore_require_restart = apt_pkg.config.find_b( |
1298 | "Unattended-Upgrade::IgnoreAppsRequireRestart", False) |
1299 | if (pkg.marked_upgrade and |
1300 | - ignore_require_restart == False and |
1301 | + ignore_require_restart is False and |
1302 | pkg.candidate.record.get("Upgrade-Requires") == "app-restart"): |
1303 | - logging.debug("pkg '%s' requires app-restart, not safe to upgrade unattended") |
1304 | + logging.debug("pkg '%s' requires app-restart, not safe to " |
1305 | + "upgrade unattended") |
1306 | return False |
1307 | return True |
1308 | |
1309 | + |
1310 | def pkgname_from_deb(debfile): |
1311 | # FIXME: add error checking here |
1312 | try: |
1313 | control = apt_inst.DebFile(debfile).control.extractdata("control") |
1314 | sections = apt_pkg.TagSection(control) |
1315 | return sections["Package"] |
1316 | - except (IOError, SystemError), e: |
1317 | + except (IOError, SystemError) as e: |
1318 | logging.error("failed to read deb file '%s' (%s)" % (debfile, e)) |
1319 | # dumb fallback |
1320 | return debfile.split("_")[0] |
1321 | |
1322 | + |
1323 | def get_md5sum_for_file_in_deb(deb_file, conf_file): |
1324 | - dpkg_cmd = ["dpkg-deb","--fsys-tarfile", deb_file] |
1325 | - tar_cmd = ["tar","-x","-O", "-f","-", "."+conf_file] |
1326 | + dpkg_cmd = ["dpkg-deb", "--fsys-tarfile", deb_file] |
1327 | + tar_cmd = ["tar", "-x", "-O", "-f", "-", "." + conf_file] |
1328 | md5_cmd = ["md5sum"] |
1329 | dpkg_p = Popen(dpkg_cmd, stdout=PIPE) |
1330 | - tar_p = Popen(tar_cmd, stdin=dpkg_p.stdout, stdout=PIPE) |
1331 | - md5_p = Popen(md5_cmd, stdin=tar_p.stdout, stdout=PIPE) |
1332 | + tar_p = Popen(tar_cmd, stdin=dpkg_p.stdout, stdout=PIPE, |
1333 | + universal_newlines=True) |
1334 | + md5_p = Popen(md5_cmd, stdin=tar_p.stdout, stdout=PIPE, |
1335 | + universal_newlines=True) |
1336 | pkg_md5sum = md5_p.communicate()[0].split()[0] |
1337 | return pkg_md5sum |
1338 | |
1339 | + |
1340 | # prefix is *only* needed for the build-in tests |
1341 | def conffile_prompt(destFile, prefix=""): |
1342 | logging.debug("check_conffile_prompt('%s')" % destFile) |
1343 | @@ -426,7 +464,7 @@ |
1344 | |
1345 | # get the conffiles for the /var/lib/dpkg/status file |
1346 | status_file = apt_pkg.config.find("Dir::State::status") |
1347 | - tagfile = apt_pkg.TagFile(open(status_file,"r")) |
1348 | + tagfile = apt_pkg.TagFile(open(status_file, "r")) |
1349 | conffiles = "" |
1350 | for section in tagfile: |
1351 | if section.get("Package") == pkgname: |
1352 | @@ -440,21 +478,22 @@ |
1353 | pkg_conffiles = "" |
1354 | deb = apt_inst.DebFile(destFile) |
1355 | try: |
1356 | - pkg_conffiles = deb.control.extractdata("conffiles").strip() |
1357 | + pkg_conffiles = deb.control.extractdata("conffiles").strip().decode( |
1358 | + "utf-8") |
1359 | except LookupError as e: |
1360 | logging.debug("No conffiles in deb '%s' (%s)" % (destFile, e)) |
1361 | |
1362 | # Conffiles: |
1363 | # /etc/bash_completion.d/m-a c7780fab6b14d75ca54e11e992a6c11c |
1364 | dpkg_status_conffiles = {} |
1365 | - for line in string.split(conffiles,"\n"): |
1366 | + for line in conffiles.splitlines(): |
1367 | # ignore empty lines |
1368 | - line = string.strip(line) |
1369 | + line = line.strip() |
1370 | if not line: |
1371 | continue |
1372 | # show what we do |
1373 | logging.debug("conffile line: '%s'", line) |
1374 | - l = string.split(line) |
1375 | + l = line.split() |
1376 | conf_file = l[0] |
1377 | md5 = l[1] |
1378 | if len(l) > 2: |
1379 | @@ -462,13 +501,13 @@ |
1380 | else: |
1381 | obs = None |
1382 | # ignore if conffile is obsolete or does not exist |
1383 | - if obs == "obsolete" or not os.path.exists(prefix+conf_file): |
1384 | + if obs == "obsolete" or not os.path.exists(prefix + conf_file): |
1385 | continue |
1386 | # ignore state "newconffile" until its clearer if there |
1387 | # might be a dpkg prompt (LP: #936870) |
1388 | if md5 == "newconffile": |
1389 | continue |
1390 | - if (not pkg_conffiles or |
1391 | + if (not pkg_conffiles or |
1392 | not conf_file in pkg_conffiles.split("\n")): |
1393 | logging.debug("'%s' not in package conffiles '%s'" % ( |
1394 | conf_file, pkg_conffiles)) |
1395 | @@ -476,7 +515,8 @@ |
1396 | # record for later |
1397 | dpkg_status_conffiles[conf_file] = md5 |
1398 | # test against the installed file |
1399 | - current_md5 = apt_pkg.md5sum(open(prefix+conf_file).read()) |
1400 | + with open(prefix + conf_file, 'rb') as fb: |
1401 | + current_md5 = apt_pkg.md5sum(fb) |
1402 | logging.debug("current md5: %s" % current_md5) |
1403 | # hashes are the same, no conffile prompt |
1404 | if current_md5 == md5: |
1405 | @@ -484,7 +524,7 @@ |
1406 | # calculate md5sum from the deb (may take a bit) |
1407 | pkg_md5sum = get_md5sum_for_file_in_deb(destFile, conf_file) |
1408 | logging.debug("pkg_md5sum: %s" % pkg_md5sum) |
1409 | - # the md5sum in the deb is unchanged, this will not |
1410 | + # the md5sum in the deb is unchanged, this will not |
1411 | # trigger a conffile prompt |
1412 | if pkg_md5sum == md5: |
1413 | continue |
1414 | @@ -493,18 +533,19 @@ |
1415 | # and that will trigger a conffile prompt, we can |
1416 | # stop processing at this point and just return True |
1417 | return True |
1418 | - |
1419 | + |
1420 | # now check if there are conffiles in the pkg that where not there |
1421 | # in the previous version in the dpkg status file |
1422 | if pkg_conffiles: |
1423 | for conf_file in pkg_conffiles.split("\n"): |
1424 | if (not conf_file in dpkg_status_conffiles and |
1425 | - os.path.exists(prefix+conf_file)): |
1426 | + os.path.exists(prefix + conf_file)): |
1427 | logging.debug("found conffile '%s' in new pkg but on dpkg " |
1428 | "status" % conf_file) |
1429 | pkg_md5sum = get_md5sum_for_file_in_deb(destFile, conf_file) |
1430 | - if pkg_md5sum != apt_pkg.md5sum(open(prefix+conf_file).read()): |
1431 | - return True |
1432 | + with open(prefix + conf_file, 'rb') as fp: |
1433 | + if pkg_md5sum != apt_pkg.md5sum(fp): |
1434 | + return True |
1435 | return False |
1436 | |
1437 | |
1438 | @@ -514,7 +555,7 @@ |
1439 | options = apt_pkg.config.value_list("DPkg::Options") |
1440 | for option in map(string.strip, options): |
1441 | if (option == "--force-confold" or |
1442 | - option == "--force-confnew"): |
1443 | + option == "--force-confnew"): |
1444 | return False |
1445 | return True |
1446 | |
1447 | @@ -535,7 +576,7 @@ |
1448 | def wrap(t, width=70, subsequent_indent=""): |
1449 | out = "" |
1450 | for s in t.split(): |
1451 | - if (len(out)-out.rfind("\n")) + len(s) > width: |
1452 | + if (len(out) - out.rfind("\n")) + len(s) > width: |
1453 | out += "\n" + subsequent_indent |
1454 | out += s + " " |
1455 | return out |
1456 | @@ -554,8 +595,9 @@ |
1457 | |
1458 | |
1459 | def _send_mail_using_mailx(to_address, subject, body): |
1460 | - mail = subprocess.Popen([ |
1461 | - MAIL_BINARY, "-s", subject, to_address], stdin=subprocess.PIPE) |
1462 | + mail = subprocess.Popen( |
1463 | + [MAIL_BINARY, "-s", subject, to_address], |
1464 | + stdin=subprocess.PIPE, universal_newlines=True) |
1465 | mail.stdin.write(body) |
1466 | mail.stdin.close() |
1467 | ret = mail.wait() |
1468 | @@ -565,14 +607,15 @@ |
1469 | def _send_mail_using_sendmail(to_address, subject, body): |
1470 | # format as a proper mail |
1471 | msg = Message() |
1472 | - charset = email.Charset.Charset("utf-8") |
1473 | - charset.body_encoding = email.Charset.QP |
1474 | + charset = email.charset.Charset("utf-8") |
1475 | + charset.body_encoding = email.charset.QP |
1476 | msg.set_charset(charset) |
1477 | msg.set_payload(body) |
1478 | msg['Subject'] = subject |
1479 | msg['To'] = to_address |
1480 | sendmail = subprocess.Popen( |
1481 | - [SENDMAIL_BINARY, "-oi", "-t"], stdin=subprocess.PIPE) |
1482 | + [SENDMAIL_BINARY, "-oi", "-t"], |
1483 | + stdin=subprocess.PIPE, universal_newlines=True) |
1484 | sendmail.stdin.write(msg.as_string()) |
1485 | sendmail.stdin.close() |
1486 | ret = sendmail.wait() |
1487 | @@ -594,16 +637,18 @@ |
1488 | # mails on on errors, just exit here |
1489 | if (res and |
1490 | apt_pkg.config.find_b("Unattended-Upgrade::MailOnlyOnError", False)): |
1491 | - return |
1492 | + return |
1493 | # Check if reboot-required flag is present |
1494 | logging.debug("Sending mail with '%s' to '%s'" % (logfile_dpkg, to_email)) |
1495 | if os.path.isfile(REBOOT_REQUIRED_FILE): |
1496 | - subject = _("[reboot required] unattended-upgrades result for '%s'") % host() |
1497 | + subject = _( |
1498 | + "[reboot required] unattended-upgrades result for '%s'") % host() |
1499 | else: |
1500 | subject = _("unattended-upgrades result for '%s'") % host() |
1501 | body = _("Unattended upgrade returned: %s\n\n") % res |
1502 | if os.path.isfile(REBOOT_REQUIRED_FILE): |
1503 | - body += _("Warning: A reboot is required to complete this upgrade.\n\n") |
1504 | + body += _( |
1505 | + "Warning: A reboot is required to complete this upgrade.\n\n") |
1506 | body += _("Packages that are upgraded:\n") |
1507 | body += " " + wrap(pkgs, 70, " ") |
1508 | body += "\n" |
1509 | @@ -613,15 +658,16 @@ |
1510 | body += "\n" |
1511 | body += "\n" |
1512 | if os.path.exists(logfile_dpkg): |
1513 | - body += _("Package installation log:")+"\n" |
1514 | - body += open(logfile_dpkg).read() |
1515 | + body += _("Package installation log:") + "\n" |
1516 | + with open(logfile_dpkg) as fp: |
1517 | + body += fp.read() |
1518 | body += "\n\n" |
1519 | body += _("Unattended-upgrades log:\n") |
1520 | body += mem_log.getvalue() |
1521 | |
1522 | - # ensure that the message a utf8 encoded string |
1523 | - if type(body) is unicode: |
1524 | - body= body.encode("utf-8", "replace") |
1525 | + # python2 needs this as utf8 encoded string (not unicode) |
1526 | + if sys.version < '3': |
1527 | + body = body.encode("utf-8") |
1528 | |
1529 | if os.path.exists(SENDMAIL_BINARY): |
1530 | ret = _send_mail_using_sendmail(to_email, subject, body) |
1531 | @@ -634,15 +680,16 @@ |
1532 | logging.debug("mail returned: %s" % ret) |
1533 | |
1534 | |
1535 | -def do_install(cache, pkgs_to_upgrade, options, logfile_dpkg): |
1536 | +def do_install(cache, pkgs_to_upgrade, blacklisted_pkgs, options, |
1537 | + logfile_dpkg): |
1538 | # set debconf to NON_INTERACTIVE, redirect output |
1539 | - os.putenv("DEBIAN_FRONTEND","noninteractive"); |
1540 | + os.putenv("DEBIAN_FRONTEND", "noninteractive") |
1541 | setup_apt_listchanges() |
1542 | - |
1543 | + |
1544 | # redirect to log |
1545 | REDIRECT_INPUT = os.devnull |
1546 | fd = os.open(REDIRECT_INPUT, os.O_RDWR) |
1547 | - os.dup2(fd,0) |
1548 | + os.dup2(fd, 0) |
1549 | |
1550 | logging.info(_("Writing dpkg log to '%s'") % logfile_dpkg) |
1551 | |
1552 | @@ -651,21 +698,25 @@ |
1553 | old_stdout = 1 |
1554 | old_stderr = 2 |
1555 | else: |
1556 | - fd = os.open(logfile_dpkg, os.O_RDWR|os.O_CREAT, 0644) |
1557 | + fd = os.open(logfile_dpkg, os.O_RDWR | os.O_CREAT, 0o644) |
1558 | old_stdout = os.dup(1) |
1559 | old_stderr = os.dup(2) |
1560 | - os.dup2(fd,1) |
1561 | - os.dup2(fd,2) |
1562 | + os.dup2(fd, 1) |
1563 | + os.dup2(fd, 2) |
1564 | |
1565 | try: |
1566 | - if (options.minimal_upgrade_steps or |
1567 | + if (options.minimal_upgrade_steps or |
1568 | # COMPAT with the mispelling |
1569 | - apt_pkg.config.find_b("Unattended-Upgrades::MinimalSteps", False) or |
1570 | - apt_pkg.config.find_b("Unattended-Upgrade::MinimalSteps", False)): |
1571 | - open("/var/run/unattended-upgrades.pid", "w").write("%s" % os.getpid()) |
1572 | + apt_pkg.config.find_b( |
1573 | + "Unattended-Upgrades::MinimalSteps", False) or |
1574 | + apt_pkg.config.find_b( |
1575 | + "Unattended-Upgrade::MinimalSteps", False)): |
1576 | + with open("/var/run/unattended-upgrades.pid", "w") as fp: |
1577 | + fp.write("%s" % os.getpid()) |
1578 | # try upgrade all "pkgs" in minimal steps |
1579 | pkg_install_success = upgrade_in_minimal_steps( |
1580 | - cache, [pkg.name for pkg in pkgs_to_upgrade], logfile_dpkg) |
1581 | + cache, [pkg.name for pkg in pkgs_to_upgrade], logfile_dpkg, |
1582 | + blacklisted_pkgs) |
1583 | else: |
1584 | pkg_install_success = upgrade_normal( |
1585 | cache, [pkg.name for pkg in pkgs_to_upgrade], logfile_dpkg) |
1586 | @@ -685,15 +736,16 @@ |
1587 | apt_pkg.config.clear("Unattended-Upgrade") |
1588 | # read rootdir (taken from apt.Cache, but we need to run it |
1589 | # here before the cache gets initialized |
1590 | - if os.path.exists(rootdir+"/etc/apt/apt.conf"): |
1591 | + if os.path.exists(rootdir + "/etc/apt/apt.conf"): |
1592 | apt_pkg.read_config_file(apt_pkg.config, |
1593 | rootdir + "/etc/apt/apt.conf") |
1594 | - if os.path.isdir(rootdir+"/etc/apt/apt.conf.d"): |
1595 | + if os.path.isdir(rootdir + "/etc/apt/apt.conf.d"): |
1596 | apt_pkg.read_config_dir(apt_pkg.config, |
1597 | rootdir + "/etc/apt/apt.conf.d") |
1598 | |
1599 | + |
1600 | def _get_logdir(): |
1601 | - logdir= apt_pkg.config.find_dir( |
1602 | + logdir = apt_pkg.config.find_dir( |
1603 | "Unattended-Upgrade::LogDir", |
1604 | # COMPAT only |
1605 | apt_pkg.config.find_dir("APT::UnattendedUpgrades::LogDir", |
1606 | @@ -722,7 +774,6 @@ |
1607 | logging.basicConfig(level=logging.INFO, |
1608 | format='%(asctime)s %(levelname)s %(message)s', |
1609 | filename=logfile) |
1610 | - |
1611 | # additional logging |
1612 | logger = logging.getLogger() |
1613 | mem_log = StringIO() |
1614 | @@ -734,7 +785,11 @@ |
1615 | mem_log_handler = logging.StreamHandler(mem_log) |
1616 | logger.addHandler(mem_log_handler) |
1617 | return mem_log |
1618 | - |
1619 | + |
1620 | + |
1621 | +def get_blacklisted_pkgs(): |
1622 | + return apt_pkg.config.value_list("Unattended-Upgrade::Package-Blacklist") |
1623 | + |
1624 | |
1625 | def main(options, rootdir=""): |
1626 | |
1627 | @@ -749,24 +804,27 @@ |
1628 | allowed_origins = get_allowed_origins() |
1629 | |
1630 | # pkgs that are (for some reason) not save to install |
1631 | - blacklisted_pkgs = apt_pkg.config.value_list("Unattended-Upgrade::Package-Blacklist") |
1632 | - logging.info(_("Initial blacklisted packages: %s"), " ".join(blacklisted_pkgs)) |
1633 | + blacklisted_pkgs = get_blacklisted_pkgs() |
1634 | + logging.info(_("Initial blacklisted packages: %s"), |
1635 | + " ".join(blacklisted_pkgs)) |
1636 | logging.info(_("Starting unattended upgrades script")) |
1637 | |
1638 | # display available origin |
1639 | - logging.info(_("Allowed origins are: %s") % map(str,allowed_origins)) |
1640 | + logging.info(_("Allowed origins are: %s") % allowed_origins) |
1641 | |
1642 | # check if the journal is dirty and if so, take emergceny action |
1643 | # the alternative is to leave the system potentially unsecure until |
1644 | # the user comes in and fixes |
1645 | if (is_dpkg_journal_dirty() and |
1646 | - apt_pkg.config.find_b("Unattended-Upgrade::AutoFixInterruptedDpkg", True)): |
1647 | + apt_pkg.config.find_b( |
1648 | + "Unattended-Upgrade::AutoFixInterruptedDpkg", True)): |
1649 | # ensure the dpkg database is not already locked (LP: #754330) |
1650 | admindir = os.path.dirname(apt_pkg.config.find("Dir::State::Status")) |
1651 | lockfd = apt_pkg.get_lock(os.path.join(admindir, "lock"), False) |
1652 | if lockfd > 0: |
1653 | - logging.warning(_("Unclean dpkg state detected, trying to correct")) |
1654 | - print _("Unclean dpkg state detected, trying to correct") |
1655 | + logging.warning( |
1656 | + _("Unclean dpkg state detected, trying to correct")) |
1657 | + print(_("Unclean dpkg state detected, trying to correct")) |
1658 | env = copy.copy(os.environ) |
1659 | env["DEBIAN_FRONTEND"] = "noninteractive" |
1660 | try: |
1661 | @@ -777,22 +835,23 @@ |
1662 | output = e.output |
1663 | logging.warning(_("dpkg --configure -a output:\n%s" % output)) |
1664 | else: |
1665 | - logging.debug("Unclean dpkg state, but locked, another package manager working?") |
1666 | - |
1667 | + logging.debug("Unclean dpkg state, but locked, another package " |
1668 | + "manager working?") |
1669 | + |
1670 | # check and get lock |
1671 | try: |
1672 | apt_pkg.pkgsystem_lock() |
1673 | - except SystemError, e: |
1674 | + except SystemError as e: |
1675 | logging.error(_("Lock could not be acquired (another package " |
1676 | "manager running?)")) |
1677 | - print _("Cache lock can not be acquired, exiting") |
1678 | + print(_("Cache lock can not be acquired, exiting")) |
1679 | sys.exit(1) |
1680 | |
1681 | # get a cache |
1682 | - cache = UnattendedUpgradesCache(rootdir=rootdir, |
1683 | + cache = UnattendedUpgradesCache(rootdir=rootdir, |
1684 | allowed_origins=allowed_origins) |
1685 | if cache._depcache.broken_count > 0: |
1686 | - print _("Cache has broken packages, exiting") |
1687 | + print(_("Cache has broken packages, exiting")) |
1688 | logging.error(_("Cache has broken packages, exiting")) |
1689 | sys.exit(1) |
1690 | # speed things up with latest apt |
1691 | @@ -802,13 +861,14 @@ |
1692 | # find out about the packages that are upgradable (in a allowed_origin) |
1693 | pkgs_to_upgrade = [] |
1694 | pkgs_kept_back = [] |
1695 | - pkgs_auto_removable = set([pkg.name for pkg in cache |
1696 | + pkgs_auto_removable = set([pkg.name for pkg in cache |
1697 | if pkg.is_auto_removable]) |
1698 | |
1699 | # now do the actual upgrade |
1700 | for pkg in cache: |
1701 | if options.debug and pkg.is_upgradable: |
1702 | - logging.debug("Checking: %s (%s)" % (pkg.name, map(str, pkg.candidate.origins))) |
1703 | + logging.debug("Checking: %s (%s)" % ( |
1704 | + pkg.name, map(str, getattr(pkg.candidate, "origins", [])))) |
1705 | if (pkg.is_upgradable and |
1706 | not is_pkgname_in_blacklist(pkg.name, blacklisted_pkgs) and |
1707 | is_allowed_origin(pkg.candidate, allowed_origins)): |
1708 | @@ -818,7 +878,7 @@ |
1709 | blacklisted_pkgs): |
1710 | # add to packages to upgrade |
1711 | pkgs_to_upgrade.append(pkg) |
1712 | - # re-eval pkgs_kept_back as the resolver may fail to |
1713 | + # re-eval pkgs_kept_back as the resolver may fail to |
1714 | # directly upgrade a pkg, but that may work during |
1715 | # a subsequent operation, see debian bug #639840 |
1716 | for pkgname in pkgs_kept_back: |
1717 | @@ -830,17 +890,18 @@ |
1718 | logging.debug("sanity check failed") |
1719 | rewind_cache(cache, pkgs_to_upgrade) |
1720 | pkgs_kept_back.append(pkg.name) |
1721 | - except SystemError, e: |
1722 | + except SystemError as e: |
1723 | # can't upgrade |
1724 | - logging.warning(_("package '%s' upgradable but fails to be marked for upgrade (%s)") % (pkg.name, e)) |
1725 | + logging.warning( |
1726 | + _("package '%s' upgradable but fails to " |
1727 | + "be marked for upgrade (%s)") % (pkg.name, e)) |
1728 | rewind_cache(cache, pkgs_to_upgrade) |
1729 | pkgs_kept_back.append(pkg.name) |
1730 | - |
1731 | |
1732 | pkgs_to_upgrade.sort(key=lambda p: p.name) |
1733 | pkgs = "\n".join([pkg.name for pkg in pkgs_to_upgrade]) |
1734 | logging.debug("pkgs that look like they should be upgraded: %s" % pkgs) |
1735 | - |
1736 | + |
1737 | # download what looks good |
1738 | if options.debug: |
1739 | fetcher = apt_pkg.Acquire(apt.progress.text.AcquireProgress()) |
1740 | @@ -851,8 +912,8 @@ |
1741 | recs = cache._records |
1742 | pm = apt_pkg.PackageManager(cache._depcache) |
1743 | try: |
1744 | - pm.get_archives(fetcher,list,recs) |
1745 | - except SystemError, e: |
1746 | + pm.get_archives(fetcher, list, recs) |
1747 | + except SystemError as e: |
1748 | logging.error(_("GetArchives() failed: '%s'") % e) |
1749 | res = fetcher.run() |
1750 | logging.debug("fetch.run() result: %s" % res) |
1751 | @@ -863,15 +924,19 @@ |
1752 | for item in fetcher.items: |
1753 | logging.debug("%s" % item) |
1754 | if item.status == item.STAT_ERROR: |
1755 | - print _("An error ocured: '%s'") % item.error_text |
1756 | + print(_("An error ocured: '%s'") % item.error_text) |
1757 | logging.error(_("An error ocured: '%s'") % item.error_text) |
1758 | if not item.complete: |
1759 | - print _("The URI '%s' failed to download, aborting") % item.desc_uri |
1760 | - logging.error(_("The URI '%s' failed to download, aborting") % item.desc_uri) |
1761 | + print(_("The URI '%s' failed to download, aborting") % |
1762 | + item.desc_uri) |
1763 | + logging.error(_("The URI '%s' failed to download, aborting") % |
1764 | + item.desc_uri) |
1765 | sys.exit(1) |
1766 | if not os.path.exists(item.destfile): |
1767 | - print _("Download finished, but file '%s' not there?!?" % item.destfile) |
1768 | - logging.error("Download finished, but file '%s' not there?!?" % item.destfile) |
1769 | + print(_("Download finished, but file '%s' not there?!?" % |
1770 | + item.destfile)) |
1771 | + logging.error("Download finished, but file '%s' not " |
1772 | + "there?!?" % item.destfile) |
1773 | sys.exit(1) |
1774 | if not item.is_trusted: |
1775 | blacklisted_pkgs.append(pkgname_from_deb(item.destfile)) |
1776 | @@ -879,18 +944,21 @@ |
1777 | # skip package (means to re-run the whole marking again |
1778 | # and making sure that the package will not be pulled in by |
1779 | # some other package again!) |
1780 | - # |
1781 | + # |
1782 | # print to stdout to ensure that this message is part of |
1783 | # the cron mail (only if no summary mail is requested) |
1784 | email = apt_pkg.config.find("Unattended-Upgrade::Mail", "") |
1785 | if not email: |
1786 | - print _("Package '%s' has conffile prompt and needs to be upgraded manually") % pkgname_from_deb(item.destfile) |
1787 | + print(_("Package '%s' has conffile prompt and needs " |
1788 | + "to be upgraded manually") % pkgname_from_deb( |
1789 | + item.destfile)) |
1790 | # log to the logfile |
1791 | - logging.warning(_("Package '%s' has conffile prompt and needs to be upgraded manually") % pkgname_from_deb(item.destfile)) |
1792 | + logging.warning(_("Package '%s' has conffile prompt and " |
1793 | + "needs to be upgraded manually") % |
1794 | + pkgname_from_deb(item.destfile)) |
1795 | blacklisted_pkgs.append(pkgname_from_deb(item.destfile)) |
1796 | pkgs_kept_back.append(pkgname_from_deb(item.destfile)) |
1797 | |
1798 | - |
1799 | # redo the selection about the packages to upgrade based on the new |
1800 | # blacklist |
1801 | logging.debug("blacklist: %s" % blacklisted_pkgs) |
1802 | @@ -904,7 +972,7 @@ |
1803 | pkg.mark_upgrade() |
1804 | if check_changes_for_sanity(cache, allowed_origins, |
1805 | blacklisted_pkgs): |
1806 | - pkgs_to_upgrade.append(pkg) |
1807 | + pkgs_to_upgrade.append(pkg) |
1808 | else: |
1809 | if not (pkg.name in pkgs_kept_back): |
1810 | pkgs_kept_back.append(pkg.name) |
1811 | @@ -916,23 +984,26 @@ |
1812 | logging.debug("dpkg is configured not to cause conffile prompts") |
1813 | |
1814 | # do auto-remove |
1815 | - if apt_pkg.config.find_b("Unattended-Upgrade::Remove-Unused-Dependencies", False): |
1816 | - now_auto_removable = set([pkg.name for pkg in cache |
1817 | + if apt_pkg.config.find_b( |
1818 | + "Unattended-Upgrade::Remove-Unused-Dependencies", False): |
1819 | + now_auto_removable = set([pkg.name for pkg in cache |
1820 | if pkg.is_auto_removable]) |
1821 | - for pkgname in now_auto_removable-pkgs_auto_removable: |
1822 | + for pkgname in now_auto_removable - pkgs_auto_removable: |
1823 | logging.debug("marking %s for remove" % pkgname) |
1824 | cache[pkgname].mark_delete() |
1825 | logging.info(_("Packages that are auto removed: '%s'") % |
1826 | - " ".join(now_auto_removable-pkgs_auto_removable)) |
1827 | + " ".join(now_auto_removable - pkgs_auto_removable)) |
1828 | |
1829 | - logging.debug("InstCount=%i DelCount=%i BrokenCout=%i" % (cache._depcache.inst_count, cache._depcache.del_count, cache._depcache.broken_count)) |
1830 | + logging.debug("InstCount=%i DelCount=%i BrokenCount=%i" % ( |
1831 | + cache._depcache.inst_count, cache._depcache.del_count, |
1832 | + cache._depcache.broken_count)) |
1833 | |
1834 | # exit if there is nothing to do and nothing to report |
1835 | if (len(pkgs_to_upgrade) == 0) and (len(pkgs_kept_back) == 0): |
1836 | logging.info(_("No packages found that can be upgraded unattended")) |
1837 | return |
1838 | |
1839 | - # check if its configured for install on shutdown, if so, the |
1840 | + # check if its configured for install on shutdown, if so, the |
1841 | # environment UNATTENDED_UPGRADES_FORCE_INSTALL_ON_SHUTDOWN will |
1842 | # be set by the unatteded-upgrades-shutdown script |
1843 | if (not "UNATTENDED_UPGRADES_FORCE_INSTALL_ON_SHUTDOWN" in os.environ and |
1844 | @@ -944,7 +1015,7 @@ |
1845 | # check if we are in dry-run mode |
1846 | if options.dry_run: |
1847 | logging.info("Option --dry-run given, *not* performing real actions") |
1848 | - apt_pkg.config.set("Debug::pkgDPkgPM","1") |
1849 | + apt_pkg.config.set("Debug::pkgDPkgPM", "1") |
1850 | |
1851 | # do the install based on the new list of pkgs |
1852 | pkgs = " ".join([pkg.name for pkg in pkgs_to_upgrade]) |
1853 | @@ -964,7 +1035,7 @@ |
1854 | shutdown_lock = apt_pkg.get_lock("/var/run/unattended-upgrades.lock") |
1855 | # do install |
1856 | pkg_install_success = do_install( |
1857 | - cache, pkgs_to_upgrade, options, logfile_dpkg) |
1858 | + cache, pkgs_to_upgrade, blacklisted_pkgs, options, logfile_dpkg) |
1859 | |
1860 | # send a mail (if needed) |
1861 | if not options.dry_run: |
1862 | @@ -972,17 +1043,18 @@ |
1863 | pkgs, pkg_install_success, pkgs_kept_back, mem_log, logfile_dpkg) |
1864 | |
1865 | # auto-reboot (if required and the config for this is set |
1866 | - if (apt_pkg.config.find_b("Unattended-Upgrade::Automatic-Reboot", False) and |
1867 | + if (apt_pkg.config.find_b( |
1868 | + "Unattended-Upgrade::Automatic-Reboot", False) and |
1869 | os.path.exists(REBOOT_REQUIRED_FILE)): |
1870 | if shutdown_lock > 0: |
1871 | os.close(shutdown_lock) |
1872 | logging.warning("Found %s, rebooting" % REBOOT_REQUIRED_FILE) |
1873 | subprocess.call(["/sbin/reboot"]) |
1874 | - |
1875 | + |
1876 | |
1877 | if __name__ == "__main__": |
1878 | - localesApp="unattended-upgrades" |
1879 | - localesDir="/usr/share/locale" |
1880 | + localesApp = "unattended-upgrades" |
1881 | + localesDir = "/usr/share/locale" |
1882 | gettext.bindtextdomain(localesApp, localesDir) |
1883 | gettext.textdomain(localesApp) |
1884 | |
1885 | @@ -996,19 +1068,20 @@ |
1886 | help=_("Simulation, download but do not install")) |
1887 | parser.add_option("", "--minimal_upgrade_steps", |
1888 | action="store_true", default=False, |
1889 | - help=_("Upgrade in minimal steps (and allow interrupting with SIGINT")) |
1890 | + help=_("Upgrade in minimal steps (and allow " |
1891 | + "interrupting with SIGINT")) |
1892 | (options, args) = parser.parse_args() |
1893 | |
1894 | if os.getuid() != 0: |
1895 | - print _("You need to be root to run this application") |
1896 | + print(_("You need to be root to run this application")) |
1897 | sys.exit(1) |
1898 | |
1899 | # nice & ionce |
1900 | os.nice(19) |
1901 | - IONICE="/usr/bin/ionice" |
1902 | + IONICE = "/usr/bin/ionice" |
1903 | if os.path.exists(IONICE): |
1904 | with open(os.devnull, "w") as fstderr: |
1905 | - subprocess.call([IONICE, "-c3", "-p",str(os.getpid())], |
1906 | + subprocess.call([IONICE, "-c3", "-p", str(os.getpid())], |
1907 | stderr=fstderr) |
1908 | |
1909 | # ensure that we are not killed when the terminal goes away e.g. on |
1910 | |
1911 | === modified file 'unattended-upgrade-shutdown' |
1912 | --- unattended-upgrade-shutdown 2012-04-26 11:40:46 +0000 |
1913 | +++ unattended-upgrade-shutdown 2012-10-08 16:08:26 +0000 |
1914 | @@ -1,4 +1,4 @@ |
1915 | -#!/usr/bin/python |
1916 | +#!/usr/bin/python3 |
1917 | # Copyright (c) 2009 Canonical Ltd |
1918 | # |
1919 | # AUTHOR: |
1920 | @@ -141,8 +141,8 @@ |
1921 | os.kill(pid, signal.SIGUSR1) |
1922 | # show log |
1923 | log_progress() |
1924 | - time.sleep(5) |
1925 | - if (time.time() - start_time) > options.delay*60: |
1926 | + time.sleep(5) |
1927 | + if (time.time() - start_time) > options.delay*60: |
1928 | logging.warning(_("Giving up on lockfile after %s delay") % options.delay) |
1929 | sys.exit(1) |
1930 |