Merge lp:~takluyver/unattended-upgrades/py3 into lp:~ubuntu-core-dev/unattended-upgrades/ubuntu

Proposed by Thomas Kluyver on 2012-06-17
Status: Merged
Approved by: Barry Warsaw on 2012-06-25
Approved revision: 285
Merge reported by: Barry Warsaw
Merged at revision: not available
Proposed branch: lp:~takluyver/unattended-upgrades/py3
Merge into: lp:~ubuntu-core-dev/unattended-upgrades/ubuntu
Diff against target: 712 lines (+115/-93) 18 files modified
To merge this branch: bzr merge lp:~takluyver/unattended-upgrades/py3
Reviewer Review Type Date Requested Status
Barry Warsaw 2012-06-17 Approve on 2012-06-25
Michael Vogt upload to debian 2012-06-25 Pending
Review via email: mp+110684@code.launchpad.net

Description of the Change

As part of Ubuntu's push for Python 3 in Quantal (https://blueprints.launchpad.net/ubuntu/+spec/foundations-q-python-versions), this makes unattended-upgrades use Python 3.

The tests pass, and I've checked that it contains equivalent files to the current package (http://packages.ubuntu.com/quantal/all/unattended-upgrades/filelist). I'm not sure how to test unattended-upgrades-shutdown, but I've ensured that the syntax is valid, and inspecting the file, it doesn't look like it will require any other changes for Python 3.

To post a comment you must log in.
Thomas Kluyver (takluyver) wrote :

I've just noticed that test_in_chroot.py isn't run with the other tests. It appears to fail on Python 2. How much attention should I pay to it?

278. By Michael Vogt on 2012-06-18

* po/da.po:
  - updated, thanks to Joe Dalton, closes: #677804

279. By Michael Vogt on 2012-06-19

* debian/unattended-upgrades.init:
  - use new style lsb-init output, closes: #678030

280. By Michael Vogt on 2012-06-19

honor verbose

Barry Warsaw (barry) wrote :

Looks like there are some text conflicts as well, but I can try to fix those.

Barry Warsaw (barry) wrote :

Oh also, the conversion to bytes in send_summary_mail() isn't right. It's generally accepted wisdom that in Python 3, Popen() should be passed universal_newlines=True which gives you unicodes. I think that's the right thing to do in this case.

Barry Warsaw (barry) wrote :

Thomas, thanks for taking this work on! I appreciate everything you can do to
help us in our porting efforts. A few comments, and then I'll work on merging
this for you.

 - I don't know how to test unattended-upgrades-shutdown, but I'll see if I
   can come up with something.

 - How hard would it be to fix the failing Python 2 test? If it fails in Py2
   it would be nice to fix, but I don't think it should necessarily block your
   work.

 - Note that unless you're holding out the possibility to revert to Python 2,
   or if you're being annoyed by pyflakes warnings, you really don't need to
   future import print_function for Python 3.

 - Line 433 would probably be better rewritten:

    with open(prefix + conf_file, 'rb') as fp:
        current_md5 = fp.read()

   I can fix that for you when I merge your branch.

Other than that, the diff looks pretty good.

Barry Warsaw (barry) wrote :

Also, test_against_real_archive fails for me even in unchanged trunk:

======================================================================
FAIL: test_against_real_archive (__main__.TestAgainstRealArchive)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./test_against_real_archive.py", line 47, in test_against_real_archive
    self.assertFalse(" ERROR " in log)
AssertionError: True is not false

----------------------------------------------------------------------
Ran 1 test in 9.248s

281. By Barry Warsaw on 2012-06-19

Merge Thomas's branch: Port to Python 3
- Additional fixes by Barry

Barry Warsaw (barry) wrote :

Thomas, for now I'm going to mark this branch as needs fixing. Please look at my branch, which merges yours into trunk and contains additional fixes that will be necessary. If you can get the remaining tests to pass, I'll be happy to re-review the branch and sponsor it for you.

lp:~barry/unattended-upgrades/py3merge

review: Needs Fixing
Thomas Kluyver (takluyver) wrote :

Thanks Barry,

On 19 June 2012 20:25, Barry Warsaw <email address hidden> wrote:
>  - How hard would it be to fix the failing Python 2 test?  If it fails in Py2
>   it would be nice to fix, but I don't think it should necessarily block your
>   work.

I should have been clearer about this - I meant 'it fails *even* in
Python 2, so I haven't tried it in Python 3 yet'. To further
complicate matters, it tests in a Lucid chroot, and there's no
python3-apt in Lucid. I don't know whether testing in an old version
is an intentional way to make sure there's updates available, or just
because the test was written then and hasn't been changed.

For reference, I've re-run it (in Python 2) and put the last part of
the output at https://gist.github.com/2957247

>  - Note that unless you're holding out the possibility to revert to Python 2,
>   or if you're being annoyed by pyflakes warnings, you really don't need to
>   future import print_function for Python 3.

Do any other distributions use unattended-upgrades? If anyone wants to
keep running it in Python 2, it's very easy for me to enable that. If
not, I agree that it's slightly tidier not to.

>  - Line 433 would probably be better rewritten:
>
>    with open(prefix + conf_file, 'rb') as fp:
>        current_md5 = fp.read()
>
>   I can fix that for you when I merge your branch.

I see you did fix it; thanks.

Thomas

Thomas Kluyver (takluyver) wrote :

Thanks for the tip about Popen(), I didn't know about that argument. It seems rather unintuitive to have universal_newlines control text/bytes mode, though.

I've checked out your branch - I don't see the failure in test_against_real_archive.py, either on Python 3 or 2. Full output is at https://gist.github.com/2957277

Barry Warsaw (barry) wrote :

On Jun 20, 2012, at 12:15 AM, Thomas Kluyver wrote:

>I've checked out your branch - I don't see the failure in
>test_against_real_archive.py, either on Python 3 or 2. Full output is at
>https://gist.github.com/2957277

That probably means it's dependent on the environment. Something in my
environment (on precise and quantal, on both a normal desktop and chroot) is
causing the test to fail. That plus the fact that the test hits the network
tell me that it's not a very good test.

The problem is, if I want to sponsor this for you, I have to build the source
package. To do that, I'd have to disable the test since the source package
won't build with a failing test. Maybe that's the best we can do with such a
bad test.

It's not clear to me what this test is actually trying to do though. Do you
think you could take a crack at improving the test so that it isn't dependent
on the environment, and/or that it doesn't hit the network? Failing that, we
might just have to bite the bullet and disable the test.

As for continuing to support Python 2, perhaps the only reason to do so is if
it needs to be backported to earlier versions that don't have Python 3. If
not, then I personally think it would be fine to drop it.

282. By Thomas Kluyver on 2012-06-24

Fix subprocess communication on Python 3

283. By Thomas Kluyver on 2012-06-24

Disable test that requires network access

284. By Thomas Kluyver on 2012-06-24

Remove Python 2 compatible imports

Thomas Kluyver (takluyver) wrote :

I think the nature of that test needs network communications - 'against real archive' seems to mean the Ubuntu package archive. I've disabled it so that "cd test/; make" doesn't see it (it checks the executable flag on the files). The package building doesn't appear to run any tests, however.

I've also dropped the backwards compatible imports, so it will only work with Python 3.

Barry Warsaw (barry) wrote :

On Jun 24, 2012, at 03:46 PM, Thomas Kluyver wrote:

>I think the nature of that test needs network communications - 'against real
>archive' seems to mean the Ubuntu package archive. I've disabled it so that
>"cd test/; make" doesn't see it (it checks the executable flag on the
>files). The package building doesn't appear to run any tests, however.
>
>I've also dropped the backwards compatible imports, so it will only work with
>Python 3.

Quick response: I'm now getting a text conflict against trunk in
debian/unattended-upgrades.init. Can you please resolve that?

I was able to build the package locally and everything else looks fine, so
once you push an non-conflicting update, I'll try to sponsor this for you.

Thanks for your work on porting this to Python 3!

285. By Thomas Kluyver on 2012-06-25

Merge trunk

Thomas Kluyver (takluyver) wrote :

Thanks, Barry. I've merged in trunk and resolved the conflict.

Barry Warsaw (barry) wrote :

Thanks Thomas, looks great. I've pushed the merge to trunk, but it's up to Michael to upload the new package. I think it has to go to Debian first, and then it'll get sync'd into Quantal.

Barry Warsaw (barry) :
review: Approve

Preview Diff

1=== added file '.bzrignore'
2--- .bzrignore 1970-01-01 00:00:00 +0000
3+++ .bzrignore 2012-06-25 20:04:38 +0000
4@@ -0,0 +1,2 @@
5+__pycache__
6+build
7
8=== modified file 'debian/changelog'
9--- debian/changelog 2012-06-25 07:11:57 +0000
10+++ debian/changelog 2012-06-25 20:04:38 +0000
11@@ -16,6 +16,9 @@
12 * debian/unattended-upgrades.init:
13 - fixes new style lsb-init output, closes: #678030
14
15+ [ Thomas Kluyver ]
16+ * Port to Python 3
17+
18 -- Michael Vogt <michael.vogt@ubuntu.com> Mon, 25 Jun 2012 08:42:23 +0200
19
20 unattended-upgrades (0.78) unstable; urgency=low
21@@ -28,7 +31,7 @@
22 - updated, thanks to Joe Dalton, closes: #677804
23 * po/de.po:
24 - updated, thanks to Chris Leick (closes: #678204)
25-
26+
27 [ Teodor MICU ]
28 * debian/unattended-upgrades.init:
29 - use new style lsb-init output, closes: #678030
30
31=== modified file 'debian/control'
32--- debian/control 2012-06-19 15:26:37 +0000
33+++ debian/control 2012-06-25 20:04:38 +0000
34@@ -3,14 +3,14 @@
35 Priority: optional
36 Maintainer: Michael Vogt <michael.vogt@ubuntu.com>
37 Build-Depends: debhelper (>= 7.0.50~), po-debconf, lsb-release
38-Build-Depends-Indep: python (>= 2.6.6-3~), python-distutils-extra
39-Standards-Version: 3.8.3
40+Build-Depends-Indep: python3, python3-distutils-extra
41+Standards-Version: 3.9.3
42 Vcs-Bzr: http://code.launchpad.net/~ubuntu-core-dev/unattended-upgrades/ubuntu/
43
44 Package: unattended-upgrades
45 Architecture: all
46-Depends: ${shlibs:Depends}, ${misc:Depends}, debconf, python,
47- python-apt (>= 0.7.90), apt-utils, apt, ucf, lsb-release,
48+Depends: ${shlibs:Depends}, ${misc:Depends}, debconf, python3,
49+ python3-apt (>= 0.7.90), apt-utils, apt, ucf, lsb-release,
50 lsb-base (>= 3.2-14)
51 Suggests: bsd-mailx
52 Description: automatic installation of security upgrades
53
54=== modified file 'debian/rules'
55--- debian/rules 2011-11-09 19:20:50 +0000
56+++ debian/rules 2012-06-25 20:04:38 +0000
57@@ -3,24 +3,26 @@
58 DIST=$(shell /usr/bin/lsb_release -i -s)
59
60 %:
61- dh $@ --with python2
62+ dh $@ --with python3
63
64 override_dh_auto_build:
65 # copy the right template into place
66 cp data/50unattended-upgrades.$(DIST) data/50unattended-upgrades
67- dh_auto_build
68+ python3 setup.py build
69+
70+override_dh_auto_install:
71+ python3 setup.py install \
72+ --root=$(CURDIR)/debian/unattended-upgrades \
73+ --install-layout=deb
74
75 override_dh_auto_clean:
76 # Sanity-check before upload.
77- set -e; if [ -e /usr/lib/python$(PYVER)/py_compile.py ]; then \
78- for f in unattended-upgrade unattended-upgrade-shutdown; do \
79- ln -nsf $$f $$f.py; \
80- python$(PYVER) /usr/lib/python$(PYVER)/py_compile.py \
81- $$f.py; \
82- rm -f $$f.py; \
83- done; \
84- fi
85- dh_auto_clean
86+ set -e; for f in unattended-upgrade unattended-upgrade-shutdown; do \
87+ ln -nsf $$f $$f.py; \
88+ py3compile $$f.py; \
89+ rm -f $$f.py; \
90+ done
91+ python3 setup.py clean -a
92
93 override_dh_installinit:
94 # we do not want to run the init script in the postinst/prerm, its
95
96=== modified file 'debian/unattended-upgrades.init'
97--- debian/unattended-upgrades.init 2012-06-25 07:07:39 +0000
98+++ debian/unattended-upgrades.init 2012-06-25 20:04:38 +0000
99@@ -33,7 +33,7 @@
100 stop)
101 if [ -e $SHUTDOWN_HELPER ]; then
102 [ "$VERBOSE" != "no" ] && log_action_begin_msg "Checking for running $DESC"
103- python $SHUTDOWN_HELPER
104+ python3 $SHUTDOWN_HELPER
105 [ "$VERBOSE" != "no" ] && log_action_end_msg $? "$NAME"
106 fi
107 ;;
108
109=== modified file 'pm/sleep.d/10_unattended-upgrades-hibernate'
110--- pm/sleep.d/10_unattended-upgrades-hibernate 2011-11-08 16:56:58 +0000
111+++ pm/sleep.d/10_unattended-upgrades-hibernate 2012-06-25 20:04:38 +0000
112@@ -17,7 +17,7 @@
113 case "${1}" in
114 hibernate)
115 if [ -e $SHUTDOWN_HELPER ]; then
116- python $SHUTDOWN_HELPER
117+ python3 $SHUTDOWN_HELPER
118 fi
119 ;;
120 resume|thaw)
121
122=== modified file 'test/Makefile'
123--- test/Makefile 2011-10-07 09:22:19 +0000
124+++ test/Makefile 2012-06-25 20:04:38 +0000
125@@ -6,9 +6,9 @@
126 set -e; \
127 find . -name 'test_*.py' | \
128 while read file; do \
129- echo "Running $$file"; \
130 if [ -x $$file ]; then \
131- python $$file ; \
132+ echo "Running $$file"; \
133+ python3 $$file ; \
134 fi \
135 done
136
137
138=== modified file 'test/create_debug_lock.py'
139--- test/create_debug_lock.py 2011-06-16 07:10:07 +0000
140+++ test/create_debug_lock.py 2012-06-25 20:04:38 +0000
141@@ -1,4 +1,4 @@
142-#!/usr/bin/python
143+#!/usr/bin/python3
144 #
145 # create a lock file so that unattended-upgrades-shutdown pauses
146 # on shutdown -- useful for testing
147
148=== modified file 'test/test_against_real_archive.py' (properties changed: +x to -x)
149--- test/test_against_real_archive.py 2012-02-28 10:48:18 +0000
150+++ test/test_against_real_archive.py 2012-06-25 20:04:38 +0000
151@@ -1,4 +1,9 @@
152-#!/usr/bin/python
153+#!/usr/bin/python3
154+"""Test unattended_upgrades against the real archive in a chroot.
155+
156+Note that this test is not run by the makefile in this folder, as it requires
157+network access, and it fails in some situations (unclear which).
158+"""
159
160 import apt
161 import apt_pkg
162@@ -42,9 +47,10 @@
163 res = unattended_upgrade.main(options, os.path.abspath("./aptroot"))
164 # check if the log file exists
165 self.assertTrue(os.path.exists(logfile))
166- log = open(logfile).read()
167+ with open(logfile) as fp:
168+ log = fp.read()
169 # check that stuff worked
170- self.assertFalse(" ERROR " in log)
171+ self.assertFalse(" ERROR " in log, log)
172 # check if we actually have the expected ugprade in it
173 self.assertTrue(
174 re.search("INFO Packages that are upgraded:.*awstats", log))
175
176=== modified file 'test/test_conffile.py'
177--- test/test_conffile.py 2011-04-28 16:29:19 +0000
178+++ test/test_conffile.py 2012-06-25 20:04:38 +0000
179@@ -1,4 +1,4 @@
180-#!/usr/bin/python
181+#!/usr/bin/python3
182
183 import apt_pkg
184 import logging
185
186=== modified file 'test/test_in_chroot.py'
187--- test/test_in_chroot.py 2012-01-02 13:14:02 +0000
188+++ test/test_in_chroot.py 2012-06-25 20:04:38 +0000
189@@ -1,4 +1,4 @@
190-#!/usr/bin/python
191+#!/usr/bin/python3
192
193 import apt
194 import logging
195@@ -56,7 +56,7 @@
196 class TestUnattendedUpgrade(unittest.TestCase):
197
198 def _create_new_debootstrap_tarball(self, tarball, target):
199- print "creating initial test tarball, this is needed only once"
200+ print("creating initial test tarball, this is needed only once")
201 # force i386
202 subprocess.call(["debootstrap",
203 "--arch=%s" % ARCH,
204@@ -73,7 +73,7 @@
205 subprocess.call(["tar", "xzf", tarball])
206
207 def test_normal_upgrade(self):
208- print "Running normal unattended upgrade in chroot"
209+ print("Running normal unattended upgrade in chroot")
210 options = MockOptions()
211 options.minimal_upgrade_steps = False
212 # run it
213@@ -82,7 +82,7 @@
214 self.assertTrue(self._verify_install_log_in_real_chroot(target))
215
216 def test_minimal_steps_upgrade(self):
217- print "Running minimal steps unattended upgrade in chroot"
218+ print("Running minimal steps unattended upgrade in chroot")
219 options = MockOptions()
220 options.minimal_upgrade_steps = True
221 # run it
222@@ -91,7 +91,7 @@
223 self.assertTrue(self._verify_install_log_in_real_chroot(target))
224
225 def test_upgrade_on_shutdown_upgrade(self):
226- print "Running unattended upgrade on shutdown (download and install) in chroot"
227+ print("Running unattended upgrade on shutdown (download and install) in chroot")
228 # ensure that it actually installs in shutdown env mode
229 options = MockOptions()
230 os.environ["UNATTENDED_UPGRADES_FORCE_INSTALL_ON_SHUTDOWN"] = "1"
231@@ -121,7 +121,7 @@
232 and does some basic verifications
233 """
234 if os.getuid() != 0:
235- print "Skipping because uid != 0"
236+ print("Skipping because uid != 0")
237 return
238
239 # clear to avoid pollution in the chroot
240@@ -171,7 +171,7 @@
241 if pid == apid:
242 ret = os.WEXITSTATUS(status)
243 break
244- #print "*******************", all_progress
245+ #print("*******************", all_progress)
246 self.assertEqual(ret, 0)
247 # this number is a bit random, we just want to be sure we have
248 # progress data
249@@ -183,7 +183,7 @@
250 # examine log
251 log = self._get_lockfile_location(target)
252 logfile = open(log).read()
253- #print logfile
254+ #print(logfile)
255 NEEDLE_PKG="ca-certificates"
256 if not re.search(
257 "Packages that are upgraded:.*%s" % NEEDLE_PKG, logfile):
258@@ -198,7 +198,7 @@
259 if not "Preparing to replace %s" % NEEDLE_PKG in dpkg_logfile:
260 logging.warn("Did not find %s upgrade in the dpkg.log" % NEEDLE_PKG)
261 return False
262- #print dpkg_logfile
263+ #print(dpkg_logfile)
264 return True
265
266
267
268=== modified file 'test/test_logdir.py'
269--- test/test_logdir.py 2011-03-04 15:04:54 +0000
270+++ test/test_logdir.py 2012-06-25 20:04:38 +0000
271@@ -1,4 +1,4 @@
272-#!/usr/bin/python
273+#!/usr/bin/python3
274
275 import apt_pkg
276 import logging
277
278=== modified file 'test/test_mail.py'
279--- test/test_mail.py 2011-02-03 19:50:16 +0000
280+++ test/test_mail.py 2012-06-25 20:04:38 +0000
281@@ -1,4 +1,4 @@
282-#!/usr/bin/python
283+#!/usr/bin/python3
284
285 import apt
286 import apt_pkg
287@@ -7,7 +7,7 @@
288 import unittest
289 import sys
290
291-from StringIO import StringIO
292+from io import StringIO
293
294 import unattended_upgrade
295 import unattended_upgrade
296@@ -36,7 +36,8 @@
297 pkgs_kept_back = []
298 mem_log = StringIO("mem_log text")
299 logfile_dpkg = "./apt-term.log"
300- open("./apt-term.log", "w").write("logfile_dpkg text")
301+ with open("./apt-term.log", "w") as fp:
302+ fp.write("logfile_dpkg text")
303 return (pkgs, res, pkgs_kept_back, mem_log, logfile_dpkg)
304
305 def _verify_common_mail_content(self, mail_txt):
306@@ -45,16 +46,19 @@
307 self.assertTrue("Packages that are upgraded:\n 2vcard" in mail_txt)
308
309 def test_summary_mail_reboot(self):
310- open("./reboot-required","w").write("")
311+ with open("./reboot-required","w") as fp:
312+ fp.write("")
313 send_summary_mail(*self._return_mock_data())
314 os.unlink("./reboot-required")
315- mail_txt = open("mail.txt").read()
316+ with open("mail.txt") as fp:
317+ mail_txt = fp.read()
318 self.assertTrue("[reboot required]" in mail_txt)
319 self._verify_common_mail_content(mail_txt)
320
321 def test_summary_mail_no_reboot(self):
322 send_summary_mail(*self._return_mock_data())
323- mail_txt = open("mail.txt").read()
324+ with open("mail.txt") as fp:
325+ mail_txt = fp.read()
326 self.assertFalse("[reboot required]" in mail_txt)
327 self._verify_common_mail_content(mail_txt)
328
329@@ -63,18 +67,21 @@
330 # for both success and failure
331 apt_pkg.config.set("Unattended-Upgrade::MailOnlyOnError", "false")
332 send_summary_mail(*self._return_mock_data(successful=True))
333- self._verify_common_mail_content(open("mail.txt").read())
334+ with open("mail.txt") as fp:
335+ self._verify_common_mail_content(fp.read())
336 os.remove("mail.txt")
337 # now with a simulated failure
338 send_summary_mail(*self._return_mock_data(successful=False))
339- self._verify_common_mail_content(open("mail.txt").read())
340+ with open("mail.txt") as fp:
341+ self._verify_common_mail_content(fp.read())
342 os.remove("mail.txt")
343 # now test with "MailOnlyOnError"
344 apt_pkg.config.set("Unattended-Upgrade::MailOnlyOnError", "true")
345 send_summary_mail(*self._return_mock_data(successful=True))
346 self.assertFalse(os.path.exists("mail.txt"))
347 send_summary_mail(*self._return_mock_data(successful=False))
348- mail_txt = open("mail.txt").read()
349+ with open("mail.txt") as fp:
350+ mail_txt = fp.read()
351 self._verify_common_mail_content(mail_txt)
352 self.assertTrue("Unattended upgrade returned: False" in mail_txt)
353 self.assertTrue(os.path.exists("mail.txt"))
354
355=== modified file 'test/test_minimal_partitions.py'
356--- test/test_minimal_partitions.py 2011-11-18 10:46:15 +0000
357+++ test/test_minimal_partitions.py 2012-06-25 20:04:38 +0000
358@@ -1,12 +1,9 @@
359-#!/usr/bin/python
360+#!/usr/bin/python3
361
362 import apt
363 import apt_pkg
364 import os
365-import logging
366 import unittest
367-import sys
368-import time
369
370 import unattended_upgrade
371
372@@ -18,7 +15,7 @@
373
374 # overwrite to log the data
375 def status_change(self, pkg, percent, status):
376- print pkg, percent
377+ print(pkg, percent)
378 self.DATA.append([pkg, percent])
379
380 class TestMinimalPartitions(unittest.TestCase):
381
382=== modified file 'test/test_origin_pattern.py'
383--- test/test_origin_pattern.py 2011-11-22 12:54:47 +0000
384+++ test/test_origin_pattern.py 2012-06-25 20:04:38 +0000
385@@ -1,4 +1,4 @@
386-#!/usr/bin/python
387+#!/usr/bin/python3
388
389 import apt
390 import apt_pkg
391
392=== modified file 'test/test_substitute.py'
393--- test/test_substitute.py 2011-02-04 11:05:12 +0000
394+++ test/test_substitute.py 2012-06-25 20:04:38 +0000
395@@ -1,4 +1,4 @@
396-#!/usr/bin/python
397+#!/usr/bin/python3
398
399 import apt
400 import apt_pkg
401@@ -7,7 +7,7 @@
402 import unittest
403 import sys
404
405-from StringIO import StringIO
406+from io import StringIO
407
408 import unattended_upgrade
409 from unattended_upgrade import substitute, get_allowed_origins
410
411=== modified file 'unattended-upgrade'
412--- unattended-upgrade 2012-05-31 07:51:04 +0000
413+++ unattended-upgrade 2012-06-25 20:04:38 +0000
414@@ -1,4 +1,4 @@
415-#!/usr/bin/python
416+#!/usr/bin/python3
417 # Copyright (c) 2005-2012 Canonical Ltd
418 #
419 # AUTHOR:
420@@ -20,11 +20,10 @@
421 # along with unattended-upgrades; if not, write to the Free Software
422 # Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
423 #
424-
425 import apt_inst
426 import apt_pkg
427
428-import ConfigParser
429+import configparser
430 import copy
431 import datetime
432 import fcntl
433@@ -33,7 +32,7 @@
434 import string
435 import sys
436
437-from StringIO import StringIO
438+from io import StringIO
439 from optparse import OptionParser
440 from subprocess import Popen, PIPE
441
442@@ -112,9 +111,8 @@
443 LOG = PROGRESS_LOG
444
445 def status_change(self, pkg, percent, status):
446- f=open(self.LOG, "w")
447- f.write(_("Progress: %s %% (%s)") % (percent, pkg))
448- f.close()
449+ with open(self.LOG, "w") as f:
450+ f.write(_("Progress: %s %% (%s)") % (percent, pkg))
451
452 def _fixup_fds(self):
453 required_fds = [ 0, 1, 2, # stdin, stdout, stderr
454@@ -135,19 +133,19 @@
455 try:
456 fd = int(fdname)
457 except Exception as e:
458- print "ERROR: can not get fd for '%s'" % fdname
459+ print("ERROR: can not get fd for '%s'" % fdname)
460 if fd in required_fds:
461 continue
462 try:
463 os.close(fd)
464- #print "closed: ", fd
465+ #print("closed: ", fd)
466 except OSError as e:
467 # there will be one fd that can not be closed
468 # as its the fd from pythons internal diropen()
469 # so its ok to ignore one close error
470 error_count += 1
471 if error_count > 1:
472- print "ERROR: os.close(%s): %s" % (fd, e)
473+ print("ERROR: os.close(%s): %s" % (fd, e))
474
475 def fork(self):
476 pid = os.fork()
477@@ -270,7 +268,7 @@
478 try:
479 with Unlocked():
480 res = cache.commit(install_progress=iprogress)
481- except SystemError,e:
482+ except SystemError as e:
483 error = e
484 if res:
485 logging.info(_("All upgrades installed"))
486@@ -334,7 +332,7 @@
487 if not res:
488 raise Exception("cache.commit() returned false")
489 cache.open()
490- except Exception, e:
491+ except Exception as e:
492 logging.error(_("Installing the upgrades failed!"))
493 logging.error(_("error message: '%s'") % e)
494 logging.error(_("dpkg returned a error! See '%s' for details") % \
495@@ -390,7 +388,7 @@
496 control = apt_inst.DebFile(debfile).control.extractdata("control")
497 sections = apt_pkg.TagSection(control)
498 return sections["Package"]
499- except (IOError, SystemError), e:
500+ except (IOError, SystemError) as e:
501 logging.error("failed to read deb file '%s' (%s)" % (debfile, e))
502 # dumb fallback
503 return debfile.split("_")[0]
504@@ -408,9 +406,9 @@
505 conffiles = section.get("Conffiles")
506 # Conffiles:
507 # /etc/bash_completion.d/m-a c7780fab6b14d75ca54e11e992a6c11c
508- for line in string.split(conffiles,"\n"):
509+ for line in conffiles.splitlines():
510 logging.debug("conffile line: %s", line)
511- l = string.split(string.strip(line))
512+ l = line.strip().split()
513 conf_file = l[0]
514 md5 = l[1]
515 if len(l) > 2:
516@@ -428,7 +426,7 @@
517 # does not have conffiles anymore
518 deb = apt_inst.DebFile(destFile)
519 try:
520- pkg_conffiles = deb.control.extractdata("conffiles")
521+ pkg_conffiles = deb.control.extractdata("conffiles").decode('utf-8')
522 except LookupError as e:
523 logging.debug("No conffiles in %s anymore? (%s)" % (destFile, e))
524 pkg_conffiles = ""
525@@ -437,7 +435,8 @@
526 logging.debug("'%s' not in package conffiles '%s'" % (conf_file, pkg_conffiles))
527 continue
528 # test against the installed file
529- current_md5 = apt_pkg.md5sum(open(prefix+conf_file).read())
530+ with open(prefix+conf_file, 'rb') as fp:
531+ current_md5 = apt_pkg.md5sum(fp.read())
532 logging.debug("current md5: %s" % current_md5)
533 # hashes are the same, no conffile prompt
534 if current_md5 == md5:
535@@ -446,9 +445,12 @@
536 dpkg_cmd = ["dpkg-deb","--fsys-tarfile",destFile]
537 tar_cmd = ["tar","-x","-O", "-f","-", "."+conf_file]
538 md5_cmd = ["md5sum"]
539- dpkg_p = Popen(dpkg_cmd, stdout=PIPE)
540- tar_p = Popen(tar_cmd, stdin=dpkg_p.stdout, stdout=PIPE)
541- md5_p = Popen(md5_cmd, stdin=tar_p.stdout, stdout=PIPE)
542+ dpkg_p = Popen(dpkg_cmd, stdout=PIPE,
543+ universal_newlines=True)
544+ tar_p = Popen(tar_cmd, stdin=dpkg_p.stdout, stdout=PIPE,
545+ universal_newlines=True)
546+ md5_p = Popen(md5_cmd, stdin=tar_p.stdout, stdout=PIPE,
547+ universal_newlines=True)
548 pkg_md5sum = md5_p.communicate()[0].split()[0]
549 logging.debug("pkg_md5sum: %s" % pkg_md5sum)
550 # the md5sum in the deb is unchanged, this will not
551@@ -496,7 +498,7 @@
552 """ deal with apt-listchanges """
553 if os.path.exists(conf):
554 # check if mail is used by apt-listchanges
555- cf = ConfigParser.ConfigParser()
556+ cf = configparser.ConfigParser()
557 cf.read(conf)
558 if cf.has_section("apt") and cf.has_option("apt", "frontend"):
559 frontend = cf.get("apt","frontend")
560@@ -520,15 +522,16 @@
561 # mails on on errors, just exit here
562 if (res and
563 apt_pkg.config.find_b("Unattended-Upgrade::MailOnlyOnError", False)):
564- return
565+ return
566 # Check if reboot-required flag is present
567 logging.debug("Sending mail with '%s' to '%s'" % (logfile_dpkg, email))
568 if os.path.isfile(REBOOT_REQUIRED_FILE):
569 subject = _("[reboot required] unattended-upgrades result for '%s'") % host()
570 else:
571 subject = _("unattended-upgrades result for '%s'") % host()
572- mail = subprocess.Popen([MAIL_BINARY, "-s", subject,
573- email], stdin=subprocess.PIPE)
574+ mail = subprocess.Popen([MAIL_BINARY, "-s", subject, email],
575+ stdin=subprocess.PIPE,
576+ universal_newlines=True)
577 s = _("Unattended upgrade returned: %s\n\n") % res
578 if os.path.isfile(REBOOT_REQUIRED_FILE):
579 s += _("Warning: A reboot is required to complete this upgrade.\n\n")
580@@ -542,7 +545,8 @@
581 s += "\n"
582 if os.path.exists(logfile_dpkg):
583 s += _("Package installation log:")+"\n"
584- s += open(logfile_dpkg).read()
585+ with open(logfile_dpkg) as fp:
586+ s += fp.read()
587 s += "\n\n"
588 s += _("Unattended-upgrades log:\n")
589 s += mem_log.getvalue()
590@@ -569,7 +573,7 @@
591 old_stdout = 1
592 old_stderr = 2
593 else:
594- fd = os.open(logfile_dpkg, os.O_RDWR|os.O_CREAT, 0644)
595+ fd = os.open(logfile_dpkg, os.O_RDWR|os.O_CREAT, 0o644)
596 old_stdout = os.dup(1)
597 old_stderr = os.dup(2)
598 os.dup2(fd,1)
599@@ -580,7 +584,8 @@
600 # COMPAT with the mispelling
601 apt_pkg.config.find_b("Unattended-Upgrades::MinimalSteps", False) or
602 apt_pkg.config.find_b("Unattended-Upgrade::MinimalSteps", False)):
603- open("/var/run/unattended-upgrades.pid", "w").write("%s" % os.getpid())
604+ with open("/var/run/unattended-upgrades.pid", "w") as fp:
605+ fp.write("%s" % os.getpid())
606 # try upgrade all "pkgs" in minimal steps
607 pkg_install_success = upgrade_in_minimal_steps(
608 cache, [pkg.name for pkg in pkgs_to_upgrade], logfile_dpkg)
609@@ -677,7 +682,7 @@
610 lockfd = apt_pkg.get_lock(os.path.join(admindir, "lock"), False)
611 if lockfd > 0:
612 logging.warning(_("Unclean dpkg state detected, trying to correct"))
613- print _("Unclean dpkg state detected, trying to correct")
614+ print(_("Unclean dpkg state detected, trying to correct"))
615 env = copy.copy(os.environ)
616 env["DEBIAN_FRONTEND"] = "noninteractive"
617 try:
618@@ -693,17 +698,17 @@
619 # check and get lock
620 try:
621 apt_pkg.pkgsystem_lock()
622- except SystemError, e:
623+ except SystemError as e:
624 logging.error(_("Lock could not be acquired (another package "
625 "manager running?)"))
626- print _("Cache lock can not be acquired, exiting")
627+ print(_("Cache lock can not be acquired, exiting"))
628 sys.exit(1)
629
630 # get a cache
631 cache = UnattendedUpgradesCache(rootdir=rootdir,
632 allowed_origins=allowed_origins)
633 if cache._depcache.broken_count > 0:
634- print _("Cache has broken packages, exiting")
635+ print(_("Cache has broken packages, exiting"))
636 logging.error(_("Cache has broken packages, exiting"))
637 sys.exit(1)
638 # speed things up with latest apt
639@@ -739,7 +744,7 @@
640 logging.debug("sanity check failed")
641 rewind_cache(cache, pkgs_to_upgrade)
642 pkgs_kept_back.append(pkg.name)
643- except SystemError, e:
644+ except SystemError as e:
645 # can't upgrade
646 logging.warning(_("package '%s' upgradable but fails to be marked for upgrade (%s)") % (pkg.name, e))
647 rewind_cache(cache, pkgs_to_upgrade)
648@@ -761,7 +766,7 @@
649 pm = apt_pkg.PackageManager(cache._depcache)
650 try:
651 pm.get_archives(fetcher,list,recs)
652- except SystemError, e:
653+ except SystemError as e:
654 logging.error(_("GetArchives() failed: '%s'") % e)
655 res = fetcher.run()
656
657@@ -771,14 +776,14 @@
658 for item in fetcher.items:
659 logging.debug("%s" % item)
660 if item.status == item.STAT_ERROR:
661- print _("An error ocured: '%s'") % item.error_text
662+ print(_("An error ocured: '%s'") % item.error_text)
663 logging.error(_("An error ocured: '%s'") % item.error_text)
664 if not item.complete:
665- print _("The URI '%s' failed to download, aborting") % item.desc_uri
666+ print(_("The URI '%s' failed to download, aborting") % item.desc_uri)
667 logging.error(_("The URI '%s' failed to download, aborting") % item.desc_uri)
668 sys.exit(1)
669 if not os.path.exists(item.destfile):
670- print _("Download finished, but file '%s' not there?!?" % item.destfile)
671+ print(_("Download finished, but file '%s' not there?!?" % item.destfile))
672 logging.error("Download finished, but file '%s' not there?!?" % item.destfile)
673 sys.exit(1)
674 if not item.is_trusted:
675@@ -792,7 +797,7 @@
676 # the cron mail (only if no summary mail is requested)
677 email = apt_pkg.config.find("Unattended-Upgrade::Mail", "")
678 if not email:
679- print _("Package '%s' has conffile prompt and needs to be upgraded manually") % pkgname_from_deb(item.destfile)
680+ print(_("Package '%s' has conffile prompt and needs to be upgraded manually") % pkgname_from_deb(item.destfile))
681 # log to the logfile
682 logging.warning(_("Package '%s' has conffile prompt and needs to be upgraded manually") % pkgname_from_deb(item.destfile))
683 blacklisted_pkgs.append(pkgname_from_deb(item.destfile))
684@@ -907,7 +912,7 @@
685 (options, args) = parser.parse_args()
686
687 if os.getuid() != 0:
688- print _("You need to be root to run this application")
689+ print(_("You need to be root to run this application"))
690 sys.exit(1)
691
692 # nice & ionce
693
694=== modified file 'unattended-upgrade-shutdown'
695--- unattended-upgrade-shutdown 2012-04-26 11:40:46 +0000
696+++ unattended-upgrade-shutdown 2012-06-25 20:04:38 +0000
697@@ -1,4 +1,4 @@
698-#!/usr/bin/python
699+#!/usr/bin/python3
700 # Copyright (c) 2009 Canonical Ltd
701 #
702 # AUTHOR:
703@@ -141,8 +141,8 @@
704 os.kill(pid, signal.SIGUSR1)
705 # show log
706 log_progress()
707- time.sleep(5)
708- if (time.time() - start_time) > options.delay*60:
709+ time.sleep(5)
710+ if (time.time() - start_time) > options.delay*60:
711 logging.warning(_("Giving up on lockfile after %s delay") % options.delay)
712 sys.exit(1)
713

Subscribers

People subscribed via source and target branches

to all changes: