Merge lp:~cjwatson/launchpad/twisted-16.5.0 into lp:launchpad

Proposed by Colin Watson on 2017-12-21
Status: Needs review
Proposed branch: lp:~cjwatson/launchpad/twisted-16.5.0
Merge into: lp:launchpad
Prerequisite: lp:~cjwatson/launchpad/modern-testtools-cleanup
Diff against target: 187 lines (+41/-29)
4 files modified
constraints.txt (+22/-12)
lib/lp/codehosting/tests/test_acceptance.py (+0/-2)
lib/lp/services/librarianserver/testing/server.py (+18/-14)
lib/lp/services/twistedsupport/tests/test_gracefulshutdown.py (+1/-1)
To merge this branch: bzr merge lp:~cjwatson/launchpad/twisted-16.5.0
Reviewer Review Type Date Requested Status
Launchpad code reviewers 2017-12-21 Pending
Review via email: mp+335534@code.launchpad.net

Commit Message

Upgrade to Twisted 16.5.0.

Description of the Change

As well as the various upgrades associated with Twisted itself, we also need lazr.sshserver 0.1.5 and txfixtures 0.4.1 for compatibility.

Going any further than 16.5.0 will require some performance testing due to https://twistedmatrix.com/trac/ticket/8079; we'll probably need to either monkey-patch gmpy into place ourselves or implement the suggested DH_compute_key changes in Twisted.

Since cryptography is rather stricter than PyCrypto, we should scan for registered SSH public keys it doesn't like before landing this and notify their owners: for example, very short DSA keys will now be refused.

To post a comment you must log in.

Unmerged revisions

18530. By Colin Watson on 2017-12-21

Upgrade to Twisted 16.5.0.

As well as the various upgrades associated with Twisted itself, we also need
lazr.sshserver 0.1.5 and txfixtures 0.4.1 for compatibility.

18529. By Colin Watson on 2017-12-21

Pass a proper channel object to http.Request.

18528. By Colin Watson on 2017-12-21

Remove atexit registration; SSHServerLayer.tearDown should be enough, and the extra registration causes duplicate teardown.

18527. By Colin Watson on 2017-12-19

Simplify BufferLogger.content.

18526. By Colin Watson on 2017-12-19

Import from the new testtools.twistedsupport rather than testtools.deferredruntest.

18525. By Colin Watson on 2017-12-19

Simplify some tests using TempDir.join.

18524. By Colin Watson on 2017-12-19

Upgrade to testtools 2.3.0 and fixtures 3.0.0.

18523. By Colin Watson on 2017-12-19

Just use "with fixture:" rather than "with self.useFixture(fixture):".

The latter form is redundant, and causes problems with newer versions of
fixtures.

18522. By Colin Watson on 2017-12-19

Remove security proxies before passing objects to assertDictEqual.

18521. By Colin Watson on 2017-12-19

Remove some stray calls to test case setUp methods.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'constraints.txt'
2--- constraints.txt 2017-12-21 21:29:58 +0000
3+++ constraints.txt 2017-12-21 21:29:59 +0000
4@@ -225,6 +225,9 @@
5 amqp==1.4.9
6 amqplib==1.0.2
7 anyjson==0.3.3
8+appdirs==1.4.3
9+asn1crypto==0.23.0
10+attrs==17.3.0
11 auditor==0.0.3
12 auditorclient==0.0.4
13 auditorfixture==0.0.7
14@@ -235,7 +238,10 @@
15 bson==0.3.3
16 bzr==2.6.0.lp.2
17 celery==3.1.18
18+cffi==1.11.2
19 Chameleon==2.11
20+constantly==15.1.0
21+cryptography==2.1.4
22 cssselect==0.9.1
23 cssutils==0.9.10
24 d2to1==0.2.12
25@@ -244,17 +250,22 @@
26 # Required by dkimpy
27 dnspython==1.10.0
28 elementtree==1.2.6-20050316
29+enum34==1.1.6
30 epydoc==3.0.1
31 extras==1.0.0
32 feedparser==4.1
33 feedvalidator==0.0.0DEV-r1049
34 fixtures==3.0.0
35 FormEncode==1.2.4
36+gmpy==1.17
37 grokcore.component==1.6
38 html5browser==0.0.9
39 httmock==1.2.3
40 httplib2==0.8
41+idna==2.6
42 importlib==1.0.2
43+incremental==17.5.0
44+ipaddress==1.0.18
45 ipython==0.13.2
46 iso8601==0.1.12
47 jsautobuild==0.2
48@@ -272,7 +283,7 @@
49 lazr.restful==0.20.0
50 lazr.restfulclient==0.13.2
51 lazr.smtptest==1.3
52-lazr.sshserver==0.1.3
53+lazr.sshserver==0.1.5
54 lazr.testing==0.1.1
55 lazr.uri==1.0.3
56 libnacl==1.3.6
57@@ -297,13 +308,16 @@
58 pbr==0.11.1
59 pgbouncer==0.0.8
60 prettytable==0.7.2
61+psutil==5.4.2
62 psycopg2==2.6.1
63-pyasn1==0.1.6
64+pyasn1==0.4.2
65+pyasn1-modules==0.2.1
66+pycparser==2.18
67 pycrypto==2.6
68 pygpgme==0.2
69 pyinotify==0.9.4
70 pymacaroons==0.9.2
71-pyOpenSSL==0.13
72+pyOpenSSL==17.5.0
73 pystache==0.5.3
74 python-dateutil==1.5
75 python-debian==0.1.23
76@@ -319,11 +333,12 @@
77 rabbitfixture==0.3.6
78 requests==2.7.0
79 requests-toolbelt==0.6.2
80+service-identity==17.0.0
81 setproctitle==1.1.7
82 setuptools-git==1.2
83 simplejson==3.8.2
84 SimpleTAL==4.3
85-six==1.9.0
86+six==1.11.0
87 soupmatchers==0.2
88 # lp:~launchpad-committers/storm/with-without-datetime
89 storm==0.19.0.99-lpwithnodatetime-r408
90@@ -332,18 +347,13 @@
91 testresources==0.2.7
92 testscenarios==0.4
93 timeline==0.0.3
94-# Build of lp:~canonical-launchpad-branches/twisted:lp-backport.
95-# p1 Support diffie-hellman-group14-sha1 key exchange in conch.ssh.
96-# p2 Add diffie-hellman-group-exchange-sha256 to twisted.conch.ssh.
97-# Add support in twisted.conch.ssh for hmac-sha2-256 and hmac-sha2-512.
98-# post3 Use PEP 440-compliant version.
99-Twisted==13.0.0post3
100+Twisted[conch,tls]==16.5.0
101 txAMQP==0.6.2
102-txfixtures==0.1.4
103+txfixtures==0.4.1
104 txlongpoll==0.2.12
105 txlongpollfixture==0.1.3
106 txpkgupload==0.2
107-unittest2==0.5.1
108+unittest2==1.1.0
109 van.testing==3.0.0
110 virtualenv-tools3==2.0.0
111 wadllib==1.3.2
112
113=== modified file 'lib/lp/codehosting/tests/test_acceptance.py'
114--- lib/lp/codehosting/tests/test_acceptance.py 2016-12-31 05:01:10 +0000
115+++ lib/lp/codehosting/tests/test_acceptance.py 2017-12-21 21:29:59 +0000
116@@ -5,7 +5,6 @@
117
118 __metaclass__ = type
119
120-import atexit
121 import os
122 import re
123 import signal
124@@ -156,7 +155,6 @@
125 tac_handler = SSHServerLayer.getTacHandler()
126 tac_handler.setUp()
127 SSHServerLayer._reset()
128- atexit.register(tac_handler.tearDown)
129 forker = SSHServerLayer.getForker()
130 forker.setUp()
131
132
133=== modified file 'lib/lp/services/librarianserver/testing/server.py'
134--- lib/lp/services/librarianserver/testing/server.py 2016-06-02 17:52:38 +0000
135+++ lib/lp/services/librarianserver/testing/server.py 2017-12-21 21:29:59 +0000
136@@ -155,20 +155,24 @@
137 return
138 chunks = self.getLogChunks()
139 # A typical startup: upload, download, restricted up, restricted down.
140- #2010-10-20 14:28:21+0530 [-] Log opened.
141- #2010-10-20 14:28:21+0530 [-] twistd 10.1.0 (/usr/bin/python 2.6.5) starting up.
142- #2010-10-20 14:28:21+0530 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
143- #2010-10-20 14:28:21+0530 [-] lp.services.librarianserver.libraryprotocol.FileUploadFactory starting on 59090
144- #2010-10-20 14:28:21+0530 [-] Starting factory <lp.services.librarianserver.libraryprotocol.FileUploadFactory instance at 0x6f8ff38>
145- #2010-10-20 14:28:21+0530 [-] twisted.web.server.Site starting on 58000
146- #2010-10-20 14:28:21+0530 [-] Starting factory <twisted.web.server.Site instance at 0x6fb2638>
147- #2010-10-20 14:28:21+0530 [-] lp.services.librarianserver.libraryprotocol.FileUploadFactory starting on 59095
148- #2010-10-20 14:28:21+0530 [-] Starting factory <lp.services.librarianserver.libraryprotocol.FileUploadFactory instance at 0x6fb25f0>
149- #2010-10-20 14:28:21+0530 [-] twisted.web.server.Site starting on 58005
150- self.upload_port = int(chunks[3].split()[-1])
151- self.download_port = int(chunks[5].split()[-1])
152- self.restricted_upload_port = int(chunks[7].split()[-1])
153- self.restricted_download_port = int(chunks[9].split()[-1])
154+ #2017-12-09 06:33:13+0530 [-] Loading /home/ubuntu/launchpad/lp-branches/devel/daemons/librarian.tac...
155+ #2017-12-09 06:33:19+0530 [-] Loaded.
156+ #2017-12-09 06:33:19+0530 [-] twistd 16.5.0 (/home/ubuntu/launchpad/lp-branches/devel/env/bin/python2.7 2.7.12) starting up.
157+ #2017-12-09 06:33:19+0530 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
158+ #2017-12-09 06:33:19+0530 [-] FileUploadFactory starting on 39851
159+ #2017-12-09 06:33:19+0530 [-] Starting factory <lp.services.librarianserver.libraryprotocol.FileUploadFactory instance at 0xf0ef80ac>
160+ #2017-12-09 06:33:19+0530 [-] Site starting on 45355
161+ #2017-12-09 06:33:19+0530 [-] Starting factory <twisted.web.server.Site instance at 0xf0ef848c>
162+ #2017-12-09 06:33:19+0530 [-] FileUploadFactory starting on 42150
163+ #2017-12-09 06:33:19+0530 [-] Starting factory <lp.services.librarianserver.libraryprotocol.FileUploadFactory instance at 0xf0ef856c>
164+ #2017-12-09 06:33:19+0530 [-] Site starting on 37341
165+ #2017-12-09 06:33:19+0530 [-] Starting factory <twisted.web.server.Site instance at 0xf0ef85ec>
166+ #2017-12-09 06:33:19+0530 [-] Not using upstream librarian
167+ #2017-12-09 06:33:19+0530 [-] daemon ready!
168+ self.upload_port = int(chunks[4].split()[-1])
169+ self.download_port = int(chunks[6].split()[-1])
170+ self.restricted_upload_port = int(chunks[8].split()[-1])
171+ self.restricted_download_port = int(chunks[10].split()[-1])
172 self.service_config = dedent("""\
173 [librarian_server]
174 root: %s
175
176=== modified file 'lib/lp/services/twistedsupport/tests/test_gracefulshutdown.py'
177--- lib/lp/services/twistedsupport/tests/test_gracefulshutdown.py 2015-10-14 15:22:01 +0000
178+++ lib/lp/services/twistedsupport/tests/test_gracefulshutdown.py 2017-12-21 21:29:59 +0000
179@@ -99,7 +99,7 @@
180
181 def make_dummy_http_request(self):
182 """Make a dummy HTTP request for tests."""
183- return http.Request('fake channel', True)
184+ return http.Request(http.HTTPChannel(), True)
185
186 def test_200_when_available(self):
187 """When the factory is available a 200 response is generated."""