Merge ~cjwatson/turnip:twisted-16.5.0 into turnip:master

[Colin Watson]

Upgrade to Twisted 16.5.0

cryptography requires libssl-dev as a system dependency, as well as pip
and setuptools upgrades to parse a new environment marker. The
twisted.web.http.Request changes in 16.3.0 require a small change to
HTTPPackClientProtocol.connectionLost, since trying to call
unregisterProducer on a finished request now crashes.

This requires https://code.launchpad.net/~cjwatson/turnip/+git/dependencies/+merge/339506, and I picked the target version to match https://code.launchpad.net/~cjwatson/launchpad/twisted-16.5.0/+merge/335534.

[William Grant (wgrant)]

review: Approve (code)

[Otto Co-Pilot (otto-copilot)]

Running landing tests failed
https://jenkins.ols.canonical.com/online-services/job/turnip/20/

diff --git a/bootstrap-requirements.txt b/bootstrap-requirements.txt
index 566a7b5..880e758 100644
--- a/bootstrap-requirements.txt
+++ b/bootstrap-requirements.txt
@@ -1,2 +1,2 @@
-pip==7.1.2
-setuptools==18.4
+pip==9.0.1
+setuptools==38.5.1
diff --git a/requirements.txt b/requirements.txt
index 914b1df..45c6863 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,15 +1,23 @@
+appdirs==1.4.3
+asn1crypto==0.24.0
 beautifulsoup4==4.3.2
-cffi==0.9.1
+cffi==1.11.4
+constantly==15.1.0
 contextlib2==0.4.0
 cornice==0.19.0
+cryptography==2.1.4
 docutils==0.12
+enum34==1.1.6
 envdir==0.7
 extras==0.0.3
 fixtures==1.0.0
 flake8==2.4.0
 gmpy==1.17
 gunicorn==19.3.0
-lazr.sshserver==0.1.4
+idna==2.6
+incremental==17.5.0
+ipaddress==1.0.19
+lazr.sshserver==0.1.7
 linecache2==1.0.0
 mccabe==0.3
 Paste==2.0.2
@@ -34,7 +42,7 @@ six==1.9.0
 testtools==1.8.1
 traceback2==1.4.0
 translationstring==1.3
-Twisted==15.5.0
+Twisted[conch]==16.5.0
 unittest2==1.0.1
 venusian==1.0
 waitress==0.8.9
diff --git a/setup.py b/setup.py
index 36a1156..38b6681 100755
--- a/setup.py
+++ b/setup.py
@@ -20,12 +20,12 @@ with open(os.path.join(here, 'NEWS')) as f:
 requires = [
     'contextlib2',
     'cornice',
-    'lazr.sshserver>=0.1.2',
+    'lazr.sshserver>=0.1.7',
     'Paste',
     'pygit2>=0.24.0,<0.25.0',
     'python-openid',
     'PyYAML',
-    'Twisted',
+    'Twisted[conch]',
     'waitress',
     'zope.interface',
     ]
diff --git a/system-dependencies.txt b/system-dependencies.txt
index 818f04f..092754d 100644
--- a/system-dependencies.txt
+++ b/system-dependencies.txt
@@ -3,5 +3,6 @@ cgit
 git
 libffi-dev
 libgit2-24
+libssl-dev
 python-dev
 python-virtualenv
diff --git a/turnip/pack/http.py b/turnip/pack/http.py
index 42fed27..99635ff 100644
--- a/turnip/pack/http.py
+++ b/turnip/pack/http.py
@@ -182,8 +182,8 @@ class HTTPPackClientProtocol(PackProtocol):
             self.factory.http_request.write(data)
 
     def connectionLost(self, reason):
-        self.factory.http_request.unregisterProducer()
         if not self.factory.http_request.finished:
+            self.factory.http_request.unregisterProducer()
             if reason.check(error.ConnectionDone):
                 # We assume that the backend will have sent an error if
                 # necessary; otherwise an empty response is permitted (and