Merge ~d0ugal/maas:twisted-patch-disconnect into maas:master

[Dougal Matthews]

LP 1741913: Patch twisted.web.http.Request to include the upstream fix

See https://github.com/twisted/twisted/commit/169fd1d93b7af06bf0f6893b193ce19970881868

d18058d... by Dougal Matthews on 2020-10-29

Make a note about the test origin and isort

[MAAS Lander (maas-lander)]

UNIT TESTS
-b twisted-patch-disconnect lp:~d0ugal/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/8574/console
COMMIT: 448505e89982e3a46784f4e14cbfb6093bd1cbde

review: Needs Fixing

[MAAS Lander (maas-lander)]

UNIT TESTS
-b twisted-patch-disconnect lp:~d0ugal/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/8575/console
COMMIT: d18058d203a07445b62414d854d1afcbb43e76a4

review: Needs Fixing

073b2d2... by Dougal Matthews on 2020-10-29

Return the parent. Not sure if this is needed.

016494d... by Dougal Matthews on 2020-10-29

Don't use super, is there a nicer way?

1474744... by Dougal Matthews on 2020-10-29

Blacken

[MAAS Lander (maas-lander)]

UNIT TESTS
-b twisted-patch-disconnect lp:~d0ugal/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/8576/console
COMMIT: 073b2d2f35193a896dfb358afb71f52b42e09d17

review: Needs Fixing

[MAAS Lander (maas-lander)]

UNIT TESTS
-b twisted-patch-disconnect lp:~d0ugal/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: 1474744a2d0e4521563936fdce61600f82bca34a

review: Approve

[Adam Collard (adam-collard)]

review: Needs Information

1d85128... by Dougal Matthews on 2020-10-29

Fix typo and add a test to check the version of Twisted needs patched

ae6faa3... by Dougal Matthews on 2020-10-29

Fix comments

bdd78ce... by Dougal Matthews on 2020-10-29

Update the test name

[MAAS Lander (maas-lander)]

UNIT TESTS
-b twisted-patch-disconnect lp:~d0ugal/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas/job/branch-tester/8581/console
COMMIT: 1d85128bec0c8c965ddff517ce75e2e0348aa53e

review: Needs Fixing

[Dougal Matthews (d0ugal)]

jenkins: !test

[MAAS Lander (maas-lander)]

UNIT TESTS
-b twisted-patch-disconnect lp:~d0ugal/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: bdd78ce833666a2bcf07d748dc1c7978cfe37294

review: Approve

[Adam Collard (adam-collard)]

+1

review: Approve

diff --git a/src/maasserver/monkey.py b/src/maasserver/monkey.py
index d60c612..1b4be21 100644
--- a/src/maasserver/monkey.py
+++ b/src/maasserver/monkey.py
@@ -1,4 +1,4 @@
-# Copyright 2016-2017 Canonical Ltd.  This software is licensed under the
+# Copyright 2016-2020 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """
@@ -10,6 +10,7 @@ __all__ = ["add_patches"]
 from collections import OrderedDict
 import inspect
 
+from twisted.web import http
 import yaml
 
 from provisioningserver.monkey import add_patches_to_twisted
@@ -95,9 +96,40 @@ def fix_ordereddict_yaml_representer():
     yaml.add_representer(OrderedDict, dumper, Dumper=yaml.SafeDumper)
 
 
+OriginalRequest = http.Request
+
+
+class PatchedRequest(OriginalRequest):
+    def write(self, data):
+        if self.finished:
+            raise RuntimeError(
+                "Request.write called on a request after "
+                "Request.finish was called."
+            )
+
+        if self._disconnected:
+            # Don't attempt to write any data to a disconnected client.
+            # The RuntimeError exception will be thrown as usual when
+            # request.finish is called
+            return
+
+        return OriginalRequest.write(self, data)
+
+
+def fix_twisted_disconnect_write():
+    """
+    Patch twisted.web.http.Request to include the upstream fix
+
+    See https://github.com/twisted/twisted/commit/169fd1d93b7af06bf0f6893b193ce19970881868
+    """
+
+    http.Request = PatchedRequest
+
+
 def add_patches():
     add_patches_to_twisted()
     fix_django_deferred_attribute()
     fix_piston_emitter_related()
     fix_piston_consumer_delete()
     fix_ordereddict_yaml_representer()
+    fix_twisted_disconnect_write()
diff --git a/src/maasserver/tests/test_monkey.py b/src/maasserver/tests/test_monkey.py
new file mode 100644
index 0000000..408af95
--- /dev/null
+++ b/src/maasserver/tests/test_monkey.py
@@ -0,0 +1,40 @@
+# Copyright 2020 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Test monkey patches."""
+
+__all__ = []
+
+from twisted import version as twisted_version
+from twisted.internet.error import ConnectionLost
+from twisted.python.failure import Failure
+from twisted.web import http
+from twisted.web.test.requesthelper import DummyChannel
+
+from maastesting.testcase import MAASTestCase
+
+
+class TestTwistedDisconnectPatch(MAASTestCase):
+    def test_write_after_connection_lost(self):
+        """
+        Calling L{Request.write} after L{Request.connectionLost} has been
+        called should not throw an exception. L{RuntimeError} will be raised
+        when finish is called on the request.
+
+        NOTE: This test is taken from the upstream fix to verify the monkey
+              patch: https://github.com/twisted/twisted/commit/169fd1d93b7af06bf0f6893b193ce19970881868
+        """
+        channel = DummyChannel()
+        req = http.Request(channel, False)
+        req.connectionLost(Failure(ConnectionLost("The end.")))
+        req.write(b"foobar")
+        self.assertRaises(RuntimeError, req.finish)
+
+    def test_twisted_version(self):
+
+        self.assertLessEqual(
+            (twisted_version.major, twisted_version.minor),
+            (19, 7),
+            "The fix_twisted_disconnect_write monkey patch is not longer "
+            "required in Twisted versions >= 19.7.0",
+        )