Merge ~bjornt/maas:bug-2066276-3.4 into maas:3.4

[Björn Tillenius]

Bug #2066276: ipv6 test failures: AttributeError: 'RRHeader' object has no attribute '_address'

fix: parse DNS results correctly

(cherry picked from commit 4f6ef7d4e2feab5445b39330949ed5a13d10d526)
(cherry picked from commit 557d3f62aa3399150b84cfc95949cd3416c6ce04)

Limit test processes in parallel to 4 by default.

It's possible to change the number of parallel processes by passing
in the MAAS_MAX_TEST_PROCESSES environment variable. For example:
MAAS_MAX_TEST_PROCESSES=8 utilities/run-py-tests-ci

This aims to reduce the number of intermittent failures in our CI.

(cherry picked from commit b2df4b603143d36e95b7e0d8e6b442fd2ac7c8ed)

Apply intermittent test failure fix from e4d49a5503.

[Björn Tillenius (bjornt)]

Self-approve backport

review: Approve

[MAAS Lander (maas-lander)]

LANDING
-b bug-2066276-3.4 lp:~bjornt/maas/+git/maas into -b 3.4 lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci.internal:8080/job/maas-tester/5586/console

[MAAS Lander (maas-lander)]

UNIT TESTS
-b bug-2066276-3.4 lp:~bjornt/maas/+git/maas into -b 3.4 lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/5587/console
COMMIT: 941ca1710023763cde8b50895a181b987282bd17

review: Needs Fixing

[MAAS Lander (maas-lander)]

LANDING
-b bug-2066276-3.4 lp:~bjornt/maas/+git/maas into -b 3.4 lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci.internal:8080/job/maas-tester/5588/console

[MAAS Lander (maas-lander)]

LANDING
-b bug-2066276-3.4 lp:~bjornt/maas/+git/maas into -b 3.4 lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci.internal:8080/job/maas-tester/5589/console

[MAAS Lander (maas-lander)]

LANDING
-b bug-2066276-3.4 lp:~bjornt/maas/+git/maas into -b 3.4 lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci.internal:8080/job/maas-tester/5591/console

[Björn Tillenius (bjornt)]

I had to pull in b2df4b603143d36e95b7e0d8e6b442fd2ac7c8ed as well, in order to get the tests to pass.

[MAAS Lander (maas-lander)]

LANDING
-b bug-2066276-3.4 lp:~bjornt/maas/+git/maas into -b 3.4 lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci.internal:8080/job/maas-tester/5615/console

[MAAS Lander (maas-lander)]

UNIT TESTS
-b bug-2066276-3.4 lp:~bjornt/maas/+git/maas into -b 3.4 lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/5617/console
COMMIT: a38369c18761084dc7d6e5cccb0c548ed3647061

review: Needs Fixing

[MAAS Lander (maas-lander)]

LANDING
-b bug-2066276-3.4 lp:~bjornt/maas/+git/maas into -b 3.4 lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci.internal:8080/job/maas-tester/5626/console

6c1e6d8... by Björn Tillenius on 2024-05-27

Apply intermittent test failure fix from e4d49a5503.

diff --git a/src/maasserver/forms/tests/test_bootresource.py b/src/maasserver/forms/tests/test_bootresource.py
index 4a3c51e..1cb5d74 100644
--- a/src/maasserver/forms/tests/test_bootresource.py
+++ b/src/maasserver/forms/tests/test_bootresource.py
@@ -134,6 +134,7 @@ class TestBootResourceForm(MAASServerTestCase):
     def test_prevents_reversed_osystem_from_driver(self):
         reserved_name = factory.make_name("name")
         OperatingSystemRegistry.register_item(reserved_name, CustomOS())
+        self.addCleanup(OperatingSystemRegistry.unregister_item, reserved_name)
         upload_type, filetype = self.pick_filetype()
         size = random.randint(1024, 2048)
         content = factory.make_string(size).encode("utf-8")
diff --git a/src/provisioningserver/utils/network.py b/src/provisioningserver/utils/network.py
index 1ea4ec0..6124f68 100644
--- a/src/provisioningserver/utils/network.py
+++ b/src/provisioningserver/utils/network.py
@@ -29,6 +29,7 @@ import netifaces
 from twisted.internet.defer import Deferred, inlineCallbacks
 from twisted.internet.interfaces import IResolver
 from twisted.names.client import getResolver
+from twisted.names.dns import AAAA
 from twisted.names.error import (
     AuthoritativeDomainError,
     DNSQueryTimeoutError,
@@ -772,8 +773,15 @@ def safe_getaddrinfo(
         resolver = getResolver()
         answers = yield resolver.lookupIPV6Address(hostname)
         return [
-            (AF_INET6, sock_type, proto, "", (ans._address, port, 0, 0))
+            (
+                AF_INET6,
+                sock_type,
+                proto,
+                "",
+                (ans.payload._address, port, 0, 0),
+            )
             for ans in answers[0]
+            if ans.type == AAAA
         ]
 
     try:
diff --git a/src/provisioningserver/utils/tests/test_network.py b/src/provisioningserver/utils/tests/test_network.py
index 1ed3e6e..24fec44 100644
--- a/src/provisioningserver/utils/tests/test_network.py
+++ b/src/provisioningserver/utils/tests/test_network.py
@@ -24,7 +24,7 @@ from testtools.matchers import (
     StartsWith,
 )
 from twisted.internet.defer import inlineCallbacks, succeed
-from twisted.names.dns import Record_AAAA
+from twisted.names.dns import AAAA, Record_AAAA, RRHeader
 from twisted.names.error import (
     AuthoritativeDomainError,
     DNSQueryTimeoutError,
@@ -2568,7 +2568,11 @@ class TestSafeGetaddrinfo(MAASTestCase):
         mock_resolver = Mock()
         mock_resolver.lookupIPV6Address = lambda _: succeed(
             (
-                [Record_AAAA(address="::")],
+                [
+                    RRHeader(
+                        name="::", type=AAAA, payload=Record_AAAA(address="::")
+                    )
+                ],
                 [],
                 [],
             )
@@ -2585,7 +2589,13 @@ class TestSafeGetaddrinfo(MAASTestCase):
         mock_resolver = Mock()
         mock_resolver.lookupIPV6Address = lambda _: succeed(
             (
-                [Record_AAAA(address="::1")],
+                [
+                    RRHeader(
+                        name="example.com",
+                        type=AAAA,
+                        payload=Record_AAAA(address="::1"),
+                    )
+                ],
                 [],
                 [],
             )
diff --git a/utilities/run-py-tests-ci b/utilities/run-py-tests-ci
index 5113d55..c8a8ff5 100755
--- a/utilities/run-py-tests-ci
+++ b/utilities/run-py-tests-ci
@@ -7,18 +7,21 @@
 
 set -o pipefail
 
+MAX_PROCESSES=${MAAS_MAX_TEST_PROCESSES:-4}
+
 MAAS_RAND_SEED="$(od -vAn -N8 -tx8 < /dev/urandom | tr -d ' ')"
 PYTHONHASHSEED="$(shuf -i 0-4294967295 -n 1)"
 export MAAS_RAND_SEED PYTHONHASHSEED
 
 echo "MAAS_RAND_SEED=${MAAS_RAND_SEED}"
 echo "PYTHONHASHSEED=${PYTHONHASHSEED}"
+echo "MAX_PROCESSES=${MAX_PROCESSES}"
 
-bin/test.parallel --emit-subunit | \
+bin/test.parallel --emit-subunit --subprocesses="${MAX_PROCESSES}" | \
     bin/subunit-1to2 | \
     bin/subunit2junitxml --no-passthrough -f -o junit.xml | \
     bin/subunit2pyunit --no-passthrough
 res1=$?
-bin/pytest -n auto --maxprocesses=6 --dist=loadscope --junit-xml=junit-pytest.xml
+bin/pytest -n auto --maxprocesses="${MAX_PROCESSES}" --dist=loadscope --junit-xml=junit-pytest.xml
 res2=$?
 exit $((res1 + res2))