Merge ~cgrabowski/maas:fix_overlapping_subnets_bind_misconfigure into maas:master

Proposed by Christian Grabowski
Status: Merged
Approved by: Christian Grabowski
Approved revision: 40348f8e30e9ffebe508d36d3d64ce712b60c00a
Merge reported by: MAAS Lander
Merged at revision: not available
Proposed branch: ~cgrabowski/maas:fix_overlapping_subnets_bind_misconfigure
Merge into: maas:master
Diff against target: 1493 lines (+876/-360)
5 files modified
src/maasserver/dns/tests/test_zonegenerator.py (+570/-55)
src/maasserver/dns/zonegenerator.py (+259/-96)
src/maastesting/noseplug.py (+3/-1)
src/provisioningserver/dns/tests/test_zoneconfig.py (+26/-152)
src/provisioningserver/dns/zoneconfig.py (+18/-56)
Reviewer Review Type Date Requested Status
MAAS Lander Approve
Jack Lloyd-Walters Approve
Review via email: mp+457267@code.launchpad.net

Commit message

fix: overlapping subnets no longer cause BIND to fail configuring

To post a comment you must log in.
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b fix_overlapping_subnets_bind_misconfigure lp:~cgrabowski/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/4217/console
COMMIT: 8498fd5425c23a1944ec179a77f0723727071874

review: Needs Fixing
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b fix_overlapping_subnets_bind_misconfigure lp:~cgrabowski/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/4218/console
COMMIT: 8bff38ac17e03b0c82b55e138ed1dda98de51316

review: Needs Fixing
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b fix_overlapping_subnets_bind_misconfigure lp:~cgrabowski/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/4226/console
COMMIT: 9fb26758287ea506e81b57e4e2b5756b153aba82

review: Needs Fixing
Revision history for this message
Christian Grabowski (cgrabowski) wrote :

jenkins: !test

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b fix_overlapping_subnets_bind_misconfigure lp:~cgrabowski/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/4228/console
COMMIT: 9fb26758287ea506e81b57e4e2b5756b153aba82

review: Needs Fixing
Revision history for this message
Christian Grabowski (cgrabowski) wrote :

jenkins: !test

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b fix_overlapping_subnets_bind_misconfigure lp:~cgrabowski/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: 9fb26758287ea506e81b57e4e2b5756b153aba82

review: Approve
Revision history for this message
Jack Lloyd-Walters (lloydwaltersj) wrote :

Looks good, a few inline nits about changing to f-strings, but nothing that affects functionality.

Revision history for this message
Jack Lloyd-Walters (lloydwaltersj) :
review: Approve
Revision history for this message
Christian Grabowski (cgrabowski) :
d68d1b0... by Christian Grabowski

fix nits

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b fix_overlapping_subnets_bind_misconfigure lp:~cgrabowski/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/4303/console
COMMIT: 2fc97d3aa94813ebe4fbddb1ac1768164ec4cc1e

review: Needs Fixing
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b fix_overlapping_subnets_bind_misconfigure lp:~cgrabowski/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/4307/console
COMMIT: d68d1b0b3897a05b00d8c33941d6443f607641a6

review: Needs Fixing
40348f8... by Christian Grabowski

fix flaky test

Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b fix_overlapping_subnets_bind_misconfigure lp:~cgrabowski/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: FAILED
LOG: http://maas-ci.internal:8080/job/maas-tester/4309/console
COMMIT: 654834eb173923c85697a3d1785303fca49957c0

review: Needs Fixing
Revision history for this message
MAAS Lander (maas-lander) wrote :

UNIT TESTS
-b fix_overlapping_subnets_bind_misconfigure lp:~cgrabowski/maas/+git/maas into -b master lp:~maas-committers/maas

STATUS: SUCCESS
COMMIT: 40348f8e30e9ffebe508d36d3d64ce712b60c00a

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/src/maasserver/dns/tests/test_zonegenerator.py b/src/maasserver/dns/tests/test_zonegenerator.py
index d213c1a..1555ae6 100644
--- a/src/maasserver/dns/tests/test_zonegenerator.py
+++ b/src/maasserver/dns/tests/test_zonegenerator.py
@@ -1,6 +1,7 @@
1# Copyright 2014-2022 Canonical Ltd. This software is licensed under the1# Copyright 2014-2022 Canonical Ltd. This software is licensed under the
2# GNU Affero General Public License version 3 (see the file LICENSE).2# GNU Affero General Public License version 3 (see the file LICENSE).
33
4import os
4import random5import random
5import socket6import socket
6from unittest.mock import ANY, call, Mock7from unittest.mock import ANY, call, Mock
@@ -37,7 +38,10 @@ from maasserver.enum import IPADDRESS_TYPE, NODE_STATUS, RDNS_MODE
37from maasserver.exceptions import UnresolvableHost38from maasserver.exceptions import UnresolvableHost
38from maasserver.models import Config, Domain, Subnet39from maasserver.models import Config, Domain, Subnet
39from maasserver.models.dnsdata import HostnameRRsetMapping40from maasserver.models.dnsdata import HostnameRRsetMapping
40from maasserver.models.staticipaddress import HostnameIPMapping41from maasserver.models.staticipaddress import (
42 HostnameIPMapping,
43 StaticIPAddress,
44)
41from maasserver.testing.config import RegionConfigurationFixture45from maasserver.testing.config import RegionConfigurationFixture
42from maasserver.testing.factory import factory46from maasserver.testing.factory import factory
43from maasserver.testing.testcase import (47from maasserver.testing.testcase import (
@@ -49,6 +53,7 @@ from maastesting.factory import factory as maastesting_factory
49from maastesting.fakemethod import FakeMethod53from maastesting.fakemethod import FakeMethod
50from maastesting.matchers import MockAnyCall, MockCalledOnceWith, MockNotCalled54from maastesting.matchers import MockAnyCall, MockCalledOnceWith, MockNotCalled
51from provisioningserver.dns.config import DynamicDNSUpdate55from provisioningserver.dns.config import DynamicDNSUpdate
56from provisioningserver.dns.testing import patch_zone_file_config_path
52from provisioningserver.dns.zoneconfig import (57from provisioningserver.dns.zoneconfig import (
53 DNSForwardZoneConfig,58 DNSForwardZoneConfig,
54 DNSReverseZoneConfig,59 DNSReverseZoneConfig,
@@ -414,7 +419,7 @@ class TestZoneGenerator(MAASServerTestCase):
414 ], # purposely out of order to assert subnets are being sorted419 ], # purposely out of order to assert subnets are being sorted
415 serial=random.randint(0, 65535),420 serial=random.randint(0, 65535),
416 ).as_list()421 ).as_list()
417 self.assertEqual(len(zones), 7)422 self.assertEqual(len(zones), 13) # 5 /24s and 8 from the /21
418 expected_domains = [423 expected_domains = [
419 "overlap",424 "overlap",
420 "36.232.10.in-addr.arpa",425 "36.232.10.in-addr.arpa",
@@ -728,14 +733,15 @@ class TestZoneGenerator(MAASServerTestCase):
728 def test_supernet_inherits_rfc2317_net(self):733 def test_supernet_inherits_rfc2317_net(self):
729 domain = Domain.objects.get_default_domain()734 domain = Domain.objects.get_default_domain()
730 subnet1 = factory.make_Subnet(host_bits=2)735 subnet1 = factory.make_Subnet(host_bits=2)
731 net = IPNetwork(subnet1.cidr)736 net1 = IPNetwork(subnet1.cidr)
732 if net.version == 6:737 if net1.version == 6:
733 prefixlen = random.randint(121, 124)738 prefixlen = random.randint(121, 124)
734 else:739 else:
735 prefixlen = random.randint(22, 24)740 prefixlen = random.randint(22, 24)
736 parent = IPNetwork("%s/%d" % (net.network, prefixlen))741 parent = IPNetwork(f"{net1.network}/{prefixlen:d}")
737 parent = IPNetwork("%s/%d" % (parent.network, prefixlen))742 parent = IPNetwork(f"{parent.network}/{prefixlen:d}")
738 subnet2 = factory.make_Subnet(cidr=parent)743 subnet2 = factory.make_Subnet(cidr=parent)
744 net2 = IPNetwork(subnet2.cidr)
739 node = factory.make_Node_with_Interface_on_Subnet(745 node = factory.make_Node_with_Interface_on_Subnet(
740 subnet=subnet1,746 subnet=subnet1,
741 vlan=subnet1.vlan,747 vlan=subnet1.vlan,
@@ -746,56 +752,51 @@ class TestZoneGenerator(MAASServerTestCase):
746 factory.make_StaticIPAddress(interface=boot_iface, subnet=subnet1)752 factory.make_StaticIPAddress(interface=boot_iface, subnet=subnet1)
747 default_ttl = random.randint(10, 300)753 default_ttl = random.randint(10, 300)
748 Config.objects.set_config("default_dns_ttl", default_ttl)754 Config.objects.set_config("default_dns_ttl", default_ttl)
755 serial = random.randint(0, 65535)
749 zones = ZoneGenerator(756 zones = ZoneGenerator(
750 domain,757 domain,
751 [subnet1, subnet2],758 [subnet1, subnet2],
752 default_ttl=default_ttl,759 default_ttl=default_ttl,
753 serial=random.randint(0, 65535),760 serial=serial,
754 ).as_list()761 ).as_list()
755 self.assertThat(762 expected = [
756 zones,763 DNSForwardZoneConfig(
757 MatchesSetwise(764 domain.name,
758 forward_zone(domain.name),765 serial=serial,
759 reverse_zone(domain.name, subnet1.cidr),766 default_ttl=default_ttl,
760 reverse_zone(domain.name, subnet2.cidr),
761 ),767 ),
762 )
763 self.assertEqual(set(), zones[1]._rfc2317_ranges)
764 self.assertEqual({net}, zones[2]._rfc2317_ranges)
765
766 def test_two_managed_interfaces_yields_one_forward_two_reverse_zones(self):
767 default_domain = Domain.objects.get_default_domain().name
768 domain = factory.make_Domain()
769 subnet1 = factory.make_Subnet()
770 subnet2 = factory.make_Subnet()
771 expected_zones = [
772 forward_zone(domain.name),
773 reverse_zone(default_domain, subnet1.cidr),
774 reverse_zone(default_domain, subnet2.cidr),
775 ]768 ]
776 subnets = Subnet.objects.all()769 for net in [net1, net2]:
777770 if (
778 expected_zones = (771 net.version == 6 and net.prefixlen < 124
779 [forward_zone(domain.name)]772 ) or net.prefixlen < 24:
780 + [773 for network in ZoneGenerator._split_large_subnet(net2):
781 reverse_zone(default_domain, subnet.get_ipnetwork())774 expected.append(
782 for subnet in subnets775 DNSReverseZoneConfig(domain.name, network=network)
783 ]776 )
784 + [777 elif net.version == 6 and net.prefixlen > 124:
785 reverse_zone(778 expected.append(
786 default_domain,779 DNSReverseZoneConfig(
787 self.rfc2317_network(subnet.get_ipnetwork()),780 domain.name, network=IPNetwork(f"{net.network}/124")
781 )
788 )782 )
789 for subnet in subnets783 expected.append(DNSReverseZoneConfig(domain.name, network=net))
790 if self.rfc2317_network(subnet.get_ipnetwork()) is not None784 elif net.version == 4 and net.prefixlen > 24:
791 ]785 expected.append(
792 )786 DNSReverseZoneConfig(
793 self.assertThat(787 domain.name, network=IPNetwork(f"{net.network}/24")
794 ZoneGenerator(788 )
795 domain, [subnet1, subnet2], serial=random.randint(0, 65535)789 )
796 ).as_list(),790 expected.append(DNSReverseZoneConfig(domain.name, network=net))
797 MatchesSetwise(*expected_zones),791 else:
792 expected.append(DNSReverseZoneConfig(domain.name, network=net))
793 self.assertCountEqual(
794 set([(zone.domain, zone._network) for zone in zones]),
795 set((e.domain, e._network) for e in expected),
796 f"{subnet1} {subnet2}",
798 )797 )
798 self.assertEqual(set(), zones[1]._rfc2317_ranges)
799 self.assertEqual({net1}, zones[2]._rfc2317_ranges)
799800
800 def test_with_many_yields_many_zones(self):801 def test_with_many_yields_many_zones(self):
801 # This demonstrates ZoneGenerator in all-singing all-dancing mode.802 # This demonstrates ZoneGenerator in all-singing all-dancing mode.
@@ -806,18 +807,29 @@ class TestZoneGenerator(MAASServerTestCase):
806 subnets = Subnet.objects.all()807 subnets = Subnet.objects.all()
807 expected_zones = set()808 expected_zones = set()
808 for domain in domains:809 for domain in domains:
809 expected_zones.add(forward_zone(domain.name))810 expected_zones.add(DNSForwardZoneConfig(domain.name))
810 for subnet in subnets:811 for subnet in subnets:
811 expected_zones.add(reverse_zone(default_domain.name, subnet.cidr))812 networks = ZoneGenerator._split_large_subnet(
812 rfc2317_net = self.rfc2317_network(subnet.get_ipnetwork())813 IPNetwork(subnet.cidr)
813 if rfc2317_net is not None:814 )
815 for network in networks:
814 expected_zones.add(816 expected_zones.add(
815 reverse_zone(default_domain.name, rfc2317_net.cidr)817 DNSReverseZoneConfig(default_domain.name, network=network)
816 )818 )
819 if rfc2317_net := self.rfc2317_network(network):
820 expected_zones.add(
821 DNSReverseZoneConfig(
822 default_domain.name,
823 network=IPNetwork(rfc2317_net.cidr),
824 )
825 )
817 actual_zones = ZoneGenerator(826 actual_zones = ZoneGenerator(
818 domains, subnets, serial=random.randint(0, 65535)827 domains, subnets, serial=random.randint(0, 65535)
819 ).as_list()828 ).as_list()
820 self.assertThat(actual_zones, MatchesSetwise(*expected_zones))829 self.assertCountEqual(
830 [(zone.domain, zone._network) for zone in actual_zones],
831 [(zone.domain, zone._network) for zone in expected_zones],
832 )
821833
822 def test_zone_generator_handles_rdns_mode_equal_enabled(self):834 def test_zone_generator_handles_rdns_mode_equal_enabled(self):
823 Domain.objects.get_or_create(name="one")835 Domain.objects.get_or_create(name="one")
@@ -898,6 +910,359 @@ class TestZoneGenerator(MAASServerTestCase):
898 ),910 ),
899 )911 )
900912
913 def test_configs_are_merged_when_overlapping(self):
914 self.patch(warn_loopback)
915 default_domain = Domain.objects.get_default_domain()
916 subnet1 = factory.make_Subnet(cidr="10.0.1.0/24")
917 subnet2 = factory.make_Subnet(cidr="10.0.0.0/21")
918 subnet1_ips = [
919 factory.make_StaticIPAddress(
920 ip=factory.pick_ip_in_Subnet(subnet1), subnet=subnet1
921 )
922 for _ in range(3)
923 ]
924 subnet2_ips = [
925 factory.make_StaticIPAddress(
926 ip=factory.pick_ip_in_Subnet(subnet2), subnet=subnet2
927 )
928 for _ in range(3)
929 ]
930 subnet1_records = [
931 factory.make_DNSResource(domain=default_domain, ip_addresses=[ip])
932 for ip in subnet1_ips
933 ]
934 subnet2_records = [
935 factory.make_DNSResource(domain=default_domain, ip_addresses=[ip])
936 for ip in subnet2_ips
937 ]
938 serial = random.randint(0, 65535)
939 dynamic_updates = [
940 DynamicDNSUpdate(
941 operation="INSERT",
942 name=record.name,
943 zone=default_domain.name,
944 rectype="A",
945 ttl=record.address_ttl,
946 answer=ip.ip,
947 )
948 for record in subnet1_records + subnet2_records
949 for ip in record.ip_addresses.all()
950 ]
951 zones = ZoneGenerator(
952 [default_domain],
953 [subnet1, subnet2],
954 serial=serial,
955 dynamic_updates=dynamic_updates,
956 ).as_list()
957
958 def _generate_mapping_for_network(network, records):
959 mapping = {}
960 for record in records:
961 if ip_set := set(
962 ip.ip
963 for ip in record.ip_addresses.all()
964 if IPAddress(ip.ip) in network
965 ):
966 mapping[
967 f"{record.name}.{default_domain.name}"
968 ] = HostnameIPMapping(
969 None,
970 record.address_ttl,
971 ip_set,
972 None,
973 1,
974 None,
975 )
976 return mapping
977
978 expected = [
979 DNSForwardZoneConfig(
980 default_domain.name,
981 mapping={
982 record.name: HostnameIPMapping(
983 None,
984 record.address_ttl,
985 set(record.ip_addresses.all()),
986 None,
987 1,
988 None,
989 )
990 for record in subnet1_records + subnet2_records
991 },
992 dynamic_updates=dynamic_updates,
993 ),
994 DNSReverseZoneConfig(
995 default_domain.name,
996 network=IPNetwork(subnet1.cidr),
997 mapping=_generate_mapping_for_network(
998 IPNetwork(subnet1.cidr), subnet1_records + subnet2_records
999 ),
1000 dynamic_updates=[
1001 DynamicDNSUpdate.as_reverse_record_update(
1002 update, IPNetwork(subnet1.cidr)
1003 )
1004 for update in dynamic_updates
1005 if update.answer_as_ip in IPNetwork(subnet1.cidr)
1006 ],
1007 ),
1008 DNSReverseZoneConfig(
1009 default_domain.name,
1010 network=IPNetwork("10.0.0.0/24"),
1011 mapping=_generate_mapping_for_network(
1012 IPNetwork("10.0.0.0/24"), subnet2_records
1013 ),
1014 dynamic_updates=[
1015 DynamicDNSUpdate.as_reverse_record_update(
1016 update, IPNetwork("10.0.0.0/24")
1017 )
1018 for update in dynamic_updates
1019 if update.answer_as_ip in IPNetwork("10.0.0.0/24")
1020 ],
1021 ),
1022 DNSReverseZoneConfig(
1023 default_domain.name,
1024 network=IPNetwork("10.0.2.0/24"),
1025 mapping=_generate_mapping_for_network(
1026 IPNetwork("10.0.2.0/24"), subnet2_records
1027 ),
1028 dynamic_updates=[
1029 DynamicDNSUpdate.as_reverse_record_update(
1030 update, IPNetwork("10.0.2.0/24")
1031 )
1032 for update in dynamic_updates
1033 if update.answer_as_ip in IPNetwork("10.0.2.0/24")
1034 ],
1035 ),
1036 DNSReverseZoneConfig(
1037 default_domain.name,
1038 network=IPNetwork("10.0.3.0/24"),
1039 mapping=_generate_mapping_for_network(
1040 IPNetwork("10.0.3.0/24"), subnet2_records
1041 ),
1042 dynamic_updates=[
1043 DynamicDNSUpdate.as_reverse_record_update(
1044 update, IPNetwork("10.0.3.0/24")
1045 )
1046 for update in dynamic_updates
1047 if update.answer_as_ip in IPNetwork("10.0.3.0/24")
1048 ],
1049 ),
1050 DNSReverseZoneConfig(
1051 default_domain.name,
1052 network=IPNetwork("10.0.4.0/24"),
1053 mapping=_generate_mapping_for_network(
1054 IPNetwork("10.0.4.0/24"), subnet2_records
1055 ),
1056 dynamic_updates=[
1057 DynamicDNSUpdate.as_reverse_record_update(
1058 update, IPNetwork("10.0.4.0/24")
1059 )
1060 for update in dynamic_updates
1061 if update.answer_as_ip in IPNetwork("10.0.4.0/24")
1062 ],
1063 ),
1064 DNSReverseZoneConfig(
1065 default_domain.name,
1066 network=IPNetwork("10.0.5.0/24"),
1067 mapping=_generate_mapping_for_network(
1068 IPNetwork("10.0.5.0/24"), subnet2_records
1069 ),
1070 dynamic_updates=[
1071 DynamicDNSUpdate.as_reverse_record_update(
1072 update, IPNetwork("10.0.5.0/24")
1073 )
1074 for update in dynamic_updates
1075 if update.answer_as_ip in IPNetwork("10.0.5.0/24")
1076 ],
1077 ),
1078 DNSReverseZoneConfig(
1079 default_domain.name,
1080 network=IPNetwork("10.0.6.0/24"),
1081 mapping=_generate_mapping_for_network(
1082 IPNetwork("10.0.6.0/24"), subnet2_records
1083 ),
1084 dynamic_updates=[
1085 DynamicDNSUpdate.as_reverse_record_update(
1086 update, IPNetwork("10.0.6.0/24")
1087 )
1088 for update in dynamic_updates
1089 if update.answer_as_ip in IPNetwork("10.0.6.0/24")
1090 ],
1091 ),
1092 DNSReverseZoneConfig(
1093 default_domain.name,
1094 network=IPNetwork("10.0.7.0/24"),
1095 mapping=_generate_mapping_for_network(
1096 IPNetwork("10.0.7.0/24"), subnet2_records
1097 ),
1098 dynamic_updates=[
1099 DynamicDNSUpdate.as_reverse_record_update(
1100 update, IPNetwork("10.0.7.0/24")
1101 )
1102 for update in dynamic_updates
1103 if update.answer_as_ip in IPNetwork("10.0.7.0/24")
1104 ],
1105 ),
1106 ]
1107
1108 for i, zone in enumerate(zones):
1109 self.assertEqual(zone.domain, expected[i].domain)
1110 self.assertEqual(zone._network, expected[i]._network)
1111 self.assertCountEqual(
1112 zone._mapping,
1113 expected[i]._mapping,
1114 )
1115 self.assertCountEqual(
1116 zone._dynamic_updates, expected[i]._dynamic_updates
1117 )
1118 if isinstance(zone, DNSReverseZoneConfig):
1119 self.assertCountEqual(
1120 zone._dynamic_ranges, expected[i]._dynamic_ranges
1121 )
1122 self.assertCountEqual(
1123 zone._rfc2317_ranges, expected[i]._rfc2317_ranges
1124 )
1125
1126 def test_configs_are_merged_when_glue_overlaps(self):
1127 self.patch(warn_loopback)
1128 default_domain = Domain.objects.get_default_domain()
1129 subnet1 = factory.make_Subnet(cidr="10.0.1.0/24")
1130 subnet2 = factory.make_Subnet(cidr="10.0.1.0/26")
1131 subnet1_ips = [
1132 factory.make_StaticIPAddress(
1133 ip=f"10.0.1.{253 + i}",
1134 subnet=subnet1, # avoid allocation collision
1135 )
1136 for i in range(3)
1137 ]
1138 subnet2_ips = [
1139 factory.make_StaticIPAddress(
1140 ip=factory.pick_ip_in_Subnet(subnet2), subnet=subnet2
1141 )
1142 for _ in range(3)
1143 ]
1144 subnet1_records = [
1145 factory.make_DNSResource(domain=default_domain, ip_addresses=[ip])
1146 for ip in subnet1_ips
1147 ]
1148 subnet2_records = [
1149 factory.make_DNSResource(domain=default_domain, ip_addresses=[ip])
1150 for ip in subnet2_ips
1151 ]
1152 serial = random.randint(0, 65535)
1153 dynamic_updates = [
1154 DynamicDNSUpdate(
1155 operation="INSERT",
1156 name=record.name,
1157 zone=default_domain.name,
1158 rectype="A",
1159 ttl=record.address_ttl,
1160 answer=ip.ip,
1161 )
1162 for record in subnet1_records + subnet2_records
1163 for ip in record.ip_addresses.all()
1164 ]
1165 zones = ZoneGenerator(
1166 [default_domain],
1167 [subnet1, subnet2],
1168 serial=serial,
1169 dynamic_updates=dynamic_updates,
1170 ).as_list()
1171
1172 def _generate_mapping_for_network(network, other_network, records):
1173 mapping = {}
1174 for record in records:
1175 ip_set = set(
1176 ip.ip
1177 for ip in record.ip_addresses.all()
1178 if IPAddress(ip.ip) in network
1179 and (
1180 IPAddress(ip.ip) not in other_network
1181 or other_network.prefixlen < network.prefixlen
1182 )
1183 )
1184 if len(ip_set) > 0:
1185 mapping[
1186 f"{record.name}.{default_domain.name}"
1187 ] = HostnameIPMapping(
1188 None,
1189 record.address_ttl,
1190 ip_set,
1191 None,
1192 1,
1193 None,
1194 )
1195 return mapping
1196
1197 expected = [
1198 DNSForwardZoneConfig(
1199 default_domain.name,
1200 mapping={
1201 record.name: HostnameIPMapping(
1202 None,
1203 record.address_ttl,
1204 set(ip.ip for ip in record.ip_addresses.all()),
1205 None,
1206 1,
1207 None,
1208 )
1209 for record in subnet1_records + subnet2_records
1210 },
1211 dynamic_updates=dynamic_updates,
1212 ),
1213 DNSReverseZoneConfig(
1214 default_domain.name,
1215 network=IPNetwork(subnet2.cidr),
1216 mapping=_generate_mapping_for_network(
1217 IPNetwork(subnet2.cidr),
1218 IPNetwork(subnet1.cidr),
1219 subnet1_records + subnet2_records,
1220 ),
1221 dynamic_updates=[
1222 DynamicDNSUpdate.as_reverse_record_update(
1223 update, IPNetwork(subnet2.cidr)
1224 )
1225 for update in dynamic_updates
1226 if update.answer_as_ip in IPNetwork(subnet2.cidr)
1227 ],
1228 ),
1229 DNSReverseZoneConfig(
1230 default_domain.name,
1231 network=IPNetwork(subnet1.cidr),
1232 mapping=_generate_mapping_for_network(
1233 IPNetwork(subnet1.cidr),
1234 IPNetwork(subnet2.cidr),
1235 subnet1_records + subnet2_records,
1236 ),
1237 dynamic_updates=[
1238 DynamicDNSUpdate.as_reverse_record_update(
1239 update, IPNetwork(subnet1.cidr)
1240 )
1241 for update in dynamic_updates
1242 if update.answer_as_ip in IPNetwork(subnet1.cidr)
1243 ],
1244 rfc2317_ranges=set([IPNetwork(subnet2.cidr)]),
1245 ),
1246 ]
1247
1248 for i, zone in enumerate(zones):
1249 self.assertEqual(zone.domain, expected[i].domain)
1250 self.assertEqual(zone._network, expected[i]._network)
1251 self.assertCountEqual(
1252 zone._mapping,
1253 expected[i]._mapping,
1254 )
1255 self.assertCountEqual(
1256 zone._dynamic_updates, expected[i]._dynamic_updates
1257 )
1258 if isinstance(zone, DNSReverseZoneConfig):
1259 self.assertCountEqual(
1260 zone._dynamic_ranges, expected[i]._dynamic_ranges
1261 )
1262 self.assertCountEqual(
1263 zone._rfc2317_ranges, expected[i]._rfc2317_ranges
1264 )
1265
9011266
902class TestZoneGeneratorTTL(MAASTransactionServerTestCase):1267class TestZoneGeneratorTTL(MAASTransactionServerTestCase):
903 """Tests for TTL in :class:ZoneGenerator`."""1268 """Tests for TTL in :class:ZoneGenerator`."""
@@ -1006,7 +1371,22 @@ class TestZoneGeneratorTTL(MAASTransactionServerTestCase):
1006 serial=random.randint(0, 65535),1371 serial=random.randint(0, 65535),
1007 ).as_list()1372 ).as_list()
1008 self.assertEqual(expected_forward, zones[0]._mapping)1373 self.assertEqual(expected_forward, zones[0]._mapping)
1009 self.assertEqual(expected_reverse, zones[1]._mapping)1374 for zone in zones[1:]:
1375 if ip_set := set(
1376 ip
1377 for ip in expected_reverse[node.fqdn].ips
1378 if IPAddress(ip) in zone._network
1379 ):
1380 expected_rev = {
1381 node.fqdn: HostnameIPMapping(
1382 node.system_id,
1383 node.address_ttl,
1384 ip_set,
1385 node.node_type,
1386 dnsrr.id,
1387 )
1388 }
1389 self.assertEqual(expected_rev, zone._mapping)
10101390
1011 @transactional1391 @transactional
1012 def test_dnsresource_address_overrides_domain(self):1392 def test_dnsresource_address_overrides_domain(self):
@@ -1058,7 +1438,22 @@ class TestZoneGeneratorTTL(MAASTransactionServerTestCase):
1058 serial=random.randint(0, 65535),1438 serial=random.randint(0, 65535),
1059 ).as_list()1439 ).as_list()
1060 self.assertEqual(expected_forward, zones[0]._mapping)1440 self.assertEqual(expected_forward, zones[0]._mapping)
1061 self.assertEqual(expected_reverse, zones[1]._mapping)1441
1442 for zone in zones[1:]:
1443 expected = {}
1444 for expected_label, expected_mapping in expected_reverse.items():
1445 if ip_set := set(
1446 ip for ip in expected_mapping.ips if ip in zone._network
1447 ):
1448 expected[expected_label] = HostnameIPMapping(
1449 system_id=expected_mapping.system_id,
1450 ttl=expected_mapping.ttl,
1451 ips=ip_set,
1452 node_type=expected_mapping.node_type,
1453 dnsresource_id=expected_mapping.dnsresource_id,
1454 user_id=expected_mapping.user_id,
1455 )
1456 self.assertEqual(expected, zone._mapping)
10621457
1063 @transactional1458 @transactional
1064 def test_dnsdata_inherits_global(self):1459 def test_dnsdata_inherits_global(self):
@@ -1167,3 +1562,123 @@ class TestZoneGeneratorTTL(MAASTransactionServerTestCase):
1167 [zone_config] = ZoneGenerator(domains=[domain], subnets=[], serial=123)1562 [zone_config] = ZoneGenerator(domains=[domain], subnets=[], serial=123)
1168 self.assertEqual(domain.name, zone_config.domain)1563 self.assertEqual(domain.name, zone_config.domain)
1169 self.assertEqual(42, zone_config.default_ttl)1564 self.assertEqual(42, zone_config.default_ttl)
1565
1566
1567class TestZoneGeneratorEndToEnd(MAASServerTestCase):
1568 def _find_most_specific_subnet(
1569 self, ip: StaticIPAddress, subnets: list[Subnet]
1570 ):
1571 networks = []
1572 for subnet in subnets:
1573 net = IPNetwork(subnet.cidr)
1574 if net.prefixlen < 24:
1575 networks += ZoneGenerator._split_large_subnet(net)
1576 else:
1577 networks.append(net)
1578 sorted_nets = sorted(networks, key=lambda net: -1 * net.prefixlen)
1579 for net in sorted_nets:
1580 if IPAddress(ip.ip) in net:
1581 return net
1582
1583 def test_ZoneGenerator_generates_config_for_zone_files(self):
1584 config_path = patch_zone_file_config_path(self)
1585 default_domain = Domain.objects.get_default_domain()
1586 domain = factory.make_Domain()
1587 subnet1 = factory.make_Subnet(cidr="10.0.1.0/24")
1588 subnet2 = factory.make_Subnet(cidr="10.0.0.0/22")
1589 subnet3 = factory.make_Subnet(cidr="10.0.1.0/27")
1590 subnet1_ips = [
1591 factory.make_StaticIPAddress(
1592 ip=factory.pick_ip_in_Subnet(subnet1), subnet=subnet1
1593 )
1594 for _ in range(3)
1595 ]
1596 subnet2_ips = [
1597 factory.make_StaticIPAddress(
1598 ip=factory.pick_ip_in_Subnet(
1599 subnet2, but_not=list(subnet1.get_ipranges_in_use())
1600 ),
1601 subnet=subnet2,
1602 )
1603 for _ in range(3)
1604 ]
1605 subnet3_ips = [
1606 factory.make_StaticIPAddress(
1607 ip=factory.pick_ip_in_Subnet(
1608 subnet3,
1609 but_not=list(subnet1.get_ipranges_in_use())
1610 + list(subnet2.get_ipranges_in_use()),
1611 ),
1612 subnet=subnet3,
1613 )
1614 for _ in range(3)
1615 ]
1616 subnet1_records = [
1617 factory.make_DNSResource(
1618 domain=random.choice((default_domain, domain)),
1619 ip_addresses=[ip],
1620 )
1621 for ip in subnet1_ips
1622 ]
1623 subnet2_records = [
1624 factory.make_DNSResource(
1625 domain=random.choice((default_domain, domain)),
1626 ip_addresses=[ip],
1627 )
1628 for ip in subnet2_ips
1629 ]
1630 subnet3_records = [
1631 factory.make_DNSResource(
1632 domain=random.choice((default_domain, domain)),
1633 ip_addresses=[ip],
1634 )
1635 for ip in subnet3_ips
1636 ]
1637 all_records = subnet1_records + subnet2_records + subnet3_records
1638 zones = ZoneGenerator(
1639 [default_domain, domain],
1640 [subnet1, subnet2, subnet3],
1641 serial=random.randint(0, 65535),
1642 ).as_list()
1643 for zone in zones:
1644 zone.write_config()
1645
1646 # check forward zones
1647 with open(
1648 os.path.join(config_path, f"zone.{default_domain.name}"), "r"
1649 ) as zf:
1650 default_domain_contents = zf.read()
1651
1652 with open(os.path.join(config_path, f"zone.{domain.name}"), "r") as zf:
1653 domain_contents = zf.read()
1654
1655 for record in all_records:
1656 if record.domain == default_domain:
1657 contents = default_domain_contents
1658 else:
1659 contents = domain_contents
1660
1661 self.assertIn(
1662 f"{record.name} 30 IN A {record.ip_addresses.first().ip}",
1663 contents,
1664 )
1665
1666 # check reverse zones
1667 for record in all_records:
1668 ip = record.ip_addresses.first()
1669 subnet = self._find_most_specific_subnet(
1670 ip, [subnet1, subnet2, subnet3]
1671 )
1672 rev_subnet = ".".join(str(subnet.network).split(".")[2::-1])
1673 if subnet.prefixlen > 24:
1674 rev_subnet = f"{str(subnet.network).split('.')[-1]}-{subnet.prefixlen}.{rev_subnet}"
1675 with open(
1676 os.path.join(config_path, f"zone.{rev_subnet}.in-addr.arpa"),
1677 "r",
1678 ) as zf:
1679 contents = zf.read()
1680 self.assertIn(
1681 f"{ip.ip.split('.')[-1]} 30 IN PTR {record.fqdn}",
1682 contents,
1683 f"{subnet} {ip.ip}",
1684 )
diff --git a/src/maasserver/dns/zonegenerator.py b/src/maasserver/dns/zonegenerator.py
index b6fd406..8272e5c 100644
--- a/src/maasserver/dns/zonegenerator.py
+++ b/src/maasserver/dns/zonegenerator.py
@@ -18,7 +18,11 @@ from maasserver.models.config import Config
18from maasserver.models.dnsdata import DNSData, HostnameRRsetMapping18from maasserver.models.dnsdata import DNSData, HostnameRRsetMapping
19from maasserver.models.dnsresource import separate_fqdn19from maasserver.models.dnsresource import separate_fqdn
20from maasserver.models.domain import Domain20from maasserver.models.domain import Domain
21from maasserver.models.staticipaddress import StaticIPAddress21from maasserver.models.iprange import IPRange
22from maasserver.models.staticipaddress import (
23 HostnameIPMapping,
24 StaticIPAddress,
25)
22from maasserver.models.subnet import Subnet26from maasserver.models.subnet import Subnet
23from maasserver.server_address import get_maas_facing_server_addresses27from maasserver.server_address import get_maas_facing_server_addresses
24from provisioningserver.dns.config import DynamicDNSUpdate28from provisioningserver.dns.config import DynamicDNSUpdate
@@ -226,6 +230,7 @@ class ZoneGenerator:
226 if self._dynamic_updates is None:230 if self._dynamic_updates is None:
227 self._dynamic_updates = []231 self._dynamic_updates = []
228 self.force_config_write = force_config_write # some data changed that nsupdate cannot update if true232 self.force_config_write = force_config_write # some data changed that nsupdate cannot update if true
233 self._existing_subnet_cfgs = {}
229234
230 @staticmethod235 @staticmethod
231 def _get_mappings():236 def _get_mappings():
@@ -357,18 +362,93 @@ class ZoneGenerator:
357 )362 )
358363
359 @staticmethod364 @staticmethod
360 def _gen_reverse_zones(365 def _split_large_subnet(network: IPNetwork) -> list[IPNetwork]:
361 subnets,366 # Generate the name of the reverse zone file:
362 serial,367 # Use netaddr's reverse_dns() to get the reverse IP name
363 ns_host_name,368 # of the first IP address in the network and then drop the first
364 mappings,369 # octets of that name (i.e. drop the octets that will be specified in
365 default_ttl,370 # the zone file).
366 dynamic_updates,371 # returns a list of (IPNetwork, zone_name, zonefile_path) tuples
367 force_config_write,372 new_networks = []
373 first = IPAddress(network.first)
374 last = IPAddress(network.last)
375 if first.version == 6:
376 # IPv6.
377 # 2001:89ab::/19 yields 8.1.0.0.2.ip6.arpa, and the full list
378 # is 8.1.0.0.2.ip6.arpa, 9.1.0.0.2.ip6.arpa
379 # The ipv6 reverse dns form is 32 elements of 1 hex digit each.
380 # How many elements of the reverse DNS name to we throw away?
381 # Prefixlen of 0-3 gives us 1, 4-7 gives us 2, etc.
382 # While this seems wrong, we always _add_ a base label back in,
383 # so it's correct.
384 rest_limit = (132 - network.prefixlen) // 4
385 # What is the prefix for each inner subnet (It will be the next
386 # smaller multiple of 4.) If it's the smallest one, then RFC2317
387 # tells us that we're adding an extra blob to the front of the
388 # reverse zone name, and we want the entire prefixlen.
389 subnet_prefix = (network.prefixlen + 3) // 4 * 4
390 if subnet_prefix == 128:
391 subnet_prefix = network.prefixlen
392 # How big is the step between subnets? Again, special case for
393 # extra small subnets.
394 step = 1 << ((128 - network.prefixlen) // 4 * 4)
395 if step < 16:
396 step = 16
397 # Grab the base (hex) and trailing labels for our reverse zone.
398 split_zone = first.reverse_dns.split(".")
399 base = int(split_zone[rest_limit - 1], 16)
400 else:
401 # IPv4.
402 # The logic here is the same as for IPv6, but with 8 instead of 4.
403 rest_limit = (40 - network.prefixlen) // 8
404 subnet_prefix = (network.prefixlen + 7) // 8 * 8
405 if subnet_prefix == 32:
406 subnet_prefix = network.prefixlen
407 step = 1 << ((32 - network.prefixlen) // 8 * 8)
408 if step < 256:
409 step = 256
410 # Grab the base (decimal) and trailing labels for our reverse
411 # zone.
412 split_zone = first.reverse_dns.split(".")
413 base = int(split_zone[rest_limit - 1])
414
415 while first <= last:
416 if first > last:
417 # if the excluding subnet pushes the base IP beyond the bounds of the generating subnet, we've reached the end and return early
418 return new_networks
419
420 new_networks.append(IPNetwork(f"{first}/{subnet_prefix:d}"))
421 base += 1
422 try:
423 first += step
424 except IndexError:
425 # IndexError occurs when we go from 255.255.255.255 to
426 # 0.0.0.0. If we hit that, we're all fine and done.
427 break
428 return new_networks
429
430 @staticmethod
431 def _filter_mapping_for_network(
432 network: IPNetwork, mappings: dict[str, HostnameIPMapping]
368 ):433 ):
369 """Generator of reverse zones, sorted by network."""434 net_mappings = {}
435 for k, v in mappings.items():
436 if ips_in_net := set(
437 ip for ip in v.ips if IPAddress(ip) in network
438 ):
439 net_mappings[k] = HostnameIPMapping(
440 v.system_id,
441 v.ttl,
442 ips_in_net,
443 v.node_type,
444 v.dnsresource_id,
445 v.user_id,
446 )
370447
371 subnets = set(subnets)448 return net_mappings
449
450 @staticmethod
451 def _generate_glue_nets(subnets: list[Subnet]):
372 # Generate the list of parent networks for rfc2317 glue. Note that we452 # Generate the list of parent networks for rfc2317 glue. Note that we
373 # need to handle the case where we are controlling both the small net453 # need to handle the case where we are controlling both the small net
374 # and a bigger network containing the /24, not just a /24 network.454 # and a bigger network containing the /24, not just a /24 network.
@@ -393,6 +473,82 @@ class ZoneGenerator:
393 )473 )
394 rfc2317_glue.setdefault(basenet, set()).add(network)474 rfc2317_glue.setdefault(basenet, set()).add(network)
395475
476 return rfc2317_glue
477
478 @staticmethod
479 def _find_glue_nets(
480 network: IPNetwork, rfc2317_glue: defaultdict[str, set[IPNetwork]]
481 ):
482 # Use the default_domain as the name for the NS host in the reverse
483 # zones. If this network is actually a parent rfc2317 glue
484 # network, then we need to generate the glue records.
485 # We need to detect the need for glue in our networks that are
486 # big.
487 if (
488 network.version == 6 and network.prefixlen < 124
489 ) or network.prefixlen < 24:
490 glue = set()
491 # This is the reason for needing the subnets sorted in
492 # increasing order of size.
493 for net in rfc2317_glue.copy().keys():
494 if net in network:
495 glue.update(rfc2317_glue[net])
496 del rfc2317_glue[net]
497 elif network in rfc2317_glue:
498 glue = rfc2317_glue[network]
499 del rfc2317_glue[network]
500 else:
501 glue = set()
502 return glue
503
504 @staticmethod
505 def _merge_into_existing_network(
506 network: IPNetwork,
507 existing: dict[IPNetwork, DNSReverseZoneConfig],
508 mapping: dict[str, HostnameIPMapping],
509 dynamic_ranges: list[IPRange] | None = [],
510 dynamic_updates: list[DynamicDNSUpdate] | None = [],
511 glue: set[IPNetwork] | None = set(),
512 is_glue_net: bool = False,
513 ):
514 # since all dynamic updates are passed and we then filter for those belonging
515 # in the network, the existing config already has all updates and we do not need
516 # to merge them, just add them if they haven't already
517 if not existing[network]._dynamic_updates:
518 existing[network]._dynamic_updates = dynamic_updates
519 existing[network]._rfc2317_ranges = existing[
520 network
521 ]._rfc2317_ranges.union(glue)
522 for k, v in mapping.items():
523 if k in existing[network]._mapping:
524 existing[network]._mapping[k].ips.union(v.ips)
525 else:
526 existing[network]._mapping[k] = v
527 existing[network]._dynamic_ranges += dynamic_ranges
528 for glue_net in glue.union(existing[network]._rfc2317_ranges):
529 for k, v in existing[network]._mapping.copy().items():
530 if ip_set := set(ip for ip in v.ips if ip not in glue_net):
531 existing[network]._mapping[k].ips = ip_set
532 else:
533 del existing[network]._mapping[k]
534
535 @staticmethod
536 def _gen_reverse_zones(
537 subnets,
538 serial,
539 ns_host_name,
540 mappings,
541 default_ttl,
542 dynamic_updates,
543 force_config_write,
544 existing_subnet_cfgs={},
545 ):
546 """Generator of reverse zones, sorted by network."""
547
548 subnets = set(subnets)
549
550 rfc2317_glue = ZoneGenerator._generate_glue_nets(subnets)
551
396 # Since get_hostname_ip_mapping(Subnet) ignores Subnet.id, so we can552 # Since get_hostname_ip_mapping(Subnet) ignores Subnet.id, so we can
397 # just do it once and be happy. LP#1600259553 # just do it once and be happy. LP#1600259
398 if len(subnets):554 if len(subnets):
@@ -412,7 +568,7 @@ class ZoneGenerator:
412 key=lambda subnet: IPNetwork(subnet.cidr).prefixlen,568 key=lambda subnet: IPNetwork(subnet.cidr).prefixlen,
413 reverse=True,569 reverse=True,
414 ):570 ):
415 network = IPNetwork(subnet.cidr)571 base_network = IPNetwork(subnet.cidr)
416 if subnet.rdns_mode == RDNS_MODE.DISABLED:572 if subnet.rdns_mode == RDNS_MODE.DISABLED:
417 # If we are not doing reverse dns for this subnet, then just573 # If we are not doing reverse dns for this subnet, then just
418 # skip to the next subnet.574 # skip to the next subnet.
@@ -421,103 +577,109 @@ class ZoneGenerator:
421 )577 )
422 continue578 continue
423579
580 networks = ZoneGenerator._split_large_subnet(base_network)
581
424 # 1. Figure out the dynamic ranges.582 # 1. Figure out the dynamic ranges.
425 dynamic_ranges = [583 dynamic_ranges = [
426 ip_range.netaddr_iprange584 ip_range.netaddr_iprange
427 for ip_range in subnet.get_dynamic_ranges()585 for ip_range in subnet.get_dynamic_ranges()
428 ]586 ]
429587
430 # 2. Start with the map of all of the nodes, including all588 for network in networks:
431 # DNSResource-associated addresses. We will prune this to just589 # 2. Start with the map of all of the nodes, including all
432 # entries for the subnet when we actually generate the zonefile.590 # DNSResource-associated addresses. We will prune this to just
433 # If we get here, then we have subnets, so we noticed that above591 # entries for the subnet when we actually generate the zonefile.
434 # and created mappings['reverse']. LP#1600259592 # If we get here, then we have subnets, so we noticed that above
435 mapping = mappings["reverse"]593 # and created mappings['reverse']. LP#1600259
436594 mapping = ZoneGenerator._filter_mapping_for_network(
437 # Use the default_domain as the name for the NS host in the reverse595 network, mappings["reverse"]
438 # zones. If this network is actually a parent rfc2317 glue596 )
439 # network, then we need to generate the glue records.
440 # We need to detect the need for glue in our networks that are
441 # big.
442 if (
443 network.version == 6 and network.prefixlen < 124
444 ) or network.prefixlen < 24:
445 glue = set()
446 # This is the reason for needing the subnets sorted in
447 # increasing order of size.
448 for net in rfc2317_glue.copy().keys():
449 if net in network:
450 glue.update(rfc2317_glue[net])
451 del rfc2317_glue[net]
452 elif network in rfc2317_glue:
453 glue = rfc2317_glue[network]
454 del rfc2317_glue[network]
455 else:
456 glue = set()
457597
458 domain_updates = [598 glue = ZoneGenerator._find_glue_nets(network, rfc2317_glue)
459 DynamicDNSUpdate.as_reverse_record_update(update, network)599 domain_updates = [
460 for update in dynamic_updates600 DynamicDNSUpdate.as_reverse_record_update(update, network)
461 if update.answer601 for update in dynamic_updates
462 and update.answer_is_ip602 if update.answer
463 and (update.answer_as_ip in network)603 and update.answer_is_ip
464 ]604 and (update.answer_as_ip in network)
605 ]
606
607 if network in existing_subnet_cfgs:
608 ZoneGenerator._merge_into_existing_network(
609 network,
610 existing_subnet_cfgs,
611 mapping,
612 dynamic_ranges=dynamic_ranges,
613 dynamic_updates=domain_updates,
614 glue=glue,
615 )
616 else:
617 existing_subnet_cfgs[network] = DNSReverseZoneConfig(
618 ns_host_name,
619 serial=serial,
620 default_ttl=default_ttl,
621 ns_host_name=ns_host_name,
622 mapping=mapping,
623 network=network,
624 dynamic_ranges=dynamic_ranges,
625 rfc2317_ranges=glue,
626 dynamic_updates=domain_updates,
627 force_config_write=force_config_write,
628 )
465629
466 yield DNSReverseZoneConfig(630 yield existing_subnet_cfgs[network]
467 ns_host_name,631
468 serial=serial,632 # Now provide any remaining rfc2317 glue networks.
469 default_ttl=default_ttl,633 for network, ranges in rfc2317_glue.items():
470 ns_host_name=ns_host_name,634 exclude_set = {
471 mapping=mapping,635 IPNetwork(s.cidr)
472 network=network,636 for s in subnets
473 dynamic_ranges=dynamic_ranges,637 if network in IPNetwork(s.cidr)
474 rfc2317_ranges=glue,638 }
475 exclude={639 domain_updates = []
476 IPNetwork(s.cidr) for s in subnets if s is not subnet640 for update in dynamic_updates:
477 },641 glue_update = True
478 dynamic_updates=domain_updates,642 for exclude_net in exclude_set:
479 force_config_write=force_config_write,643 if (
480 )644 update.answer
481 # Now provide any remaining rfc2317 glue networks.645 and update.answer_is_ip
482 for network, ranges in rfc2317_glue.items():646 and update.answer_as_ip in exclude_net
483 exclude_set = {647 ):
484 IPNetwork(s.cidr)648 glue_update = False
485 for s in subnets649 break
486 if network in IPNetwork(s.cidr)
487 }
488 domain_updates = []
489 for update in dynamic_updates:
490 glue_update = True
491 for exclude_net in exclude_set:
492 if (650 if (
493 update.answer651 glue_update
652 and update.answer
494 and update.answer_is_ip653 and update.answer_is_ip
495 and update.answer_as_ip in exclude_net654 and update.answer_as_ip in network
496 ):655 ):
497 glue_update = False656 domain_updates.append(
498 break657 DynamicDNSUpdate.as_reverse_record_update(
499 if (658 update, network
500 glue_update659 )
501 and update.answer
502 and update.answer_is_ip
503 and update.answer_as_ip in network
504 ):
505 domain_updates.append(
506 DynamicDNSUpdate.as_reverse_record_update(
507 update, network
508 )660 )
661
662 if network in existing_subnet_cfgs:
663 ZoneGenerator._merge_into_existing_network(
664 network,
665 existing_subnet_cfgs,
666 mapping,
667 dynamic_updates=domain_updates,
668 glue=ranges,
669 is_glue_net=True,
509 )670 )
510 yield DNSReverseZoneConfig(671 else:
511 ns_host_name,672 existing_subnet_cfgs[network] = DNSReverseZoneConfig(
512 serial=serial,673 ns_host_name,
513 default_ttl=default_ttl,674 serial=serial,
514 network=network,675 default_ttl=default_ttl,
515 ns_host_name=ns_host_name,676 network=network,
516 rfc2317_ranges=ranges,677 ns_host_name=ns_host_name,
517 exclude=exclude_set,678 rfc2317_ranges=ranges,
518 dynamic_updates=domain_updates,679 dynamic_updates=domain_updates,
519 force_config_write=force_config_write,680 force_config_write=force_config_write,
520 )681 )
682 yield existing_subnet_cfgs[network]
521683
522 def __iter__(self):684 def __iter__(self):
523 """Iterate over zone configs.685 """Iterate over zone configs.
@@ -553,6 +715,7 @@ class ZoneGenerator:
553 default_ttl,715 default_ttl,
554 self._dynamic_updates,716 self._dynamic_updates,
555 self.force_config_write,717 self.force_config_write,
718 existing_subnet_cfgs=self._existing_subnet_cfgs,
556 ),719 ),
557 )720 )
558721
diff --git a/src/maastesting/noseplug.py b/src/maastesting/noseplug.py
index 02949ec..ff21c32 100644
--- a/src/maastesting/noseplug.py
+++ b/src/maastesting/noseplug.py
@@ -474,7 +474,9 @@ class CleanTestToolsFailure(Plugin):
474 ec, ev, tb = err474 ec, ev, tb = err
475 if ec is not _StringException:475 if ec is not _StringException:
476 return err476 return err
477 return Exception, Exception(*ev.args), tb477 if hasattr(ev, "args"):
478 return Exception, Exception(*ev.args), tb
479 return Exception, Exception(ev), tb
478480
479 formatError = formatFailure481 formatError = formatFailure
480482
diff --git a/src/provisioningserver/dns/tests/test_zoneconfig.py b/src/provisioningserver/dns/tests/test_zoneconfig.py
index 1433493..06e6c87 100644
--- a/src/provisioningserver/dns/tests/test_zoneconfig.py
+++ b/src/provisioningserver/dns/tests/test_zoneconfig.py
@@ -521,17 +521,16 @@ class TestDNSReverseZoneConfig(MAASTestCase):
521521
522 def test_computes_zone_file_config_file_paths(self):522 def test_computes_zone_file_config_file_paths(self):
523 domain = factory.make_name("zone")523 domain = factory.make_name("zone")
524 reverse_file_name = [524 # in order to merge changes, maasserver.dns.zone_generator.ZoneGenerator will split large subnets
525 "zone.%d.168.192.in-addr.arpa" % i for i in range(4)525 # meaning there's a 1:1 zonefile and DNSReverseZoneConfig
526 ]526 reverse_file_name = "zone.0.168.192.in-addr.arpa"
527 dns_zone_config = DNSReverseZoneConfig(527 dns_zone_config = DNSReverseZoneConfig(
528 domain, network=IPNetwork("192.168.0.0/22")528 domain, network=IPNetwork("192.168.0.0/24")
529 )
530 self.assertEqual(
531 os.path.join(get_zone_file_config_dir(), reverse_file_name),
532 dns_zone_config.zone_info[0].target_path,
529 )533 )
530 for i in range(4):
531 self.assertEqual(
532 os.path.join(get_zone_file_config_dir(), reverse_file_name[i]),
533 dns_zone_config.zone_info[i].target_path,
534 )
535534
536 def test_computes_zone_file_config_file_paths_for_small_network(self):535 def test_computes_zone_file_config_file_paths_for_small_network(self):
537 domain = factory.make_name("zone")536 domain = factory.make_name("zone")
@@ -555,20 +554,8 @@ class TestDNSReverseZoneConfig(MAASTestCase):
555 # A special case is the small subnet (less than 256 hosts for IPv4,554 # A special case is the small subnet (less than 256 hosts for IPv4,
556 # less than 16 hosts for IPv6), in which case, we follow RFC2317 with555 # less than 16 hosts for IPv6), in which case, we follow RFC2317 with
557 # the modern adjustment of using '-' instead of '/'.556 # the modern adjustment of using '-' instead of '/'.
558 zn = "%d.0.0.0.0.0.0.0.0.0.0.0.4.0.1.f.1.0.8.a.b.0.1.0.0.2.ip6.arpa"
559 expected = [557 expected = [
560 # IPv4 networks.558 # IPv4 networks.
561 # /22 ==> 4 /24 reverse zones
562 (
563 IPNetwork("192.168.0.1/22"),
564 [
565 DomainInfo(
566 IPNetwork("192.168.%d.0/24" % i),
567 "%d.168.192.in-addr.arpa" % i,
568 )
569 for i in range(4)
570 ],
571 ),
572 # /24 ==> 1 reverse zone559 # /24 ==> 1 reverse zone
573 (560 (
574 IPNetwork("192.168.0.1/24"),561 IPNetwork("192.168.0.1/24"),
@@ -625,27 +612,6 @@ class TestDNSReverseZoneConfig(MAASTestCase):
625 )612 )
626 ],613 ],
627 ),614 ),
628 # /2 with hex digits ==> 4 /4 reverse zones
629 (
630 IPNetwork("8000::/2"),
631 [
632 DomainInfo(IPNetwork("8000::/4"), "8.ip6.arpa"),
633 DomainInfo(IPNetwork("9000::/4"), "9.ip6.arpa"),
634 DomainInfo(IPNetwork("a000::/4"), "a.ip6.arpa"),
635 DomainInfo(IPNetwork("b000::/4"), "b.ip6.arpa"),
636 ],
637 ),
638 # /103 ==> 2 /104 reverse zones
639 (
640 IPNetwork("2001:ba8:1f1:400::/103"),
641 [
642 DomainInfo(
643 IPNetwork("2001:ba8:1f1:400:0:0:%d00:0000/104" % i),
644 zn % i,
645 )
646 for i in range(2)
647 ],
648 ),
649 # /125 ==> 1 reverse zone, based on RFC2317615 # /125 ==> 1 reverse zone, based on RFC2317
650 (616 (
651 IPNetwork("2001:ba8:1f1:400::/125"),617 IPNetwork("2001:ba8:1f1:400::/125"),
@@ -818,7 +784,7 @@ class TestDNSReverseZoneConfig(MAASTestCase):
818 target_dir = patch_zone_file_config_path(self)784 target_dir = patch_zone_file_config_path(self)
819 domain = factory.make_string()785 domain = factory.make_string()
820 ns_host_name = factory.make_name("ns")786 ns_host_name = factory.make_name("ns")
821 network = IPNetwork("192.168.0.1/22")787 network = IPNetwork("192.168.0.1/24")
822 dynamic_network = IPNetwork("192.168.0.1/28")788 dynamic_network = IPNetwork("192.168.0.1/28")
823 dns_zone_config = DNSReverseZoneConfig(789 dns_zone_config = DNSReverseZoneConfig(
824 domain,790 domain,
@@ -830,25 +796,24 @@ class TestDNSReverseZoneConfig(MAASTestCase):
830 ],796 ],
831 )797 )
832 dns_zone_config.write_config()798 dns_zone_config.write_config()
833 for sub in range(4):799 reverse_file_name = "zone.0.168.192.in-addr.arpa"
834 reverse_file_name = f"zone.{sub}.168.192.in-addr.arpa"800 expected_GEN_direct = dns_zone_config.get_GENERATE_directives(
835 expected_GEN_direct = dns_zone_config.get_GENERATE_directives(801 dynamic_network,
836 dynamic_network,802 domain,
837 domain,803 DomainInfo(
838 DomainInfo(804 IPNetwork("192.168.0.0/24"),
839 IPNetwork(f"192.168.{sub}.0/24"),805 "0.168.192.in-addr.arpa",
840 f"{sub}.168.192.in-addr.arpa",806 ),
841 ),807 )
842 )808 with open(os.path.join(target_dir, reverse_file_name), "r") as fh:
843 with open(os.path.join(target_dir, reverse_file_name), "r") as fh:809 contents = fh.read()
844 contents = fh.read()810 needles = [f"30 IN NS {ns_host_name}"] + [
845 needles = [f"30 IN NS {ns_host_name}"] + [811 f"$GENERATE {iterator_values} {reverse_dns} IN PTR {hostname}"
846 f"$GENERATE {iterator_values} {reverse_dns} IN PTR {hostname}"812 for iterator_values, reverse_dns, hostname in expected_GEN_direct
847 for iterator_values, reverse_dns, hostname in expected_GEN_direct813 ]
848 ]
849814
850 for needle in needles:815 for needle in needles:
851 self.assertIn(needle, contents)816 self.assertIn(needle, contents)
852817
853 def test_writes_reverse_dns_zone_config_for_small_network(self):818 def test_writes_reverse_dns_zone_config_for_small_network(self):
854 target_dir = patch_zone_file_config_path(self)819 target_dir = patch_zone_file_config_path(self)
@@ -1106,97 +1071,6 @@ class TestDNSReverseZoneConfig(MAASTestCase):
1106 ],1071 ],
1107 )1072 )
11081073
1109 def test_dynamic_updates_are_only_sent_for_specific_domain_info(self):
1110 patch_zone_file_config_path(self)
1111 domain = factory.make_string()
1112 network = IPNetwork("10.246.64.0/21")
1113 subnetwork1 = IPNetwork("10.246.64.0/24")
1114 subnetwork2 = IPNetwork("10.246.65.0/24")
1115 ip1 = factory.pick_ip_in_network(subnetwork1)
1116 ip2 = factory.pick_ip_in_network(subnetwork2)
1117 hostname1 = f"{factory.make_string()}.{domain}"
1118 hostname2 = f"{factory.make_string()}.{domain}"
1119 fwd_updates = [
1120 DynamicDNSUpdate(
1121 operation="INSERT",
1122 zone=domain,
1123 name=hostname1,
1124 rectype="A",
1125 answer=ip1,
1126 ),
1127 DynamicDNSUpdate(
1128 operation="INSERT",
1129 zone=domain,
1130 name=hostname2,
1131 rectype="A",
1132 answer=ip2,
1133 ),
1134 ]
1135 rev_updates = [
1136 DynamicDNSUpdate.as_reverse_record_update(update, network)
1137 for update in fwd_updates
1138 ]
1139 # gets changed to a /24 and any other space in the original
1140 # subnet is split into a separate zone for a given /24
1141 zone = DNSReverseZoneConfig(
1142 domain,
1143 serial=random.randint(1, 100),
1144 network=network,
1145 dynamic_updates=rev_updates,
1146 )
1147
1148 run_command = self.patch(actions, "run_command")
1149 zone.write_config()
1150 zone.write_config()
1151
1152 expected_stdin1 = "\n".join(
1153 [
1154 "server localhost",
1155 "zone 64.246.10.in-addr.arpa",
1156 f"update add {IPAddress(ip1).reverse_dns} {zone.default_ttl} PTR {hostname1}",
1157 f"update add 64.246.10.in-addr.arpa {zone.default_ttl} SOA 64.246.10.in-addr.arpa. nobody.example.com. {zone.serial} 600 1800 604800 {zone.default_ttl}",
1158 "send\n",
1159 ]
1160 )
1161
1162 expected_stdin2 = "\n".join(
1163 [
1164 "server localhost",
1165 "zone 65.246.10.in-addr.arpa",
1166 f"update add {IPAddress(ip2).reverse_dns} {zone.default_ttl} PTR {hostname2}",
1167 f"update add 65.246.10.in-addr.arpa {zone.default_ttl} SOA 65.246.10.in-addr.arpa. nobody.example.com. {zone.serial} 600 1800 604800 {zone.default_ttl}",
1168 "send\n",
1169 ]
1170 )
1171
1172 expected_stdin3 = "\n".join(
1173 [
1174 "server localhost",
1175 "zone 71.246.10.in-addr.arpa",
1176 f"update add 71.246.10.in-addr.arpa {zone.default_ttl} SOA 71.246.10.in-addr.arpa. nobody.example.com. {zone.serial} 600 1800 604800 {zone.default_ttl}",
1177 "send\n",
1178 ]
1179 )
1180
1181 run_command.assert_any_call(
1182 "nsupdate",
1183 "-k",
1184 get_nsupdate_key_path(),
1185 stdin=expected_stdin1.encode("ascii"),
1186 )
1187 run_command.assert_any_call(
1188 "nsupdate",
1189 "-k",
1190 get_nsupdate_key_path(),
1191 stdin=expected_stdin2.encode("ascii"),
1192 )
1193 run_command.assert_any_call(
1194 "nsupdate",
1195 "-k",
1196 get_nsupdate_key_path(),
1197 stdin=expected_stdin3.encode("ascii"),
1198 )
1199
12001074
1201class TestDNSReverseZoneConfig_GetGenerateDirectives(MAASTestCase):1075class TestDNSReverseZoneConfig_GetGenerateDirectives(MAASTestCase):
1202 """Tests for `DNSReverseZoneConfig.get_GENERATE_directives()`."""1076 """Tests for `DNSReverseZoneConfig.get_GENERATE_directives()`."""
diff --git a/src/provisioningserver/dns/zoneconfig.py b/src/provisioningserver/dns/zoneconfig.py
index 2e37e19..cf8a8ef 100644
--- a/src/provisioningserver/dns/zoneconfig.py
+++ b/src/provisioningserver/dns/zoneconfig.py
@@ -413,30 +413,11 @@ class DNSReverseZoneConfig(DomainConfigBase):
413 self._network = kwargs.pop("network", None)413 self._network = kwargs.pop("network", None)
414 self._dynamic_ranges = kwargs.pop("dynamic_ranges", [])414 self._dynamic_ranges = kwargs.pop("dynamic_ranges", [])
415 self._rfc2317_ranges = kwargs.pop("rfc2317_ranges", [])415 self._rfc2317_ranges = kwargs.pop("rfc2317_ranges", [])
416 self._exclude = kwargs.pop("exclude", set())416 zone_info = self.compose_zone_info(self._network)
417 zone_info = self.compose_zone_info(
418 self._network, exclude=self._exclude
419 )
420 super().__init__(domain, zone_info=zone_info, **kwargs)417 super().__init__(domain, zone_info=zone_info, **kwargs)
421418
422 @classmethod419 @classmethod
423 def _skip_if_overlaps(cls, first, base, step, network, exclude):420 def compose_zone_info(cls, network):
424 for other_network in exclude:
425 if (
426 first in other_network
427 and network.prefixlen < other_network.prefixlen
428 ): # allow the more specific overlapping subnet to create the zone config
429 try:
430 base += 1
431 first += step
432 except IndexError:
433 # IndexError occurs when we go from 255.255.255.255 to
434 # 0.0.0.0. If we hit that, we're all fine and done.
435 break
436 return (first, base)
437
438 @classmethod
439 def compose_zone_info(cls, network, exclude=()):
440 """Return the names of the reverse zones."""421 """Return the names of the reverse zones."""
441 # Generate the name of the reverse zone file:422 # Generate the name of the reverse zone file:
442 # Use netaddr's reverse_dns() to get the reverse IP name423 # Use netaddr's reverse_dns() to get the reverse IP name
@@ -444,9 +425,7 @@ class DNSReverseZoneConfig(DomainConfigBase):
444 # octets of that name (i.e. drop the octets that will be specified in425 # octets of that name (i.e. drop the octets that will be specified in
445 # the zone file).426 # the zone file).
446 # returns a list of (IPNetwork, zone_name, zonefile_path) tuples427 # returns a list of (IPNetwork, zone_name, zonefile_path) tuples
447 info = []
448 first = IPAddress(network.first)428 first = IPAddress(network.first)
449 last = IPAddress(network.last)
450 if first.version == 6:429 if first.version == 6:
451 # IPv6.430 # IPv6.
452 # 2001:89ab::/19 yields 8.1.0.0.2.ip6.arpa, and the full list431 # 2001:89ab::/19 yields 8.1.0.0.2.ip6.arpa, and the full list
@@ -488,40 +467,23 @@ class DNSReverseZoneConfig(DomainConfigBase):
488 split_zone = first.reverse_dns.split(".")467 split_zone = first.reverse_dns.split(".")
489 zone_rest = ".".join(split_zone[rest_limit:-1])468 zone_rest = ".".join(split_zone[rest_limit:-1])
490 base = int(split_zone[rest_limit - 1])469 base = int(split_zone[rest_limit - 1])
491 while first <= last:470
492 (first, base) = cls._skip_if_overlaps(471 # Rest_limit has bounds of 1..labelcount+1 (5 or 33).
493 first, base, step, network, exclude472 # If we're stripping any elements, then we just want base.name.
494 )473 if rest_limit > 1:
495 if first > last:474 if first.version == 6:
496 # if the excluding subnet pushes the base IP beyond the bounds of the generating subnet, we've reached the end and return early475 new_zone = f"{base:x}.{zone_rest}"
497 return info
498
499 # Rest_limit has bounds of 1..labelcount+1 (5 or 33).
500 # If we're stripping any elements, then we just want base.name.
501 if rest_limit > 1:
502 if first.version == 6:
503 new_zone = f"{base:x}.{zone_rest}"
504 else:
505 new_zone = "%d.%s" % (base, zone_rest)
506 # We didn't actually strip any elemnts, so base goes back with
507 # the prefixlen attached.
508 elif first.version == 6:
509 new_zone = "%x-%d.%s" % (base, network.prefixlen, zone_rest)
510 else:476 else:
511 new_zone = "%d-%d.%s" % (base, network.prefixlen, zone_rest)477 new_zone = f"{base:d}.{zone_rest}"
512 info.append(478 # We didn't actually strip any elemnts, so base goes back with
513 DomainInfo(479 # the prefixlen attached.
514 IPNetwork("%s/%d" % (first, subnet_prefix)), new_zone480 elif first.version == 6:
515 )481 new_zone = f"{base:x}-{network.prefixlen:d}.{zone_rest}"
516 )482 else:
517 base += 1483 new_zone = f"{base:d}-{network.prefixlen:d}.{zone_rest}"
518 try:484 return [
519 first += step485 DomainInfo(IPNetwork(f"{first}/{subnet_prefix:d}"), new_zone),
520 except IndexError:486 ]
521 # IndexError occurs when we go from 255.255.255.255 to
522 # 0.0.0.0. If we hit that, we're all fine and done.
523 break
524 return info
525487
526 @classmethod488 @classmethod
527 def get_PTR_mapping(cls, mapping, network):489 def get_PTR_mapping(cls, mapping, network):

Subscribers

People subscribed via source and target branches