Merge lp:~mpontillo/maas/lease-parser-ignore-free-leases--1.9 into lp:maas/1.9
- lease-parser-ignore-free-leases--1.9
- Merge into 1.9
Status: | Merged |
---|---|
Approved by: | Mike Pontillo |
Approved revision: | no longer in the source branch. |
Merged at revision: | 4597 |
Proposed branch: | lp:~mpontillo/maas/lease-parser-ignore-free-leases--1.9 |
Merge into: | lp:maas/1.9 |
Diff against target: |
449 lines (+192/-51) 5 files modified
src/maasserver/rpc/leases.py (+2/-0) src/maasserver/rpc/tests/test_regionservice.py (+6/-1) src/provisioningserver/dhcp/leases_parser.py (+32/-11) src/provisioningserver/dhcp/leases_parser_fast.py (+24/-6) src/provisioningserver/dhcp/tests/test_leases_parser.py (+128/-33) |
To merge this branch: | bzr merge lp:~mpontillo/maas/lease-parser-ignore-free-leases--1.9 |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Blake Rouse (community) | Approve | ||
Review via email: mp+316300@code.launchpad.net |
Commit message
DHCP lease parser fixes.
* Fix improper handling of DHCP leases with a 'free' binding state,
and/or no 'ends' time.
* Fix lease parser to properly parse static host mappings with a '-'
in the mapping key (such as the entries that MAAS itself writes).
* Fix DNS zones to be updated when lease parser finishes and updates
the region.
* Fix lease parser to properly remove released leases.
* Fix traceback that could occur when the fast lease parser attempts
interpret a released lease.
Description of the change
Mike Pontillo (mpontillo) wrote : | # |
Mike Pontillo (mpontillo) wrote : | # |
To prevent the possibility of stale leases remaining in the StaticIPAddress table as DISCOVERED, an additional fix was needed to filter stale leases (in case an IP address was assigned to another host).
One issue remains that I found in testing: after the lease parser runs, the zone file isn't udpated. (Rather, I saw it get updated ~3 seconds before the lease parser finished and uploaded the leases.) Therefore I had an out-of-date zone file even though the lease parser had already determined the correct (hostname, ip) mapping, and that mapping was shown in the UI. (Workaround: `service maas-regiond restart`.)
Blake Rouse (blake-rouse) wrote : | # |
Looks good. As for the restart of maas-regiond. Did it never get updated? As without it updating the zone files issues will still exists resolving dynamic IP addresses.
Mike Pontillo (mpontillo) wrote : | # |
Blake, thanks for taking a look. It turned out there was a separate issue with BIND that also prevented the DNS zones from being properly updated. LaMont has landed a fix (and has queued a corresponding SRU for BIND).
Blake Rouse (blake-rouse) wrote : | # |
Awesome. Glad this is resolved. Looks good!
MAAS Lander (maas-lander) wrote : | # |
The attempt to merge lp:~mpontillo/maas/lease-parser-ignore-free-leases--1.9 into lp:maas/1.9 failed. Below is the output from the failed tests.
Ign http://
Get:1 http://
Get:2 http://
Hit http://
Hit http://
Hit http://
Get:3 http://
Get:4 http://
Get:5 http://
Get:6 http://
Get:7 http://
Get:8 http://
Get:9 http://
Get:10 http://
Get:11 http://
Get:12 http://
Get:13 http://
Get:14 http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Fetched 2,982 kB in 1s (1,802 kB/s)
Reading package lists...
sudo DEBIAN_
MAAS Lander (maas-lander) wrote : | # |
The attempt to merge lp:~mpontillo/maas/lease-parser-ignore-free-leases--1.9 into lp:maas/1.9 failed. Below is the output from the failed tests.
Ign http://
Get:1 http://
Get:2 http://
Hit http://
Hit http://
Hit http://
Get:3 http://
Get:4 http://
Get:5 http://
Get:6 http://
Get:7 http://
Get:8 http://
Get:9 http://
Get:10 http://
Get:11 http://
Get:12 http://
Get:13 http://
Get:14 http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Fetched 2,982 kB in 1s (1,718 kB/s)
Reading package lists...
sudo DEBIAN_
Mike Pontillo (mpontillo) wrote : | # |
Hm. I don't understand why this keeps failing. I just SSH'd to the lander and tried the same test, and it passed.
MAAS Lander (maas-lander) wrote : | # |
The attempt to merge lp:~mpontillo/maas/lease-parser-ignore-free-leases--1.9 into lp:maas/1.9 failed. Below is the output from the failed tests.
Get:1 http://
Get:2 http://
Get:3 http://
Get:4 http://
Get:5 http://
Ign http://
Get:6 http://
Hit http://
Hit http://
Hit http://
Get:7 http://
Get:8 http://
Get:9 http://
Get:10 http://
Get:11 http://
Get:12 http://
Get:13 http://
Get:14 http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Hit http://
Fetched 2,982 kB in 2s (1,104 kB/s)
Reading package lists...
sudo DEBIAN_
Preview Diff
1 | === modified file 'src/maasserver/rpc/leases.py' | |||
2 | --- src/maasserver/rpc/leases.py 2015-08-27 16:07:30 +0000 | |||
3 | +++ src/maasserver/rpc/leases.py 2017-02-07 17:35:36 +0000 | |||
4 | @@ -16,6 +16,7 @@ | |||
5 | 16 | "update_leases", | 16 | "update_leases", |
6 | 17 | ] | 17 | ] |
7 | 18 | 18 | ||
8 | 19 | from maasserver.dns.config import dns_update_all_zones | ||
9 | 19 | from maasserver.models.nodegroup import NodeGroup | 20 | from maasserver.models.nodegroup import NodeGroup |
10 | 20 | from maasserver.models.staticipaddress import StaticIPAddress | 21 | from maasserver.models.staticipaddress import StaticIPAddress |
11 | 21 | from maasserver.utils.orm import transactional | 22 | from maasserver.utils.orm import transactional |
12 | @@ -49,4 +50,5 @@ | |||
13 | 49 | else: | 50 | else: |
14 | 50 | leases = convert_mappings_to_leases(mappings) | 51 | leases = convert_mappings_to_leases(mappings) |
15 | 51 | StaticIPAddress.objects.update_leases(nodegroup, leases) | 52 | StaticIPAddress.objects.update_leases(nodegroup, leases) |
16 | 53 | dns_update_all_zones() | ||
17 | 52 | return {} | 54 | return {} |
18 | 53 | 55 | ||
19 | === modified file 'src/maasserver/rpc/tests/test_regionservice.py' | |||
20 | --- src/maasserver/rpc/tests/test_regionservice.py 2015-11-06 16:27:35 +0000 | |||
21 | +++ src/maasserver/rpc/tests/test_regionservice.py 2017-02-07 17:35:36 +0000 | |||
22 | @@ -57,6 +57,7 @@ | |||
23 | 57 | from maasserver.models.nodegroup import NodeGroup | 57 | from maasserver.models.nodegroup import NodeGroup |
24 | 58 | from maasserver.rpc import ( | 58 | from maasserver.rpc import ( |
25 | 59 | events as events_module, | 59 | events as events_module, |
26 | 60 | leases as leases_module, | ||
27 | 60 | regionservice, | 61 | regionservice, |
28 | 61 | ) | 62 | ) |
29 | 62 | from maasserver.rpc.regionservice import ( | 63 | from maasserver.rpc.regionservice import ( |
30 | @@ -84,6 +85,7 @@ | |||
31 | 84 | from maasserver.utils.threads import deferToDatabase | 85 | from maasserver.utils.threads import deferToDatabase |
32 | 85 | from maastesting.djangotestcase import DjangoTransactionTestCase | 86 | from maastesting.djangotestcase import DjangoTransactionTestCase |
33 | 86 | from maastesting.matchers import ( | 87 | from maastesting.matchers import ( |
34 | 88 | MockCalledOnce, | ||
35 | 87 | MockCalledOnceWith, | 89 | MockCalledOnceWith, |
36 | 88 | MockCallsMatch, | 90 | MockCallsMatch, |
37 | 89 | MockNotCalled, | 91 | MockNotCalled, |
38 | @@ -424,9 +426,11 @@ | |||
39 | 424 | 426 | ||
40 | 425 | @wait_for_reactor | 427 | @wait_for_reactor |
41 | 426 | @inlineCallbacks | 428 | @inlineCallbacks |
43 | 427 | def test__stores_leases(self): | 429 | def test__stores_leases_and_updates_dns(self): |
44 | 430 | dns_update = self.patch(leases_module, 'dns_update_all_zones') | ||
45 | 428 | interface, ngi, nodegroup = yield deferToDatabase( | 431 | interface, ngi, nodegroup = yield deferToDatabase( |
46 | 429 | self.make_interface_on_managed_cluster_interface) | 432 | self.make_interface_on_managed_cluster_interface) |
47 | 433 | |||
48 | 430 | mapping = { | 434 | mapping = { |
49 | 431 | "ip": ngi.ip_range_low, | 435 | "ip": ngi.ip_range_low, |
50 | 432 | "mac": interface.mac_address.get_raw(), | 436 | "mac": interface.mac_address.get_raw(), |
51 | @@ -436,6 +440,7 @@ | |||
52 | 436 | b"uuid": nodegroup.uuid, b"mappings": [mapping]}) | 440 | b"uuid": nodegroup.uuid, b"mappings": [mapping]}) |
53 | 437 | 441 | ||
54 | 438 | self.assertThat(response, Equals({})) | 442 | self.assertThat(response, Equals({})) |
55 | 443 | self.assertThat(dns_update, MockCalledOnce()) | ||
56 | 439 | 444 | ||
57 | 440 | [(ip, mac)] = yield deferToDatabase( | 445 | [(ip, mac)] = yield deferToDatabase( |
58 | 441 | self.get_leases_for, nodegroup=nodegroup) | 446 | self.get_leases_for, nodegroup=nodegroup) |
59 | 442 | 447 | ||
60 | === modified file 'src/provisioningserver/dhcp/leases_parser.py' | |||
61 | --- src/provisioningserver/dhcp/leases_parser.py 2016-05-11 18:50:52 +0000 | |||
62 | +++ src/provisioningserver/dhcp/leases_parser.py 2017-02-07 17:35:36 +0000 | |||
63 | @@ -38,7 +38,9 @@ | |||
64 | 38 | ) | 38 | ) |
65 | 39 | 39 | ||
66 | 40 | 40 | ||
68 | 41 | ip = Regex("[:0-9a-fA-F][:.0-9a-fA-F]{2,38}") | 41 | LEASE_TIME_FORMAT = '%w %Y/%m/%d %H:%M:%S' |
69 | 42 | |||
70 | 43 | ip = Regex("[:0-9a-fA-F][-:.0-9a-fA-F]{2,38}") | ||
71 | 42 | mac = Regex("[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5}") | 44 | mac = Regex("[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){5}") |
72 | 43 | hardware_type = Regex('[A-Za-z0-9_-]+') | 45 | hardware_type = Regex('[A-Za-z0-9_-]+') |
73 | 44 | args = Regex('[^"{;]+') | QuotedString('"') | 46 | args = Regex('[^"{;]+') | QuotedString('"') |
74 | @@ -125,11 +127,20 @@ | |||
75 | 125 | expiry, or None if the lease has no expiry date. | 127 | expiry, or None if the lease has no expiry date. |
76 | 126 | """ | 128 | """ |
77 | 127 | assert is_lease(lease) | 129 | assert is_lease(lease) |
80 | 128 | ends = getattr(lease, 'ends', None) | 130 | end_time = getattr(lease, 'ends', '') |
81 | 129 | if ends is None or len(ends) == 0 or ends.lower() == 'never': | 131 | if end_time is None or len(end_time) == 0: |
82 | 132 | binding_state = getattr(lease, 'binding state', '') | ||
83 | 133 | binding_state = binding_state.lower() | ||
84 | 134 | if binding_state == 'free': | ||
85 | 135 | # For a 'free' lease, the release time is the 'starts' time. | ||
86 | 136 | start_time = getattr(lease, 'starts', '') | ||
87 | 137 | if start_time is None or len(start_time) == 0: | ||
88 | 138 | return None | ||
89 | 139 | return datetime.strptime(start_time, LEASE_TIME_FORMAT) | ||
90 | 140 | elif end_time.lower() == 'never': | ||
91 | 130 | return None | 141 | return None |
92 | 131 | else: | 142 | else: |
94 | 132 | return datetime.strptime(ends, '%w %Y/%m/%d %H:%M:%S') | 143 | return datetime.strptime(end_time, LEASE_TIME_FORMAT) |
95 | 133 | 144 | ||
96 | 134 | 145 | ||
97 | 135 | def has_expired(lease, now): | 146 | def has_expired(lease, now): |
98 | @@ -153,13 +164,17 @@ | |||
99 | 153 | def gather_leases(hosts_and_leases): | 164 | def gather_leases(hosts_and_leases): |
100 | 154 | """Find current leases among `hosts_and_leases`.""" | 165 | """Find current leases among `hosts_and_leases`.""" |
101 | 155 | now = datetime.utcnow() | 166 | now = datetime.utcnow() |
109 | 156 | # If multiple leases for the same address are valid at the same | 167 | # Ensure we have the most recent MAC leased to each IP address. |
110 | 157 | # time, for whatever reason, the list will contain all of them. | 168 | leases = OrderedDict() |
111 | 158 | return [ | 169 | for lease in filter(is_lease, hosts_and_leases): |
112 | 159 | (lease.host, lease.hardware.mac) | 170 | lease_mac = get_lease_mac(lease) |
113 | 160 | for lease in filter(is_lease, hosts_and_leases) | 171 | lease_ip = lease.host |
114 | 161 | if not has_expired(lease, now) | 172 | if not has_expired(lease, now) and lease_mac is not None: |
115 | 162 | ] | 173 | leases[lease_ip] = lease_mac |
116 | 174 | else: | ||
117 | 175 | if lease_ip in leases: | ||
118 | 176 | del leases[lease_ip] | ||
119 | 177 | return [(ip, mac) for ip, mac in leases.items()] | ||
120 | 163 | 178 | ||
121 | 164 | 179 | ||
122 | 165 | def get_host_mac(host): | 180 | def get_host_mac(host): |
123 | @@ -173,6 +188,12 @@ | |||
124 | 173 | return None | 188 | return None |
125 | 174 | else: | 189 | else: |
126 | 175 | return host | 190 | return host |
127 | 191 | # In this case, it's stored the same way a lease MAC is stored. | ||
128 | 192 | return get_lease_mac(host) | ||
129 | 193 | |||
130 | 194 | |||
131 | 195 | def get_lease_mac(host): | ||
132 | 196 | """Get the MAC address from a lease declaration.""" | ||
133 | 176 | hardware = getattr(host, 'hardware', None) | 197 | hardware = getattr(host, 'hardware', None) |
134 | 177 | if hardware in (None, '', b''): | 198 | if hardware in (None, '', b''): |
135 | 178 | return None | 199 | return None |
136 | 179 | 200 | ||
137 | === modified file 'src/provisioningserver/dhcp/leases_parser_fast.py' | |||
138 | --- src/provisioningserver/dhcp/leases_parser_fast.py 2015-09-09 15:02:30 +0000 | |||
139 | +++ src/provisioningserver/dhcp/leases_parser_fast.py 2017-02-07 17:35:36 +0000 | |||
140 | @@ -23,7 +23,10 @@ | |||
141 | 23 | 'parse_leases', | 23 | 'parse_leases', |
142 | 24 | ] | 24 | ] |
143 | 25 | 25 | ||
145 | 26 | from collections import defaultdict | 26 | from collections import ( |
146 | 27 | defaultdict, | ||
147 | 28 | OrderedDict, | ||
148 | 29 | ) | ||
149 | 27 | from datetime import datetime | 30 | from datetime import datetime |
150 | 28 | from itertools import chain | 31 | from itertools import chain |
151 | 29 | import re | 32 | import re |
152 | @@ -31,6 +34,7 @@ | |||
153 | 31 | from provisioningserver.dhcp.leases_parser import ( | 34 | from provisioningserver.dhcp.leases_parser import ( |
154 | 32 | get_host_ip, | 35 | get_host_ip, |
155 | 33 | get_host_mac, | 36 | get_host_mac, |
156 | 37 | get_lease_mac, | ||
157 | 34 | has_expired, | 38 | has_expired, |
158 | 35 | is_host, | 39 | is_host, |
159 | 36 | is_lease, | 40 | is_lease, |
160 | @@ -43,7 +47,7 @@ | |||
161 | 43 | ^\s* # Ignore leading whitespace on each line. | 47 | ^\s* # Ignore leading whitespace on each line. |
162 | 44 | (host|lease) # Look only for host or lease stanzas. | 48 | (host|lease) # Look only for host or lease stanzas. |
163 | 45 | \s+ # Mandatory whitespace. | 49 | \s+ # Mandatory whitespace. |
165 | 46 | ([0-9a-fA-F.:]+) # Capture the IP/MAC address for this stanza. | 50 | ([0-9a-fA-F.:-]+) # Capture the IP/MAC address for this stanza. |
166 | 47 | \s*{ # Optional whitespace then an opening brace. | 51 | \s*{ # Optional whitespace then an opening brace. |
167 | 48 | ''', | 52 | ''', |
168 | 49 | re.MULTILINE | re.DOTALL | re.VERBOSE) | 53 | re.MULTILINE | re.DOTALL | re.VERBOSE) |
169 | @@ -71,15 +75,29 @@ | |||
170 | 71 | 75 | ||
171 | 72 | 76 | ||
172 | 73 | def parse_leases(leases_contents): | 77 | def parse_leases(leases_contents): |
174 | 74 | results = [] | 78 | leases = OrderedDict() |
175 | 79 | hosts = [] | ||
176 | 75 | now = datetime.utcnow() | 80 | now = datetime.utcnow() |
177 | 76 | for entry in extract_leases(leases_contents): | 81 | for entry in extract_leases(leases_contents): |
178 | 77 | if is_lease(entry): | 82 | if is_lease(entry): |
181 | 78 | if not has_expired(entry, now): | 83 | mac = get_lease_mac(entry) |
182 | 79 | results.append((entry.host, entry.hardware.mac)) | 84 | if not has_expired(entry, now) and mac is not None: |
183 | 85 | leases[entry.host] = entry.hardware.mac | ||
184 | 86 | else: | ||
185 | 87 | # Expired or released lease. | ||
186 | 88 | if entry.host in leases: | ||
187 | 89 | del leases[entry.host] | ||
188 | 80 | elif is_host(entry): | 90 | elif is_host(entry): |
189 | 81 | mac = get_host_mac(entry) | 91 | mac = get_host_mac(entry) |
190 | 82 | ip = get_host_ip(entry) | 92 | ip = get_host_ip(entry) |
191 | 83 | if ip and mac: | 93 | if ip and mac: |
193 | 84 | results.append((ip, mac)) | 94 | # A host entry came later than a lease entry for the same IP |
194 | 95 | # address. Letting them both stay will confuse MAAS by allowing | ||
195 | 96 | # an ephemeral lease to exist at the same time as a static | ||
196 | 97 | # host map. | ||
197 | 98 | if ip in leases: | ||
198 | 99 | del leases[ip] | ||
199 | 100 | hosts.append((ip, mac)) | ||
200 | 101 | results = leases.items() | ||
201 | 102 | results.extend(hosts) | ||
202 | 85 | return results | 103 | return results |
203 | 86 | 104 | ||
204 | === modified file 'src/provisioningserver/dhcp/tests/test_leases_parser.py' | |||
205 | --- src/provisioningserver/dhcp/tests/test_leases_parser.py 2016-05-11 18:50:52 +0000 | |||
206 | +++ src/provisioningserver/dhcp/tests/test_leases_parser.py 2017-02-07 17:35:36 +0000 | |||
207 | @@ -41,26 +41,32 @@ | |||
208 | 41 | 41 | ||
209 | 42 | class Lease(object): | 42 | class Lease(object): |
210 | 43 | 43 | ||
212 | 44 | def __init__(self, lease_or_host, host, fixed_address, hardware, ends): | 44 | def __init__( |
213 | 45 | self, lease_or_host, host, fixed_address, hardware, starts, ends, | ||
214 | 46 | binding_state=None): | ||
215 | 45 | self.lease_or_host = lease_or_host | 47 | self.lease_or_host = lease_or_host |
216 | 46 | self.host = host | 48 | self.host = host |
217 | 47 | self.hardware = hardware | 49 | self.hardware = hardware |
218 | 50 | self.starts = starts | ||
219 | 48 | setattr(self, 'fixed-address', fixed_address) | 51 | setattr(self, 'fixed-address', fixed_address) |
220 | 49 | self.ends = ends | 52 | self.ends = ends |
221 | 53 | if binding_state is not None: | ||
222 | 54 | setattr(self, 'binding state', binding_state) | ||
223 | 50 | 55 | ||
224 | 51 | def __iter__(self): | 56 | def __iter__(self): |
225 | 52 | return iter(self.__dict__.keys()) | 57 | return iter(self.__dict__.keys()) |
226 | 53 | 58 | ||
227 | 54 | 59 | ||
230 | 55 | def fake_parsed_lease(ip=None, mac=None, ends=None, | 60 | def fake_parsed_lease(ip=None, mac=None, starts=None, ends=None, |
231 | 56 | entry_type='lease'): | 61 | entry_type='lease', state=None): |
232 | 57 | """Fake a lease as produced by the parser.""" | 62 | """Fake a lease as produced by the parser.""" |
233 | 58 | if ip is None: | 63 | if ip is None: |
234 | 59 | ip = factory.make_ipv4_address() | 64 | ip = factory.make_ipv4_address() |
235 | 60 | if mac is None: | 65 | if mac is None: |
236 | 61 | mac = factory.make_mac_address() | 66 | mac = factory.make_mac_address() |
237 | 62 | Hardware = namedtuple('Hardware', ['mac']) | 67 | Hardware = namedtuple('Hardware', ['mac']) |
239 | 63 | lease = Lease(entry_type, ip, ip, Hardware(mac), ends) | 68 | lease = Lease( |
240 | 69 | entry_type, ip, ip, Hardware(mac), starts, ends, binding_state=state) | ||
241 | 64 | return lease | 70 | return lease |
242 | 65 | 71 | ||
243 | 66 | 72 | ||
244 | @@ -312,7 +318,13 @@ | |||
245 | 312 | """ % params)) | 318 | """ % params)) |
246 | 313 | self.assertEqual([(params['ip'], params['mac'])], leases) | 319 | self.assertEqual([(params['ip'], params['mac'])], leases) |
247 | 314 | 320 | ||
249 | 315 | def test_parse_leases_takes_all_leases_for_address(self): | 321 | def test_parse_leases_takes_current_lease_for_address(self): |
250 | 322 | # Note: a previous version of this test case checked that two leases | ||
251 | 323 | # can exist at the same time for a single IP address. This has been | ||
252 | 324 | # removed in order to prevent stale entries from polluting MAAS's | ||
253 | 325 | # discoveries with old information. To support bonds, static host | ||
254 | 326 | # mappings will still allow more than one MAC address to be assigned to | ||
255 | 327 | # the same IP address. | ||
256 | 316 | params = { | 328 | params = { |
257 | 317 | 'ip': factory.make_ipv4_address(), | 329 | 'ip': factory.make_ipv4_address(), |
258 | 318 | 'old_owner': factory.make_mac_address(), | 330 | 'old_owner': factory.make_mac_address(), |
259 | @@ -321,15 +333,35 @@ | |||
260 | 321 | leases = self.parse(dedent("""\ | 333 | leases = self.parse(dedent("""\ |
261 | 322 | lease %(ip)s { | 334 | lease %(ip)s { |
262 | 323 | hardware ethernet %(old_owner)s; | 335 | hardware ethernet %(old_owner)s; |
263 | 336 | ends 0 1990/01/01 00:00:00; | ||
264 | 324 | } | 337 | } |
265 | 325 | lease %(ip)s { | 338 | lease %(ip)s { |
266 | 326 | hardware ethernet %(new_owner)s; | 339 | hardware ethernet %(new_owner)s; |
267 | 327 | } | 340 | } |
268 | 328 | """ % params)) | 341 | """ % params)) |
269 | 329 | self.assertEqual( | 342 | self.assertEqual( |
273 | 330 | [(params['ip'], params['old_owner']), | 343 | [(params['ip'], params['new_owner'])], leases) |
274 | 331 | (params['ip'], params['new_owner'])], | 344 | |
275 | 332 | leases) | 345 | def test_multiple_host_declarations_are_reported(self): |
276 | 346 | params = { | ||
277 | 347 | 'ip': factory.make_ipv4_address(), | ||
278 | 348 | 'bondmac1': factory.make_mac_address(), | ||
279 | 349 | 'bondmac2': factory.make_mac_address(), | ||
280 | 350 | } | ||
281 | 351 | leases = self.parse(dedent("""\ | ||
282 | 352 | host %(bondmac1)s { | ||
283 | 353 | hardware ethernet %(bondmac1)s; | ||
284 | 354 | fixed-address %(ip)s; | ||
285 | 355 | } | ||
286 | 356 | host %(bondmac2)s { | ||
287 | 357 | hardware ethernet %(bondmac2)s; | ||
288 | 358 | fixed-address %(ip)s; | ||
289 | 359 | } | ||
290 | 360 | """ % params)) | ||
291 | 361 | self.assertEqual([ | ||
292 | 362 | (params['ip'], params['bondmac1']), | ||
293 | 363 | (params['ip'], params['bondmac2']) | ||
294 | 364 | ], leases) | ||
295 | 333 | 365 | ||
296 | 334 | def test_parse_leases_recognizes_host_deleted_statement_as_rubout(self): | 366 | def test_parse_leases_recognizes_host_deleted_statement_as_rubout(self): |
297 | 335 | params = { | 367 | params = { |
298 | @@ -362,6 +394,39 @@ | |||
299 | 362 | 394 | ||
300 | 363 | class TestLeasesParserFast(MAASTestCase): | 395 | class TestLeasesParserFast(MAASTestCase): |
301 | 364 | 396 | ||
302 | 397 | def test_handles_dash_separator_for_host_mapping(self): | ||
303 | 398 | ip = factory.make_ipv4_address() | ||
304 | 399 | mac = factory.make_mac_address() | ||
305 | 400 | mac_dash = mac.replace(":", "-") | ||
306 | 401 | leases = leases_parser_fast.parse_leases(dedent("""\ | ||
307 | 402 | host %s { | ||
308 | 403 | hardware ethernet %s; | ||
309 | 404 | fixed-address %s; | ||
310 | 405 | } | ||
311 | 406 | """ % (mac_dash, mac, ip))) | ||
312 | 407 | self.assertEqual([(ip, mac)], leases) | ||
313 | 408 | |||
314 | 409 | def test_multiple_host_declarations_are_reported(self): | ||
315 | 410 | params = { | ||
316 | 411 | 'ip': factory.make_ipv4_address(), | ||
317 | 412 | 'bondmac1': factory.make_mac_address(), | ||
318 | 413 | 'bondmac2': factory.make_mac_address(), | ||
319 | 414 | } | ||
320 | 415 | leases = leases_parser_fast.parse_leases(dedent("""\ | ||
321 | 416 | host %(bondmac1)s { | ||
322 | 417 | hardware ethernet %(bondmac1)s; | ||
323 | 418 | fixed-address %(ip)s; | ||
324 | 419 | } | ||
325 | 420 | host %(bondmac2)s { | ||
326 | 421 | hardware ethernet %(bondmac2)s; | ||
327 | 422 | fixed-address %(ip)s; | ||
328 | 423 | } | ||
329 | 424 | """ % params)) | ||
330 | 425 | self.assertEqual([ | ||
331 | 426 | (params['ip'], params['bondmac1']), | ||
332 | 427 | (params['ip'], params['bondmac2']) | ||
333 | 428 | ], leases) | ||
334 | 429 | |||
335 | 365 | def test_expired_lease_does_not_shadow_earlier_host_stanza(self): | 430 | def test_expired_lease_does_not_shadow_earlier_host_stanza(self): |
336 | 366 | params = { | 431 | params = { |
337 | 367 | 'ip': factory.make_ipv4_address(), | 432 | 'ip': factory.make_ipv4_address(), |
338 | @@ -402,33 +467,55 @@ | |||
339 | 402 | } | 467 | } |
340 | 403 | """ % params)) | 468 | """ % params)) |
341 | 404 | # The lease hasn't expired, so shadows the earlier host stanza. | 469 | # The lease hasn't expired, so shadows the earlier host stanza. |
365 | 405 | self.assertEqual( | 470 | # (But since the host mapping has not been removed, it takes precedence |
366 | 406 | [(params["ip"], params["mac1"]), (params["ip"], params["mac2"])], | 471 | # by coming later in the list, since that should better describe the |
367 | 407 | leases) | 472 | # intent of the MAAS administrator.) |
345 | 408 | |||
346 | 409 | def test_host_stanza_shadows_earlier_active_lease(self): | ||
347 | 410 | params = { | ||
348 | 411 | 'ip': factory.make_ipv4_address(), | ||
349 | 412 | 'mac1': factory.make_mac_address(), | ||
350 | 413 | 'mac2': factory.make_mac_address(), | ||
351 | 414 | } | ||
352 | 415 | leases = leases_parser_fast.parse_leases(dedent("""\ | ||
353 | 416 | lease %(ip)s { | ||
354 | 417 | starts 5 2010/01/01 00:00:01; | ||
355 | 418 | hardware ethernet %(mac2)s; | ||
356 | 419 | } | ||
357 | 420 | host %(ip)s { | ||
358 | 421 | dynamic; | ||
359 | 422 | hardware ethernet %(mac1)s; | ||
360 | 423 | fixed-address %(ip)s; | ||
361 | 424 | } | ||
362 | 425 | """ % params)) | ||
363 | 426 | # The lease hasn't expired, but the host entry is later, so it | ||
364 | 427 | # shadows the earlier lease stanza. | ||
368 | 428 | self.assertEqual( | 473 | self.assertEqual( |
369 | 429 | [(params["ip"], params["mac2"]), (params["ip"], params["mac1"])], | 474 | [(params["ip"], params["mac2"]), (params["ip"], params["mac1"])], |
370 | 430 | leases) | 475 | leases) |
371 | 431 | 476 | ||
372 | 477 | def test_host_stanza_replaces_earlier_active_lease(self): | ||
373 | 478 | params = { | ||
374 | 479 | 'ip': factory.make_ipv4_address(), | ||
375 | 480 | 'mac1': factory.make_mac_address(), | ||
376 | 481 | 'mac2': factory.make_mac_address(), | ||
377 | 482 | } | ||
378 | 483 | leases = leases_parser_fast.parse_leases(dedent("""\ | ||
379 | 484 | lease %(ip)s { | ||
380 | 485 | starts 5 2010/01/01 00:00:01; | ||
381 | 486 | hardware ethernet %(mac2)s; | ||
382 | 487 | } | ||
383 | 488 | host %(ip)s { | ||
384 | 489 | dynamic; | ||
385 | 490 | hardware ethernet %(mac1)s; | ||
386 | 491 | fixed-address %(ip)s; | ||
387 | 492 | } | ||
388 | 493 | """ % params)) | ||
389 | 494 | # The lease hasn't expired, but the host entry is later. So the host | ||
390 | 495 | # mapping takes precedence. | ||
391 | 496 | self.assertEqual( | ||
392 | 497 | [(params["ip"], params["mac1"])], leases) | ||
393 | 498 | |||
394 | 499 | def test_released_lease_with_no_end_time_is_released(self): | ||
395 | 500 | params = { | ||
396 | 501 | 'ip': factory.make_ipv4_address(), | ||
397 | 502 | 'mac1': factory.make_mac_address(), | ||
398 | 503 | 'mac2': factory.make_mac_address(), | ||
399 | 504 | } | ||
400 | 505 | leases = leases_parser_fast.parse_leases(dedent("""\ | ||
401 | 506 | lease %(ip)s { | ||
402 | 507 | starts 5 2010/01/01 00:00:01; | ||
403 | 508 | hardware ethernet %(mac2)s; | ||
404 | 509 | } | ||
405 | 510 | lease %(ip)s { | ||
406 | 511 | dynamic; | ||
407 | 512 | binding state free; | ||
408 | 513 | starts 0 1990/01/01 00:00:00; | ||
409 | 514 | } | ||
410 | 515 | """ % params)) | ||
411 | 516 | # The lease was added and then removed, so we expect a no-op. | ||
412 | 517 | self.assertEqual([], leases) | ||
413 | 518 | |||
414 | 432 | 519 | ||
415 | 433 | class TestLeasesParserFunctions(MAASTestCase): | 520 | class TestLeasesParserFunctions(MAASTestCase): |
416 | 434 | 521 | ||
417 | @@ -440,6 +527,14 @@ | |||
418 | 440 | hour=03, minute=04, second=05), | 527 | hour=03, minute=04, second=05), |
419 | 441 | get_expiry_date(lease)) | 528 | get_expiry_date(lease)) |
420 | 442 | 529 | ||
421 | 530 | def test_get_expiry_date_uses_start_date_for_free_lease(self): | ||
422 | 531 | lease = fake_parsed_lease(starts='0 2011/01/02 03:04:05', state='free') | ||
423 | 532 | self.assertEqual( | ||
424 | 533 | datetime( | ||
425 | 534 | year=2011, month=01, day=02, | ||
426 | 535 | hour=03, minute=04, second=05), | ||
427 | 536 | get_expiry_date(lease)) | ||
428 | 537 | |||
429 | 443 | def test_get_expiry_date_returns_None_for_never(self): | 538 | def test_get_expiry_date_returns_None_for_never(self): |
430 | 444 | self.assertIsNone( | 539 | self.assertIsNone( |
431 | 445 | get_expiry_date(fake_parsed_lease(ends='never'))) | 540 | get_expiry_date(fake_parsed_lease(ends='never'))) |
432 | @@ -492,7 +587,7 @@ | |||
433 | 492 | ] | 587 | ] |
434 | 493 | self.assertEqual([(ip, new_owner)], gather_leases(leases)) | 588 | self.assertEqual([(ip, new_owner)], gather_leases(leases)) |
435 | 494 | 589 | ||
437 | 495 | def test_gather_leases_ignores_ordering(self): | 590 | def test_ordering_is_important_to_gather_leases(self): |
438 | 496 | earlier = '1 2001/01/01 00:00:00' | 591 | earlier = '1 2001/01/01 00:00:00' |
439 | 497 | ip = factory.make_ipv4_address() | 592 | ip = factory.make_ipv4_address() |
440 | 498 | old_owner = factory.make_mac_address() | 593 | old_owner = factory.make_mac_address() |
441 | @@ -501,7 +596,7 @@ | |||
442 | 501 | fake_parsed_lease(ip=ip, mac=new_owner), | 596 | fake_parsed_lease(ip=ip, mac=new_owner), |
443 | 502 | fake_parsed_lease(ip=ip, mac=old_owner, ends=earlier), | 597 | fake_parsed_lease(ip=ip, mac=old_owner, ends=earlier), |
444 | 503 | ] | 598 | ] |
446 | 504 | self.assertEqual([(ip, new_owner)], gather_leases(leases)) | 599 | self.assertEqual([], gather_leases(leases)) |
447 | 505 | 600 | ||
448 | 506 | def test_gather_leases_ignores_host_declarations(self): | 601 | def test_gather_leases_ignores_host_declarations(self): |
449 | 507 | self.assertEqual([], gather_leases([fake_parsed_host()])) | 602 | self.assertEqual([], gather_leases([fake_parsed_host()])) |
The way this code works is subtle. There is a difference between a "free" DHCP lease (free because it has not been renewed) and a "released" DHCP lease. And that difference is that when you don't explicitly release the address, you see something like:
lease 172.16.99.101 {
starts 5 2017/02/03 02:05:01;
ends 5 2017/02/03 02:05:31;
tstp 5 2017/02/03 02:05:31;
cltt 5 2017/02/03 02:05:01;
binding state free;
hardware ethernet 52:54:00:3d:e7:29;
}
So the lease is still valid, in truth, in the above case. (because it has not been renewed.) An explicitly released address will look like this:
lease 172.16.99.102 {
starts 5 2017/02/03 02:06:06;
cltt 5 2017/02/03 02:06:06;
binding state free;
hardware ethernet 52:54:00:8f:10:eb;
}
Notice that there is no 'ends' time.
So the root cause of the bug is: a free lease without an 'ends' time was considered an 'infinite' lease. This meant MAAS confusion*, because if IP address X was leased to host 'foo' and then explicitly released (and therefore did not have an 'ends' time, because the lease was cut short), and later host 'bar' leases IP address X, it will appear that two hosts have leased the same IP address. Ouch,
*: Bad pun, sorry.