Merge ~mpontillo/maas:beaconing--full-ipv6-support into maas:master

Proposed by Mike Pontillo
Status: Merged
Approved by: Mike Pontillo
Approved revision: bde965df28ebe3c0f8a8a5081aaf235f9a097722
Merge reported by: MAAS Lander
Merged at revision: not available
Proposed branch: ~mpontillo/maas:beaconing--full-ipv6-support
Merge into: maas:master
Diff against target: 725 lines (+389/-73)
8 files modified
src/provisioningserver/utils/ethernet.py (+1/-0)
src/provisioningserver/utils/network.py (+31/-0)
src/provisioningserver/utils/send_beacons.py (+13/-3)
src/provisioningserver/utils/services.py (+6/-2)
src/provisioningserver/utils/tcpip.py (+113/-25)
src/provisioningserver/utils/tests/test_network.py (+65/-0)
src/provisioningserver/utils/tests/test_send_beacons.py (+2/-2)
src/provisioningserver/utils/tests/test_tcpip.py (+158/-41)
Reviewer Review Type Date Requested Status
Blake Rouse (community) Approve
Review via email: mp+329228@code.launchpad.net

Commit message

Add support for IPv6 PCAP interpretation

 * Allows IPv6-based beacons to be received and processed.
   (They were already being sent, but could not be interpreted.)
 * Drive-by fix to allow the maas-rack send-beacons command to
   properly populate the payload for unicast beacons.

To post a comment you must log in.
Revision history for this message
Mike Pontillo (mpontillo) wrote :

I added get_ifname_ifdata_for_destination() to do a drive-by fix for the "maas-rack send-beacons" command that I discovered while testing. It'll need test cases before this can land, but the IPv6 portion of this branch can be reviewed.

Revision history for this message
Blake Rouse (blake-rouse) wrote :

Looks good. Will wait on the tests for a complete review, but +1 on the IPv6 stuff.

Revision history for this message
Mike Pontillo (mpontillo) wrote :

This is ready for another review. I added TestGetIfnameIfdataForDestination, updated test_send_beacons.py, and added support (and tests for) IPv4-mapped addresses in get_source_address().

Revision history for this message
Blake Rouse (blake-rouse) wrote :

Looks good.

review: Approve
Revision history for this message
MAAS Lander (maas-lander) wrote :

LANDING
-b beaconing--full-ipv6-support lp:~mpontillo/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/222/consoleText

Revision history for this message
MAAS Lander (maas-lander) wrote :

LANDING
-b beaconing--full-ipv6-support lp:~mpontillo/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/224/consoleText

Revision history for this message
Mike Pontillo (mpontillo) wrote :

Unrelated test failure. http://paste.ubuntu.com/25342124/

Revision history for this message
MAAS Lander (maas-lander) wrote :

LANDING
-b beaconing--full-ipv6-support lp:~mpontillo/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/225/consoleText

Revision history for this message
MAAS Lander (maas-lander) wrote :

LANDING
-b beaconing--full-ipv6-support lp:~mpontillo/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/227/consoleText

Revision history for this message
MAAS Lander (maas-lander) wrote :

LANDING
-b beaconing--full-ipv6-support lp:~mpontillo/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/229/consoleText

Revision history for this message
MAAS Lander (maas-lander) wrote :

LANDING
-b beaconing--full-ipv6-support lp:~mpontillo/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/232/consoleText

Revision history for this message
Mike Pontillo (mpontillo) wrote :

So many unrelated failures. We'll need to go fix all these.

http://paste.ubuntu.com/25348440/

Revision history for this message
MAAS Lander (maas-lander) wrote :

LANDING
-b beaconing--full-ipv6-support lp:~mpontillo/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/233/consoleText

Revision history for this message
Mike Pontillo (mpontillo) wrote :

This one is a bit disturbing.

http://paste.ubuntu.com/25348719/

Revision history for this message
MAAS Lander (maas-lander) wrote :

LANDING
-b beaconing--full-ipv6-support lp:~mpontillo/maas into -b master lp:~maas-committers/maas

STATUS: FAILED BUILD
LOG: http://maas-ci-jenkins.internal:8080/job/maas/job/branch-tester/234/consoleText

Revision history for this message
Mike Pontillo (mpontillo) wrote :
Revision history for this message
Andres Rodriguez (andreserl) wrote :

same error with the tags. I also have noticed something strange with tags
in machines. Sometimes they just disappear.

On Mon, Aug 21, 2017 at 12:22 PM, Mike Pontillo <<email address hidden>
> wrote:

> http://paste.ubuntu.com/25363581/
> --
> https://code.launchpad.net/~mpontillo/maas/+git/maas/+merge/329228
> Your team MAAS Committers is subscribed to branch maas:master.
>

--
Andres Rodriguez
Engineering Manager, MAAS
Canonical USA, Inc.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/src/provisioningserver/utils/ethernet.py b/src/provisioningserver/utils/ethernet.py
2index c19de8c..22fa74c 100644
3--- a/src/provisioningserver/utils/ethernet.py
4+++ b/src/provisioningserver/utils/ethernet.py
5@@ -35,6 +35,7 @@ VLAN_HEADER_LEN = 4
6 class ETHERTYPE:
7 """Enumeration to represent ethertypes that MAAS needs to understand."""
8 IPV4 = hex_str_to_bytes('0800')
9+ IPV6 = hex_str_to_bytes('86dd')
10 ARP = hex_str_to_bytes('0806')
11 VLAN = hex_str_to_bytes('8100')
12
13diff --git a/src/provisioningserver/utils/network.py b/src/provisioningserver/utils/network.py
14index b228dd3..faf799f 100644
15--- a/src/provisioningserver/utils/network.py
16+++ b/src/provisioningserver/utils/network.py
17@@ -83,6 +83,14 @@ ENI_PARSED_METHODS = [
18 "dhcp",
19 ]
20
21+# Hard-coded loopback interface information, since the loopback interface isn't
22+# included in `get_all_interfaces_definition()`.
23+LOOPBACK_INTERFACE_INFO = {
24+ "enabled": True,
25+ "index": 1,
26+ "links": [{"address": "::1/128"}, {"address": "127.0.0.1/8"}]
27+}
28+
29
30 REVERSE_RESOLVE_RETRIES = (1, 2, 4, 8, 16)
31
32@@ -1239,6 +1247,27 @@ def enumerate_assigned_ips(ifdata):
33 return (link['address'].split('/')[0] for link in links)
34
35
36+def get_ifname_ifdata_for_destination(
37+ destination_ip: IPAddressOrNetwork, interfaces: dict):
38+ """Returns an (ifname, ifdata) tuple for the given destination.
39+
40+ :param destination_ip: The destination IP address.
41+ :param interfaces: The output of `get_all_interfaces_definition()`.
42+ :returns: tuple of (ifname, ifdata)
43+ :raise: ValueError if not found
44+ """
45+ source_ip = get_source_address(destination_ip)
46+ if source_ip is None:
47+ raise ValueError("No route to host: %s" % destination_ip)
48+ if source_ip == "::1" or source_ip == "127.0.0.1":
49+ return "lo", LOOPBACK_INTERFACE_INFO
50+ for ifname, ifdata in interfaces.items():
51+ for candidate in enumerate_assigned_ips(ifdata):
52+ if candidate == source_ip:
53+ return ifname, ifdata
54+ raise ValueError("Source IP not found in interface links: %s" % source_ip)
55+
56+
57 def enumerate_ipv4_addresses(ifdata):
58 """Yields each IPv4 address assigned to an interface.
59
60@@ -1396,6 +1425,8 @@ def get_source_address(destination_ip: IPAddressOrNetwork):
61 destination_ip = IPAddress(destination_ip.first + 1)
62 else:
63 destination_ip = make_ipaddress(destination_ip)
64+ if destination_ip.is_ipv4_mapped():
65+ destination_ip = destination_ip.ipv4()
66 af = AF_INET if destination_ip.version == 4 else AF_INET6
67 with socket.socket(af, socket.SOCK_DGRAM) as sock:
68 peername = str(destination_ip)
69diff --git a/src/provisioningserver/utils/send_beacons.py b/src/provisioningserver/utils/send_beacons.py
70index 5e3ecf9..94c770e 100644
71--- a/src/provisioningserver/utils/send_beacons.py
72+++ b/src/provisioningserver/utils/send_beacons.py
73@@ -22,8 +22,14 @@ from provisioningserver.utils.beaconing import (
74 BEACON_PORT,
75 create_beacon_payload,
76 )
77-from provisioningserver.utils.network import get_all_interfaces_definition
78-from provisioningserver.utils.services import BeaconingSocketProtocol
79+from provisioningserver.utils.network import (
80+ get_all_interfaces_definition,
81+ get_ifname_ifdata_for_destination,
82+)
83+from provisioningserver.utils.services import (
84+ BeaconingSocketProtocol,
85+ interface_info_to_beacon_remote_payload,
86+)
87 from twisted.internet import reactor
88
89
90@@ -82,7 +88,11 @@ def do_beaconing(args, interfaces=None):
91 protocol.send_multicast_beacons(interfaces, verbose=args.verbose)
92 else:
93 log.msg("Sending unicast beacon to '%s'..." % destination_ip)
94- beacon = create_beacon_payload("solicitation")
95+ ifname, ifdata = get_ifname_ifdata_for_destination(
96+ destination_ip, interfaces)
97+ remote = interface_info_to_beacon_remote_payload(ifname, ifdata)
98+ payload = {"remote": remote}
99+ beacon = create_beacon_payload("solicitation", payload=payload)
100 protocol.send_beacon(beacon, (destination_ip, BEACON_PORT))
101 reactor.callLater(args.timeout, lambda: reactor.stop())
102 reactor.run()
103diff --git a/src/provisioningserver/utils/services.py b/src/provisioningserver/utils/services.py
104index cb5c81b..8613586 100644
105--- a/src/provisioningserver/utils/services.py
106+++ b/src/provisioningserver/utils/services.py
107@@ -364,6 +364,8 @@ def interface_info_to_beacon_remote_payload(ifname, ifdata, rx_vid=None):
108 # It will be obvious to the receiver that the source interface
109 # was enabled. ;-)
110 remote.pop('enabled', None)
111+ # Remote doesn't need to know which interfaces are monitored.
112+ remote.pop('monitored', None)
113 # Don't need all the links, just the one that originated this
114 # packet.
115 remote.pop('links', None)
116@@ -501,6 +503,8 @@ class BeaconingSocketProtocol(DatagramProtocol):
117 # Loopback interface always has ifindex == 1.
118 join_ipv6_beacon_group(sock, 1)
119 for _, ifdata in self.interfaces.items():
120+ # Always try to join the IPv6 group on each interface.
121+ join_ipv6_beacon_group(sock, ifdata['index'])
122 # Merely joining the group with the default parameters is not
123 # enough, since we want to join the group on *all* interfaces.
124 # So we need to join each group using an assigned IPv4 address
125@@ -513,7 +517,6 @@ class BeaconingSocketProtocol(DatagramProtocol):
126 # secondary IP address on the same interface will produce
127 # an "Address already in use" error.
128 break
129- join_ipv6_beacon_group(sock, ifdata['index'])
130
131 def updateInterfaces(self, interfaces):
132 self.interfaces = interfaces
133@@ -567,7 +570,8 @@ class BeaconingSocketProtocol(DatagramProtocol):
134 # If the packet cannot be sent for whatever reason, OSError will
135 # be raised, and we won't record sending a beacon we didn't
136 # actually send.
137- self.tx_queue[beacon.payload['uuid']] = beacon
138+ uuid = beacon.payload['uuid']
139+ self.tx_queue[uuid] = beacon
140 age_out_uuid_queue(self.tx_queue)
141 return True
142 except OSError as e:
143diff --git a/src/provisioningserver/utils/tcpip.py b/src/provisioningserver/utils/tcpip.py
144index dbf84dc..26c3fe1 100644
145--- a/src/provisioningserver/utils/tcpip.py
146+++ b/src/provisioningserver/utils/tcpip.py
147@@ -9,6 +9,7 @@ __all__ = [
148 ]
149
150 from collections import namedtuple
151+from ipaddress import ip_address
152 import struct
153 import time
154
155@@ -18,23 +19,6 @@ from provisioningserver.utils.ethernet import (
156 ETHERTYPE,
157 )
158
159-# Definitions for IPv4 packets used with `struct`.
160-# See https://tools.ietf.org/html/rfc791#section-3.1 for more details.
161-IPV4_PACKET = '!BBHHHBBHLL'
162-IPv4Packet = namedtuple('IPv4Packet', (
163- 'version__ihl',
164- 'tos',
165- 'total_length',
166- 'fragment_id',
167- 'flags__fragment_offset',
168- 'ttl',
169- 'protocol',
170- 'header_checksum',
171- 'src_ip',
172- 'dst_ip',
173-))
174-IPV4_HEADER_MIN_LENGTH = 20
175-
176 # Definition for a decoded network packet.
177 Packet = namedtuple("Packet", (
178 'timestamp',
179@@ -55,7 +39,39 @@ class PacketProcessingError(Exception):
180
181 class PROTOCOL:
182 """Enumeration to represent IP protocols that MAAS needs to understand."""
183+ # https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml
184+ IPV6_HOP_BY_HOP = 0x00
185+ ICMP = 0x01
186+ IGMP = 0x02
187+ TCP = 0x06
188 UDP = 0x11
189+ IPV6_ENCAPSULATION = 0x29
190+ IPV6_ROUTING_HEADER = 0x2B
191+ IPV6_FRAGMENT_HEADER = 0x2C
192+ MOBILITY = 0x37
193+ IPV6_ICMP = 0x3A
194+ IPV6_NO_NEXT_HEADER = 0x3B
195+ IPV6_DESTINATION_OPTIONS = 0x3C
196+ IPV6_MOBILITY = 0x87
197+ IPV6_SHIM6 = 0x8C
198+
199+
200+# Definitions for IPv4 packets used with `struct`.
201+# See https://tools.ietf.org/html/rfc791#section-3.1 for more details.
202+IPV4_PACKET = '!BBHHHBBHLL'
203+IPv4Packet = namedtuple('IPv4Packet', (
204+ 'version__ihl',
205+ 'tos',
206+ 'total_length',
207+ 'fragment_id',
208+ 'flags__fragment_offset',
209+ 'ttl',
210+ 'protocol',
211+ 'header_checksum',
212+ 'src_ip',
213+ 'dst_ip',
214+))
215+IPV4_HEADER_MIN_LENGTH = 20
216
217
218 class IPv4:
219@@ -110,6 +126,7 @@ class IPv4:
220 "Truncated IPv4 header; IHL indicates to read %d bytes; got "
221 "%d bytes." % (self.ihl, len(pkt_bytes)))
222 return
223+ self.protocol = packet.protocol
224 # Everything beyond the IHL is the upper-layer payload. (No need to
225 # understand IP options at this time.)
226 self.payload = pkt_bytes[self.ihl:]
227@@ -126,6 +143,74 @@ class IPv4:
228 return self.valid
229
230
231+IPV6_PACKET = '!LHBB16s16s'
232+IPv6Packet = namedtuple('IPv6Packet', (
233+ 'version__traffic_class__flow_label',
234+ 'payload_length',
235+ 'next_header',
236+ 'hop_limit',
237+ 'src_ip',
238+ 'dst_ip',
239+))
240+IPV6_HEADER_MIN_LENGTH = 40
241+
242+
243+class IPv6:
244+ """Representation of an IPv6 packet."""
245+
246+ def __init__(self, pkt_bytes: bytes):
247+ """Decodes the specified IPv6 packet.
248+
249+ The IP payload will be placed in the `payload` ivar if the packet
250+ is valid. If the packet is valid, the `valid` ivar will be set to True.
251+ If the packet is not valid, the `valid` ivar will be set to False, and
252+ the `invalid_reason` will contain a description of why the packet is
253+ not valid.
254+
255+ This class does not validate the header checksum, and as such, should
256+ only be used for testing.
257+
258+ :param pkt_bytes: The input bytes of the IPv6 packet.
259+ """
260+ self.valid = True
261+ self.invalid_reason = None
262+ self.protocol = None
263+ if len(pkt_bytes) < IPV6_HEADER_MIN_LENGTH:
264+ self.valid = False
265+ self.invalid_reason = (
266+ "Truncated IPv6 header; need at least %d bytes." % (
267+ IPV6_HEADER_MIN_LENGTH))
268+ return
269+ packet = IPv6Packet._make(
270+ struct.unpack(
271+ IPV6_PACKET, pkt_bytes[0:IPV6_HEADER_MIN_LENGTH]))
272+ self.packet = packet
273+ # Mask out the version_ihl field to get the IP version and IHL
274+ # (Internet Header Length) separately.
275+ self.version = (
276+ packet.version__traffic_class__flow_label & 0xF0000000) >> 28
277+ if self.version != 6:
278+ self.valid = False
279+ self.invalid_reason = (
280+ "Invalid version field; expected IPv6, got IPv%d." % (
281+ self.version))
282+ return
283+ # XXX mpontillo 2017-08-15: should process next headers.
284+ # (not required for beaconing, since there won't be any)
285+ self.protocol = packet.next_header
286+ self.payload = pkt_bytes[IPV6_HEADER_MIN_LENGTH:]
287+
288+ @property
289+ def src_ip(self):
290+ return ip_address(self.packet.src_ip)
291+
292+ @property
293+ def dst_ip(self):
294+ return ip_address(self.packet.dst_ip)
295+
296+ def is_valid(self):
297+ return self.valid
298+
299 # Definitions for UDP packets used with `struct`.
300 # https://tools.ietf.org/html/rfc768
301 UDP_PACKET = '!HHHH'
302@@ -196,20 +281,23 @@ def decode_ethernet_udp_packet(packet, pcap_header=None):
303 ethernet = Ethernet(packet, time=timestamp)
304 if not ethernet.is_valid():
305 raise PacketProcessingError("Invalid Ethernet packet.")
306- # XXX Need to support IPv6 as well.
307- if ethernet.ethertype != ETHERTYPE.IPV4:
308+ ethertype = ethernet.ethertype
309+ supported_ethertypes = (ETHERTYPE.IPV4, ETHERTYPE.IPV6)
310+ if ethertype not in supported_ethertypes:
311 raise PacketProcessingError(
312- "Invalid ethertype; expected %r, got %r." % (
313- ETHERTYPE.IPV4, ethernet.ethertype))
314+ "Invalid ethertype; expected one of %r, got %r." % (
315+ supported_ethertypes, ethertype))
316 # Interpret Layer 3
317- ip = IPv4(ethernet.payload)
318+ if ethertype == ETHERTYPE.IPV4:
319+ ip = IPv4(ethernet.payload)
320+ else:
321+ ip = IPv6(ethernet.payload)
322 if not ip.is_valid():
323 raise PacketProcessingError(ip.invalid_reason)
324- if ip.packet.protocol != PROTOCOL.UDP:
325- # Ignore non-IPv4 packets.
326+ if ip.protocol != PROTOCOL.UDP:
327 raise PacketProcessingError(
328 "Invalid protocol; expected %d (UDP), got %d." % (
329- PROTOCOL.UDP, ip.packet.protocol))
330+ PROTOCOL.UDP, ip.protocol))
331 # Interpret Layer 4
332 udp = UDP(ip.payload)
333 if not udp.is_valid():
334diff --git a/src/provisioningserver/utils/tests/test_network.py b/src/provisioningserver/utils/tests/test_network.py
335index 9ce2410..8f1d0e6 100644
336--- a/src/provisioningserver/utils/tests/test_network.py
337+++ b/src/provisioningserver/utils/tests/test_network.py
338@@ -59,6 +59,7 @@ from provisioningserver.utils.network import (
339 get_all_interfaces_definition,
340 get_default_monitored_interfaces,
341 get_eui_organization,
342+ get_ifname_ifdata_for_destination,
343 get_interface_children,
344 get_mac_organization,
345 get_source_address,
346@@ -70,6 +71,7 @@ from provisioningserver.utils.network import (
347 ip_range_within_network,
348 IPRangeStatistics,
349 is_loopback_address,
350+ LOOPBACK_INTERFACE_INFO,
351 MAASIPRange,
352 MAASIPSet,
353 make_iprange,
354@@ -1647,6 +1649,65 @@ class TestHasIPv4Address(InterfaceLinksTestCase):
355 Equals(True))
356
357
358+class TestGetIfnameIfdataForDestination(MAASTestCase):
359+ """Tests for `get_ifname_ifdata_for_destination()`."""
360+
361+ def setUp(self):
362+ super().setUp()
363+ self.interfaces = {
364+ "eth0": {
365+ "links": [{"address": "192.168.0.1/24"}]
366+ },
367+ "eth1": {
368+ "links": [
369+ {"address": "2001:db8::1/64"},
370+ {"address": "172.16.0.1/24"},
371+ ]
372+ },
373+ }
374+ self.get_source_address_mock = self.patch(
375+ network_module, 'get_source_address')
376+
377+ def test__returns_interface_for_expected_source_ip(self):
378+ self.get_source_address_mock.return_value = "2001:db8::1"
379+ ifname, ifdata = get_ifname_ifdata_for_destination(
380+ "2001:db8::2", self.interfaces)
381+ self.assertThat(ifname, Equals("eth1"))
382+ self.assertThat(ifdata, Equals(self.interfaces['eth1']))
383+ self.get_source_address_mock.return_value = "192.168.0.1"
384+ ifname, ifdata = get_ifname_ifdata_for_destination(
385+ "192.168.0.2", self.interfaces)
386+ self.assertThat(ifname, Equals("eth0"))
387+ self.assertThat(ifdata, Equals(self.interfaces['eth0']))
388+ self.get_source_address_mock.return_value = "172.16.0.1"
389+ ifname, ifdata = get_ifname_ifdata_for_destination(
390+ "172.16.0.2", self.interfaces)
391+ self.assertThat(ifname, Equals("eth1"))
392+ self.assertThat(ifdata, Equals(self.interfaces['eth1']))
393+
394+ def test__handles_loopback_addresses(self):
395+ self.get_source_address_mock.return_value = "127.0.0.1"
396+ ifname, ifdata = get_ifname_ifdata_for_destination(
397+ "127.0.0.1", self.interfaces)
398+ self.assertThat(ifname, Equals("lo"))
399+ self.assertThat(ifdata, Equals(LOOPBACK_INTERFACE_INFO))
400+ self.get_source_address_mock.return_value = "::1"
401+ ifname, ifdata = get_ifname_ifdata_for_destination(
402+ "::1", self.interfaces)
403+ self.assertThat(ifname, Equals("lo"))
404+ self.assertThat(ifdata, Equals(LOOPBACK_INTERFACE_INFO))
405+
406+ def test__raises_valueerror_if_no_route_to_host(self):
407+ self.get_source_address_mock.return_value = None
408+ with ExpectedException(ValueError):
409+ get_ifname_ifdata_for_destination("2001:db8::2", self.interfaces)
410+
411+ def test__raises_valueerror_if_source_ip_not_found(self):
412+ self.get_source_address_mock.return_value = "192.168.0.2"
413+ with ExpectedException(ValueError):
414+ get_ifname_ifdata_for_destination("192.168.0.3", self.interfaces)
415+
416+
417 class TestEnumerateAddresses(InterfaceLinksTestCase):
418 """Tests for `enumerate_assigned_ips` and `enumerate_ipv4_addresses()`."""
419
420@@ -2236,6 +2297,10 @@ class TestGetSourceAddress(MAASTestCase):
421 self.assertThat(
422 get_source_address("127.0.0.1"), Equals("127.0.0.1"))
423
424+ def test__converts_ipv4_mapped_ipv6_to_ipv4(self):
425+ self.assertThat(
426+ get_source_address("::ffff:127.0.0.1"), Equals("127.0.0.1"))
427+
428 def test__supports_ipv6(self):
429 self.assertThat(
430 get_source_address("::1"), Equals("::1"))
431diff --git a/src/provisioningserver/utils/tests/test_send_beacons.py b/src/provisioningserver/utils/tests/test_send_beacons.py
432index 2a15108..c781e13 100644
433--- a/src/provisioningserver/utils/tests/test_send_beacons.py
434+++ b/src/provisioningserver/utils/tests/test_send_beacons.py
435@@ -147,12 +147,12 @@ class TestSendBeaconsProtocolInteraction(
436
437 def test__sends_unicast_beacon(self):
438 self.run_command(
439- '-v', '-s', '1.1.1.1', '-t', '42', '-p', '4242', '2.2.2.2')
440+ '-v', '-s', '1.1.1.1', '-t', '42', '-p', '4242', '127.0.0.1')
441 self.assertThat(self.protocol_mock, MockCalledOnceWith(
442 ANY, debug=True, interface='1.1.1.1', port=4242,
443 process_incoming=True, interfaces=TEST_INTERFACES))
444 self.assertThat(
445 self.fake_protocol.send_multicast_beacons, MockNotCalled())
446 self.assertThat(self.fake_protocol.send_beacon, MockCalledOnceWith(
447- ANY, ("::ffff:2.2.2.2", 5240)
448+ ANY, ("::ffff:127.0.0.1", 5240)
449 ))
450diff --git a/src/provisioningserver/utils/tests/test_tcpip.py b/src/provisioningserver/utils/tests/test_tcpip.py
451index 8d3096c..cfe801f 100644
452--- a/src/provisioningserver/utils/tests/test_tcpip.py
453+++ b/src/provisioningserver/utils/tests/test_tcpip.py
454@@ -17,6 +17,7 @@ from provisioningserver.utils.pcap import PCAP
455 from provisioningserver.utils.tcpip import (
456 decode_ethernet_udp_packet,
457 IPv4,
458+ IPv6,
459 PacketProcessingError,
460 UDP,
461 )
462@@ -27,11 +28,7 @@ from testtools.matchers import Equals
463 def make_ipv4_packet(
464 total_length=None, version=None, ihl=None, payload=None,
465 truncated=False):
466- """Construct an IPv4 packet using the specified parameters.
467-
468- If the specified `vid` is not None, it is interpreted as an integer VID,
469- and the appropriate Ethertype fields are adjusted.
470- """
471+ """Construct an IPv4 packet using the specified parameters."""
472 if payload is None:
473 payload = b''
474 if total_length is None:
475@@ -72,6 +69,39 @@ def make_ipv4_packet(
476 return ipv4_packet
477
478
479+def make_ipv6_packet(
480+ payload_length=None, version=None, payload=None, protocol=0x11,
481+ truncated=False):
482+ """Construct an IPv6 packet using the specified parameters."""
483+ if payload is None:
484+ payload = b''
485+ if payload_length is None:
486+ payload_length = len(payload)
487+ if version is None:
488+ version = 6
489+ version__traffic_class__flow_label = (version << 28).to_bytes(4, "big")
490+
491+ ipv6_packet = (
492+ # Version, traffic class, flow label
493+ version__traffic_class__flow_label +
494+ # Total length in bytes
495+ payload_length.to_bytes(2, "big") +
496+ # Next header (default is UDP)
497+ protocol.to_bytes(1, "big") +
498+ # Hop limit (TTL)
499+ hex_str_to_bytes('00') +
500+ # Source address
501+ hex_str_to_bytes('00000000000000000000000000000000') +
502+ # Destination address
503+ hex_str_to_bytes('00000000000000000000000000000000')
504+ )
505+ assert len(ipv6_packet) == 40, "Length was %d" % len(ipv6_packet)
506+ if truncated:
507+ return ipv6_packet[:19]
508+ ipv6_packet = ipv6_packet + payload
509+ return ipv6_packet
510+
511+
512 class TestIPv4(MAASTestCase):
513
514 def test__parses_ipv4_packet(self):
515@@ -107,6 +137,33 @@ class TestIPv4(MAASTestCase):
516 ipv4.invalid_reason, DocTestMatches("Truncated..."))
517
518
519+class TestIPv6(MAASTestCase):
520+
521+ def test__parses_ipv6_packet(self):
522+ payload = factory.make_bytes(48)
523+ packet = make_ipv6_packet(payload=payload)
524+ ipv6 = IPv6(packet)
525+ self.assertThat(ipv6.is_valid(), Equals(True))
526+ self.assertThat(ipv6.version, Equals(6))
527+ self.assertThat(ipv6.packet.payload_length, Equals(len(payload)))
528+ self.assertThat(ipv6.payload, Equals(payload))
529+
530+ def test__fails_for_non_ipv6_packet(self):
531+ payload = factory.make_bytes(48)
532+ packet = make_ipv6_packet(payload=payload, version=5)
533+ ipv6 = IPv6(packet)
534+ self.assertThat(ipv6.is_valid(), Equals(False))
535+ self.assertThat(
536+ ipv6.invalid_reason, DocTestMatches("Invalid version..."))
537+
538+ def test__fails_for_truncated_packet(self):
539+ packet = make_ipv6_packet(truncated=True)
540+ ipv6 = IPv6(packet)
541+ self.assertThat(ipv6.is_valid(), Equals(False))
542+ self.assertThat(
543+ ipv6.invalid_reason, DocTestMatches("Truncated..."))
544+
545+
546 def make_udp_packet(
547 total_length=None, payload=None, truncated_header=False,
548 truncated_payload=False):
549@@ -173,38 +230,91 @@ class TestUDP(MAASTestCase):
550 udp.invalid_reason, DocTestMatches("UDP packet truncated..."))
551
552
553-GOOD_ETHERNET_UDP_PCAP = (
554+GOOD_ETHERNET_IPV4_UDP_PCAP = (
555 b'\xd4\xc3\xb2\xa1\x02\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00'
556 b'\x00@\x00\x00\x01\x00\x00\x00v\xe19Y\xadF\x08\x00^\x00\x00\x00^\x00\x00'
557 b'\x00\x01\x00^\x00\x00v\x00\x16>\x91zz\x08\x00E\x00\x00P\xe2E@\x00\x01'
558 b'\x11\xe0\xce\xac\x10*\x02\xe0\x00\x00v\xda\xc2\x14x\x00<h(4\x00\x00\x00'
559 b'\x02uuid\x00%\x00\x00\x0078d1a4f0-4ca4-11e7-b2bb-00163e917a7a\x00\x00')
560
561-GOOD_ETHERNET_UDP_PACKET = (
562- b'\x01\x00^\x00\x00v\x00\x16>\x91zz\x08\x00E\x00\x00P\xe2E@\x00\x01'
563- b'\x11\xe0\xce\xac\x10*\x02\xe0\x00\x00v\xda\xc2\x14x\x00<h(4\x00\x00\x00'
564- b'\x02uuid\x00%\x00\x00\x0078d1a4f0-4ca4-11e7-b2bb-00163e917a7a\x00\x00')
565+GOOD_ETHERNET_HEADER_IPV4 = b'\x01\x00^\x00\x00v\x00\x16>\x91zz\x08\x00'
566+
567+GOOD_ETHERNET_HEADER_IPV6 = b'\x01\x00^\x00\x00v\x00\x16>\x91zz\x86\xdd'
568+
569+BAD_ETHERNET_HEADER_WRONG_ETHERTYPE = (
570+ b'\x01\x00^\x00\x00v\x00\x16>\x91zz\x07\xFF')
571+
572+BAD_ETHERNET_TRUNCATED_HEADER = b'\x01\x00^\x00\x00v'
573+
574+GOOD_IPV4_HEADER = (
575+ b'E\x00\x00P\xe2E@\x00\x01\x11\xe0\xce\xac\x10*\x02\xe0\x00\x00v'
576+)
577+
578+BAD_IPV4_HEADER_WRONG_PROTOCOL = (
579+ b'E\x00\x00P\xe2E@\x00\x01\x12\xe0\xce\xac\x10*\x02\xe0\x00\x00v'
580+)
581+
582+GOOD_UDP_PAYLOAD = (
583+ b'\xda\xc2\x14x\x00<h(4\x00\x00\x00\x02uuid\x00%\x00\x00\x0078d1a4f0-4ca4-'
584+ b'11e7-b2bb-00163e917a7a\x00\x00'
585+)
586
587 TRUNCATED_UDP_PAYLOAD = (
588- b'\x01\x00^\x00\x00v\x00\x16>\x91zz\x08\x00E\x00\x00P\xe2E@\x00\x01'
589- b'\x11\xe0\xce\xac\x10*\x02\xe0\x00\x00v\xda\xc2\x14x\x00<h(4\x00\x00\x00'
590- b'\x02uuid\x00%\x00\x00\x0078d1a4f0-4ca4-11e7-b2bb-00163e917a7a\x00')
591+ b'\xda\xc2\x14x\x00<h(4\x00\x00\x00\x02uuid\x00%\x00\x00\x0078d1a4f0-4ca4-'
592+ b'11e7-b2bb-00163e917a7a\x00'
593+)
594
595-BAD_ETHERTYPE = (
596- b'\x01\x00^\x00\x00v\x00\x16>\x91zz\x07\xFFE\x00\x00P\xe2E@\x00\x01'
597- b'\x11\xe0\xce\xac\x10*\x02\xe0\x00\x00v\xda\xc2\x14x\x00<h(4\x00\x00\x00'
598- b'\x02uuid\x00%\x00\x00\x0078d1a4f0-4ca4-11e7-b2bb-00163e917a7a\x00\x00')
599+GOOD_ETHERNET_IPV4_UDP_PACKET = (
600+ GOOD_ETHERNET_HEADER_IPV4 +
601+ GOOD_IPV4_HEADER +
602+ GOOD_UDP_PAYLOAD
603+)
604
605-BAD_IPV4_UDP_HEADER = (
606- b'\x01\x00^\x00\x00v\x00\x16>\x91zz\x08\x00E\x00\x00P\xe2E@\x00\x01'
607- b'\x11\xe0\xce\xac\x10*\x02\xe0\x00\x00v\xdb')
608+GOOD_ETHERNET_IPV6_UDP_PACKET = (
609+ GOOD_ETHERNET_HEADER_IPV6 +
610+ make_ipv6_packet(payload=GOOD_UDP_PAYLOAD)
611+)
612
613-NOT_UDP_PROTOCOL = (
614- b'\x01\x00^\x00\x00v\x00\x16>\x91zz\x08\x00E\x00\x00P\xe2E@\x00\x01'
615- b'\x12\xe0\xce\xac\x10*\x02\xe0\x00\x00v\xda\xc2\x14x\x00<h(4\x00\x00\x00'
616- b'\x02uuid\x00%\x00\x00\x0078d1a4f0-4ca4-11e7-b2bb-00163e917a7a\x00\x00')
617+BAD_ETHERNET_IPV4_TRUNCATED_UDP_PAYLOAD = (
618+ GOOD_ETHERNET_HEADER_IPV4 +
619+ GOOD_IPV4_HEADER +
620+ TRUNCATED_UDP_PAYLOAD
621+)
622
623-TRUNCATED_ETHERNET_HEADER = b'\x01\x00^\x00\x00v'
624+BAD_ETHERNET_IPV6_TRUNCATED_UDP_PAYLOAD = (
625+ GOOD_ETHERNET_HEADER_IPV6 +
626+ make_ipv6_packet(
627+ payload_length=len(TRUNCATED_UDP_PAYLOAD) + 1,
628+ payload=TRUNCATED_UDP_PAYLOAD)
629+)
630+
631+BAD_ETHERNET_ETHERTYPE = (
632+ BAD_ETHERNET_HEADER_WRONG_ETHERTYPE +
633+ GOOD_IPV4_HEADER +
634+ GOOD_UDP_PAYLOAD
635+)
636+
637+BAD_IPV4_TRUNCATED_UDP_HEADER = (
638+ GOOD_ETHERNET_HEADER_IPV4 +
639+ GOOD_IPV4_HEADER +
640+ b'\xdb'
641+)
642+
643+BAD_IPV6_TRUNCATED_UDP_HEADER = (
644+ GOOD_ETHERNET_HEADER_IPV6 +
645+ make_ipv6_packet(payload_length=20, payload=b'\x00')
646+)
647+
648+BAD_IPV4_NOT_UDP_PROTOCOL = (
649+ GOOD_ETHERNET_HEADER_IPV4 +
650+ BAD_IPV4_HEADER_WRONG_PROTOCOL +
651+ GOOD_UDP_PAYLOAD
652+)
653+
654+BAD_IPV6_NOT_UDP_PROTOCOL = (
655+ GOOD_ETHERNET_HEADER_IPV6 +
656+ make_ipv6_packet(payload=TRUNCATED_UDP_PAYLOAD, protocol=0x12)
657+)
658
659 EXPECTED_PCAP_TIME = 1496965494
660
661@@ -217,41 +327,48 @@ EXPECTED_PAYLOAD = (
662 class TestDecodeEthernetUDPPacket(MAASTestCase):
663
664 def test__gets_time_from_pcap_header(self):
665- pcap_file = BytesIO(GOOD_ETHERNET_UDP_PCAP)
666+ pcap_file = BytesIO(GOOD_ETHERNET_IPV4_UDP_PCAP)
667 pcap = PCAP(pcap_file)
668 for header, packet_bytes in pcap:
669 packet = decode_ethernet_udp_packet(packet_bytes, header)
670 self.expectThat(packet.timestamp, Equals(EXPECTED_PCAP_TIME))
671 self.expectThat(packet.payload, Equals(EXPECTED_PAYLOAD))
672
673- def test__decodes_from_bytes(self):
674+ def test__decodes_ipv4_from_bytes(self):
675 expected_time = EXPECTED_PCAP_TIME + randint(1, 100)
676 self.patch(time, "time").return_value = expected_time
677- packet = decode_ethernet_udp_packet(GOOD_ETHERNET_UDP_PACKET)
678+ packet = decode_ethernet_udp_packet(GOOD_ETHERNET_IPV4_UDP_PACKET)
679 self.expectThat(packet.timestamp, Equals(expected_time))
680 self.expectThat(packet.payload, Equals(EXPECTED_PAYLOAD))
681
682 def test__fails_for_bad_ethertype(self):
683- self.patch(time, "time").return_value = EXPECTED_PCAP_TIME
684 with ExpectedException(PacketProcessingError, '.*Invalid ethertype.*'):
685- decode_ethernet_udp_packet(BAD_ETHERTYPE)
686+ decode_ethernet_udp_packet(BAD_ETHERNET_ETHERTYPE)
687
688 def test__fails_for_bad_ethernet_packet(self):
689- self.patch(time, "time").return_value = EXPECTED_PCAP_TIME
690 with ExpectedException(PacketProcessingError, '.*Invalid Ethernet.*'):
691- decode_ethernet_udp_packet(TRUNCATED_ETHERNET_HEADER)
692+ decode_ethernet_udp_packet(BAD_ETHERNET_TRUNCATED_HEADER)
693+
694+ def test__fails_for_bad_ipv4_udp_header(self):
695+ with ExpectedException(PacketProcessingError, '.*Truncated UDP.*'):
696+ decode_ethernet_udp_packet(BAD_IPV4_TRUNCATED_UDP_HEADER)
697+
698+ def test__fails_if_not_udp_protocol_ipv4(self):
699+ with ExpectedException(PacketProcessingError, '.*Invalid protocol*'):
700+ decode_ethernet_udp_packet(BAD_IPV4_NOT_UDP_PROTOCOL)
701+
702+ def test__fails_if_ipv4_udp_packet_truncated(self):
703+ with ExpectedException(PacketProcessingError, '.*UDP packet trunc.*'):
704+ decode_ethernet_udp_packet(BAD_ETHERNET_IPV4_TRUNCATED_UDP_PAYLOAD)
705
706- def test__fails_for_bad_udp_header(self):
707- self.patch(time, "time").return_value = EXPECTED_PCAP_TIME
708+ def test__fails_for_bad_ipv6_udp_header(self):
709 with ExpectedException(PacketProcessingError, '.*Truncated UDP.*'):
710- decode_ethernet_udp_packet(BAD_IPV4_UDP_HEADER)
711+ decode_ethernet_udp_packet(BAD_IPV6_TRUNCATED_UDP_HEADER)
712
713- def test__fails_if_not_udp_protocol(self):
714- self.patch(time, "time").return_value = EXPECTED_PCAP_TIME
715+ def test__fails_if_not_udp_protocol_ipv6(self):
716 with ExpectedException(PacketProcessingError, '.*Invalid protocol*'):
717- decode_ethernet_udp_packet(NOT_UDP_PROTOCOL)
718+ decode_ethernet_udp_packet(BAD_IPV6_NOT_UDP_PROTOCOL)
719
720- def test__fails_if_udp_packet_truncated(self):
721- self.patch(time, "time").return_value = EXPECTED_PCAP_TIME
722+ def test__fails_if_ipv6_udp_packet_truncated(self):
723 with ExpectedException(PacketProcessingError, '.*UDP packet trunc.*'):
724- decode_ethernet_udp_packet(TRUNCATED_UDP_PAYLOAD)
725+ decode_ethernet_udp_packet(BAD_ETHERNET_IPV6_TRUNCATED_UDP_PAYLOAD)

Subscribers

People subscribed via source and target branches