Merge lp:~mpontillo/maas/interpret-dhcp-packets-correctly--bug-1628645 into lp:~maas-committers/maas/trunk

[Mike Pontillo]

Fix external DHCP detection code.

 * Parse DHCP options to reliably get DHCP server address.
 * Change DHCP flags field to indicate preference for
   unicast replies from the server.
 * Fix race condition that caused DHCP offers to be missed.
 * When sending the DHCP discovery packet, always send retries,
   to increase the changes we see all servers.
 * Change client unique identifier option to make it more
   unique, and make it obvious that MAAS is sending it.
 * Add code to interpret IPv4, UDP, and DHCP packets.
   (Used for support/debugging only; a debug command uses it.)
 * Added a `maas-rack observe-dhcp` command.
   (For support/debugging only; not used at runtime.)

There are a several major issues with the current external DHCP detection code:

(1) We are not parsing the DHCP options; we use a hard-coded index into the packet to get the DHCP server identifier (IP address). (Fixed in this branch.)

(2) The way we bind to the sending socket doesn't work for interfaces with no address. So we cannot discover external DHCP servers on interfaces with no IP address configured. (NOT fixed in this branch.***)

(3) The way we bind to the receive socket causes a race condition that makes it likely the DHCP offer will come in before we are listening to the receive socket. (Fixed in this branch.)

(4) Since we set the 'broadcast' flag in the DHCP discover packet, we don't reliably receive replies from all servers. (It ignores packets from the DD-WRT router I tested it with, for example.) This is because the DHCP server assumes it cannot talk to the client on a unicast IP/MAC and broadcasts the response from a bogus IP address instead, which the Linux TCP/IP stack never sends up to the socket. One way to fix this is to listen to DHCP replies via something like 'tcpdump' or libpcap instead of a UDP socket. Another way to fix this is to unset the broadcast flag. (This branch fixes that by unsetting the broadcast flag.)

(5) UDP is unreliable, and we only send a single DHCP discover packet out before assuming no servers responded. (Fixed in this branch.)

(6) The deferToThread() covers too much code. It should
only cover code that could block. (Fixed in this branch.)

(7) It is difficult to debug MAAS's interpretation of DHCP offer packets. (Fixed in this branch by adding a 'maas-rack observe-dhcp' command.)

***:
Fixing (2) requires additional privileges. To be specific, we need a way of sending a DHCP probe out directly on the interface, and a way of listening for the replies without relying on a socket bound to a unicast IP address. The listening portion is partially done in this branch (via `maas-rack observe-dhcp`) to ease debugging and support, but changing the design to use it in MAAS is a drastic design change that should not happen in this branch.

[Gavin Panella (allenap)]

I saw this branch linked from the channel and thought to start reviewing it... but eventually realised that it was Work In Progress. So, no vote, but a few comments. I didn't get to the end but none of my comments are blockers so it's looking good so far.

[Mike Pontillo (mpontillo)]

Thanks for taking the time to look at this monster WIP branch, Gavin. Some replies to your comments below.

[Mike Pontillo (mpontillo)]

Per your comment inline, I think it would be interesting to see how twisted behaves with a monotonic clock. It's worth trying, but I suppose if any code anywhere assumes it's a UNIX timestamp, it's game over.

This branch is still a work in progress, but today I heavily refactored the code in `detect.py`. In particular, would you give me your thoughts on the `DHCPRequestMonitor` class?

One thing I tried to do was use @inlineCallbacks to run a generator inside a thread using deferToThread(), but that failed horribly, so I changed the method back to just returning a `set`. Not sure if that's even possible.

[Blake Rouse (blake-rouse)]

Wow this is really large branch. I really just started scrolling quickly at the end because it was so large and I got bored. The code lots good and well tested. The packet creation and parsing is very complicated. It sucks to see all this code being placed in MAAS that we need to maintain when there is already python libraries that do this kind of thing.

I noticed you typed a lot of your methods, but every single one was missing the @typed decorator. You should add that to all the typed methods. I commented on some, but just gave up as I went as it was so large.

Since you have tested this a lot with your router, it can only be an improvement over what it does today. I trusty you!

review: Approve

=== modified file 'Makefile'
--- Makefile	2016-09-22 02:53:33 +0000
+++ Makefile	2016-10-05 01:31:45 +0000
@@ -99,6 +99,7 @@
 sudoers:
 	utilities/grant-nmap-permissions
 	utilities/install-arp-observer
+	utilities/install-dhcp-observer
 
 bin/buildout: bootstrap-buildout.py
 	@utilities/configure-buildout --quiet

=== added file 'scripts/maas-dhcp-monitor'
--- scripts/maas-dhcp-monitor	1970-01-01 00:00:00 +0000
+++ scripts/maas-dhcp-monitor	2016-10-05 01:31:45 +0000
@@ -0,0 +1,20 @@
+#!/bin/sh -euf
+# Copyright 2016 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+# Utility script to wrap `tcpdump`, so that this script can be called with 
+# `sudo` without allowing MAAS access to read arbitrary network traffic.
+# This script is designed to be as minimal as possible, to prevent arbitrary
+# code execution.
+
+if [ $# -ne 1 ]; then
+    echo "Write DHCP traffic to stdout using tcpdump's binary PCAP format." >&2
+    echo "" >&2
+    echo "Usage:" >&2
+    echo "    $0 <interface>" >&2
+    exit 32
+fi 
+
+exec /usr/sbin/tcpdump --interface "$1" --no-promiscuous-mode \
+    --packet-buffered --immediate-mode --snapshot-length=1500 -n -w - \
+    "(port 67 or port 68) or (vlan and (port 67 or port 68))"

=== modified file 'src/maastesting/factory.py'
--- src/maastesting/factory.py	2016-09-23 01:54:50 +0000
+++ src/maastesting/factory.py	2016-10-05 01:31:45 +0000
@@ -654,5 +654,72 @@
             strings += [release['series'], version_str]
         return random.choice(strings)
 
+    def make_dhcp_packet(
+            self, transaction_id: bytes = None,
+            truncated: bool=False, truncated_option_value: bool=False,
+            bad_cookie: bool=False, truncated_option_length: bool=False,
+            include_server_identifier: bool=False, server_ip: str="127.1.1.1",
+            include_end_option: bool=True) -> bytes:
+        """Returns a [possibly invalid] DHCP packet."""
+        if transaction_id is None:
+            transaction_id = self.make_bytes(size=4)
+        options = b''
+        if include_server_identifier:
+            # 0x36 == 54 (Server Identifier option)
+            ip_bytes = int(IPAddress(server_ip).value).to_bytes(4, "big")
+            options += b"\x36\x04" + ip_bytes
+        if truncated_option_value:
+            options += b"\x36\x04\x7f\x01"
+            include_end_option = False
+        if truncated_option_length:
+            options += b"\x36"
+            include_end_option = False
+        # Currently, we only validation the transaction ID, and the fact that
+        # the reply packet has a "Server Identifier" option. This might be
+        # considered a bug, but in practice it works out.
+        packet = (
+            # Message type: 0x02 (BOOTP operation: reply).
+            b'\x02'
+            # Hardware type: Ethernet
+            b'\x01'
+            # Hardware address length: 6
+            b'\x06'
+            # Hops: 0
+            b'\x00' +
+            # Transaction ID
+            transaction_id +
+            # Seconds
+            b'\x00\x00'
+            # Flags
+            b'\x00\x00'
+            # Client IP address: 0.0.0.0
+            b'\x00\x00\x00\x00'
+            # Your (client) IP address: 0.0.0.0
+            b'\x00\x00\x00\x00'
+            # Next server IP address: 0.0.0.0
+            b'\x00\x00\x00\x00'
+            # Relay agent IP address: 0.0.0.0
+            b'\x00\x00\x00\x00' +
+            # Client hardware address
+            b'\x01\x02\x03\x04\x05\x06'
+            # Hardware address padding
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' +
+            # Server host name
+            (b'\x00' * 67) +
+            # Boot filename
+            (b'\x00' * 125) +
+            # Cookie
+            (b'\x63\x82\x53\x63' if not bad_cookie else b'xxxx') +
+            # "DHCP Offer" option
+            b'\x35\x01\x02' +
+            options +
+            # End options.
+            (b'\xff' if include_end_option else b'')
+        )
+        if truncated:
+            packet = packet[:200]
+        return packet
+
+
 # Create factory singleton.
 factory = Factory()

=== modified file 'src/provisioningserver/__main__.py'
--- src/provisioningserver/__main__.py	2016-09-17 00:41:01 +0000
+++ src/provisioningserver/__main__.py	2016-10-05 01:31:45 +0000
@@ -12,6 +12,7 @@
 import provisioningserver.upgrade_cluster
 import provisioningserver.utils.arp
 import provisioningserver.utils.avahi
+import provisioningserver.utils.dhcp
 import provisioningserver.utils.scan_network
 from provisioningserver.utils.script import (
     AtomicDeleteScript,
@@ -29,6 +30,7 @@
     'install-uefi-config': provisioningserver.boot.install_grub,
     'observe-arp': provisioningserver.utils.arp,
     'observe-mdns': provisioningserver.utils.avahi,
+    'observe-dhcp': provisioningserver.utils.dhcp,
     'scan-network': provisioningserver.utils.scan_network,
     'register': provisioningserver.register_command,
     'support-dump': provisioningserver.support_dump,

=== modified file 'src/provisioningserver/dhcp/detect.py'
--- src/provisioningserver/dhcp/detect.py	2016-04-11 16:23:26 +0000
+++ src/provisioningserver/dhcp/detect.py	2016-10-05 01:31:45 +0000
@@ -1,28 +1,58 @@
-
 # Copyright 2013-2016 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """Utilities and helpers to help discover DHCP servers on your network."""
 
 __all__ = [
-    'probe_dhcp',
-    ]
+    'probe_interface',
+]
 
 from contextlib import contextmanager
 import errno
 import fcntl
+from os import strerror
 from random import randint
 import socket
 import struct
+import time
+from typing import (
+    List,
+    Optional,
+)
 
+import attr
+from netaddr import IPAddress
 from provisioningserver.logger import get_maas_logger
-
-
-maaslog = get_maas_logger("dhcp.detect")
-
-
-def make_transaction_ID():
-    """Generate a random DHCP transaction identifier."""
+from provisioningserver.utils.dhcp import DHCP
+from twisted.internet import reactor
+from twisted.internet.defer import (
+    CancelledError,
+    Deferred,
+    DeferredList,
+    FirstError,
+    inlineCallbacks,
+)
+from twisted.internet.interfaces import IReactorThreads
+from twisted.internet.task import deferLater
+from twisted.internet.threads import (
+    blockingCallFromThread,
+    deferToThread,
+)
+from twisted.python import log
+from twisted.python.failure import Failure
+
+
+_LOG_SYSTEM = "dhcp.detect"
+maaslog = get_maas_logger(_LOG_SYSTEM)
+
+
+def _log_msg(*args, **kwargs):
+    """Log to the Twisted logger using a unique `system` name."""
+    log.msg(*args, system=_LOG_SYSTEM, **kwargs)
+
+
+def make_dhcp_transaction_id() -> bytes:
+    """Generate and return a random DHCP transaction identifier."""
     transaction_id = b''
     for _ in range(4):
         transaction_id += struct.pack(b'!B', randint(0, 255))
@@ -32,71 +62,117 @@
 class DHCPDiscoverPacket:
     """A representation of a DHCP_DISCOVER packet.
 
-    :param my_mac: The MAC address to which the dhcp server should respond.
+    :param mac: The MAC address to which the dhcp server should respond.
         Normally this is the MAC of the interface you're using to send the
         request.
     """
 
-    def __init__(self, my_mac):
-        self.transaction_ID = make_transaction_ID()
-        self.packed_mac = self.string_mac_to_packed(my_mac)
-        self._build()
-
-    @classmethod
-    def string_mac_to_packed(cls, mac):
+    def __init__(
+            self, mac: str=None, transaction_id: bytes=None, seconds: int=0):
+        super().__init__()
+        self.mac_bytes = None
+        self.mac_str = None
+        self.seconds = seconds
+        if transaction_id is None:
+            self.transaction_id = make_dhcp_transaction_id()
+        else:
+            self.transaction_id = transaction_id
+        if mac is not None:
+            self.set_mac(mac)
+
+    def __hash__(self):
+        # Needed for unit tests.
+        return hash((self.mac_bytes, self.seconds, self.transaction_id))
+
+    def __eq__(self, other):
+        # Needed for unit tests.
+        return (
+            (self.mac_bytes, self.seconds, self.transaction_id) ==
+            (other.mac_bytes, other.seconds, other.transaction_id)
+        )
+
+    @staticmethod
+    def mac_string_to_bytes(mac: str) -> bytes:
         """Convert a string MAC address to 6 hex octets.
 
         :param mac: A MAC address in the format AA:BB:CC:DD:EE:FF
         :return: a byte string of length 6
         """
-        packed = b''
+        mac_bytes = b''
         for pair in mac.split(':'):
             hex_octet = int(pair, 16)
-            packed += struct.pack(b'!B', hex_octet)
-        return packed
-
-    def _build(self):
-        self.packet = b''
-        self.packet += b'\x01'  # Message type: Boot Request (1)
-        self.packet += b'\x01'  # Hardware type: Ethernet
-        self.packet += b'\x06'  # Hardware address length: 6
-        self.packet += b'\x00'  # Hops: 0
-        self.packet += self.transaction_ID
-        self.packet += b'\x00\x00'  # Seconds elapsed: 0
-
-        # Bootp flags: 0x8000 (Broadcast) + reserved flags
-        self.packet += b'\x80\x00'
-
-        self.packet += b'\x00\x00\x00\x00'  # Client IP address: 0.0.0.0
-        self.packet += b'\x00\x00\x00\x00'  # Your (client) IP address: 0.0.0.0
-        self.packet += b'\x00\x00\x00\x00'  # Next server IP address: 0.0.0.0
-        self.packet += b'\x00\x00\x00\x00'  # Relay agent IP address: 0.0.0.0
-        self.packet += self.packed_mac
-
-        # Client hardware address padding: 00000000000000000000
-        self.packet += b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
-
-        self.packet += b'\x00' * 67  # Server host name not given
-        self.packet += b'\x00' * 125  # Boot file name not given
-        self.packet += b'\x63\x82\x53\x63'  # Magic cookie: DHCP
-
-        # Option: (t=53,l=1) DHCP Message Type = DHCP Discover
-        self.packet += b'\x35\x01\x01'
-
-        self.packet += b'\x3d\x06' + self.packed_mac
-
-        # Option: (t=55,l=3) Parameter Request List
-        self.packet += b'\x37\x03\x03\x01\x06'
-
-        self.packet += b'\xff'   # End Option
-
-
-class DHCPOfferPacket:
-    """A representation of a DHCP_OFFER packet."""
-
-    def __init__(self, data):
-        self.transaction_ID = data[4:8]
-        self.dhcp_server_ID = socket.inet_ntoa(data[245:249])
+            mac_bytes += struct.pack(b'!B', hex_octet)
+        return mac_bytes
+
+    def set_mac(self, mac: str) -> None:
+        """Sets the MAC address used for the client hardware address, and
+        client unique identifier option.
+        """
+        self.mac_bytes = self.mac_string_to_bytes(mac)
+        self.mac_str = mac
+
+    @property
+    def client_uid_option(self) -> bytes:
+        """Returns a `bytes` object representing the client UID.
+
+        The `set_mac()` method must have been called prior to using this.
+        """
+        # Option: (6=61,l=~23) Client Unique Identifier
+        # Make our unique identifier a little more unique by adding "MAAS-",
+        # so it will look like "\x00MAAS-00:00:00:00:00:00\x00"
+        # See https://tools.ietf.org/html/rfc2132#section-9.14 for details.
+        client_id = b"\x00MAAS-" + self.mac_str.encode("ascii")
+        client_id_len = len(client_id).to_bytes(1, "big")
+        return b'\x3d' + client_id_len + client_id
+
+    @property
+    def packet(self) -> bytes:
+        """Builds and returns the packet based on specified MAC and seconds."""
+        return (
+            # Message type: Boot Request (1)
+            b'\x01'
+            # Hardware type: Ethernet
+            b'\x01'
+            # Hardware address length: 6
+            b'\x06'
+            # Hops: 0
+            b'\x00' +
+            self.transaction_id +
+            self.seconds.to_bytes(2, "big") +
+            # Flags: the most significant bit is the broadcast bit.
+            #     0x8000 means "force the server to use broadcast".
+            #     0x0000 means "it's okay to unicast replies".
+            # We will miss packets from some DHCP servers if we don't prefer
+            # unicast. (For example, a DD-WRT router was observed sending to
+            # the broadcast address from a link-local IPv4 address, which was
+            # rejected by the IP stack before the socket could recv() it.)
+            b'\x00\x00'
+            # Client IP address: 0.0.0.0
+            b'\x00\x00\x00\x00'
+            # Your (client) IP address: 0.0.0.0
+            b'\x00\x00\x00\x00'
+            # Next server IP address: 0.0.0.0
+            b'\x00\x00\x00\x00'
+            # Relay agent IP address: 0.0.0.0
+            b'\x00\x00\x00\x00' +
+            # Client hardware address
+            self.mac_bytes +
+            # Client hardware address padding: 00000000000000000000
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' +
+            # Server host name not given
+            (b'\x00' * 67) +
+            # Boot file name not given
+            (b'\x00' * 125) +
+            # Magic cookie: DHCP
+            b'\x63\x82\x53\x63'
+            # Option: (t=53,l=1) DHCP Message Type = DHCP Discover
+            b'\x35\x01\x01' +
+            self.client_uid_option +
+            # Option: (t=55,l=3) Parameter Request List
+            b'\x37\x03\x03\x01\x06' +
+            # End Option
+            b'\xff'
+        )
 
 
 # UDP ports for the BOOTP protocol.  Used for discovery requests.
@@ -110,22 +186,55 @@
 SIOCGIFHWADDR = 0x8927
 
 
-def get_interface_MAC(sock, interface):
+def get_interface_mac(sock: socket.socket, ifname: str) -> str:
     """Obtain a network interface's MAC address, as a string."""
-    ifreq = struct.pack(b'256s', interface.encode('ascii')[:15])
-    info = fcntl.ioctl(sock.fileno(), SIOCGIFHWADDR, ifreq)
-    mac = ''.join('%02x:' % char for char in info[18:24])[:-1]
+    ifreq = struct.pack(b'256s', ifname.encode('utf-8')[:15])
+    try:
+        info = fcntl.ioctl(sock.fileno(), SIOCGIFHWADDR, ifreq)
+    except OSError as e:
+        if e.errno is not None and e.errno == errno.ENODEV:
+            raise InterfaceNotFound(
+                "Interface not found: '%s'." % ifname)
+        else:
+            raise MACAddressNotAvailable(
+                "Failed to get MAC address for '%s': %s." % (
+                    ifname, strerror(e.errno)))
+    else:
+        # Of course we're sure these are the correct indexes into the `ifreq`.
+        # Also, your lack of faith is disturbing.
+        mac = ''.join('%02x:' % char for char in info[18:24])[:-1]
     return mac
 
 
-def get_interface_IP(sock, interface):
+def get_interface_ip(sock: socket.socket, ifname: str) -> str:
     """Obtain an IP address for a network interface, as a string."""
-    ifreq = struct.pack(
-        b'16sH14s', interface.encode('ascii')[:15],
-        socket.AF_INET, b'\x00' * 14)
-    info = fcntl.ioctl(sock, SIOCGIFADDR, ifreq)
-    ip = struct.unpack(b'16sH2x4s8x', info)[2]
-    return socket.inet_ntoa(ip)
+    ifreq_tuple = (ifname.encode('utf-8')[:15], socket.AF_INET, b'\x00' * 14)
+    ifreq = struct.pack(b'16sH14s', *ifreq_tuple)
+    try:
+        info = fcntl.ioctl(sock, SIOCGIFADDR, ifreq)
+    except OSError as e:
+        if e.errno == errno.ENODEV:
+            raise InterfaceNotFound(
+                "Interface not found: '%s'." % ifname)
+        elif e.errno == errno.EADDRNOTAVAIL:
+            raise IPAddressNotAvailable(
+                "No IP address found on interface '%s'." % ifname)
+        else:
+            raise IPAddressNotAvailable(
+                "Failed to get IP address for '%s': %s." % (
+                    ifname, strerror(e.errno)))
+    else:
+        # Parse the `struct ifreq` that comes back from the ioctl() call.
+        #     16x --> char ifr_name[IFNAMSIZ];
+        # ... next is a union of structures; we're interested in the
+        # `sockaddr_in` that is returned from this particular ioctl().
+        #     2x  --> short sin_family;
+        #     2x  --> unsigned short sin_port;
+        #     4s  --> struct in_addr sin_addr;
+        #     8x  --> char sin_zero[8];
+        addr, = struct.unpack(b'16x2x2x4s8x', info)
+        ip = socket.inet_ntoa(addr)
+    return ip
 
 
 @contextmanager
@@ -142,37 +251,237 @@
         sock.close()
 
 
-def request_dhcp(interface):
-    """Broadcast a DHCP discovery request.  Return DHCP transaction ID."""
+class DHCPProbeException(Exception):
+    """Class of known-possible exceptions during DHCP probing.
+
+    These exceptions are logged without including the traceback.
+    """
+
+
+class IPAddressNotAvailable(DHCPProbeException):
+    """Raised when an interface's IP address could not be determined."""
+
+
+class MACAddressNotAvailable(DHCPProbeException):
+    """Raised when an interface's MAC address could not be determined."""
+
+
+class InterfaceNotFound(DHCPProbeException):
+    """Raised when an interface could not be found."""
+
+
+def send_dhcp_request_packet(
+        request: DHCPDiscoverPacket, ifname: str) -> None:
+    """Sends out the specified DHCP discover packet to the given interface.
+
+    Optionally takes a `retry_call` to cancel if a fatal error occurs before
+    the first packet can be sent, such as inability to get a source IP
+    address.
+    """
     with udp_socket() as sock:
         sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
-        mac = get_interface_MAC(sock, interface)
-        bind_address = get_interface_IP(sock, interface)
-        discover = DHCPDiscoverPacket(mac)
+        mac = get_interface_mac(sock, ifname)
+        request.set_mac(mac)
+        bind_address = get_interface_ip(sock, ifname)
         sock.bind((bind_address, BOOTP_CLIENT_PORT))
-        sock.sendto(discover.packet, ('<broadcast>', BOOTP_SERVER_PORT))
-    return discover.transaction_ID
-
-
-def receive_offers(transaction_id):
-    """Receive DHCP offers.  Return set of offering servers."""
-    servers = set()
-    with udp_socket() as sock:
-        # The socket we use for receiving DHCP offers must be bound to IF_ANY.
-        sock.bind(('', BOOTP_CLIENT_PORT))
-        try:
-            while True:
-                sock.settimeout(3)
-                data = sock.recv(1024)
-                offer = DHCPOfferPacket(data)
-                if offer.transaction_ID == transaction_id:
-                    servers.add(offer.dhcp_server_ID)
-        except socket.timeout:
-            # No more offers.  Done.
-            return servers
-
-
-def probe_dhcp(interface):
+        sock.sendto(request.packet, ('<broadcast>', BOOTP_SERVER_PORT))
+
+
+# Packets will be sent at the following intervals (in seconds).
+# The length of `DHCP_REQUEST_TIMING` indicates the number of packets
+# that will be sent. The values should get progressively larger, to mimic
+# the exponential back-off retry behavior of a real DHCP client.
+DHCP_REQUEST_TIMING = (0, 2, 4, 8)
+
+# Wait `REPLY_TIMEOUT` seconds to receive responses.
+# This value should be a little larger than the largest value in the
+# `DHCP_REQUEST_TIMING` tuple, to account for network and server delays.
+REPLY_TIMEOUT = 10
+
+# How long we should wait each iteration before waking up and checking if the
+# timeout has elapsed.
+SOCKET_TIMEOUT = 0.5
+
+
+class DHCPRequestMonitor:
+
+    def __init__(self, ifname: str, clock: IReactorThreads=None):
+        if clock is None:
+            clock = reactor
+        self.clock = clock    # type: IReactorThreads
+        self.ifname = ifname  # type: str
+        self.servers = None   # type: set
+        self.dhcpRequestsDeferredList = None  # type: DeferredList
+        self.deferredDHCPRequests = []        # type: List[Deferred]
+        self.transaction_id = make_dhcp_transaction_id()  # type: bytes
+
+    def send_requests_and_await_replies(self):
+        """Sends out DHCP requests and waits for their replies.
+
+        This method is intended to run under `deferToThread()`.
+
+        Calls the reactor using `blockingCallFromThread` to queue the request
+        packets.
+
+        Blocks for ~10 seconds while checking for DHCP offers.
+
+        :returns: `set` of `DHCPServer` objects.
+        """
+        # Since deferToThread() might be delayed until the next thread is
+        # available, it's better to kick off the DHCP requests from the
+        # spawned thread rather than hoping the thread starts running after
+        # we kick off the requests.
+        blockingCallFromThread(self.clock, self.deferDHCPRequests)
+        servers = set()
+        # Convert the transaction_id to an integer so we can test it against
+        # what the parsed DHCP packet will return.
+        xid = int.from_bytes(self.transaction_id, byteorder='big')
+        with udp_socket() as sock:
+            sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
+            # Note: the empty string is the equivalent of INADDR_ANY.
+            sock.bind(('', BOOTP_CLIENT_PORT))
+            # The timeout has to be relatively small, since we wake up every
+            # timeout interval to check the elapsed time.
+            sock.settimeout(0.5)
+            runtime = 0
+            # Use a monotonic clock to ensure leaping backward in time won't
+            # cause an infinite loop.
+            start_time = time.monotonic()
+            while runtime < REPLY_TIMEOUT:
+                try:
+                    # Use recvfrom() to check the source IP for the request.
+                    # It could be interesting in cases where DHCP relay is in
+                    # use.
+                    data, (address, port) = sock.recvfrom(2048)
+                except socket.timeout:
+                    continue
+                else:
+                    offer = DHCP(data)
+                    if not offer.is_valid():
+                        _log_msg(
+                            "Invalid DHCP response received from %s on '%s': "
+                            "%s" % (
+                                address, self.ifname, offer.invalid_reason))
+                    elif offer.packet.xid == xid:
+                        # Offer matches our transaction ID, so check if it has
+                        # a Server Identifier option.
+                        server = offer.server_identifier
+                        if server is not None:
+                            servers.add(DHCPServer(server, address))
+                finally:
+                    runtime = time.monotonic() - start_time
+        return servers
+
+    @staticmethod
+    def cancelAll(deferreds: List[Deferred]):
+        for deferred in deferreds:
+            deferred.cancel()
+
+    def deferredDHCPRequestErrback(
+            self, failure: Failure) -> Optional[Failure]:
+        if failure.check(FirstError):
+            # If an error occurred, cancel any other pending requests.
+            # (The error is likely to occur for those requests, too.)
+            # Unfortunately we can't cancel using the DeferredList, since
+            # the DeferredList considers itself "called" the moment the first
+            # errback is invoked.
+            self.cancelAll(self.deferredDHCPRequests)
+            # Suppress further error handling. The original Deferred's errback
+            # has already been called.
+            return None
+        elif failure.check(DHCPProbeException):
+            _log_msg("DHCP probe failed. %s" % failure.getErrorMessage())
+        elif failure.check(CancelledError):
+            # Intentionally cancelled; no need to spam the log.
+            pass
+        else:
+            log.err(
+                failure, "DHCP probe on '%s' failed with an unknown error." % (
+                    self.ifname))
+        # Make sure the error is propagated to the DeferredList.
+        # We need this so that the DeferredList knows to call us with
+        # FirstError, which is our indicator to cancel the remaining calls.
+        # (It's set to consumeErrors, so it won't spam the log.)
+        return failure
+
+    def deferDHCPRequests(self) -> None:
+        """Queues some DHCP requests to fire off later.
+
+        Delays calls slightly so we have a chance to open a listen socket.
+
+        Uses the `clock`, `ifname`, and `transaction_id` ivars.
+
+        Stores a `DeferredList` of periodic DHCP requests calls in the
+        `dhcpRequestsDeferredList` ivar, and the list of `Deferred` objects in
+        the `deferredDHCPRequests` ivar.
+        """
+        self.deferredDHCPRequests = []
+        for seconds in DHCP_REQUEST_TIMING:
+            packet = DHCPDiscoverPacket(
+                transaction_id=self.transaction_id, seconds=seconds)
+            # Wait 0.1 seconds before sending the request, so we have a chance
+            # to open a listen socket.
+            seconds += 0.1
+            deferred = deferLater(
+                self.clock, seconds, send_dhcp_request_packet, packet,
+                self.ifname)
+            deferred.addErrback(self.deferredDHCPRequestErrback)
+            self.deferredDHCPRequests.append(deferred)
+        # Use fireOnOneErrback so that we know to cancel the remaining attempts
+        # to send requests if one of them fails.
+        self.dhcpRequestsDeferredList = DeferredList(
+            self.deferredDHCPRequests, fireOnOneErrback=True,
+            consumeErrors=True)
+        self.dhcpRequestsDeferredList.addErrback(
+            self.deferredDHCPRequestErrback)
+
+    @inlineCallbacks
+    def run(self) -> Deferred:
+        """Queues DHCP requests to be sent, then waits (in a separate thread)
+        for replies.
+
+        Requests will be sent using an exponential back-off algorithm, to mimic
+        a real DHCP client. But we'll just pretend we didn't see any of the
+        replies, and hope for more servers to respond.
+
+        The set of `DHCPServer`s that responded to the request(s) is stored
+        in the `servers` ivar, which in turn makes the `dhcp_servers` and
+        `dhcp_addresses` properties useful.
+        """
+        servers = yield deferToThread(self.send_requests_and_await_replies)
+        if len(servers) > 0:
+            _log_msg(
+                "External DHCP server(s) discovered on interface '%s': %s"
+                % (self.ifname, ', '.join(str(server) for server in servers)))
+        self.servers = servers
+
+    @property
+    def dhcp_servers(self):
+        return set(str(server.server) for server in self.servers)
+
+    @property
+    def dhcp_addresses(self):
+        return set(str(server.address) for server in self.servers)
+
+
+@attr.s
+class DHCPServer:
+    server = attr.ib(convert=IPAddress)
+    address = attr.ib(convert=IPAddress)
+
+    def __str__(self):
+        """Returns eitehr a longer format string (if the address we received
+        the packet from is different from the DHCP server address specified in
+        the packet) or a single IP address (if the address we received the
+        packet from and the server address are the same).
+        """
+        if self.server == self.address:
+            return str(self.server)
+        return "%s (via %s)" % (self.server, self.address)
+
+
+@inlineCallbacks
+def probe_interface(interface):
     """Look for a DHCP server on the network.
 
     This must be run with provileges to broadcast from the BOOTP port, which
@@ -182,45 +491,12 @@
     :param interface: Network interface name, e.g. "eth0", attached to the
         network you wish to probe.
     :return: Set of discovered DHCP servers.
-
-    :exception IOError: If the interface does not have an IP address.
-    """
-    # There is a small race window here, after we close the first socket and
-    # before we bind the second one.  Hopefully executing a few lines of code
-    # will be faster than communication over the network.
-    # UDP is not reliable at any rate.  If detection is important, we should
-    # send out repeated requests.
-    transaction_id = request_dhcp(interface)
-    return receive_offers(transaction_id)
-
-
-def probe_interface(interface):
-    """Probe the given interface for DHCP servers.
-
-    :param interface: interface name
-    :return: A set of IP addresses of detected servers.
-
-    :note: Any servers running on the IP address of the local host are
-        filtered out as they will be the MAAS DHCP server.
-    """
-    try:
-        servers = probe_dhcp(interface)
-    except IOError as e:
-        servers = set()
-        if e.errno == errno.EADDRNOTAVAIL:
-            # Errno EADDRNOTAVAIL is "Cannot assign requested address"
-            # which we need to ignore; it means the interface has no IP
-            # and there's no need to scan this interface as it's not in
-            # use.
-            maaslog.debug(
-                "Ignoring DHCP scan for %s, it has no IP address", interface)
-        elif e.errno == errno.ENODEV:
-            # Errno ENODEV is "no such device". This seems an odd situation
-            # since we're scanning detected devices, so this is probably
-            # a bug.
-            maaslog.error(
-                "Ignoring DHCP scan for %s, it no longer exists. Check "
-                "your cluster interfaces configuration.", interface)
-        else:
-            raise
-    return servers
+    """
+    dhcp_request_monitor = DHCPRequestMonitor(interface)
+    yield dhcp_request_monitor.run()
+    # The caller expects a set of addresses in unicode format.
+    # XXX We might want to consider using the address we got from
+    # recvfrom(), since that is likely the address relaying to this
+    # interface. (Those are stored in the `dchp_addresses` ivar.)
+    # Further investigation required.
+    return dhcp_request_monitor.dhcp_servers

=== modified file 'src/provisioningserver/dhcp/tests/test_detect.py'
--- src/provisioningserver/dhcp/tests/test_detect.py	2016-05-12 19:07:37 +0000
+++ src/provisioningserver/dhcp/tests/test_detect.py	2016-10-05 01:31:45 +0000
@@ -6,121 +6,180 @@
 __all__ = []
 
 import errno
-import fcntl
+import random
 import socket
 from unittest import mock
+from unittest.mock import call
 
 from maastesting.factory import factory
+from maastesting.matchers import (
+    DocTestMatches,
+    Equals,
+    HasLength,
+    MockCallsMatch,
+)
+from maastesting.runtest import MAASTwistedRunTest
 from maastesting.testcase import MAASTestCase
+from maastesting.twisted import TwistedLoggerFixture
 from provisioningserver.dhcp.detect import (
     BOOTP_CLIENT_PORT,
     BOOTP_SERVER_PORT,
+    DHCP_REQUEST_TIMING,
     DHCPDiscoverPacket,
-    DHCPOfferPacket,
-    get_interface_IP,
-    get_interface_MAC,
-    make_transaction_ID,
+    DHCPProbeException,
+    DHCPRequestMonitor,
+    DHCPServer,
+    get_interface_ip,
+    get_interface_mac,
+    InterfaceNotFound,
+    IPAddressNotAvailable,
+    MACAddressNotAvailable,
+    make_dhcp_transaction_id,
     probe_interface,
-    receive_offers,
-    request_dhcp,
+    send_dhcp_request_packet,
     udp_socket,
 )
 import provisioningserver.dhcp.detect as detect_module
-from provisioningserver.testing.testcase import PservTestCase
-
-
-class MakeTransactionID(MAASTestCase):
-    """Tests for `make_transaction_ID`."""
-
-    def test_produces_well_formed_ID(self):
-        # The dhcp transaction should be 4 bytes long.
-        transaction_id = make_transaction_ID()
+from testtools import ExpectedException
+from testtools.matchers import Contains
+from twisted.internet import reactor
+from twisted.internet.defer import (
+    CancelledError,
+    DeferredList,
+    inlineCallbacks,
+)
+from twisted.internet.task import (
+    Clock,
+    deferLater,
+)
+from twisted.python.failure import Failure
+
+
+class MakeDHCPTransactionID(MAASTestCase):
+    """Tests for `make_dhcp_transaction_id`."""
+
+    def test_produces_well_formed_id(self):
+        # The DHCP transaction ID should be 4 bytes long.
+        transaction_id = make_dhcp_transaction_id()
         self.assertIsInstance(transaction_id, bytes)
         self.assertEqual(4, len(transaction_id))
 
     def test_randomises(self):
         self.assertNotEqual(
-            make_transaction_ID(),
-            make_transaction_ID())
+            make_dhcp_transaction_id(),
+            make_dhcp_transaction_id())
 
 
 class TestDHCPDiscoverPacket(MAASTestCase):
 
-    def test_init_sets_transaction_ID(self):
-        transaction_id = make_transaction_ID()
-        self.patch(detect_module, 'make_transaction_ID').return_value = (
+    def test_init_sets_transaction_id(self):
+        transaction_id = make_dhcp_transaction_id()
+        self.patch(detect_module, 'make_dhcp_transaction_id').return_value = (
             transaction_id)
-
         discover = DHCPDiscoverPacket(factory.make_mac_address())
-
-        self.assertEqual(transaction_id, discover.transaction_ID)
-
-    def test_init_sets_packed_mac(self):
+        self.assertEqual(transaction_id, discover.transaction_id)
+
+    def test_init_sets_mac_bytes(self):
         mac = factory.make_mac_address()
         discover = DHCPDiscoverPacket(mac)
         self.assertEqual(
-            discover.string_mac_to_packed(mac),
-            discover.packed_mac)
+            discover.mac_string_to_bytes(mac),
+            discover.mac_bytes)
 
-    def test_init_sets_packet(self):
+    def test__packet_property_after_init_with_mac_and_no_transaction_id(self):
         discover = DHCPDiscoverPacket(factory.make_mac_address())
         self.assertIsNotNone(discover.packet)
 
-    def test_string_mac_to_packed(self):
+    def test__converts_byte_string_to_bytes(self):
         discover = DHCPDiscoverPacket
         expected = b"\x01\x22\x33\x99\xaa\xff"
         input = "01:22:33:99:aa:ff"
-        self.assertEqual(expected, discover.string_mac_to_packed(input))
+        self.assertEqual(expected, discover.mac_string_to_bytes(input))
 
-    def test__build(self):
+    def test__builds_packet(self):
         mac = factory.make_mac_address()
-        discover = DHCPDiscoverPacket(mac)
-        discover._build()
-
+        seconds = random.randint(0, 1024)
+        xid = factory.make_bytes(4)
+        discover = DHCPDiscoverPacket(
+            transaction_id=xid, mac=mac, seconds=seconds)
+        seconds_bytes = seconds.to_bytes(2, "big")
         expected = (
-            b'\x01\x01\x06\x00' + discover.transaction_ID +
-            b'\x00\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00' +
-            b'\x00\x00\x00\x00\x00\x00\x00\x00' +
-            discover.packed_mac +
+            b'\x01\x01\x06\x00' + xid + seconds_bytes +
+            b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' +
+            b'\x00\x00\x00\x00\x00\x00' +
+            discover.mac_bytes +
             b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' +
             b'\x00' * 67 +
             b'\x00' * 125 +
-            b'\x63\x82\x53\x63\x35\x01\x01\x3d\x06' + discover.packed_mac +
+            b'\x63\x82\x53\x63\x35\x01\x01' +
+            discover.client_uid_option +
             b'\x37\x03\x03\x01\x06\xff')
-
         self.assertEqual(expected, discover.packet)
 
 
-class TestDHCPOfferPacket(MAASTestCase):
-
-    def test_decodes_dhcp_server(self):
-        buffer = b'\x00' * 245 + b'\x10\x00\x00\xaa'
-        offer = DHCPOfferPacket(buffer)
-        self.assertEqual('16.0.0.170', offer.dhcp_server_ID)
-
-
 class TestGetInterfaceMAC(MAASTestCase):
-    """Tests for `get_interface_MAC`."""
+    """Tests for `get_interface_mac`."""
 
-    def test_loopback_has_zero_MAC(self):
+    def test__loopback_has_zero_mac(self):
         # It's a lame test, but what other network interfaces can we reliably
         # test this on?
         sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
-        self.assertEqual('00:00:00:00:00:00', get_interface_MAC(sock, 'lo'))
+        self.assertEqual('00:00:00:00:00:00', get_interface_mac(sock, 'lo'))
+
+    def test__invalid_interface_raises_interfacenotfound(self):
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+        with ExpectedException(InterfaceNotFound):
+            get_interface_mac(sock, factory.make_unicode_string(size=15))
+
+    def test__no_mac_raises_macaddressnotavailable(self):
+        mock_ioerror = IOError()
+        mock_ioerror.errno = errno.EOPNOTSUPP
+        self.patch(detect_module.fcntl, 'ioctl').side_effect = mock_ioerror
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+        # Drive-by test for the strerror() call.
+        with ExpectedException(
+                MACAddressNotAvailable, '.*not supported.*'):
+            get_interface_mac(sock, 'lo')
 
 
 class TestGetInterfaceIP(MAASTestCase):
-    """Tests for `get_interface_IP`."""
+    """Tests for `get_interface_ip`."""
 
     def test_loopback_has_localhost_address(self):
         sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
-        self.assertEqual('127.0.0.1', get_interface_IP(sock, 'lo'))
+        self.assertEqual('127.0.0.1', get_interface_ip(sock, 'lo'))
+
+    def test__invalid_interface_raises_interfacenotfound(self):
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+        with ExpectedException(detect_module.InterfaceNotFound):
+            get_interface_ip(sock, factory.make_unicode_string(size=15))
+
+    def test__no_ip_raises_ipaddressnotavailable(self):
+        mock_ioerror = IOError()
+        mock_ioerror.errno = errno.EADDRNOTAVAIL
+        self.patch(detect_module.fcntl, 'ioctl').side_effect = mock_ioerror
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+        with ExpectedException(
+                IPAddressNotAvailable, '.*No IP address.*'):
+            get_interface_ip(sock, 'lo')
+
+    def test__unknown_errno_ip_raises_ipaddressnotavailable(self):
+        mock_ioerror = IOError()
+        mock_ioerror.errno = errno.EACCES
+        self.patch(detect_module.fcntl, 'ioctl').side_effect = mock_ioerror
+        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+        # Drive-by test for the strerror() call.
+        with ExpectedException(
+                IPAddressNotAvailable,
+                'Failed.*Permission denied.'):
+            get_interface_ip(sock, 'lo')
 
 
 def patch_socket(testcase):
     """Patch `socket.socket` to return a mock."""
     sock = mock.MagicMock()
-    testcase.patch(socket, 'socket', mock.MagicMock(return_value=sock))
+    testcase.patch(
+        detect_module.socket, 'socket', mock.MagicMock(return_value=sock))
     return sock
 
 
@@ -152,175 +211,242 @@
             sock.setsockopt.mock_calls)
 
 
-class TestRequestDHCP(MAASTestCase):
-    """Tests for `request_dhcp`."""
-
-    def patch_interface_MAC(self):
-        """Patch `get_interface_MAC` to return a fixed value."""
-        mac = factory.make_mac_address()
-        self.patch(detect_module, 'get_interface_MAC').return_value = mac
-        return mac
-
-    def patch_interface_IP(self):
-        """Patch `get_interface_IP` to return a fixed value."""
-        ip = factory.make_ipv4_address()
-        self.patch(detect_module, 'get_interface_IP').return_value = ip
-        return ip
-
-    def patch_transaction_ID(self):
-        """Patch `make_transaction_ID` to return a fixed value."""
-        transaction_id = make_transaction_ID()
-        self.patch(
-            detect_module, 'make_transaction_ID').return_value = transaction_id
-        return transaction_id
-
-    def test_sends_discover_packet(self):
-        sock = patch_socket(self)
-        self.patch_interface_MAC()
-        self.patch_interface_IP()
-        interface = factory.make_name('interface')
-
-        request_dhcp(interface)
-
-        [call] = sock.sendto.mock_calls
-        _, args, _ = call
-        self.assertEqual(
-            ('<broadcast>', BOOTP_SERVER_PORT),
-            args[1])
-
-    def test_returns_transaction_id(self):
-        patch_socket(self)
-        self.patch_interface_MAC()
-        self.patch_interface_IP()
-        transaction_id = self.patch_transaction_ID()
-        interface = factory.make_name('interface')
-
-        self.assertEqual(transaction_id, request_dhcp(interface))
-
-
-class FakePacketReceiver:
-    """Fake callable to substitute for a socket's `recv`.
-
-    Returns the given packets on successive calls.  When it runs out,
-    raises a timeout.
-    """
-
-    def __init__(self, packets=None):
-        if packets is None:
-            packets = []
-        self.calls = []
-        self.packets = list(packets)
-
-    def __call__(self, recv_size):
-        self.calls.append(recv_size)
-        if len(self.packets) == 0:
-            raise socket.timeout()
-        else:
-            return self.packets.pop(0)
-
-
-class TestReceiveOffers(MAASTestCase):
-    """Tests for `receive_offers`."""
-
-    def patch_recv(self, sock, num_packets=0):
-        """Patch up socket's `recv` to return `num_packets` arbitrary packets.
-
-        After that, further calls to `recv` will raise a timeout.
-        """
-        packets = [factory.make_bytes() for _ in range(num_packets)]
-        receiver = FakePacketReceiver(packets)
-        self.patch(sock, 'recv', receiver)
-        return receiver
-
-    def patch_offer_packet(self):
-        """Patch a mock `DHCPOfferPacket`."""
-        transaction_id = factory.make_bytes(4)
-        packet = mock.MagicMock()
-        packet.transaction_ID = transaction_id
-        packet.dhcp_server_ID = factory.make_ipv4_address()
-        self.patch(detect_module, 'DHCPOfferPacket').return_value = packet
-        return packet
-
-    def test_receives_from_socket(self):
-        sock = patch_socket(self)
-        receiver = self.patch_recv(sock)
-        transaction_id = self.patch_offer_packet().transaction_ID
-
-        receive_offers(transaction_id)
-
-        self.assertEqual(
-            [mock.call(socket.AF_INET, socket.SOCK_DGRAM)],
-            socket.socket.mock_calls)
-        self.assertEqual(
-            [mock.call(('', BOOTP_CLIENT_PORT))],
-            sock.bind.mock_calls)
-        self.assertEqual([1024], receiver.calls)
-
-    def test_returns_empty_if_nothing_received(self):
-        sock = patch_socket(self)
-        self.patch_recv(sock)
-        transaction_id = self.patch_offer_packet().transaction_ID
-
-        self.assertEqual(set(), receive_offers(transaction_id))
-
-    def test_processes_offer(self):
-        sock = patch_socket(self)
-        self.patch_recv(sock, 1)
-        packet = self.patch_offer_packet()
-
-        self.assertEqual(
-            {packet.dhcp_server_ID},
-            receive_offers(packet.transaction_ID))
-
-    def test_ignores_other_transactions(self):
-        sock = patch_socket(self)
-        self.patch_recv(sock, 1)
-        self.patch_offer_packet()
-        other_transaction_id = factory.make_bytes(4)
-
-        self.assertEqual(set(), receive_offers(other_transaction_id))
-
-    def test_propagates_errors_other_than_timeout(self):
-        class InducedError(Exception):
-            """Deliberately induced error for testing."""
-
-        sock = patch_socket(self)
-        sock.recv = mock.MagicMock(side_effect=InducedError)
-
-        self.assertRaises(
-            InducedError,
-            receive_offers, factory.make_bytes(4))
-
-
-class TestPeriodicTask(PservTestCase):
-
-    def test_probe_interface_returns_empty_set_when_nothing_detected(self):
-        self.patch(detect_module, 'probe_dhcp').return_value = set()
-        results = probe_interface("eth0")
-        self.assertEqual(set(), results)
-
-    def test_probe_interface_returns_empty_set_when_IP_missing(self):
-        # If the interface being probed has no IP address, the
-        # request_dhcr() method will raise IOError with errno 99. Make
-        # sure this is caught and ignored.
-        ioerror = IOError(
-            errno.EADDRNOTAVAIL, "Cannot assign requested address")
-        self.patch(fcntl, 'ioctl').side_effect = ioerror
-        results = probe_interface("eth0")
-        self.assertEqual(set(), results)
-
-    def test_probe_interface_returns_empty_set_when_device_missing(self):
-        # If the interface being probed does not exist, the
-        # request_dhcp() method will raise IOError with errno 19. Make
-        # sure this is caught and ignored.
-        ioerror = IOError(errno.ENODEV, "No such device")
-        self.patch(fcntl, 'ioctl').side_effect = ioerror
-        results = probe_interface("eth0")
-        self.assertEqual(set(), results)
-
-    def test_probe_interface_returns_populated_set(self):
-        # Test that the detected DHCP server is returned.
-        self.patch(
-            detect_module, 'probe_dhcp').return_value = {'10.2.2.2'}
-        results = probe_interface("eth0")
-        self.assertEqual({'10.2.2.2'}, results)
+class TestSendDHCPRequestPacket(MAASTestCase):
+
+    def test__sends_expected_packet(self):
+        mock_socket = patch_socket(self)
+        mock_socket.bind = mock.MagicMock()
+        mock_socket.sendto = mock.MagicMock()
+        self.patch(detect_module.get_interface_ip).return_value = '127.0.0.1'
+        self.patch(
+            detect_module.get_interface_mac).return_value = '00:00:00:00:00:00'
+        request = DHCPDiscoverPacket()
+        send_dhcp_request_packet(request, 'lo')
+        self.assertThat(
+            mock_socket.bind, MockCallsMatch(
+                call(('127.0.0.1', BOOTP_CLIENT_PORT))))
+        self.assertThat(
+            mock_socket.sendto, MockCallsMatch(
+                call(request.packet, ('<broadcast>', BOOTP_SERVER_PORT))))
+
+
+def make_Failure(exception_type, *args, **kwargs):
+    try:
+        raise exception_type(*args, **kwargs)
+    except:
+        return Failure()
+
+
+class TestDHCPRequestMonitor(MAASTestCase):
+
+    run_tests_with = MAASTwistedRunTest.make_factory(debug=False, timeout=5)
+
+    def test__send_requests_and_await_replies(self):
+        # This test is a bit large because it covers the entire functionality
+        # of the `send_requests_and_await_replies()` method. (It could be
+        # split apart into multiple tests, but the large amount of setup work
+        # and interdependencies makes that a maintenance burden.)
+        mock_socket = patch_socket(self)
+        mock_socket.bind = mock.MagicMock()
+        mock_socket.recvfrom = mock.MagicMock()
+        mock_socket.setsockopt = mock.MagicMock()
+        mock_socket.settimeout = mock.MagicMock()
+        # Pretend we were successful at deferring the DHCP requests.
+        self.patch_autospec(detect_module, 'blockingCallFromThread')
+        # This method normally blocks for ~10 seconds, so take control of the
+        # monotonic clock and make sure the last call to `recvfrom()` happens
+        # just as we hit the reply timeout.
+        mock_time_monotonic = self.patch(detect_module.time.monotonic)
+        mock_time_monotonic.side_effect = (
+            # Start time (before loop starts).
+            10,
+            # First reply (truncated packet).
+            11,
+            # Second reply (not a match to our transaction).
+            12,
+            # Third reply (Matching reply with server identifier option).
+            13,
+            # First socket timeout (need to make sure the loop continues).
+            14,
+            # Second socket timeout (hey, we're done!).
+            10 + detect_module.REPLY_TIMEOUT
+        )
+        mock_xid = factory.make_bytes(4)
+        valid_dhcp_reply = factory.make_dhcp_packet(
+            transaction_id=mock_xid,
+            include_server_identifier=True, server_ip='127.1.1.1')
+        mock_get_xid = self.patch(detect_module.make_dhcp_transaction_id)
+        mock_get_xid.return_value = mock_xid
+        # Valid DHCP packet, but not a match because it doesn't have a
+        # Server Identifier option.
+        valid_non_match = DHCPDiscoverPacket(
+            mac="01:02:03:04:05:06", transaction_id=mock_xid).packet
+        mock_socket.recvfrom.side_effect = (
+            # Truncated packet, to test logging.
+            (b'', ('127.0.0.1', BOOTP_SERVER_PORT)),
+            (valid_non_match, ('127.0.0.2', BOOTP_SERVER_PORT)),
+            (valid_dhcp_reply, ('127.0.0.3', BOOTP_SERVER_PORT)),
+            socket.timeout,
+            socket.timeout,
+        )
+        logger = self.useFixture(TwistedLoggerFixture())
+        monitor = DHCPRequestMonitor('lo', Clock())
+        result = monitor.send_requests_and_await_replies()
+        self.assertThat(
+            mock_socket.setsockopt, MockCallsMatch(
+                call(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1),
+                call(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)))
+        self.assertThat(
+            mock_socket.bind, MockCallsMatch(call(('', 68))))
+        self.assertThat(
+            mock_socket.settimeout, MockCallsMatch(
+                call(detect_module.SOCKET_TIMEOUT)))
+        self.assertThat(
+            mock_socket.recvfrom, MockCallsMatch(
+                call(2048), call(2048), call(2048), call(2048), call(2048)))
+        # One of the response packets was truncated.
+        self.assertThat(logger.output, DocTestMatches(
+            "Invalid DHCP response...Truncated..."))
+        self.assertThat(result, HasLength(1))
+        # Ensure we record the fact that the reply packet came from a different
+        # IP address than the server claimed to be.
+        self.assertThat(result, Contains(DHCPServer('127.1.1.1', '127.0.0.3')))
+
+    @inlineCallbacks
+    def test__cancelAll(self):
+        self.errbacks_called = 0
+
+        def mock_errback(result: Failure):
+            self.assertTrue(result.check(CancelledError))
+            self.errbacks_called += 1
+        a = deferLater(reactor, 6, lambda: 'a')
+        b = deferLater(reactor, 6, lambda: 'b')
+        a.addBoth(mock_errback)
+        b.addBoth(mock_errback)
+        deferreds = [a, b]
+        DHCPRequestMonitor.cancelAll(deferreds)
+        deferredList = DeferredList(deferreds)
+        yield deferredList
+        self.assertThat(self.errbacks_called, Equals(2))
+
+    @inlineCallbacks
+    def test__deferredDHCPRequestErrback_cancels_all_on_FirstError(self):
+        mock_cancelAll = self.patch(DHCPRequestMonitor, 'cancelAll')
+
+        def raise_ioerror():
+            raise IOError()
+        a = deferLater(reactor, 0.0, raise_ioerror)
+        b = deferLater(reactor, 6, lambda: 'b')
+        monitor = DHCPRequestMonitor('lo')
+        monitor.deferredDHCPRequests = [a, b]
+        deferredList = DeferredList(
+            monitor.deferredDHCPRequests,
+            consumeErrors=True, fireOnOneErrback=True)
+        deferredList.addErrback(monitor.deferredDHCPRequestErrback)
+        yield deferredList
+        # Still have one call left in the reactor, since we mocked cancelAll().
+        b.cancel()
+        self.assertThat(mock_cancelAll, MockCallsMatch(call([a, b])))
+
+    def test__deferredDHCPRequestErrback_logs_known_exceptions(self):
+        logger = self.useFixture(TwistedLoggerFixture())
+        monitor = DHCPRequestMonitor('lo')
+        error = factory.make_string()
+        monitor.deferredDHCPRequestErrback(
+            make_Failure(DHCPProbeException, error))
+        self.assertThat(logger.output, DocTestMatches(
+            "DHCP probe failed. %s" % error))
+
+    def test__deferredDHCPRequestErrback_logs_unknown_exceptions(self):
+        logger = self.useFixture(TwistedLoggerFixture())
+        monitor = DHCPRequestMonitor('lo')
+        error = factory.make_string()
+        monitor.deferredDHCPRequestErrback(
+            make_Failure(IOError, error))
+        self.assertThat(logger.output, DocTestMatches(
+            "...unknown error...Traceback...%s" % error))
+
+    def test__deferredDHCPRequestErrback_ignores_cancelled(self):
+        logger = self.useFixture(TwistedLoggerFixture())
+        monitor = DHCPRequestMonitor('lo')
+        error = factory.make_string()
+        monitor.deferredDHCPRequestErrback(
+            make_Failure(CancelledError, error))
+        self.assertThat(logger.output, DocTestMatches(""))
+
+    def test__deferDHCPRequests(self):
+        clock = Clock()
+        monitor = DHCPRequestMonitor('lo', clock)
+        mock_addErrback = mock.MagicMock()
+        mock_deferredListResult = mock.MagicMock()
+        mock_deferLater = self.patch(detect_module, 'deferLater')
+        mock_deferLater.return_value = mock.MagicMock()
+        mock_deferLater.return_value.addErrback = mock_addErrback
+        mock_DeferredList = self.patch(detect_module, 'DeferredList')
+        mock_DeferredList.return_value = mock_deferredListResult
+        mock_deferredListResult.addErrback = mock_addErrback
+        expected_calls = [
+            call(clock, seconds + 0.1, send_dhcp_request_packet,
+                 DHCPDiscoverPacket(
+                     transaction_id=monitor.transaction_id, seconds=seconds),
+                 'lo')
+            for seconds in DHCP_REQUEST_TIMING
+        ]
+        monitor.deferDHCPRequests()
+        self.assertThat(
+            mock_deferLater, MockCallsMatch(*expected_calls))
+        self.assertThat(mock_DeferredList, MockCallsMatch(
+            call(
+                monitor.deferredDHCPRequests, fireOnOneErrback=True,
+                consumeErrors=True)))
+        # Expect addErrback to be called both on each individual Deferred, plus
+        # one more time on the DeferredList.
+        expected_errback_calls = [
+            call(monitor.deferredDHCPRequestErrback)
+            for _ in range(len(DHCP_REQUEST_TIMING) + 1)
+        ]
+        self.assertThat(
+            mock_addErrback, MockCallsMatch(*expected_errback_calls))
+
+    @inlineCallbacks
+    def test__run_logs_result_and_makes_properties_available(self):
+        logger = self.useFixture(TwistedLoggerFixture())
+        monitor = DHCPRequestMonitor('lo')
+        mock_send_and_await = self.patch(
+            monitor, 'send_requests_and_await_replies')
+        mock_send_and_await.return_value = {
+            DHCPServer('127.0.0.1', '127.0.0.1'),
+            DHCPServer('127.1.1.1', '127.2.2.2'),
+        }
+        yield monitor.run()
+        self.assertThat(mock_send_and_await, MockCallsMatch(call()))
+        self.assertThat(logger.output, DocTestMatches(
+            "External DHCP server(s) discovered on interface 'lo': 127.0.0.1, "
+            "127.1.1.1 (via 127.2.2.2)"))
+        self.assertThat(monitor.dhcp_servers, Equals(
+            {"127.0.0.1", "127.1.1.1"}))
+        self.assertThat(monitor.dhcp_addresses, Equals(
+            {"127.0.0.1", "127.2.2.2"}))
+
+    @inlineCallbacks
+    def test__run_skips_logging_if_no_servers_found(self):
+        logger = self.useFixture(TwistedLoggerFixture())
+        monitor = DHCPRequestMonitor('lo')
+        mock_send_and_await = self.patch(
+            monitor, 'send_requests_and_await_replies')
+        mock_send_and_await.return_value = {}
+        yield monitor.run()
+        self.assertThat(mock_send_and_await, MockCallsMatch(call()))
+        self.assertThat(logger.output, DocTestMatches(""))
+
+    @inlineCallbacks
+    def test__run_via_probe_interface_returns_servers(self):
+        mock_send_and_await = self.patch(
+            DHCPRequestMonitor, 'send_requests_and_await_replies')
+        mock_send_and_await.return_value = {
+            DHCPServer('127.0.0.1', '127.0.0.1'),
+            DHCPServer('127.1.1.1', '127.2.2.2'),
+        }
+        result = yield probe_interface('lo')
+        self.assertThat(mock_send_and_await, MockCallsMatch(call()))
+        self.assertThat(result, Equals({"127.0.0.1", "127.1.1.1"}))

=== modified file 'src/provisioningserver/rackdservices/dhcp_probe_service.py'
--- src/provisioningserver/rackdservices/dhcp_probe_service.py	2016-04-11 16:23:26 +0000
+++ src/provisioningserver/rackdservices/dhcp_probe_service.py	2016-10-05 01:31:45 +0000
@@ -7,10 +7,10 @@
     "DHCPProbeService",
     ]
 
-
 from datetime import timedelta
 import socket
 
+from provisioningserver.config import is_dev_environment
 from provisioningserver.dhcp.detect import probe_interface
 from provisioningserver.logger.log import get_maas_logger
 from provisioningserver.rpc.exceptions import NoConnectionsAvailable
@@ -21,15 +21,19 @@
     retries,
 )
 from twisted.application.internet import TimerService
-from twisted.internet.defer import inlineCallbacks
+from twisted.internet.defer import (
+    inlineCallbacks,
+    maybeDeferred,
+)
 from twisted.internet.threads import deferToThread
 from twisted.protocols.amp import UnhandledCommand
+from twisted.python import log
 
 
 maaslog = get_maas_logger("dhcp.probe")
 
 
-class DHCPProbeService(TimerService, object):
+class DHCPProbeService(TimerService):
     """Service to probe for DHCP servers on the rack controller interface's.
 
     Built on top of Twisted's `TimerService`.
@@ -46,6 +50,9 @@
         self.clock = reactor
         self.client_service = client_service
 
+    def log(self, string):
+        log.msg(string, system=type(self).__name__)
+
     def _get_interfaces(self):
         """Return the interfaces for this rack controller."""
         d = deferToThread(get_all_interfaces_definition)
@@ -81,10 +88,7 @@
         return d
 
     @inlineCallbacks
-    def probe_dhcp(self):
-        """Find all the interfaces on this rack controller and probe for
-        DHCP servers.
-        """
+    def _tryGetClient(self):
         client = None
         for elapsed, remaining, wait in retries(15, 5, self.clock):
             try:
@@ -92,31 +96,46 @@
                 break
             except NoConnectionsAvailable:
                 yield pause(wait, self.clock)
-        else:
+        return client
+
+    @inlineCallbacks
+    def probe_dhcp(self):
+        """Find all the interfaces on this rack controller and probe for
+        DHCP servers.
+        """
+        client = yield self._tryGetClient()
+        if client is None:
             maaslog.error(
-                "Can't initiate DHCP probe, no RPC connection to region.")
+                "Can't initiate DHCP probe; no RPC connection to region.")
             return
 
         # Iterate over interfaces and probe each one.
         interfaces = yield self._get_interfaces()
+        self.log(
+            "Probe for external DHCP servers started on interfaces: %s." % (
+                ', '.join(interfaces)))
         for interface in interfaces:
             try:
-                servers = yield deferToThread(probe_interface, interface)
-            except socket.error:
-                maaslog.error(
-                    "Failed to probe sockets; did you configure authbind as "
-                    "per HACKING.txt?")
-                break
+                servers = yield maybeDeferred(probe_interface, interface)
+            except socket.error as e:
+                error = (
+                    "Failed to probe for external DHCP servers on interface "
+                    "'%s'." % interface)
+                if is_dev_environment():
+                    error += " (Did you configure authbind per HACKING.txt?)"
+                log.err(e, error, system=type(self).__name__)
+                continue
             else:
                 if len(servers) > 0:
-                    # Only send one, if it gets cleared out then the
-                    # next detection pass will send a different one, if it
-                    # still exists.
+                    # XXX For now, only send the region one server, since
+                    # it can only track one per VLAN (this could be considered
+                    # a bug).
                     yield self._inform_region_of_dhcp(
                         client, interface, servers.pop())
                 else:
                     yield self._inform_region_of_dhcp(
                         client, interface, None)
+        self.log("External DHCP probe complete.")
 
     @inlineCallbacks
     def try_probe_dhcp(self):

=== modified file 'src/provisioningserver/rackdservices/tests/test_dhcp_probe_service.py'
--- src/provisioningserver/rackdservices/tests/test_dhcp_probe_service.py	2016-07-30 01:17:54 +0000
+++ src/provisioningserver/rackdservices/tests/test_dhcp_probe_service.py	2016-10-05 01:31:45 +0000
@@ -7,7 +7,6 @@
 
 
 from unittest.mock import (
-    call,
     Mock,
     sentinel,
 )
@@ -17,7 +16,6 @@
     get_mock_calls,
     HasLength,
     MockCalledOnceWith,
-    MockCallsMatch,
     MockNotCalled,
 )
 from maastesting.testcase import MAASTwistedRunTest
@@ -32,6 +30,7 @@
 from provisioningserver.rpc.testing import MockLiveClusterToRegionRPCFixture
 from provisioningserver.testing.testcase import PservTestCase
 from twisted.internet import defer
+from twisted.internet.defer import inlineCallbacks
 from twisted.internet.task import Clock
 
 
@@ -73,28 +72,7 @@
         # Now there were two calls.
         self.assertThat(get_mock_calls(probe_dhcp), HasLength(2))
 
-    def test_probe_is_initiated_in_new_thread(self):
-        clock = Clock()
-        interface_name = factory.make_name("eth")
-        interfaces = {
-            interface_name: {
-                "enabled": True,
-            }
-        }
-
-        deferToThread = self.patch(dhcp_probe_service, 'deferToThread')
-        deferToThread.side_effect = [
-            defer.succeed(interfaces),
-            defer.succeed(None),
-        ]
-        service = DHCPProbeService(Mock(), clock)
-        service.startService()
-        self.assertThat(
-            deferToThread, MockCallsMatch(
-                call(dhcp_probe_service.get_all_interfaces_definition),
-                call(dhcp_probe_service.probe_interface, interface_name)))
-
-    @defer.inlineCallbacks
+    @inlineCallbacks
     def test_exits_gracefully_if_cant_report_foreign_dhcp_server(self):
         clock = Clock()
         interface_name = factory.make_name("eth")
@@ -109,8 +87,10 @@
             dhcp_probe_service, 'deferToThread')
         deferToThread.side_effect = [
             defer.succeed(interfaces),
-            defer.succeed(['192.168.0.100']),
         ]
+        probe_interface = self.patch(
+            dhcp_probe_service, 'probe_interface')
+        probe_interface.return_value = ['192.168.0.100']
         protocol, connecting = self.patch_rpc_methods()
         self.addCleanup((yield connecting))
 
@@ -154,7 +134,7 @@
                 "Unable to probe for DHCP servers: %s",
                 error_message))
 
-    @defer.inlineCallbacks
+    @inlineCallbacks
     def test_reports_foreign_dhcp_servers_to_region(self):
         clock = Clock()
         interface_name = factory.make_name("eth")
@@ -172,9 +152,10 @@
         foreign_dhcp_ip = factory.make_ipv4_address()
         deferToThread.side_effect = [
             defer.succeed(interfaces),
-            defer.succeed([foreign_dhcp_ip]),
         ]
-
+        probe_interface = self.patch(
+            dhcp_probe_service, 'probe_interface')
+        probe_interface.return_value = [foreign_dhcp_ip]
         client = getRegionClient()
         rpc_service = Mock()
         rpc_service.getClient.return_value = client
@@ -192,7 +173,7 @@
                 interface_name=interface_name,
                 dhcp_ip=foreign_dhcp_ip))
 
-    @defer.inlineCallbacks
+    @inlineCallbacks
     def test_reports_lack_of_foreign_dhcp_servers_to_region(self):
         clock = Clock()
         interface_name = factory.make_name("eth")
@@ -209,8 +190,10 @@
             dhcp_probe_service, 'deferToThread')
         deferToThread.side_effect = [
             defer.succeed(interfaces),
-            defer.succeed([]),
         ]
+        probe_interface = self.patch(
+            dhcp_probe_service, 'probe_interface')
+        probe_interface.return_value = []
 
         client = getRegionClient()
         rpc_service = Mock()

=== modified file 'src/provisioningserver/rackdservices/tests/test_networks_monitoring_service.py'
--- src/provisioningserver/rackdservices/tests/test_networks_monitoring_service.py	2016-09-14 17:19:51 +0000
+++ src/provisioningserver/rackdservices/tests/test_networks_monitoring_service.py	2016-10-05 01:31:45 +0000
@@ -27,7 +27,7 @@
 
 class TestRackNetworksMonitoringService(MAASTestCase):
 
-    run_tests_with = MAASTwistedRunTest.make_factory(debug=True, timeout=5)
+    run_tests_with = MAASTwistedRunTest.make_factory(debug=False, timeout=5)
 
     @inlineCallbacks
     def test_runs_refresh_first_time(self):

=== added file 'src/provisioningserver/utils/dhcp.py'
--- src/provisioningserver/utils/dhcp.py	1970-01-01 00:00:00 +0000
+++ src/provisioningserver/utils/dhcp.py	2016-10-05 01:31:45 +0000
@@ -0,0 +1,305 @@
+# Copyright 2016 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Utilities for working with DHCP packets."""
+
+__all__ = [
+    "DHCP",
+    "add_arguments",
+    "run"
+]
+
+from collections import namedtuple
+from datetime import datetime
+from io import BytesIO
+import os
+from pprint import pformat
+import stat
+import struct
+import subprocess
+import sys
+from textwrap import dedent
+
+from netaddr import IPAddress
+from provisioningserver.utils.ethernet import (
+    Ethernet,
+    ETHERTYPE,
+)
+from provisioningserver.utils.network import (
+    bytes_to_ipaddress,
+    format_eui,
+)
+from provisioningserver.utils.pcap import (
+    PCAP,
+    PCAPError,
+)
+from provisioningserver.utils.script import ActionScriptError
+from provisioningserver.utils.tcpip import (
+    IPv4,
+    PROTOCOL,
+    UDP,
+)
+
+# The SEEN_AGAIN_THRESHOLD is a time (in seconds) that determines how often
+# to report (IP, MAC) bindings that have been seen again (or "REFRESHED").
+# While it is important for MAAS to know about "NEW" and "MOVED" bindings
+# immediately, "REFRESHED" bindings occur too often to be useful, and
+# are thus throttled by this value.
+SEEN_AGAIN_THRESHOLD = 600
+
+# Definitions for DHCP packet used with `struct`.
+# See https://tools.ietf.org/html/rfc2131#section-2 for packet format.
+DHCP_PACKET = '!BBBBLHHLLLL16s64s128sBBBB'
+DHCPPacket = namedtuple('DHCPPacket', (
+    'op',
+    'htype',
+    'len',
+    'hops',
+    'xid',
+    'secs',
+    'flags',
+    'ciaddr',
+    'yiaddr',
+    'siaddr',
+    'giaddr',
+    'chaddr',
+    'sname',
+    'file',
+    'cookie1',
+    'cookie2',
+    'cookie3',
+    'cookie4',
+))
+
+# This is the size of the struct; DHCP options are not included here.
+SIZEOF_DHCP_PACKET = 240
+
+
+class InvalidDHCPPacket(Exception):
+    """Raised internally when a DHCP packet is not valid."""
+
+
+class DHCP:
+    """Representation of a DHCP packet."""
+
+    def __init__(self, pkt_bytes: bytes):
+        """
+        Create a DHCP packet, given the specified upper-layer packet bytes.
+
+        :param pkt_bytes: The input bytes of the DHCP packet.
+        :type pkt_bytes: bytes
+        """
+        super().__init__()
+        self.valid = None
+        self.invalid_reason = None
+        self.options = None
+        if len(pkt_bytes) < SIZEOF_DHCP_PACKET:
+            self.valid = False
+            self.invalid_reason = "Truncated DHCP packet."
+            return
+        packet = DHCPPacket._make(
+            struct.unpack(DHCP_PACKET, pkt_bytes[0:SIZEOF_DHCP_PACKET]))
+        # https://tools.ietf.org/html/rfc2131#section-3
+        expected_cookie = (99, 130, 83, 99)
+        actual_cookie = (
+            packet.cookie1, packet.cookie2, packet.cookie3, packet.cookie4)
+        if expected_cookie != actual_cookie:
+            self.valid = False
+            self.invalid_reason = "Invalid DHCP cookie."
+            return
+        self.packet = packet
+        option_bytes = pkt_bytes[SIZEOF_DHCP_PACKET:]
+        try:
+            self.options = {
+                option: value
+                for option, value in self._parse_options(option_bytes)
+            }
+        except InvalidDHCPPacket as exception:
+            self.valid = False
+            self.invalid_reason = str(exception)
+        if self.valid is None:
+            self.valid = True
+
+    def _parse_options(self, option_bytes: bytes):
+        """Yields tuples of DHCP options found in the given `bytes`.
+
+        :returns: Iterator of (option_code: int, option: bytes).
+        :raises InvalidDHCPPacket: If the options are invalid.
+        """
+        stream = BytesIO(option_bytes)
+        while True:
+            option_bytes = stream.read(1)
+            if len(option_bytes) != 1:
+                break
+            option_code = option_bytes[0]
+            # RFC 1533 (https://tools.ietf.org/html/rfc1533#section-3) defines
+            # 255 as the "end option" and 0 as the "pad option"; both are one
+            # byte in length.
+            if option_code == 255:
+                break
+            if option_code == 0:
+                continue
+            # Each option field is a one-byte quantity indicating how many
+            # bytes are expected to follow.
+            length_bytes = stream.read(1)
+            if len(length_bytes) != 1:
+                raise InvalidDHCPPacket(
+                    "Truncated length field in DHCP option.")
+            option_length = length_bytes[0]
+            option_value = stream.read(option_length)
+            if len(option_value) != option_length:
+                raise InvalidDHCPPacket(
+                    "Truncated DHCP option value.")
+            yield option_code, option_value
+
+    def is_valid(self):
+        return self.valid
+
+    @property
+    def server_identifier(self) -> IPAddress:
+        """Returns the DHCP server identifier option.
+
+        This returns the IP address of the DHCP server.
+
+        :return: netaddr.IPAddress
+        """
+        server_identifier_bytes = self.options.get(54, None)
+        if server_identifier_bytes is not None:
+            return bytes_to_ipaddress(server_identifier_bytes)
+        return None
+
+    def write(self, out=sys.stdout):
+        """Output text-based details about this DHCP packet to the specified
+        file or stream.
+
+        :param out: An object with `write(str)` and `flush()` methods.
+        """
+        packet = pformat(self.packet)
+        out.write(packet)
+        out.write('\n')
+        options = pformat(self.options)
+        out.write(options)
+        out.write("\nServer identifier: %s\n\n" % self.server_identifier)
+        out.flush()
+
+
+def observe_dhcp_packets(input=sys.stdin.buffer, out=sys.stdout):
+    """Read stdin and look for tcpdump binary DHCP output.
+
+    :param input: Stream to read PCAP data from.
+    :type input: a file or stream supporting `read(int)`
+    :param out: Stream to write to.
+    :type input: a file or stream supporting `write(str)` and `flush()`.
+    """
+    try:
+        pcap = PCAP(input)
+        if pcap.global_header.data_link_type != 1:
+            # Not an Ethernet interface. Need to exit here, because our
+            # assumptions about the link layer header won't be correct.
+            return 4
+        for header, packet in pcap:
+            # Interpret Layer 2
+            ethernet = Ethernet(packet, time=header.timestamp_seconds)
+            out.write(str(datetime.now()))
+            out.write("\n")
+            if not ethernet.is_valid():
+                out.write("Invalid Ethernet packet.\n\n")
+                out.flush()
+                # Ignore packets with a truncated Ethernet header.
+                continue
+            if ethernet.ethertype != ETHERTYPE.IPV4:
+                out.write("Invalid ethertype; expected %04x, got %04x.\n\n" % (
+                    ethernet.ethertype, ETHERTYPE.IPV4))
+                out.flush()
+                # Ignore non-IPv4 packets.
+                continue
+            # Interpret Layer 3
+            ipv4 = IPv4(ethernet.payload)
+            if not ipv4.is_valid():
+                out.write(ipv4.invalid_reason)
+                out.write("\n\n")
+                out.flush()
+                continue
+            if ipv4.packet.protocol != PROTOCOL.UDP:
+                # Ignore non-IPv4 packets.
+                out.write("Invalid IPv4 protocol; expected %d, got %d.\n\n" % (
+                    ipv4.packet.protocol, PROTOCOL.UDP))
+                out.flush()
+                continue
+            # Interpret Layer 4
+            udp = UDP(ipv4.payload)
+            if not udp.is_valid():
+                out.write(udp.invalid_reason)
+                out.write("\n\n")
+                out.flush()
+                continue
+            dhcp = DHCP(udp.payload)
+            if not dhcp.is_valid():
+                out.write(dhcp.invalid_reason)
+            out.write(
+                "     Source MAC address: %s\n" % format_eui(ethernet.src_eui))
+            out.write(
+                "Destination MAC address: %s\n" % format_eui(ethernet.dst_eui))
+            if ethernet.vid is not None:
+                out.write("     Seen on 802.1Q VID: %s\n" % ethernet.vid)
+            out.write("      Source IP address: %s\n" % ipv4.src_ip)
+            out.write(" Destination IP address: %s\n" % ipv4.dst_ip)
+            dhcp.write(out=out)
+            out.flush()
+    except EOFError:
+        # Capture aborted before it could even begin. Note that this does not
+        # occur if the end-of-stream occurs normally. (In that case, the
+        # program will just exit.)
+        return 3
+    except PCAPError:
+        # Capture aborted due to an I/O error.
+        return 2
+    return None
+
+
+def add_arguments(parser):
+    """Add this command's options to the `ArgumentParser`.
+
+    Specified by the `ActionScript` interface.
+    """
+    parser.description = dedent("""\
+        Observes DHCP traffic specified interface.
+        """)
+    parser.add_argument(
+        'interface', type=str, nargs='?',
+        help="Ethernet interface from which to capture traffic. Optional if "
+             "an input file is specified.")
+    parser.add_argument(
+        '-i', '--input-file', type=str, required=False,
+        help="File to read PCAP output from. Use - for stdin. Default is to "
+             "call `sudo /usr/lib/maas/maas-dhcp-monitor` to get input.")
+
+
+def run(args, output=sys.stdout, stdin=sys.stdin,
+        stdin_buffer=sys.stdin.buffer):
+    """Observe an Ethernet interface and print DHCP packets."""
+    network_monitor = None
+    if args.input_file is None:
+        if args.interface is None:
+            raise ActionScriptError("Required argument: interface")
+        network_monitor = subprocess.Popen(
+            ["sudo", "--non-interactive", "/usr/lib/maas/maas-dhcp-monitor",
+             args.interface], stdin=subprocess.DEVNULL, stdout=subprocess.PIPE,
+            stderr=subprocess.DEVNULL
+        )
+        infile = network_monitor.stdout
+    else:
+        if args.input_file == '-':
+            mode = os.fstat(stdin.fileno()).st_mode
+            if not stat.S_ISFIFO(mode):
+                raise ActionScriptError("Expected stdin to be a pipe.")
+            infile = stdin_buffer
+        else:
+            infile = open(args.input_file, "rb")
+    return_code = observe_dhcp_packets(input=infile, out=output)
+    if return_code is not None:
+        raise SystemExit(return_code)
+    if network_monitor is not None:
+        return_code = network_monitor.poll()
+        if return_code is not None:
+            raise SystemExit(return_code)

=== modified file 'src/provisioningserver/utils/ethernet.py'
--- src/provisioningserver/utils/ethernet.py	2016-07-17 22:17:00 +0000
+++ src/provisioningserver/utils/ethernet.py	2016-10-05 01:31:45 +0000
@@ -10,6 +10,7 @@
 from collections import namedtuple
 import struct
 
+from netaddr import EUI
 from provisioningserver.utils.network import (
     bytes_to_int,
     hex_str_to_bytes,
@@ -33,6 +34,7 @@
 
 class ETHERTYPE:
     """Enumeration to represent ethertypes that MAAS needs to understand."""
+    IPV4 = hex_str_to_bytes('0800')
     ARP = hex_str_to_bytes('0806')
     VLAN = hex_str_to_bytes('8100')
 
@@ -93,6 +95,16 @@
         self.dst_mac = packet.dst_mac
         self.time = time
 
+    @property
+    def src_eui(self):
+        """Returns a netaddr.EUI representing the source MAC address."""
+        return EUI(bytes_to_int(self.src_mac))
+
+    @property
+    def dst_eui(self):
+        """Returns a netaddr.EUI representing the destination MAC address."""
+        return EUI(bytes_to_int(self.dst_mac))
+
     def is_valid(self):
         """Returns True if this is a valid Ethernet packet, False otherwise."""
         return self.valid

=== modified file 'src/provisioningserver/utils/network.py'
--- src/provisioningserver/utils/network.py	2016-10-04 04:36:39 +0000
+++ src/provisioningserver/utils/network.py	2016-10-05 01:31:45 +0000
@@ -32,6 +32,7 @@
     getaddrinfo,
     IPPROTO_TCP,
 )
+import struct
 
 from netaddr import (
     EUI,
@@ -726,7 +727,7 @@
 
 
 def inet_ntop(value):
-    """Convert IPv4 and IPv6 addresses from binary to text form.
+    """Convert IPv4 and IPv6 addresses from integer to text form.
     (See also inet_ntop(3), the C function with the same name and function.)"""
     return str(IPAddress(value))
 
@@ -784,9 +785,22 @@
 
 
 def ipv4_to_bytes(ipv4_address):
+    """Converts the specified IPv4 address (in text or integer form) to bytes.
+    """
     return bytes.fromhex("%08x" % IPAddress(ipv4_address).value)
 
 
+def bytes_to_ipaddress(ip_address_bytes):
+    if len(ip_address_bytes) == 4:
+        return IPAddress(struct.unpack('!L', ip_address_bytes)[0])
+    if len(ip_address_bytes) == 16:
+        most_significant, least_significant = struct.unpack(
+            "!QQ", ip_address_bytes)
+        return IPAddress((most_significant << 64) | least_significant)
+    else:
+        raise ValueError("Invalid IP address size: expected 4 or 16 bytes.")
+
+
 def format_eui(eui):
     """Returns the specified netaddr.EUI object formatted in the MAAS style."""
     return str(eui).replace('-', ':').lower()

=== added file 'src/provisioningserver/utils/tcpip.py'
--- src/provisioningserver/utils/tcpip.py	1970-01-01 00:00:00 +0000
+++ src/provisioningserver/utils/tcpip.py	2016-10-05 01:31:45 +0000
@@ -0,0 +1,162 @@
+# Copyright 2016 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Utilities for working with TCP/IP packets. (That is, layers 3 and 4.)"""
+
+__all__ = [
+    "IPv4",
+    "UDP",
+]
+
+from collections import namedtuple
+import struct
+
+from netaddr import IPAddress
+
+# Definitions for IPv4 packets used with `struct`.
+# See https://tools.ietf.org/html/rfc791#section-3.1 for more details.
+IPV4_PACKET = '!BBHHHBBHLL'
+IPv4Packet = namedtuple('IPv4Packet', (
+    'version__ihl',
+    'tos',
+    'total_length',
+    'fragment_id',
+    'flags__fragment_offset',
+    'ttl',
+    'protocol',
+    'header_checksum',
+    'src_ip',
+    'dst_ip',
+))
+IPV4_HEADER_MIN_LENGTH = 20
+
+
+class PROTOCOL:
+    """Enumeration to represent IP protocols that MAAS needs to understand."""
+    UDP = 0x11
+
+
+class IPv4:
+    """Representation of an IPv4 packet."""
+
+    def __init__(self, pkt_bytes: bytes):
+        """Decodes the specified IPv4 packet.
+
+        The IP payload will be placed in the `payload` ivar if the packet
+        is valid. If the packet is valid, the `valid` ivar will be set to True.
+        If the packet is not valid, the `valid` ivar will be set to False, and
+        the `invalid_reason` will contain a description of why the packet is
+        not valid.
+
+        This class does not validate the header checksum, and as such, should
+        only be used for testing.
+
+        :param pkt_bytes: The input bytes of the IPv4 packet.
+        """
+        self.valid = True
+        self.invalid_reason = None
+        if len(pkt_bytes) < IPV4_HEADER_MIN_LENGTH:
+            self.valid = False
+            self.invalid_reason = (
+                "Truncated IPv4 header; need at least %d bytes." % (
+                    IPV4_HEADER_MIN_LENGTH))
+            return
+        packet = IPv4Packet._make(
+            struct.unpack(
+                IPV4_PACKET, pkt_bytes[0:IPV4_HEADER_MIN_LENGTH]))
+        self.packet = packet
+        # Mask out the version_ihl field to get the IP version and IHL
+        # (Internet Header Length) separately.
+        self.version = (packet.version__ihl & 0xF0) >> 4
+        # The IHL is a count of 4-bit words that comprise the header.
+        self.ihl = (packet.version__ihl & 0xF) * 4
+        if self.version != 4:
+            self.valid = False
+            self.invalid_reason = (
+                "Invalid version field; expected IPv4, got IPv%d." % (
+                    self.version))
+            return
+        if self.ihl < 20:
+            self.invalid_reason = (
+                "Invalid IPv4 IHL field; expected at least 20 bytes; got %d "
+                "bytes." % self.ihl)
+            self.valid = False
+            return
+        if len(pkt_bytes) < self.ihl:
+            self.valid = False
+            self.invalid_reason = (
+                "Truncated IPv4 header; IHL indicates to read %d bytes; got "
+                "%d bytes." % (self.ihl, len(pkt_bytes)))
+            return
+        # Everything beyond the IHL is the upper-layer payload. (No need to
+        # understand IP options at this time.)
+        self.payload = pkt_bytes[self.ihl:]
+
+    @property
+    def src_ip(self):
+        return IPAddress(self.packet.src_ip)
+
+    @property
+    def dst_ip(self):
+        return IPAddress(self.packet.dst_ip)
+
+    def is_valid(self):
+        return self.valid
+
+
+# Definitions for UDP packets used with `struct`.
+# https://tools.ietf.org/html/rfc768
+UDP_PACKET = '!HHHH'
+UDPPacket = namedtuple('IPPacket', (
+    'src_port',
+    'dst_port',
+    'length',
+    'checksum',
+))
+UDP_HEADER_LENGTH = 8
+
+
+class UDP:
+    """Representation of a UDP packet."""
+
+    def __init__(self, pkt_bytes: bytes):
+        """Decodes the specified UDP packet.
+
+        The UDP payload will be placed in the `payload` ivar if the packet
+        is valid. If the packet is valid, the `valid` ivar will be set to True.
+        If the packet is not valid, the `valid` ivar will be set to False, and
+        the `invalid_reason` will contain a description of why the packet is
+        not valid.
+
+        This class does not validate the UDP checksum, and as such, should only
+        be used for testing.
+        """
+        self.valid = True
+        self.invalid_reason = None
+        if len(pkt_bytes) < UDP_HEADER_LENGTH:
+            self.valid = False
+            self.invalid_reason = (
+                "Truncated UDP header; need at least %d bytes." % (
+                    UDP_HEADER_LENGTH))
+            return
+        packet = UDPPacket._make(
+            struct.unpack(
+                UDP_PACKET, pkt_bytes[0:UDP_HEADER_LENGTH]))
+        self.packet = packet
+        if packet.length < UDP_HEADER_LENGTH:
+            self.valid = False
+            self.invalid_reason = (
+                "Invalid UDP packet; got length of %d bytes; expected at "
+                "least %d bytes. " % (packet.length, UDP_HEADER_LENGTH))
+            return
+        # UDP length includes UDP header, so subtract it to get payload length.
+        payload_length = packet.length - UDP_HEADER_LENGTH
+        self.payload = pkt_bytes[UDP_HEADER_LENGTH:]
+        if len(self.payload) != payload_length:
+            self.valid = False
+            self.invalid_reason = (
+                "UDP packet truncated; expected %d bytes; got %d bytes." % (
+                    payload_length, len(self.payload)))
+
+    def is_valid(self):
+        return self.valid

=== added file 'src/provisioningserver/utils/tests/test_dhcp.py'
--- src/provisioningserver/utils/tests/test_dhcp.py	1970-01-01 00:00:00 +0000
+++ src/provisioningserver/utils/tests/test_dhcp.py	2016-10-05 01:31:45 +0000
@@ -0,0 +1,67 @@
+# Copyright 2016 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for ``provisioningserver.utils.dhcp``."""
+
+__all__ = []
+
+from maastesting.factory import factory
+from maastesting.matchers import DocTestMatches
+from maastesting.testcase import MAASTestCase
+from netaddr import IPAddress
+from provisioningserver.utils.dhcp import DHCP
+from testtools.matchers import (
+    Equals,
+    Is,
+)
+
+
+class TestDHCP(MAASTestCase):
+
+    def test__is_valid_returns_false_for_truncated_packet(self):
+        packet = factory.make_dhcp_packet(truncated=True)
+        dhcp = DHCP(packet)
+        self.assertThat(dhcp.is_valid(), Equals(False))
+        self.assertThat(dhcp.invalid_reason, DocTestMatches(
+            "Truncated DHCP packet."))
+
+    def test__is_valid_returns_false_for_invalid_cookie(self):
+        packet = factory.make_dhcp_packet(bad_cookie=True)
+        dhcp = DHCP(packet)
+        self.assertThat(dhcp.is_valid(), Equals(False))
+        self.assertThat(dhcp.invalid_reason, DocTestMatches(
+            "Invalid DHCP cookie."))
+
+    def test__is_valid_returns_false_for_truncated_option_length(self):
+        packet = factory.make_dhcp_packet(truncated_option_length=True)
+        dhcp = DHCP(packet)
+        self.assertThat(dhcp.is_valid(), Equals(False))
+        self.assertThat(dhcp.invalid_reason, DocTestMatches(
+            "Truncated length field in DHCP option."))
+
+    def test__is_valid_returns_false_for_truncated_option_value(self):
+        packet = factory.make_dhcp_packet(truncated_option_value=True)
+        dhcp = DHCP(packet)
+        self.assertThat(dhcp.is_valid(), Equals(False))
+        self.assertThat(dhcp.invalid_reason, DocTestMatches(
+            "Truncated DHCP option value."))
+
+    def test__is_valid_return_true_for_valid_packet(self):
+        packet = factory.make_dhcp_packet()
+        dhcp = DHCP(packet)
+        self.assertThat(dhcp.is_valid(), Equals(True))
+
+    def test__returns_server_identifier_if_included(self):
+        server_ip = factory.make_ip_address(ipv6=False)
+        packet = factory.make_dhcp_packet(
+            include_server_identifier=True, server_ip=server_ip)
+        dhcp = DHCP(packet)
+        self.assertThat(dhcp.is_valid(), Equals(True))
+        self.assertThat(dhcp.server_identifier, Equals(IPAddress(server_ip)))
+
+    def test__server_identifier_none_if_not_included(self):
+        packet = factory.make_dhcp_packet(
+            include_server_identifier=False)
+        dhcp = DHCP(packet)
+        self.assertThat(dhcp.is_valid(), Equals(True))
+        self.assertThat(dhcp.server_identifier, Is(None))

=== added file 'src/provisioningserver/utils/tests/test_tcpip.py'
--- src/provisioningserver/utils/tests/test_tcpip.py	1970-01-01 00:00:00 +0000
+++ src/provisioningserver/utils/tests/test_tcpip.py	2016-10-05 01:31:45 +0000
@@ -0,0 +1,165 @@
+# Copyright 2016 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for ``provisioningserver.utils.tcpip``."""
+
+__all__ = []
+
+from maastesting.factory import factory
+from maastesting.matchers import DocTestMatches
+from maastesting.testcase import MAASTestCase
+from provisioningserver.utils.network import hex_str_to_bytes
+from provisioningserver.utils.tcpip import (
+    IPv4,
+    UDP,
+)
+from testtools.matchers import Equals
+
+
+def make_ipv4_packet(
+        total_length=None, version=None, ihl=None, payload=None,
+        truncated=False):
+    """Construct an IPv4 packet using the specified parameters.
+
+    If the specified `vid` is not None, it is interpreted as an integer VID,
+    and the appropriate Ethertype fields are adjusted.
+    """
+    if payload is None:
+        payload = b''
+    if total_length is None:
+        total_length = 20 + len(payload)
+    if version is None:
+        version = 4
+    if ihl is None:
+        ihl = 5
+    version__ihl = ((version << 4) | ihl).to_bytes(1, "big")
+
+    ipv4_packet = (
+        # Version, IHL
+        version__ihl +
+        # TOS
+        hex_str_to_bytes('00') +
+        # Total length in bytes
+        total_length.to_bytes(2, "big") +
+        # Identification
+        hex_str_to_bytes('0000') +
+        # Flags, fragment offset
+        hex_str_to_bytes('0000') +
+        # TTL
+        hex_str_to_bytes('00') +
+        # Protocol (just make it UDP for now)
+        hex_str_to_bytes('11') +
+        # Header checksum
+        hex_str_to_bytes('0000') +
+        # Source address
+        hex_str_to_bytes('00000000') +
+        # Destination address
+        hex_str_to_bytes('00000000')
+        # No options.
+    )
+    assert len(ipv4_packet) == 20, "Length was %d" % len(ipv4_packet)
+    if truncated:
+        return ipv4_packet[:19]
+    ipv4_packet = ipv4_packet + payload
+    return ipv4_packet
+
+
+class TestIPv4(MAASTestCase):
+
+    def test__parses_ipv4_packet(self):
+        payload = factory.make_bytes(48)
+        packet = make_ipv4_packet(payload=payload)
+        ipv4 = IPv4(packet)
+        self.assertThat(ipv4.is_valid(), Equals(True))
+        self.assertThat(ipv4.version, Equals(4))
+        self.assertThat(ipv4.ihl, Equals(20))
+        self.assertThat(ipv4.payload, Equals(payload))
+
+    def test__fails_for_non_ipv4_packet(self):
+        payload = factory.make_bytes(48)
+        packet = make_ipv4_packet(payload=payload, version=5)
+        ipv4 = IPv4(packet)
+        self.assertThat(ipv4.is_valid(), Equals(False))
+        self.assertThat(
+            ipv4.invalid_reason, DocTestMatches("Invalid version..."))
+
+    def test__fails_for_bad_ihl(self):
+        payload = factory.make_bytes(48)
+        packet = make_ipv4_packet(payload=payload, ihl=0)
+        ipv4 = IPv4(packet)
+        self.assertThat(ipv4.is_valid(), Equals(False))
+        self.assertThat(
+            ipv4.invalid_reason, DocTestMatches("Invalid IPv4 IHL..."))
+
+    def test__fails_for_truncated_packet(self):
+        packet = make_ipv4_packet(truncated=True)
+        ipv4 = IPv4(packet)
+        self.assertThat(ipv4.is_valid(), Equals(False))
+        self.assertThat(
+            ipv4.invalid_reason, DocTestMatches("Truncated..."))
+
+
+def make_udp_packet(
+        total_length=None, payload=None, truncated_header=False,
+        truncated_payload=False):
+    """Construct an IPv4 packet using the specified parameters.
+
+    If the specified `vid` is not None, it is interpreted as an integer VID,
+    and the appropriate Ethertype fields are adjusted.
+    """
+    if payload is None:
+        payload = b''
+    if total_length is None:
+        total_length = 8 + len(payload)
+
+    udp_packet = (
+        # Source port
+        hex_str_to_bytes('0000') +
+        # Destination port
+        hex_str_to_bytes('0000') +
+        # UDP header length + payload length
+        total_length.to_bytes(2, "big") +
+        # Checksum
+        hex_str_to_bytes('0000')
+    )
+    assert len(udp_packet) == 8, "Length was %d" % len(udp_packet)
+    if truncated_header:
+        return udp_packet[:7]
+    udp_packet = udp_packet + payload
+    if truncated_payload:
+        return udp_packet[:-1]
+    return udp_packet
+
+
+class TestUDP(MAASTestCase):
+
+    def test__parses_udp_packet(self):
+        payload = factory.make_bytes(48)
+        packet = make_udp_packet(payload=payload)
+        udp = UDP(packet)
+        self.assertThat(udp.is_valid(), Equals(True))
+        self.assertThat(udp.payload, Equals(payload))
+
+    def test__fails_for_truncated_udp_header(self):
+        packet = make_udp_packet(truncated_header=True)
+        udp = UDP(packet)
+        self.assertThat(udp.is_valid(), Equals(False))
+        self.assertThat(
+            udp.invalid_reason, DocTestMatches("Truncated UDP header..."))
+
+    def test__fails_for_bad_length(self):
+        payload = factory.make_bytes(48)
+        packet = make_udp_packet(total_length=0, payload=payload)
+        udp = UDP(packet)
+        self.assertThat(udp.is_valid(), Equals(False))
+        self.assertThat(
+            udp.invalid_reason, DocTestMatches(
+                "Invalid UDP packet; got length..."))
+
+    def test__fails_for_truncated_payload(self):
+        payload = factory.make_bytes(48)
+        packet = make_udp_packet(truncated_payload=True, payload=payload)
+        udp = UDP(packet)
+        self.assertThat(udp.is_valid(), Equals(False))
+        self.assertThat(
+            udp.invalid_reason, DocTestMatches("UDP packet truncated..."))

=== added file 'utilities/install-dhcp-observer'
--- utilities/install-dhcp-observer	1970-01-01 00:00:00 +0000
+++ utilities/install-dhcp-observer	2016-10-05 01:31:45 +0000
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+echo "Copying /usr/lib/maas/maas-dhcp-monitor..."
+sudo mkdir -p /usr/lib/maas
+sudo cp "$SCRIPT_DIR/../scripts/maas-dhcp-monitor" /usr/lib/maas/maas-dhcp-monitor
+SUDOERS_LINE="$USER ALL= NOPASSWD: /usr/lib/maas/maas-dhcp-monitor"
+SUDOERS_FILE=/etc/sudoers.d/99-maas-dev-$USER
+echo "Installing sudoers file: $SUDOERS_FILE"
+echo "$SUDOERS_LINE" | sudo tee $SUDOERS_FILE
+sudo chmod 440 $SUDOERS_FILE
+echo "Done. You should now be able to do the following to get raw 'tcpdump' output:"
+echo "    sudo /usr/lib/maas/maas-dhcp-monitor <interface>"
+echo "In addition, the following command should now work in the dev env:"
+echo "    bin/maas-rack observe-dhcp <interface>"