Merge lp:~allenap/maas/dns-serials--bug-1571645--garbage into lp:~maas-committers/maas/trunk

[Gavin Panella]

Periodically remove old DNS publication records.

[Mike Pontillo (mpontillo)]

So that HA regions don't step on each other and all try to do this at the same time,, should we introduce a random component in the timing?

[Mike Pontillo (mpontillo)]

Ignore my previous comment. I totally missed that it was already a random time interval. Well done. =)

That said, I'm really on the fence about this branch.

I think many "enterprise" customers would want full audit logging, which is admittedly a long way away. The previous branch does a good job of keeping a record of every change made, and why.

I personally believe that 7 days is enough, should we need the information in this table to diagnose an issue with MAAS.

On the other hand, it feels wrong to add complexity to MAAS in order to delete auditing data, which many customers might want to keep around.

review: Abstain

[Gavin Panella (allenap)]

You make good points. It may be fine to do without cleaning garbage; each row is not more than ~300B so it's going to take a long time before this really becomes a problem.

If we decide not to use this, how would you feel if I landed all the code in here (with tests, naturally) but without hooking it up as a service, so that it does not run by default?

[Blake Rouse (blake-rouse)]

I like this and I think we should run this service. I would not consider this an audit log, and we have no way of exposing this information to an administrator. We do log in the maaslog every time the DNS is updated. The DNS service code easily load the reason and include that in the log message.

review: Approve

[MAAS Lander (maas-lander)]

The attempt to merge lp:~allenap/maas/dns-serials--bug-1571645--garbage into lp:maas failed. Below is the output from the failed tests.

Get:1 http://security.ubuntu.com/ubuntu xenial-security InRelease [92.2 kB]
Hit:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [93.3 kB]
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Fetched 185 kB in 0s (393 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-coverage python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-mock python3-netaddr python3-netifaces python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
libpq-dev is already the newest version (9.5.2-1).
make is already the newest version (4.1-6).
postgresql is already the ...

[MAAS Lander (maas-lander)]

The attempt to merge lp:~allenap/maas/dns-serials--bug-1571645--garbage into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Get:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease [93.3 kB]
Get:3 http://security.ubuntu.com/ubuntu xenial-security InRelease [92.2 kB]
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Fetched 185 kB in 0s (453 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-coverage python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-mock python3-netaddr python3-netifaces python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
libpq-dev is already the newest version (9.5.2-1).
make is already the newest version (4.1-6).
postgresql is already the ...

[Gavin Panella (allenap)]

Clear review slot.

review: Abstain

[MAAS Lander (maas-lander)]

The attempt to merge lp:~allenap/maas/dns-serials--bug-1571645--garbage into lp:maas failed. Below is the output from the failed tests.

Hit:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Hit:2 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial InRelease
Hit:3 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-updates InRelease
Hit:4 http://prodstack-zone-2.clouds.archive.ubuntu.com/ubuntu xenial-backports InRelease
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
    --no-install-recommends install apache2 archdetect-deb authbind bash bind9 bind9utils build-essential bzr bzr-builddeb chromium-browser chromium-chromedriver curl daemontools debhelper dh-apport dh-systemd distro-info dnsutils firefox freeipmi-tools git gjs ipython isc-dhcp-common libjs-angularjs libjs-jquery libjs-jquery-hotkeys libjs-yui3-full libjs-yui3-min libpq-dev make nodejs-legacy npm postgresql pxelinux python3-all python3-apt python3-bson python3-convoy python3-crochet python3-cssselect python3-curtin python3-dev python3-distro-info python3-django python3-django-nose python3-django-piston3 python3-dnspython python3-docutils python3-formencode python3-hivex python3-httplib2 python3-jinja2 python3-jsonschema python3-lxml python3-netaddr python3-netifaces python3-novaclient python3-oauth python3-oauthlib python3-openssl python3-paramiko python3-petname python3-pexpect python3-psycopg2 python3-pyinotify python3-pyparsing python3-pyvmomi python3-requests python3-seamicroclient python3-setuptools python3-simplestreams python3-sphinx python3-tempita python3-twisted python3-txtftp python3-tz python3-yaml python3-zope.interface python-bson python-crochet python-django python-django-piston python-djorm-ext-pgarray python-formencode python-lxml python-netaddr python-netifaces python-pocket-lint python-psycopg2 python-simplejson python-tempita python-twisted python-yaml socat syslinux-common tgt ubuntu-cloudimage-keyring wget xvfb
Reading package lists...
Building dependency tree...
Reading state information...
apache2 is already the newest version (2.4.18-2ubuntu3).
archdetect-deb is already the newest version (1.117ubuntu2).
authbind is already the newest version (2.1.1+nmu1).
bash is already the newest version (4.3-14ubuntu1).
build-essential is already the newest version (12.1ubuntu2).
bzr is already the newest version (2.7.0-2ubuntu1).
curl is already the newest version (7.47.0-1ubuntu2).
debhelper is already the newest version (9.20160115ubuntu3).
distro-info is already the newest version (0.14build1).
freeipmi-tools is already the newest version (1.4.11-1ubuntu1).
git is already the newest version (1:2.7.4-0ubuntu1).
isc-dhcp-common is already the newest version (4.3.3-5ubuntu12).
libjs-angularjs is already the newest version (1.2.28-1ubuntu2).
libjs-jquery is already the newest version (1.11.3+dfsg-4).
libjs-yui3-full is already the newest version (3.5.1-1ubuntu3).
libjs-yui3-min is already the newest version (3.5.1-1ubuntu3).
make is already the newest version (4.1-6).
postgresql is already the newest version (9.5+173).
pxelinux is already the newest version (3:6.03+dfsg-11ubuntu1).
python-django is already...

[Gavin Panella (allenap)]

Fix unrelated, spuriously failing test.

=== modified file 'src/maasserver/api/tests/test_interfaces.py'
--- src/maasserver/api/tests/test_interfaces.py	2016-05-24 22:05:45 +0000
+++ src/maasserver/api/tests/test_interfaces.py	2016-05-26 13:31:29 +0000
@@ -26,7 +26,7 @@
     ContainsDict,
     Equals,
     MatchesDict,
-    MatchesListwise,
+    MatchesSetwise,
 )
 
 
@@ -703,7 +703,7 @@
             nic.name
             for nic in children
             ), parsed_interface["children"])
-        self.assertThat(parsed_interface["links"], MatchesListwise(links))
+        self.assertThat(parsed_interface["links"], MatchesSetwise(*links))
         json_discovered = parsed_interface["discovered"][0]
         self.assertEqual(dhcp_subnet.id, json_discovered["subnet"]["id"])
         self.assertEqual(discovered_ip, json_discovered["ip_address"])
@@ -1393,7 +1393,7 @@
             "params": Equals(interface.params),
             "effective_mtu": Equals(interface.get_effective_mtu()),
         }))
-        self.assertThat(parsed_interface["links"], MatchesListwise(links))
+        self.assertThat(parsed_interface["links"], MatchesSetwise(*links))
         json_discovered = parsed_interface["discovered"][0]
         self.assertEqual(dhcp_subnet.id, json_discovered["subnet"]["id"])
         self.assertEqual(discovered_ip, json_discovered["ip_address"])

=== added file 'src/maasserver/dns/publication.py'
--- src/maasserver/dns/publication.py	1970-01-01 00:00:00 +0000
+++ src/maasserver/dns/publication.py	2016-05-26 13:31:29 +0000
@@ -0,0 +1,72 @@
+# Copyright 2016 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Services related to DNS publication."""
+
+__all__ = [
+    "DNSPublicationGarbageService",
+]
+
+from datetime import (
+    datetime,
+    timedelta,
+)
+import random
+
+from maasserver.models.dnspublication import DNSPublication
+from maasserver.utils.orm import transactional
+from maasserver.utils.threads import deferToDatabase
+from provisioningserver.utils.twisted import callOut
+from pytz import UTC
+from twisted.application.service import Service
+from twisted.internet import reactor
+from twisted.internet.task import LoopingCall
+from twisted.python import log
+
+
+class DNSPublicationGarbageService(Service):
+    """Periodically delete DNS publications more than 7 days old."""
+
+    clock = None
+
+    def startService(self):
+        super().startService()
+        self._loop = LoopingCall(self._tryCollectGarbage)
+        self._loop.clock = reactor if self.clock is None else self.clock
+        self._loopDone = self._loop.start(self._getInterval(), now=False)
+        self._loopDone.addErrback(log.err, "Garbage loop failed.")
+
+    def stopService(self):
+        if self._loop.running:
+            self._loop.stop()
+        return self._loopDone.addBoth(
+            callOut, super().stopService)
+
+    def _getInterval(self):
+        """Return a random interval between 3 and 6 hours.
+
+        :return: The number of seconds.
+        """
+        return random.randrange(
+            timedelta(hours=3).total_seconds(),
+            timedelta(hours=6).total_seconds())
+
+    def _updateInterval(self):
+        """Update the loop's interval.
+
+        Only call this when the loop is running, otherwise it will crash.
+        Also, it only really makes sense to call it when the loop's function
+        is executing otherwise it will have no effect until the next loop.
+        """
+        self._loop.interval = self._getInterval()
+
+    def _tryCollectGarbage(self):
+        cutoff = datetime.utcnow().replace(tzinfo=UTC) - timedelta(days=7)
+        d = deferToDatabase(self._collectGarbage, cutoff)  # In a transaction.
+        d.addBoth(callOut, self._updateInterval)  # Always adjust the schedule.
+        d.addErrback(log.err, "Failure when removing old DNS publications.")
+        return d
+
+    @transactional
+    def _collectGarbage(self, cutoff):
+        return DNSPublication.objects.collect_garbage(cutoff)

=== added file 'src/maasserver/dns/tests/test_publication.py'
--- src/maasserver/dns/tests/test_publication.py	1970-01-01 00:00:00 +0000
+++ src/maasserver/dns/tests/test_publication.py	2016-05-26 13:31:29 +0000
@@ -0,0 +1,126 @@
+# Copyright 2016 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Tests for `maasserver.dns.publication`."""
+
+__all__ = []
+
+from datetime import (
+    datetime,
+    timedelta,
+)
+
+from crochet import wait_for
+from maasserver.dns import publication
+from maasserver.models.dnspublication import DNSPublication
+from maasserver.testing.testcase import MAASTransactionServerTestCase
+from maastesting.factory import factory
+from maastesting.matchers import (
+    DocTestMatches,
+    GreaterThanOrEqual,
+    MockCalledOnceWith,
+    MockNotCalled,
+)
+from maastesting.runtest import MAASCrochetRunTest
+from maastesting.testcase import MAASTestCase
+from maastesting.twisted import TwistedLoggerFixture
+from provisioningserver.utils.twisted import pause
+from pytz import UTC
+from testtools.matchers import (
+    LessThan,
+    MatchesAll,
+)
+from twisted.internet.defer import (
+    fail,
+    inlineCallbacks,
+)
+from twisted.internet.task import Clock
+
+
+IsExpectedInterval = MatchesAll(
+    GreaterThanOrEqual(3 * 60 * 60), LessThan(6 * 60 * 60),
+    first_only=True)
+
+
+def patch_utcnow(test):
+    utcnow = test.patch(publication, "datetime").utcnow
+    ref = utcnow.return_value = datetime.utcnow()
+    return ref
+
+
+class TestDNSPublicationGarbageService(MAASTestCase):
+    """Tests for `DNSPublicationGarbageService`."""
+
+    run_tests_with = MAASCrochetRunTest
+
+    def test_starting_and_stopping(self):
+        deferToDatabase = self.patch(publication, "deferToDatabase")
+
+        utcnow = patch_utcnow(self)
+        cutoff = utcnow.replace(tzinfo=UTC) - timedelta(days=7)
+
+        dnsgc = publication.DNSPublicationGarbageService()
+        dnsgc.clock = clock = Clock()
+
+        dnsgc.startService()
+        self.assertTrue(dnsgc.running)
+        self.assertTrue(dnsgc._loop.running)
+        self.assertThat(deferToDatabase, MockNotCalled())
+        self.assertThat(dnsgc._loop.interval, IsExpectedInterval)
+
+        clock.advance(dnsgc._loop.interval)
+        self.assertThat(
+            deferToDatabase, MockCalledOnceWith(
+                dnsgc._collectGarbage, cutoff))
+        self.assertThat(dnsgc._loop.interval, IsExpectedInterval)
+
+        dnsgc.stopService()
+        self.assertFalse(dnsgc.running)
+        self.assertFalse(dnsgc._loop.running)
+
+    def test_failures_are_logged(self):
+        deferToDatabase = self.patch(publication, "deferToDatabase")
+        deferToDatabase.return_value = fail(factory.make_exception())
+
+        dnsgc = publication.DNSPublicationGarbageService()
+        dnsgc.clock = clock = Clock()
+
+        with TwistedLoggerFixture() as logger:
+            dnsgc.startService()
+            clock.advance(dnsgc._loop.interval)
+            dnsgc.stopService()
+
+        self.assertThat(logger.output, DocTestMatches(
+            """\
+            Failure when removing old DNS publications.
+            Traceback (most recent call last):...
+            Failure: maastesting.factory.TestException#...
+            """))
+
+        self.assertFalse(dnsgc.running)
+
+
+class TestDNSPublicationGarbageServiceWithDatabase(
+        MAASTransactionServerTestCase):
+    """Tests for `DNSPublicationGarbageService` with the database."""
+
+    run_tests_with = MAASCrochetRunTest
+
+    @wait_for(30.0)
+    @inlineCallbacks
+    def test_garbage_is_collected(self):
+        dnsgc = publication.DNSPublicationGarbageService()
+
+        utcnow = patch_utcnow(self)
+        cutoff = utcnow.replace(tzinfo=UTC) - timedelta(days=7)
+
+        self.patch(dnsgc, "_getInterval").side_effect = [0, 999]
+        self.patch(DNSPublication.objects, "collect_garbage")
+
+        yield dnsgc.startService()
+        yield pause(0.0)  # Let the reactor tick.
+        yield dnsgc.stopService()
+
+        self.assertThat(
+            DNSPublication.objects.collect_garbage,
+            MockCalledOnceWith(cutoff))

=== modified file 'src/maasserver/eventloop.py'
--- src/maasserver/eventloop.py	2016-04-01 18:16:57 +0000
+++ src/maasserver/eventloop.py	2016-05-26 13:31:29 +0000
@@ -94,6 +94,11 @@
     return nonces_cleanup.NonceCleanupService()
 
 
+def make_DNSPublicationGarbageService():
+    from maasserver.dns import publication
+    return publication.DNSPublicationGarbageService()
+
+
 def make_StatusMonitorService():
     from maasserver import status_monitor
     return status_monitor.StatusMonitorService()
@@ -198,6 +203,11 @@
             "factory": make_NonceCleanupService,
             "requires": [],
         },
+        "dns-publication-cleanup": {
+            "only_on_master": True,
+            "factory": make_DNSPublicationGarbageService,
+            "requires": [],
+        },
         "status-monitor": {
             "only_on_master": True,
             "factory": make_StatusMonitorService,

=== modified file 'src/maasserver/models/dnspublication.py'
--- src/maasserver/models/dnspublication.py	2016-04-29 10:17:54 +0000
+++ src/maasserver/models/dnspublication.py	2016-05-26 13:31:29 +0000
@@ -7,6 +7,8 @@
     "DNSPublication",
 ]
 
+from datetime import datetime
+
 from django.core.validators import (
     MaxValueValidator,
     MinValueValidator,
@@ -54,14 +56,17 @@
             # use migrations to provide an initial publication.
             raise self.model.DoesNotExist() from None
 
-    def collect_garbage(self):
+    def collect_garbage(self, cutoff: datetime=None):
         """Delete all but the most recently inserted `DNSPublication`."""
         try:
             publication = self.get_most_recent()
         except self.model.DoesNotExist:
             pass  # Nothing to do.
         else:
-            self.filter(id__lt=publication.id).delete()
+            candidates = self.filter(id__lt=publication.id)
+            if cutoff is not None:
+                candidates = candidates.filter(created__lt=cutoff)
+            candidates.delete()
 
 
 class DNSPublication(Model):

=== modified file 'src/maasserver/models/tests/test_dnspublication.py'
--- src/maasserver/models/tests/test_dnspublication.py	2016-04-28 17:37:11 +0000
+++ src/maasserver/models/tests/test_dnspublication.py	2016-05-26 13:31:29 +0000
@@ -118,3 +118,50 @@
         self.assertThat(DNSPublication.objects.all(), HasLength(0))
         DNSPublication.objects.collect_garbage()
         self.assertThat(DNSPublication.objects.all(), HasLength(0))
+
+    def test_collect_garbage_leaves_records_older_than_specified(self):
+        publications = {
+            timedelta(days=1): DNSPublication(source="1 day ago"),
+            timedelta(minutes=1): DNSPublication(source="1 minute ago"),
+            timedelta(seconds=0): DNSPublication(source="now"),
+        }
+
+        with connection.cursor() as cursor:
+            cursor.execute("SELECT now()")
+            [now] = cursor.fetchone()
+
+        # Work from oldest to youngest so that the youngest gets the highest
+        # primary key; the primary key is used to determine the most recent.
+        for delta in sorted(publications, reverse=True):
+            publication = publications[delta]
+            publication.save()
+            # Use SQL to set `created`; Django's field validation prevents it.
+            with connection.cursor() as cursor:
+                cursor.execute(
+                    "UPDATE maasserver_dnspublication SET created = %s"
+                    " WHERE id = %s", [now - delta, publication.id])
+
+        def get_ages():
+            pubs = DNSPublication.objects.all()
+            return {now - pub.created for pub in pubs}
+
+        deltas = set(publications)
+        self.assertThat(get_ages(), Equals(deltas))
+
+        one_second = timedelta(seconds=1)
+        # Work from oldest to youngest again, collecting garbage each time.
+        while len(deltas) > 1:
+            delta = max(deltas)
+            # Publications of exactly the specified age are not deleted.
+            DNSPublication.objects.collect_garbage(now - delta)
+            self.assertThat(get_ages(), Equals(deltas))
+            # Publications of just a second over are deleted.
+            DNSPublication.objects.collect_garbage(now - delta + one_second)
+            self.assertThat(get_ages(), Equals(deltas - {delta}))
+            # We're done with this one.
+            deltas.discard(delta)
+
+        # The most recent publication will never be deleted.
+        DNSPublication.objects.collect_garbage()
+        self.assertThat(get_ages(), Equals(deltas))
+        self.assertThat(deltas, HasLength(1))

=== modified file 'src/maasserver/tests/test_plugin.py'
--- src/maasserver/tests/test_plugin.py	2016-04-12 17:36:35 +0000
+++ src/maasserver/tests/test_plugin.py	2016-05-26 13:31:29 +0000
@@ -93,6 +93,7 @@
         self.assertIsInstance(service, MultiService)
         expected_services = [
             "database-tasks",
+            "dns-publication-cleanup",
             "import-resources",
             "import-resources-progress",
             "nonce-cleanup",