Merge lp:~jtv/maas/use-main-archive-for-intel-arches into lp:~maas-committers/maas/trunk

[Jeroen T. Vermeulen]

Address one of several problems reported in bug 1310076: Curtin userdata should use the main archive for all i386/amd64 architectures, not just for the ‘generic’ ones. It was accidentally using the ports archive for HWE kernels.

This is not enough to fix bug 1310076, but it addresses one of the problems reported there.

Jeroen

[Raphaël Badin (rvb)]

[0]

src/maasserver/tests/test_preseed.py

63 + :param main_arch: A main architecture, such as `i386` or `armhf`. A
64 + subarchitecture will be made up.

Why is is called *main*_arch?

review: Approve

[Raphaël Badin (rvb)]

Btw, this partially fixes what I think is a critical bug so I suggest you backport this to 1.5.

[Jeroen T. Vermeulen (jtv)]

> 63 + :param main_arch: A main architecture, such as `i386` or
> `armhf`. A
> 64 + subarchitecture will be made up.
>
> Why is is called *main*_arch?

To distinguish it from the full combination of main architecture and subarchitecture — which we also call "arch."

[MAAS Lander (maas-lander)]

The attempt to merge lp:~jtv/maas/use-main-archive-for-intel-arches into lp:maas failed. Below is the output from the failed tests.

Ign http://security.ubuntu.com trusty-security InRelease
Hit http://security.ubuntu.com trusty-security Release.gpg
Hit http://security.ubuntu.com trusty-security Release
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Hit http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Hit http://nova.clouds.archive.ubuntu.com trusty-updates Release
Hit http://security.ubuntu.com trusty-security/main Sources
Hit http://security.ubuntu.com trusty-security/universe Sources
Hit http://security.ubuntu.com trusty-security/main amd64 Packages
Hit http://security.ubuntu.com trusty-security/universe amd64 Packages
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Ign http://security.ubuntu.com trusty-security/main Translation-en_US
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Ign http://security.ubuntu.com trusty-security/universe Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en_US
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 bind9 bind9utils build-essential bzr-builddeb curl daemontools debhelper dh-apport distro-info dnsutils firefox freeipmi-tools ipython isc-dhcp-common libjs-raphael libjs-yui3-full libjs-yui3-min libpq-dev make postgresql python-amqplib python-bzrlib python-celery python-convoy python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-formencode python-httplib2 python-jinja2 python-jsonschema python-lockfile python-lxml python-netaddr python-netifaces python-oauth python-oops python-oops-amqp python-oops-datedir-repo python-oops-twisted python-...

[MAAS Lander (maas-lander)]

The attempt to merge lp:~jtv/maas/use-main-archive-for-intel-arches into lp:maas failed. Below is the output from the failed tests.

Ign http://security.ubuntu.com trusty-security InRelease
Get:1 http://security.ubuntu.com trusty-security Release.gpg [933 B]
Get:2 http://security.ubuntu.com trusty-security Release [58.5 kB]
Ign http://nova.clouds.archive.ubuntu.com trusty InRelease
Ign http://nova.clouds.archive.ubuntu.com trusty-updates InRelease
Hit http://nova.clouds.archive.ubuntu.com trusty Release.gpg
Get:3 http://nova.clouds.archive.ubuntu.com trusty-updates Release.gpg [933 B]
Hit http://nova.clouds.archive.ubuntu.com trusty Release
Get:4 http://nova.clouds.archive.ubuntu.com trusty-updates Release [58.5 kB]
Get:5 http://security.ubuntu.com trusty-security/main Sources [970 B]
Hit http://nova.clouds.archive.ubuntu.com trusty/main Sources
Get:6 http://security.ubuntu.com trusty-security/universe Sources [14 B]
Get:7 http://security.ubuntu.com trusty-security/main amd64 Packages [977 B]
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Sources
Hit http://nova.clouds.archive.ubuntu.com trusty/main amd64 Packages
Get:8 http://security.ubuntu.com trusty-security/universe amd64 Packages [14 B]
Hit http://nova.clouds.archive.ubuntu.com trusty/universe amd64 Packages
Hit http://security.ubuntu.com trusty-security/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/main Translation-en
Hit http://security.ubuntu.com trusty-security/universe Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en
Ign http://security.ubuntu.com trusty-security/main Translation-en_US
Ign http://security.ubuntu.com trusty-security/universe Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty/main Translation-en_US
Get:9 http://nova.clouds.archive.ubuntu.com trusty-updates/main Sources [3,405 B]
Get:10 http://nova.clouds.archive.ubuntu.com trusty-updates/universe Sources [648 B]
Get:11 http://nova.clouds.archive.ubuntu.com trusty-updates/main amd64 Packages [10.1 kB]
Get:12 http://nova.clouds.archive.ubuntu.com trusty-updates/universe amd64 Packages [1,602 B]
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en
Hit http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en
Ign http://nova.clouds.archive.ubuntu.com trusty/universe Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty-updates/main Translation-en_US
Ign http://nova.clouds.archive.ubuntu.com trusty-updates/universe Translation-en_US
Fetched 137 kB in 0s (785 kB/s)
Reading package lists...
sudo DEBIAN_FRONTEND=noninteractive apt-get -y \
     --no-install-recommends install apache2 bind9 bind9utils build-essential bzr-builddeb curl daemontools debhelper dh-apport distro-info dnsutils firefox freeipmi-tools ipython isc-dhcp-common libjs-raphael libjs-yui3-full libjs-yui3-min libpq-dev make postgresql python-amqplib python-bzrlib python-celery python-convoy python-crochet python-cssselect python-curtin python-dev python-distro-info python-django python-django-piston python-django-south python-djorm-ext-pgarray python-docutils python-formencode python-httplib2 python-jinja2 python-js...

[Jeroen T. Vermeulen (jtv)]

The test failures were a bug in a Django update. I landed a workaround in MAAS, so we can land branches again!

=== modified file 'contrib/preseeds_v2/curtin_userdata'
--- contrib/preseeds_v2/curtin_userdata	2014-04-15 14:56:09 +0000
+++ contrib/preseeds_v2/curtin_userdata	2014-04-21 11:47:42 +0000
@@ -28,7 +28,7 @@
 power_state:
   mode: reboot
 
-{{if node.architecture in {'i386/generic', 'amd64/generic'} }}
+{{if node.split_arch()[0] in {'i386', 'amd64'} }}
 apt_mirrors:
   ubuntu_archive: http://{{main_archive_hostname}}/{{main_archive_directory}}
   ubuntu_security: http://{{main_archive_hostname}}/{{main_archive_directory}}

=== modified file 'src/maasserver/models/node.py'
--- src/maasserver/models/node.py	2014-04-03 15:49:39 +0000
+++ src/maasserver/models/node.py	2014-04-21 11:47:42 +0000
@@ -899,3 +899,8 @@
                 "expression. This expression must instead be updated to set "
                 "this node to install with the fast installer.")
         self.tags.add(uti_tag)
+
+    def split_arch(self):
+        """Return architecture and subarchitecture, as a tuple."""
+        arch, subarch = self.architecture.split('/')
+        return (arch, subarch)

=== modified file 'src/maasserver/models/tests/test_node.py'
--- src/maasserver/models/tests/test_node.py	2014-04-03 12:54:58 +0000
+++ src/maasserver/models/tests/test_node.py	2014-04-21 11:47:42 +0000
@@ -870,6 +870,13 @@
             "The use-fastpath-installer tag is defined with an expression",
             unicode(error))
 
+    def test_split_arch_returns_arch_as_tuple(self):
+        main_arch = factory.make_name('arch')
+        sub_arch = factory.make_name('subarch')
+        full_arch = '%s/%s' % (main_arch, sub_arch)
+        node = factory.make_node(architecture=full_arch)
+        self.assertEqual((main_arch, sub_arch), node.split_arch())
+
 
 class NodeRoutersTest(MAASServerTestCase):
 

=== modified file 'src/maasserver/tests/test_preseed.py'
--- src/maasserver/tests/test_preseed.py	2014-04-10 13:50:25 +0000
+++ src/maasserver/tests/test_preseed.py	2014-04-21 11:47:42 +0000
@@ -62,6 +62,7 @@
     AllMatch,
     Contains,
     ContainsAll,
+    HasLength,
     IsInstance,
     MatchesAll,
     Not,
@@ -614,6 +615,60 @@
                 ]
             ))
 
+    def make_fastpath_node(self, main_arch=None):
+        """Return a `Node`, with FPI enabled, and the given main architecture.
+
+        :param main_arch: A main architecture, such as `i386` or `armhf`.  A
+            subarchitecture will be made up.
+        """
+        if main_arch is None:
+            main_arch = factory.make_name('arch')
+        arch = '%s/%s' % (main_arch, factory.make_name('subarch'))
+        node = factory.make_node(architecture=arch)
+        node.use_fastpath_installer()
+        return node
+
+    def extract_archive_setting(self, userdata):
+        """Extract the `ubuntu_archive` setting from `userdata`."""
+        userdata_lines = []
+        for line in userdata.splitlines():
+            line = line.strip()
+            if line.startswith('ubuntu_archive'):
+                userdata_lines.append(line)
+        self.assertThat(userdata_lines, HasLength(1))
+        [userdata_line] = userdata_lines
+        key, value = userdata_line.split(':', 1)
+        return value.strip()
+
+    def summarise_url(self, url):
+        """Return just the hostname and path from `url`, normalised."""
+        # This is needed because the userdata deliberately makes some minor
+        # changes to the archive URLs, making it harder to recognise which
+        # archive they use: slashes are added, schemes are hard-coded.
+        parsed_result = urlparse(url)
+        return parsed_result.netloc, parsed_result.path.strip('/')
+
+    def test_get_curtin_config_uses_main_archive_for_i386(self):
+        node = self.make_fastpath_node('i386')
+        userdata = get_curtin_config(node)
+        self.assertEqual(
+            self.summarise_url(Config.objects.get_config('main_archive')),
+            self.summarise_url(self.extract_archive_setting(userdata)))
+
+    def test_get_curtin_config_uses_main_archive_for_amd64(self):
+        node = self.make_fastpath_node('amd64')
+        userdata = get_curtin_config(node)
+        self.assertEqual(
+            self.summarise_url(Config.objects.get_config('main_archive')),
+            self.summarise_url(self.extract_archive_setting(userdata)))
+
+    def test_get_curtin_config_uses_ports_archive_for_other_arch(self):
+        node = self.make_fastpath_node()
+        userdata = get_curtin_config(node)
+        self.assertEqual(
+            self.summarise_url(Config.objects.get_config('ports_archive')),
+            self.summarise_url(self.extract_archive_setting(userdata)))
+
     def test_get_curtin_context(self):
         node = factory.make_node()
         node.use_fastpath_installer()