Merge lp:~smoser/maas/preserve-sources-list into lp:maas/trunk

[Scott Moser]

specify 'preserve_sources_list' to cloud-init in install preseed

This moves the cloud-init/local-cloud-config out of the preseed explicitly
and instead renders it in compose_cloud_init_preseed.

We now put 2 items in this debconf escaped yaml, essentially feeding
cloud-init this yaml:
  manage_etc_hosts: localhost
  preserve_source_list: true

preserve_sources_list is required so cloud-init on first boot does not:
 a.) break arm mirrors (12.04 arm installs)
 b.) override a local mirror that was specified somewhere else.

In the future this will make adding other cloud-init config simpler.

Also, remove incorrect and useless [ignored] string in commissioning user-data.

specify 'preserve_sources_list' to cloud-init in install preseed

This moves the cloud-init/local-cloud-config out of the preseed explicitly
and instead renders it in compose_cloud_init_preseed.

We now put 2 items in this debconf escaped yaml, essentially feeding
cloud-init this yaml:
  manage_etc_hosts: localhost
  preserve_source_list: true

preserve_sources_list is required so cloud-init on first boot does not:
 a.) break arm mirrors (12.04 arm installs)
 b.) override a local mirror that was specified somewhere else.

In the future this will make adding other cloud-init config simpler.

Also, remove incorrect and useless [ignored] string in commissioning user-data.

[Julian Edwards (julian-edwards)]

On Wednesday 03 October 2012 16:43:24 you wrote:
> The proposal to merge lp:~smoser/maas/preserve-sources-list into lp:maas has
> been updated.
>
> Status: Needs review => Work in progress
>
> For more details, see:
> https://code.launchpad.net/~smoser/maas/preserve-sources-list/+merge/127825

FWIW you don't need to do this if you open up the green expander at the bottom
of the MP creation page and untick "ready for review".

[Robie Basak (racb)]

This doesn't break anything, but after deployment sources.list is still wrong on ARM. Enlistment and commissioning work fine as they did before.

[Robie Basak (racb)]

It works now. Deploying Precise on highbank from MAAS trunk on a Quantal machine using a Precise ephemeral image, sources.list is now correct after deployment and "apt-get update" Just Works.

Thanks, Scott!

[Robie Basak (racb)]

review: Approve (qa)

[Julian Edwards (julian-edwards)]

I am landing this via https://code.launchpad.net/~julian-edwards/maas/smoser/+merge/128149

=== modified file 'contrib/preseeds_v2/commissioning'
--- contrib/preseeds_v2/commissioning	2012-06-19 16:01:07 +0000
+++ contrib/preseeds_v2/commissioning	2012-10-04 14:03:22 +0000
@@ -1,2 +1,1 @@
 {{preseed_data}}
-cloud-init   cloud-init/local-cloud-config string manage_etc_hosts: localhost

=== modified file 'contrib/preseeds_v2/generic'
--- contrib/preseeds_v2/generic	2012-10-02 19:08:05 +0000
+++ contrib/preseeds_v2/generic	2012-10-04 14:03:22 +0000
@@ -27,7 +27,6 @@
 
 {{def preseed}}
 {{preseed_data}}
-cloud-init   cloud-init/local-cloud-config string manage_etc_hosts: localhost
 {{enddef}}
 
 {{def post_scripts}}

=== modified file 'src/maasserver/compose_preseed.py'
--- src/maasserver/compose_preseed.py	2012-08-16 13:38:51 +0000
+++ src/maasserver/compose_preseed.py	2012-10-04 14:03:22 +0000
@@ -29,12 +29,20 @@
         'oauth_token_secret': token.secret,
         })
 
+    local_config_yaml = yaml.safe_dump({
+           "manage_etc_hosts": "localhost",
+           "apt_preserve_sources_list": True,
+        })
+    # this is debconf escaping
+    local_config = local_config_yaml.replace("\\", "\\\\").replace("\n", "\\n")
+
     # Preseed data to send to cloud-init.  We set this as MAAS_PRESEED in
     # ks_meta, and it gets fed straight into debconf.
-    metadata_preseed_items = [
+    preseed_items = [
         ('datasources', 'multiselect', 'MAAS'),
         ('maas-metadata-url', 'string', absolute_reverse('metadata')),
         ('maas-metadata-credentials', 'string', credentials),
+        ('local-cloud-config', 'string', local_config)
         ]
 
     return '\n'.join(
@@ -43,7 +51,7 @@
             item_type,
             item_value,
             )
-        for item_name, item_type, item_value in metadata_preseed_items)
+        for item_name, item_type, item_value in preseed_items)
 
 
 def compose_commissioning_preseed(token):

=== modified file 'src/maasserver/tests/test_compose_preseed.py'
--- src/maasserver/tests/test_compose_preseed.py	2012-08-16 13:57:02 +0000
+++ src/maasserver/tests/test_compose_preseed.py	2012-10-04 14:03:22 +0000
@@ -68,3 +68,29 @@
         self.assertEqual(token.consumer.key, maas_dict['consumer_key'])
         self.assertEqual(token.key, maas_dict['token_key'])
         self.assertEqual(token.secret, maas_dict['token_secret'])
+
+    def test_compose_preseed_valid_local_cloud_config(self):
+        node = factory.make_node(status=NODE_STATUS.READY)
+        preseed = compose_preseed(node)
+        token = NodeKey.objects.get_token_for_node(node)
+
+        keyname = "cloud-init/local-cloud-config"
+        self.assertIn(keyname, preseed)
+
+        # expected input is 'cloud-init/local-cloud-config string VALUE'
+        # where one or more spaces in between tokens, and VALUE ending at newline
+        value = preseed[preseed.find(keyname) + len(keyname):]
+        value = value[value.find("string") + len("string"):].lstrip()
+        if "\n" in value:
+            value = value[:value.find("\n")]
+
+        # now debconf-unescape it
+        value = value.replace("\\n", "\n").replace("\\\\", "\\")
+
+        # at this point it should be valid yaml
+        data = yaml.safe_load(value)
+
+        self.assertIn("manage_etc_hosts", data)
+        self.assertEqual(data["manage_etc_hosts"], "localhost")
+        self.assertIn("apt_preserve_sources_list", data)
+        self.assertEqual(data["apt_preserve_sources_list"], True)