Merge lp:~barry/ubuntu-system-image/server-si30-configd into lp:ubuntu-system-image/server

[Barry Warsaw]

  * Add support for system-image client 3.0 via symlinks from config.d.
  * Add some helpful links to README.rst
  * Fix some typos.

[Barry Warsaw (barry)]

I think the conflicts are because the prereq branch moves some things around. Probably best to merge that first, and then resubmit this against trunk.

[Barry Warsaw (barry)]

Updated against the current toxify branch (which transitively updates to the current trunk). The conflicts are bogus. This branch is ready again.

[Barry Warsaw (barry)]

Merged latest toxify branch. This branch should be ready to go too. The conflicts are bogus.

[Steve Langasek (vorlon)]

See inline comment.

review: Needs Information

[Barry Warsaw (barry)]

Resubmitted against server trunk since prereq branch has landed. Hopefully this will eliminate the bogus conflicts.

[Steve Langasek (vorlon)]

Hi Barry,

Porting over my review inline comment from the previous MP.

review: Needs Information

[Barry Warsaw (barry)]

On Apr 28, 2015, at 11:32 PM, Steve Langasek wrote:

>This appears to be testing that the target of the symlink is an absolute path
>of system/etc/system-image/client.ini. This may be what the current code
>does, but is that the right result? Have you tested that this will be
>unpacked correctly on the client?

I'm actually trying to stand up a local server on a vm, but wow, entropy
creation in vm seems unbelievably slow!

[lp:~barry/ubuntu-system-image/server-si30-configd]

276. By Barry Warsaw on 2015-05-08

* Use relative paths for the etc/system-image/config.d symlinks
* Provide some better debugging messages.
* Handle resource management better.
* Watch out for initrd.img not being gzip'd, but LP: #1452856

[lp:~barry/ubuntu-system-image/server-si30-configd]

277. By Barry Warsaw on 2015-05-08

Fix the test.

[Barry Warsaw (barry)]

r277 is ready for re-review.

[lp:~barry/ubuntu-system-image/server-si30-configd]

278. By Barry Warsaw on 2015-05-08

Undo a change that doesn't really help.

[Barry Warsaw (barry)]

Make that r278.

=== modified file 'README.rst'
--- README.rst	2015-04-28 07:15:04 +0000
+++ README.rst	2015-05-08 16:39:53 +0000
@@ -99,3 +99,15 @@
 
 Note however that the Python 3 test suite will have failures on Precise due to
 other missing packages (e.g. `python3-gpgme`).  If it hurts, don't do it.
+
+
+See also
+========
+
+For more information on setting up a server see:
+
+    https://wiki.ubuntu.com/ImageBasedUpgrades/ServerSetup
+
+For more information on operating a server see:
+
+    https://wiki.ubuntu.com/ImageBasedUpgrades/ServerOperation

=== modified file 'bin/copy-image'
--- bin/copy-image	2015-04-20 15:35:09 +0000
+++ bin/copy-image	2015-05-08 16:39:53 +0000
@@ -34,7 +34,7 @@
     parser.add_argument("device", metavar="DEVICE")
     parser.add_argument("version", metavar="VERSION", type=int)
     parser.add_argument("-k", "--keep-version", action="store_true",
-                        help="Keep the original verison number")
+                        help="Keep the original version number")
     parser.add_argument("-p", "--phased-percentage", type=int,
                         help="Set the phased percentage for the copied image",
                         default=100)

=== modified file 'bin/si-shell'
--- bin/si-shell	2015-03-11 20:55:00 +0000
+++ bin/si-shell	2015-05-08 16:39:53 +0000
@@ -58,7 +58,7 @@
     # Start the shell
     banner = """Welcome to the system-image shell.
 The configuration is available as: conf
-The system-image tree is availabe as: pub
+The system-image tree is available as: pub
 """
 
     class CompleterConsole(code.InteractiveConsole):

=== modified file 'lib/systemimage/tests/test_tools.py'
--- lib/systemimage/tests/test_tools.py	2015-03-31 20:51:38 +0000
+++ lib/systemimage/tests/test_tools.py	2015-05-08 16:39:53 +0000
@@ -300,3 +300,19 @@
         tools.repack_recovery_keyring(self.config, "%s/recovery.tar.xz" %
                                                    self.temp_directory,
                                       "archive-master")
+
+    def test_system_image_30_symlinks(self):
+        # To support system-image 3.0, generate symlinks for config.d
+        version_tarball = "%s/version.tar" % self.temp_directory
+        tools.generate_version_tarball(self.config, "testing", "test",
+                                       "1.2.3.4",
+                                       version_tarball)
+        version_tarfile = tarfile.open(version_tarball, "r:")
+        default_ini = version_tarfile.getmember(
+            "system/etc/system-image/config.d/00_default.ini")
+        self.assertEqual(default_ini.type, tarfile.SYMTYPE)
+        self.assertEqual(default_ini.linkname, "../client.ini")
+        channel_ini = version_tarfile.getmember(
+            "system/etc/system-image/config.d/01_channel.ini")
+        self.assertEqual(channel_ini.type, tarfile.SYMTYPE)
+        self.assertEqual(channel_ini.linkname, "../channel.ini")

=== modified file 'lib/systemimage/tools.py'
--- lib/systemimage/tools.py	2015-04-20 15:38:11 +0000
+++ lib/systemimage/tools.py	2015-05-08 16:39:53 +0000
@@ -124,6 +124,32 @@
 
     tarball.addfile(channel_file, BytesIO(channel.encode("utf-8")))
 
+    # With system-image 3.0, we need a couple of additional files.  For now we
+    # can just use symlink, but once support for system-image < 3.0 is
+    # removed, these symlinks should become real files and the channel_file
+    # should be removed.
+    #
+    # We use relative paths in the links so that we don't have to worry
+    # about the recovery "system/" prefix.
+    path_00_default = os.path.join(
+        os.path.dirname(channel_path),
+        "config.d", "00_default.ini")
+    default_file = tarfile.TarInfo()
+    default_file.name = path_00_default
+    default_file.type = tarfile.SYMTYPE
+    default_file.linkname = "../client.ini"
+    tarball.addfile(default_file)
+
+    path_01_channel = os.path.join(
+        os.path.dirname(channel_path),
+        "config.d", "01_channel.ini")
+    channel_file = tarfile.TarInfo()
+    channel_file.name = path_01_channel
+    channel_file.type = tarfile.SYMTYPE
+    channel_file.linkname = os.path.join(
+        "..", os.path.basename(channel_path))
+    tarball.addfile(channel_file)
+
     tarball.close()
 
 
@@ -169,13 +195,11 @@
     if os.path.exists(destination):
         raise Exception("Destination already exists: %s" % destination)
 
-    logger.debug("Ungzipping file: %s" % destination)
+    logger.debug("Ungzipping {} to: {}".format(path, destination))
 
-    compressed = gzip.open(path, "rb")
-    uncompressed = open(destination, "wb+")
-    uncompressed.writelines(compressed)
-    uncompressed.close()
-    compressed.close()
+    with gzip.open(path, "rb") as compressed:
+        with open(destination, "wb+") as uncompressed:
+            uncompressed.writelines(compressed)
 
     return destination