Merge lp:~cjwatson/launchpad/custom-uefi into lp:launchpad

[Colin Watson]

Add UEFI custom upload format.

== Summary ==

Bug 1016594: Ubuntu Engineering needs Launchpad to support signing UEFI boot loaders and publishing them in the Ubuntu archive.

== Proposed fix ==

Create a new custom upload format for UEFI. This provides a reasonable place for us to sign images on publication.

Make the key/certificate location configurable in launchpad-lazr.conf, for fairly obvious reasons. Filesystem locations may differ, we're only going to want the real key on production, and developer machines don't need a key at all.

Never auto-approve uploads containing UEFI files for signing, since signed images are going to be rather high-value. This means that we don't have to invent some new permissioning system for who's allowed to upload UEFI images or which packages they're allowed to go in; instead, we can just rely on them all being held for manual approval.

== LOC Rationale ==

+311. This feature is critical to UE, and I have 3039 lines of credit right now, so I'd very much like to use some of that. Most of my other work at the moment is coming out LoC-negative.

== Tests ==

bin/test -vvct test_uefi -t test_ftparchive -t test_generate_extra_overrides -t test_nascentuploadfile -t test_custom_uploads_copier

== Demo and Q/A ==

Once sbsigntool has been installed on dogfood along with a test key, we should change efilinux to add a raw-uefi custom tarball and upload that to dogfood. It should be extracted to /dists/quantal/main/uefi/efilinux-<ARCH>/<VERSION>/ and the .efi file signed by the test key, and a current -> <VERSION> symlink should be created.

== Lint ==

Pre-existing lint in *-lazr.conf files, which I don't think I should touch:

./configs/development/launchpad-lazr.conf
      64: Line exceeds 80 characters.
      93: Line exceeds 80 characters.
./lib/lp/services/config/schema-lazr.conf
     457: Line exceeds 80 characters.
    1050: Line exceeds 80 characters.
    1057: Line exceeds 80 characters.
    1595: Line exceeds 80 characters.

[Brad Crittenden (bac)]

Hi Colin,

What an interesting branch! Well, not the branch per se but the whole endeavor. I knew this work was coming but hadn't paid attention to the details so it was a good read.

<supertrivial>
In lib/lp/archivepublisher/uefi.py the items in __all__ should be alphabetized.
</supertrivial>

The comment in uefi.py:

The filename should be something like:
        <TYPE>_<VERSION>_<ARCH>.tar.gz

seems a bit breezy to me. When I first read it I thought the naming convention was optional. Should the description really read "The filename must be of the form:" ?

In fact, if the filename does not follow that pattern there could be problems in setTargetDirectory. Perhaps there needs to be checks in there and a test to exercise it.

On a related note, it looks like setTargetDirectory and getSeriesKey both embed the same knowledge about the structure of the filename. A common method could be used by both so that the filename parsing (simple as it is) isn't repeated.

I know it was not your doing, but I also know you share an aversion to poor spelling, so could you fix the occurrences of 'wheter' in lib/lp/soyuz/interfaces/queue.py (and perhaps the one other place it appears).

Thanks Colin.

review: Approve (code)

[William Grant (wgrant)]

6 [archivepublisher]
7 run_parts_location: none
8 +uefi_key_location: none
9 +uefi_cert_location: none

This is global to the entire config, so it'll affect any other archive that's published under it (eg. partner).

658 + UEFI = DBItem(6, """
659 + uefi
660 +
661 + A signed UEFI boot loader image.
662 + """)

Isn't the custom upload *un*signed?

=== modified file 'lib/lp/archivepublisher/config.py'
--- lib/lp/archivepublisher/config.py	2012-01-01 02:58:52 +0000
+++ lib/lp/archivepublisher/config.py	2012-07-03 12:54:32 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2011 Canonical Ltd.  This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 #
 # This is the python package that defines the
@@ -81,6 +81,11 @@
         pubconf.miscroot = None
         pubconf.germinateroot = None
 
+    if archive.is_main:
+        pubconf.uefiroot = pubconf.archiveroot + '-uefi'
+    else:
+        pubconf.uefiroot = None
+
     pubconf.poolroot = os.path.join(pubconf.archiveroot, 'pool')
     pubconf.distsroot = os.path.join(pubconf.archiveroot, 'dists')
 

=== modified file 'lib/lp/archivepublisher/customupload.py'
--- lib/lp/archivepublisher/customupload.py	2012-06-29 08:40:05 +0000
+++ lib/lp/archivepublisher/customupload.py	2012-07-03 12:54:32 +0000
@@ -101,11 +101,11 @@
 
         self.tmpdir = None
 
-    def process(self, archive_root, tarfile_path, distroseries):
+    def process(self, pubconf, tarfile_path, distroseries):
         """Process the upload and install it into the archive."""
         self.tarfile_path = tarfile_path
         try:
-            self.setTargetDirectory(archive_root, tarfile_path, distroseries)
+            self.setTargetDirectory(pubconf, tarfile_path, distroseries)
             self.checkForConflicts()
             self.extract()
             self.installFiles()
@@ -113,7 +113,15 @@
         finally:
             self.cleanup()
 
-    def setTargetDirectory(self, archive_root, tarfile_path, distroseries):
+    @staticmethod
+    def parsePath(tarfile_path):
+        """Parse tarfile_path, returning its useful components.
+
+        :raises ValueError: If tarfile_path is incorrectly formed.
+        """
+        raise NotImplementedError
+
+    def setTargetDirectory(self, pubconf, tarfile_path, distroseries):
         """Set self.targetdir based on parameters.
 
         This should also set self.version and self.arch (if applicable) as a

=== modified file 'lib/lp/archivepublisher/ddtp_tarball.py'
--- lib/lp/archivepublisher/ddtp_tarball.py	2012-05-28 13:13:53 +0000
+++ lib/lp/archivepublisher/ddtp_tarball.py	2012-07-03 12:54:32 +0000
@@ -12,7 +12,10 @@
 
 __metaclass__ = type
 
-__all__ = ['process_ddtp_tarball']
+__all__ = [
+    'DdtpTarballUpload',
+    'process_ddtp_tarball',
+    ]
 
 import os
 
@@ -22,7 +25,7 @@
 class DdtpTarballUpload(CustomUpload):
     """DDTP (Debian Description Translation Project) tarball upload
 
-    The tarball should be name as:
+    The tarball filename must be of the form:
 
      <NAME>_<COMPONENT>_<VERSION>.tar.gz
 
@@ -44,13 +47,16 @@
     """
     custom_type = "ddtp-tarball"
 
-    def setTargetDirectory(self, archive_root, tarfile_path, distroseries):
-        tarfile_base = os.path.basename(tarfile_path)
-        name, component, self.version = tarfile_base.split('_')
+    @staticmethod
+    def parsePath(tarfile_path):
+        name, component, version = os.path.basename(tarfile_path).split("_")
+        return name, component, version
+
+    def setTargetDirectory(self, pubconf, tarfile_path, distroseries):
+        _, component, self.version = self.parsePath(tarfile_path)
         self.arch = None
-
-        self.targetdir = os.path.join(archive_root, 'dists',
-                                      distroseries, component)
+        self.targetdir = os.path.join(
+            pubconf.archiveroot, 'dists', distroseries, component)
 
     def checkForConflicts(self):
         # We just overwrite older files, so no conflicts are possible.
@@ -65,7 +71,7 @@
         pass
 
 
-def process_ddtp_tarball(archive_root, tarfile_path, distroseries):
+def process_ddtp_tarball(pubconf, tarfile_path, distroseries):
     """Process a raw-ddtp-tarball tarfile.
 
     Unpacking it into the given archive for the given distroseries.
@@ -73,4 +79,4 @@
     anything goes wrong.
     """
     upload = DdtpTarballUpload()
-    upload.process(archive_root, tarfile_path, distroseries)
+    upload.process(pubconf, tarfile_path, distroseries)

=== modified file 'lib/lp/archivepublisher/debian_installer.py'
--- lib/lp/archivepublisher/debian_installer.py	2012-05-30 10:25:43 +0000
+++ lib/lp/archivepublisher/debian_installer.py	2012-07-03 12:54:32 +0000
@@ -22,7 +22,7 @@
 class DebianInstallerUpload(CustomUpload):
     """ Debian Installer custom upload.
 
-    The debian-installer filename should be something like:
+    The debian-installer filename must be of the form:
 
         <BASE>_<VERSION>_<ARCH>.tar.gz
 
@@ -40,20 +40,21 @@
     """
     custom_type = "installer"
 
-    def setTargetDirectory(self, archive_root, tarfile_path, distroseries):
-        tarfile_base = os.path.basename(tarfile_path)
-        _, self.version, self.arch = tarfile_base.split("_")
-        self.arch = self.arch.split(".")[0]
+    @staticmethod
+    def parsePath(tarfile_path):
+        base, version, arch = os.path.basename(tarfile_path).split("_")
+        return base, version, arch.split(".")[0]
 
+    def setTargetDirectory(self, pubconf, tarfile_path, distroseries):
+        _, self.version, self.arch = self.parsePath(tarfile_path)
         self.targetdir = os.path.join(
-            archive_root, 'dists', distroseries, 'main',
+            pubconf.archiveroot, 'dists', distroseries, 'main',
             'installer-%s' % self.arch)
 
     @classmethod
     def getSeriesKey(cls, tarfile_path):
         try:
-            _, _, arch = os.path.basename(tarfile_path).split("_")
-            return arch.split(".")[0]
+            return cls.parsePath(tarfile_path)[2]
         except ValueError:
             return None
 
@@ -70,7 +71,7 @@
         return filename.startswith('%s/' % self.version)
 
 
-def process_debian_installer(archive_root, tarfile_path, distroseries):
+def process_debian_installer(pubconf, tarfile_path, distroseries):
     """Process a raw-installer tarfile.
 
     Unpacking it into the given archive for the given distroseries.
@@ -78,4 +79,4 @@
     wrong.
     """
     upload = DebianInstallerUpload()
-    upload.process(archive_root, tarfile_path, distroseries)
+    upload.process(pubconf, tarfile_path, distroseries)

=== modified file 'lib/lp/archivepublisher/dist_upgrader.py'
--- lib/lp/archivepublisher/dist_upgrader.py	2012-06-29 08:40:05 +0000
+++ lib/lp/archivepublisher/dist_upgrader.py	2012-07-03 12:54:32 +0000
@@ -34,7 +34,7 @@
     Dist-Upgrader is a tarball containing files for performing automatic
     distroseries upgrades, driven by architecture.
 
-    The tarball should be name as:
+    The tarball filename must be of the form:
 
       <NAME>_<VERSION>_<ARCH>.tar.gz
 
@@ -57,19 +57,21 @@
     """
     custom_type = "dist-upgrader"
 
-    def setTargetDirectory(self, archive_root, tarfile_path, distroseries):
-        tarfile_base = os.path.basename(tarfile_path)
-        name, self.version, self.arch = tarfile_base.split("_")
-        self.arch = self.arch.split(".")[0]
+    @staticmethod
+    def parsePath(tarfile_path):
+        name, version, arch = os.path.basename(tarfile_path).split("_")
+        return name, version, arch.split(".")[0]
 
-        self.targetdir = os.path.join(archive_root, 'dists', distroseries,
-                                      'main', 'dist-upgrader-%s' % self.arch)
+    def setTargetDirectory(self, pubconf, tarfile_path, distroseries):
+        _, self.version, self.arch = self.parsePath(tarfile_path)
+        self.targetdir = os.path.join(
+            pubconf.archiveroot, 'dists', distroseries, 'main',
+            'dist-upgrader-%s' % self.arch)
 
     @classmethod
     def getSeriesKey(cls, tarfile_path):
         try:
-            _, _, arch = os.path.basename(tarfile_path).split("_")
-            return arch.split(".")[0]
+            return cls.parsePath(tarfile_path)[2]
         except ValueError:
             return None
 
@@ -95,7 +97,7 @@
         return version and not filename.startswith('current')
 
 
-def process_dist_upgrader(archive_root, tarfile_path, distroseries):
+def process_dist_upgrader(pubconf, tarfile_path, distroseries):
     """Process a raw-dist-upgrader tarfile.
 
     Unpacking it into the given archive for the given distroseries.
@@ -103,4 +105,4 @@
     wrong.
     """
     upload = DistUpgraderUpload()
-    upload.process(archive_root, tarfile_path, distroseries)
+    upload.process(pubconf, tarfile_path, distroseries)

=== modified file 'lib/lp/archivepublisher/model/ftparchive.py'
--- lib/lp/archivepublisher/model/ftparchive.py	2012-03-27 11:08:12 +0000
+++ lib/lp/archivepublisher/model/ftparchive.py	2012-07-03 12:54:32 +0000
@@ -21,6 +21,7 @@
 from lp.services.database.decoratedresultset import DecoratedResultSet
 from lp.services.database.stormexpr import Concatenate
 from lp.services.librarian.model import LibraryFileAlias
+from lp.services.osutils import write_file
 from lp.services.webapp.interfaces import (
     DEFAULT_FLAVOR,
     IStoreSelector,
@@ -42,12 +43,6 @@
     return (os.path.basename(filename).split("_"))[0]
 
 
-def f_touch(*parts):
-    """Touch the file named by the arguments concatenated as a path."""
-    fname = os.path.join(*parts)
-    open(fname, "w").close()
-
-
 def safe_mkdir(path):
     """Ensures the path exists, creating it if it doesn't."""
     if not os.path.exists(path):
@@ -215,15 +210,15 @@
             (comp, "debian-installer"),
             (comp, "src"),
             ):
-            f_touch(os.path.join(
+            write_file(os.path.join(
                 self._config.overrideroot,
-                ".".join(("override", suite) + path)))
+                ".".join(("override", suite) + path)), "")
 
         # Create empty file lists.
         def touch_list(*parts):
-            f_touch(os.path.join(
+            write_file(os.path.join(
                 self._config.overrideroot,
-                "_".join((suite, ) + parts)))
+                "_".join((suite, ) + parts)), "")
         touch_list(comp, "source")
 
         arch_tags = [

=== modified file 'lib/lp/archivepublisher/tests/test_config.py'
--- lib/lp/archivepublisher/tests/test_config.py	2012-05-21 12:55:59 +0000
+++ lib/lp/archivepublisher/tests/test_config.py	2012-07-03 12:54:32 +0000
@@ -48,6 +48,7 @@
             archiveroot + "-germinate", primary_config.germinateroot)
         self.assertEqual(
             self.root + "/ubuntutest-temp", primary_config.temproot)
+        self.assertEqual(archiveroot + "-uefi", primary_config.uefiroot)
 
     def test_partner_config(self):
         # Partner archive configuration is correct.
@@ -69,6 +70,7 @@
         self.assertIsNone(partner_config.germinateroot)
         self.assertEqual(
             self.root + "/ubuntutest-temp", partner_config.temproot)
+        self.assertEqual(archiveroot + "-uefi", partner_config.uefiroot)
 
     def test_debug_config(self):
         # The publisher configuration for DEBUG archives points to
@@ -88,6 +90,7 @@
         self.assertIsNone(debug_config.miscroot)
         self.assertIsNone(debug_config.germinateroot)
         self.assertEqual(self.root + "/ubuntutest-temp", debug_config.temproot)
+        self.assertEqual(archiveroot + "-uefi", debug_config.uefiroot)
 
     def test_copy_config(self):
         # In the case of copy archives (used for rebuild testing) the
@@ -109,6 +112,7 @@
         self.assertEqual(
             archiveroot + "-germinate", copy_config.germinateroot)
         self.assertEqual(archiveroot + "-temp", copy_config.temproot)
+        self.assertIsNone(copy_config.uefiroot)
 
 
 class TestGetPubConfigPPA(TestCaseWithFactory):
@@ -144,6 +148,7 @@
         self.assertIsNone(self.ppa_config.germinateroot)
         self.assertEqual(
             "/var/tmp/archive/ubuntutest-temp", self.ppa_config.temproot)
+        self.assertIsNone(self.ppa_config.uefiroot)
 
     def test_private_ppa_separate_root(self):
         # Private PPAs are published to a different location.
@@ -172,3 +177,4 @@
         self.assertIsNone(p3a_config.germinateroot)
         self.assertEqual(
             "/var/tmp/archive/ubuntutest-temp", p3a_config.temproot)
+        self.assertIsNone(p3a_config.uefiroot)

=== modified file 'lib/lp/archivepublisher/tests/test_ddtp_tarball.py'
--- lib/lp/archivepublisher/tests/test_ddtp_tarball.py	2012-05-25 13:28:31 +0000
+++ lib/lp/archivepublisher/tests/test_ddtp_tarball.py	2012-07-03 12:54:32 +0000
@@ -14,11 +14,18 @@
 from lp.testing import TestCase
 
 
+class FakeConfig:
+    """A fake publisher configuration."""
+    def __init__(self, archiveroot):
+        self.archiveroot = archiveroot
+
+
 class TestDdtpTarball(TestCase):
 
     def setUp(self):
         super(TestDdtpTarball, self).setUp()
         self.temp_dir = self.makeTemporaryDirectory()
+        self.pubconf = FakeConfig(self.temp_dir)
         self.suite = "distroseries"
         # CustomUpload.installFiles requires a umask of 022.
         old_umask = os.umask(022)
@@ -33,7 +40,7 @@
     def process(self):
         self.archive.close()
         self.buffer.close()
-        process_ddtp_tarball(self.temp_dir, self.path, self.suite)
+        process_ddtp_tarball(self.pubconf, self.path, self.suite)
 
     def getTranslationsPath(self, filename):
         return os.path.join(

=== modified file 'lib/lp/archivepublisher/tests/test_debian_installer.py'
--- lib/lp/archivepublisher/tests/test_debian_installer.py	2012-05-30 10:25:43 +0000
+++ lib/lp/archivepublisher/tests/test_debian_installer.py	2012-07-03 12:54:32 +0000
@@ -21,11 +21,18 @@
 from lp.testing import TestCase
 
 
+class FakeConfig:
+    """A fake publisher configuration."""
+    def __init__(self, archiveroot):
+        self.archiveroot = archiveroot
+
+
 class TestDebianInstaller(TestCase):
 
     def setUp(self):
         super(TestDebianInstaller, self).setUp()
         self.temp_dir = self.makeTemporaryDirectory()
+        self.pubconf = FakeConfig(self.temp_dir)
         self.suite = "distroseries"
         # CustomUpload.installFiles requires a umask of 022.
         old_umask = os.umask(022)
@@ -51,7 +58,7 @@
     def process(self):
         self.archive.close()
         self.buffer.close()
-        process_debian_installer(self.temp_dir, self.path, self.suite)
+        process_debian_installer(self.pubconf, self.path, self.suite)
 
     def getInstallerPath(self, versioned_filename=None):
         installer_path = os.path.join(

=== modified file 'lib/lp/archivepublisher/tests/test_dist_upgrader.py'
--- lib/lp/archivepublisher/tests/test_dist_upgrader.py	2012-05-30 10:25:43 +0000
+++ lib/lp/archivepublisher/tests/test_dist_upgrader.py	2012-07-03 12:54:32 +0000
@@ -22,11 +22,18 @@
 from lp.testing import TestCase
 
 
+class FakeConfig:
+    """A fake publisher configuration."""
+    def __init__(self, archiveroot):
+        self.archiveroot = archiveroot
+
+
 class TestDistUpgrader(TestCase):
 
     def setUp(self):
         super(TestDistUpgrader, self).setUp()
         self.temp_dir = self.makeTemporaryDirectory()
+        self.pubconf = FakeConfig(self.temp_dir)
         self.suite = "distroseries"
         # CustomUpload.installFiles requires a umask of 022.
         old_umask = os.umask(022)
@@ -41,7 +48,7 @@
     def process(self):
         self.archive.close()
         self.buffer.close()
-        process_dist_upgrader(self.temp_dir, self.path, self.suite)
+        process_dist_upgrader(self.pubconf, self.path, self.suite)
 
     def getUpgraderPath(self):
         return os.path.join(

=== modified file 'lib/lp/archivepublisher/tests/test_ftparchive.py'
--- lib/lp/archivepublisher/tests/test_ftparchive.py	2012-03-27 12:07:15 +0000
+++ lib/lp/archivepublisher/tests/test_ftparchive.py	2012-07-03 12:54:32 +0000
@@ -16,7 +16,6 @@
 from lp.archivepublisher.diskpool import DiskPool
 from lp.archivepublisher.model.ftparchive import (
     AptFTPArchiveFailure,
-    f_touch,
     FTPArchiveHandler,
     )
 from lp.archivepublisher.publishing import Publisher
@@ -27,10 +26,7 @@
     BufferLogger,
     DevNullLogger,
     )
-from lp.testing import (
-    TestCase,
-    TestCaseWithFactory,
-    )
+from lp.testing import TestCaseWithFactory
 from lp.testing.dbuser import switch_dbuser
 from lp.testing.layers import (
     LaunchpadZopelessLayer,
@@ -493,26 +489,3 @@
         distro = distroarchseries.distroseries.distribution
         fa = FTPArchiveHandler(DevNullLogger(), None, None, distro, None)
         self.assertRaises(AptFTPArchiveFailure, fa.runApt, "bogus-config")
-
-
-class TestFTouch(TestCase):
-    """Tests for f_touch function."""
-
-    def setUp(self):
-        TestCase.setUp(self)
-        self.test_folder = self.useTempDir()
-
-    def test_f_touch_new_file(self):
-        # Test f_touch correctly creates a new file.
-        f_touch(self.test_folder, "file_to_touch")
-        self.assertTrue(os.path.exists("%s/file_to_touch" % self.test_folder))
-
-    def test_f_touch_existing_file(self):
-        # Test f_touch truncates existing files.
-        with open("%s/file_to_truncate" % self.test_folder, "w") as f:
-            f.write("I'm some test contents")
-
-        f_touch(self.test_folder, "file_to_leave_alone")
-
-        with open("%s/file_to_leave_alone" % self.test_folder, "r") as f:
-            self.assertEqual("", f.read())

=== modified file 'lib/lp/archivepublisher/tests/test_generate_extra_overrides.py'
--- lib/lp/archivepublisher/tests/test_generate_extra_overrides.py	2012-05-21 19:03:16 +0000
+++ lib/lp/archivepublisher/tests/test_generate_extra_overrides.py	2012-07-03 12:54:32 +0000
@@ -29,6 +29,7 @@
 from lp.services.osutils import (
     ensure_directory_exists,
     open_for_writing,
+    write_file,
     )
 from lp.services.scripts.base import LaunchpadScriptFailure
 from lp.services.utils import file_exists
@@ -47,12 +48,6 @@
         return handle.read()
 
 
-def touch(path):
-    """Create an empty file at path."""
-    with open_for_writing(path, "a"):
-        pass
-
-
 class TestAtomicFile(TestCaseWithFactory):
     """Tests for the AtomicFile helper class."""
 
@@ -562,7 +557,7 @@
         other_file = "other-file"
         output = partial(os.path.join, self.script.config.germinateroot)
         for base in (seed_old_file, seed_new_file, other_file):
-            touch(output(base))
+            write_file(output(base), "")
         self.script.removeStaleOutputs(series_name, set([seed_new_file]))
         self.assertFalse(os.path.exists(output(seed_old_file)))
         self.assertTrue(os.path.exists(output(seed_new_file)))

=== added file 'lib/lp/archivepublisher/tests/test_uefi.py'
--- lib/lp/archivepublisher/tests/test_uefi.py	1970-01-01 00:00:00 +0000
+++ lib/lp/archivepublisher/tests/test_uefi.py	2012-07-03 12:54:32 +0000
@@ -0,0 +1,132 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""Test UEFI custom uploads."""
+
+__metaclass__ = type
+
+import os
+
+from lp.archivepublisher.customupload import (
+    CustomUploadAlreadyExists,
+    CustomUploadBadUmask,
+    )
+from lp.archivepublisher.uefi import (
+    UefiConfigurationError,
+    UefiNothingToSign,
+    UefiUpload,
+    )
+from lp.services.osutils import write_file
+from lp.services.tarfile_helpers import LaunchpadWriteTarFile
+from lp.testing import TestCase
+from lp.testing.fakemethod import FakeMethod
+
+
+class FakeConfig:
+    """A fake publisher configuration."""
+    def __init__(self, archiveroot, uefiroot):
+        self.archiveroot = archiveroot
+        self.uefiroot = uefiroot
+
+
+class TestUefi(TestCase):
+
+    def setUp(self):
+        super(TestUefi, self).setUp()
+        self.temp_dir = self.makeTemporaryDirectory()
+        self.uefi_dir = self.makeTemporaryDirectory()
+        self.pubconf = FakeConfig(self.temp_dir, self.uefi_dir)
+        self.suite = "distroseries"
+        # CustomUpload.installFiles requires a umask of 022.
+        old_umask = os.umask(022)
+        self.addCleanup(os.umask, old_umask)
+
+    def setUpKeyAndCert(self):
+        self.key = os.path.join(self.uefi_dir, "uefi.key")
+        self.cert = os.path.join(self.uefi_dir, "uefi.crt")
+        write_file(self.key, "")
+        write_file(self.cert, "")
+
+    def openArchive(self, loader_type, version, arch):
+        self.path = os.path.join(
+            self.temp_dir, "%s_%s_%s.tar.gz" % (loader_type, version, arch))
+        self.buffer = open(self.path, "wb")
+        self.archive = LaunchpadWriteTarFile(self.buffer)
+
+    def process(self):
+        self.archive.close()
+        self.buffer.close()
+        upload = UefiUpload()
+        upload.sign = FakeMethod()
+        upload.process(self.pubconf, self.path, self.suite)
+        return upload
+
+    def getUefiPath(self, loader_type, arch):
+        return os.path.join(
+            self.temp_dir, "dists", self.suite, "main", "uefi",
+            "%s-%s" % (loader_type, arch))
+
+    def test_unconfigured(self):
+        # If there is no key/cert configuration, processing fails.
+        self.pubconf = FakeConfig(self.temp_dir, None)
+        self.openArchive("test", "1.0", "amd64")
+        self.assertRaises(UefiConfigurationError, self.process)
+
+    def test_missing_key_and_cert(self):
+        # If the configured key/cert are missing, processing fails.
+        self.openArchive("test", "1.0", "amd64")
+        self.archive.add_file("1.0/empty.efi", "")
+        self.assertRaises(UefiConfigurationError, self.process)
+
+    def test_no_efi_files(self):
+        # Tarballs containing no *.efi files are rejected.
+        self.setUpKeyAndCert()
+        self.openArchive("empty", "1.0", "amd64")
+        self.archive.add_file("hello", "world")
+        self.assertRaises(UefiNothingToSign, self.process)
+
+    def test_already_exists(self):
+        # If the target directory already exists, processing fails.
+        self.setUpKeyAndCert()
+        self.openArchive("test", "1.0", "amd64")
+        self.archive.add_file("1.0/empty.efi", "")
+        os.makedirs(os.path.join(self.getUefiPath("test", "amd64"), "1.0"))
+        self.assertRaises(CustomUploadAlreadyExists, self.process)
+
+    def test_bad_umask(self):
+        # The umask must be 022 to avoid incorrect permissions.
+        self.setUpKeyAndCert()
+        self.openArchive("test", "1.0", "amd64")
+        self.archive.add_file("1.0/dir/file.efi", "foo")
+        os.umask(002)  # cleanup already handled by setUp
+        self.assertRaises(CustomUploadBadUmask, self.process)
+
+    def test_correct_signing_command(self):
+        # getSigningCommand returns the correct command.
+        self.setUpKeyAndCert()
+        upload = UefiUpload()
+        upload.setTargetDirectory(
+            self.pubconf, "test_1.0_amd64.tar.gz", "distroseries")
+        expected_command = [
+            "sbsign", "--key", self.key, "--cert", self.cert, "t.efi"]
+        self.assertEqual(expected_command, upload.getSigningCommand("t.efi"))
+
+    def test_signs_image(self):
+        # Each image in the tarball is signed.
+        self.setUpKeyAndCert()
+        self.openArchive("test", "1.0", "amd64")
+        self.archive.add_file("1.0/empty.efi", "")
+        upload = self.process()
+        self.assertEqual(1, upload.sign.call_count)
+        self.assertEqual(1, len(upload.sign.calls[0][0]))
+        self.assertEqual(
+            "empty.efi", os.path.basename(upload.sign.calls[0][0][0]))
+
+    def test_installed(self):
+        # Files in the tarball are installed correctly.
+        self.setUpKeyAndCert()
+        self.openArchive("test", "1.0", "amd64")
+        self.archive.add_file("1.0/empty.efi", "")
+        self.process()
+        self.assertTrue(os.path.exists(os.path.join(
+            self.getUefiPath("test", "amd64"), "1.0", "empty.efi")))

=== added file 'lib/lp/archivepublisher/uefi.py'
--- lib/lp/archivepublisher/uefi.py	1970-01-01 00:00:00 +0000
+++ lib/lp/archivepublisher/uefi.py	2012-07-03 12:54:32 +0000
@@ -0,0 +1,141 @@
+# Copyright 2012 Canonical Ltd.  This software is licensed under the
+# GNU Affero General Public License version 3 (see the file LICENSE).
+
+"""The processing of UEFI boot loader images.
+
+UEFI Secure Boot requires boot loader images to be signed, and we want to
+have signed images in the archive so that they can be used for upgrades.
+This cannot be done on the build daemons because they are insufficiently
+secure to hold signing keys, so we sign them as a custom upload instead.
+"""
+
+__metaclass__ = type
+
+__all__ = [
+    "process_uefi",
+    "UefiUpload",
+    ]
+
+import os
+import subprocess
+
+from lp.archivepublisher.customupload import (
+    CustomUpload,
+    CustomUploadError,
+    )
+from lp.services.osutils import remove_if_exists
+
+
+class UefiConfigurationError(CustomUploadError):
+    """No signing key location is configured."""
+    def __init__(self, message):
+        CustomUploadError.__init__(
+            self, "UEFI signing configuration error: %s" % message)
+
+
+class UefiNothingToSign(CustomUploadError):
+    """The tarball contained no *.efi files."""
+    def __init__(self, tarfile_path):
+        CustomUploadError.__init__(
+            self, "UEFI upload '%s' contained no *.efi files" % tarfile_path)
+
+
+class UefiUpload(CustomUpload):
+    """UEFI boot loader custom upload.
+
+    The filename must be of the form:
+
+        <TYPE>_<VERSION>_<ARCH>.tar.gz
+
+    where:
+
+      * TYPE: loader type (e.g. 'efilinux');
+      * VERSION: encoded version;
+      * ARCH: targeted architecture tag (e.g. 'amd64').
+
+    The contents are extracted in the archive in the following path:
+
+        <ARCHIVE>/dists/<SUITE>/main/uefi/<TYPE>-<ARCH>/<VERSION>
+
+    A 'current' symbolic link points to the most recent version.  The
+    tarfile must contain at least one file matching the wildcard *.efi, and
+    any such files are signed using the archive's UEFI signing key.
+
+    Signing keys may be installed in the "uefiroot" directory specified in
+    publisher configuration.  In this directory, the private key is
+    "uefi.key" and the certificate is "uefi.crt".
+    """
+    custom_type = "UEFI"
+
+    @staticmethod
+    def parsePath(tarfile_path):
+        loader_type, version, arch = os.path.basename(tarfile_path).split("_")
+        return loader_type, version, arch.split(".")[0]
+
+    def setTargetDirectory(self, pubconf, tarfile_path, distroseries):
+        if pubconf.uefiroot is None:
+            raise UefiConfigurationError(
+                "no UEFI root configured for this archive")
+        self.key = os.path.join(pubconf.uefiroot, "uefi.key")
+        self.cert = os.path.join(pubconf.uefiroot, "uefi.crt")
+        if not os.access(self.key, os.R_OK):
+            raise UefiConfigurationError(
+                "UEFI private key %s not readable" % self.key)
+        if not os.access(self.cert, os.R_OK):
+            raise UefiConfigurationError(
+                "UEFI certificate %s not readable" % self.cert)
+
+        loader_type, self.version, self.arch = self.parsePath(tarfile_path)
+        self.targetdir = os.path.join(
+            pubconf.archiveroot, "dists", distroseries, "main", "uefi",
+            "%s-%s" % (loader_type, self.arch))
+
+    @classmethod
+    def getSeriesKey(cls, tarfile_path):
+        try:
+            loader_type, _, arch = cls.parsePath(tarfile_path)
+            return loader_type, arch
+        except ValueError:
+            return None
+
+    def findEfiFilenames(self):
+        """Find all the *.efi files in an extracted tarball."""
+        for dirpath, dirnames, filenames in os.walk(self.tmpdir):
+            for filename in filenames:
+                if filename.endswith(".efi"):
+                    yield os.path.join(dirpath, filename)
+
+    def getSigningCommand(self, image):
+        """Return the command used to sign an image."""
+        return ["sbsign", "--key", self.key, "--cert", self.cert, image]
+
+    def sign(self, image):
+        """Sign an image."""
+        subprocess.check_call(self.getSigningCommand(image))
+
+    def extract(self):
+        """Copy the custom upload to a temporary directory, and sign it.
+
+        No actual extraction is required.
+        """
+        super(UefiUpload, self).extract()
+        efi_filenames = list(self.findEfiFilenames())
+        if not efi_filenames:
+            raise UefiNothingToSign(self.tarfile_path)
+        for efi_filename in efi_filenames:
+            remove_if_exists("%s.signed" % efi_filename)
+            self.sign(efi_filename)
+
+    def shouldInstall(self, filename):
+        return filename.startswith("%s/" % self.version)
+
+
+def process_uefi(pubconf, tarfile_path, distroseries):
+    """Process a raw-uefi tarfile.
+
+    Unpacking it into the given archive for the given distroseries.
+    Raises CustomUploadError (or some subclass thereof) if anything goes
+    wrong.
+    """
+    upload = UefiUpload()
+    upload.process(pubconf, tarfile_path, distroseries)

=== modified file 'lib/lp/archiveuploader/nascentuploadfile.py'
--- lib/lp/archiveuploader/nascentuploadfile.py	2012-06-29 08:40:05 +0000
+++ lib/lp/archiveuploader/nascentuploadfile.py	2012-07-03 12:54:32 +0000
@@ -31,6 +31,10 @@
 from zope.component import getUtility
 
 from lp.app.errors import NotFoundError
+from lp.archivepublisher.debian_installer import DebianInstallerUpload
+from lp.archivepublisher.dist_upgrader import DistUpgraderUpload
+from lp.archivepublisher.ddtp_tarball import DdtpTarballUpload
+from lp.archivepublisher.uefi import UefiUpload
 from lp.archiveuploader.utils import (
     determine_source_file_type,
     prefix_multi_line_string,
@@ -268,6 +272,14 @@
             PackageUploadCustomFormat.STATIC_TRANSLATIONS,
         'raw-meta-data':
             PackageUploadCustomFormat.META_DATA,
+        'raw-uefi': PackageUploadCustomFormat.UEFI,
+        }
+
+    custom_handlers = {
+        PackageUploadCustomFormat.DEBIAN_INSTALLER: DebianInstallerUpload,
+        PackageUploadCustomFormat.DIST_UPGRADER: DistUpgraderUpload,
+        PackageUploadCustomFormat.DDTP_TARBALL: DdtpTarballUpload,
+        PackageUploadCustomFormat.UEFI: UefiUpload,
         }
 
     @property
@@ -284,6 +296,16 @@
         if self.section_name not in self.custom_sections:
             yield UploadError(
                 "Unsupported custom section name %r" % self.section_name)
+        else:
+            handler = self.custom_handlers.get(
+                self.custom_sections[self.section_name])
+            if handler is not None:
+                try:
+                    handler.parsePath(self.filename)
+                except ValueError:
+                    yield UploadError(
+                        "Invalid filename %r for section name %r" % (
+                            self.filename, self.section_name))
 
     def storeInDatabase(self):
         """Create and return the corresponding LibraryFileAlias reference."""
@@ -294,6 +316,13 @@
             restricted=self.policy.archive.private)
         return libraryfile
 
+    def autoApprove(self):
+        """Return whether this custom upload can be automatically approved."""
+        # UEFI uploads are signed, and must therefore be approved by a human.
+        if self.custom_type == PackageUploadCustomFormat.UEFI:
+            return False
+        return True
+
 
 class PackageUploadFile(NascentUploadFile):
     """Base class to model sources and binary files contained in a upload. """

=== modified file 'lib/lp/archiveuploader/tests/test_nascentuploadfile.py'
--- lib/lp/archiveuploader/tests/test_nascentuploadfile.py	2012-04-27 14:20:20 +0000
+++ lib/lp/archiveuploader/tests/test_nascentuploadfile.py	2012-07-03 12:54:32 +0000
@@ -25,6 +25,7 @@
 from lp.buildmaster.enums import BuildStatus
 from lp.registry.interfaces.pocket import PackagePublishingPocket
 from lp.services.log.logger import BufferLogger
+from lp.services.osutils import write_file
 from lp.soyuz.enums import (
     PackagePublishingStatus,
     PackageUploadCustomFormat,
@@ -92,6 +93,36 @@
         self.assertEquals("bla.txt", libraryfile.filename)
         self.assertEquals("application/octet-stream", libraryfile.mimetype)
 
+    def test_debian_installer_verify(self):
+        # debian-installer uploads are required to have sensible filenames.
+        uploadfile = self.createCustomUploadFile(
+            "debian-installer-images_20120627_i386.tar.gz", "data",
+            "main/raw-installer", "extra")
+        self.assertEqual([], list(uploadfile.verify()))
+        uploadfile = self.createCustomUploadFile(
+            "bla.txt", "data", "main/raw-installer", "extra")
+        errors = list(uploadfile.verify())
+        self.assertEqual(1, len(errors))
+        self.assertIsInstance(errors[0], UploadError)
+
+    def test_no_handler_no_verify(self):
+        # Uploads without special handlers have no filename checks.
+        uploadfile = self.createCustomUploadFile(
+            "bla.txt", "data", "main/raw-meta-data", "extra")
+        self.assertEqual([], list(uploadfile.verify()))
+
+    def test_debian_installer_auto_approved(self):
+        # debian-installer uploads are auto-approved.
+        uploadfile = self.createCustomUploadFile(
+            "bla.txt", "data", "main/raw-installer", "extra")
+        self.assertTrue(uploadfile.autoApprove())
+
+    def test_uefi_not_auto_approved(self):
+        # UEFI uploads are auto-approved.
+        uploadfile = self.createCustomUploadFile(
+            "bla.txt", "data", "main/raw-uefi", "extra")
+        self.assertFalse(uploadfile.autoApprove())
+
 
 class PackageUploadFileTestCase(NascentUploadFileTestCase):
     """Base class for all tests of classes deriving from PackageUploadFile."""
@@ -286,8 +317,7 @@
             "data.tar.%s" % data_format,
             ]
         for member in members:
-            with open(os.path.join(tempdir, member), "w") as f:
-                pass
+            write_file(os.path.join(tempdir, member), "")
         retcode = subprocess.call(
             ["ar", "rc", filename] + members, cwd=tempdir)
         self.assertEqual(0, retcode)

=== modified file 'lib/lp/archiveuploader/uploadpolicy.py'
--- lib/lp/archiveuploader/uploadpolicy.py	2012-06-19 03:26:57 +0000
+++ lib/lp/archiveuploader/uploadpolicy.py	2012-07-03 12:54:32 +0000
@@ -317,6 +317,14 @@
             raise AssertionError(
                 "Upload is not sourceful, binaryful or mixed.")
 
+    def autoApprove(self, upload):
+        """Check that all custom files in this upload can be auto-approved."""
+        if upload.binaryful:
+            for custom_file in upload.changes.custom_files:
+                if not custom_file.autoApprove():
+                    return False
+        return True
+
 
 class SyncUploadPolicy(AbstractUploadPolicy):
     """This policy is invoked when processing sync uploads."""

=== modified file 'lib/lp/services/osutils.py'
--- lib/lp/services/osutils.py	2012-06-29 08:40:05 +0000
+++ lib/lp/services/osutils.py	2012-07-03 12:54:32 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2011 Canonical Ltd.  This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """Utilities for doing the sort of thing the os module does."""
@@ -15,6 +15,7 @@
     'remove_tree',
     'two_stage_kill',
     'until_no_eintr',
+    'write_file',
     ]
 
 from contextlib import contextmanager
@@ -207,6 +208,5 @@
 
 
 def write_file(path, content):
-    f = open(path, 'w')
-    f.write(content)
-    f.close()
+    with open_for_writing(path, 'w') as f:
+        f.write(content)

=== modified file 'lib/lp/soyuz/browser/queue.py'
--- lib/lp/soyuz/browser/queue.py	2012-06-29 08:40:05 +0000
+++ lib/lp/soyuz/browser/queue.py	2012-07-03 12:54:32 +0000
@@ -1,4 +1,4 @@
-# Copyright 2009-2011 Canonical Ltd.  This software is licensed under the
+# Copyright 2009-2012 Canonical Ltd.  This software is licensed under the
 # GNU Affero General Public License version 3 (see the file LICENSE).
 
 """Browser views for package queue."""
@@ -578,6 +578,7 @@
             (self.contains_installer, ("Installer", 'ubuntu-icon')),
             (self.contains_upgrader, ("Upgrader", 'ubuntu-icon')),
             (self.contains_ddtp, (ddtp, 'ubuntu-icon')),
+            (self.contains_uefi, ("Signed UEFI boot loader", 'ubuntu-icon')),
             ]
         return [
             self.composeIcon(*details)

=== modified file 'lib/lp/soyuz/browser/tests/builder-views.txt'
--- lib/lp/soyuz/browser/tests/builder-views.txt	2012-05-29 01:45:39 +0000
+++ lib/lp/soyuz/browser/tests/builder-views.txt	2012-07-03 12:54:32 +0000
@@ -283,7 +283,7 @@
 
  * title: the title that will be presented for this category in the UI;
 
- * virtualized: wheter the category represents the virtualized or
+ * virtualized: whether the category represents the virtualized or
    non-virtualized build-farm;
 
  * groups: a property that return all `BuilderGroup` instanced

=== modified file 'lib/lp/soyuz/configure.zcml'
--- lib/lp/soyuz/configure.zcml	2012-06-22 05:25:35 +0000
+++ lib/lp/soyuz/configure.zcml	2012-07-03 12:54:32 +0000
@@ -173,6 +173,7 @@
                 contains_installer
                 contains_upgrader
                 contains_ddtp
+                contains_uefi
                 displayname
                 displayarchs
                 displayversion

=== modified file 'lib/lp/soyuz/enums.py'
--- lib/lp/soyuz/enums.py	2012-05-28 15:12:00 +0000
+++ lib/lp/soyuz/enums.py	2012-07-03 12:54:32 +0000
@@ -474,6 +474,12 @@
         the Software Center.
         """)
 
+    UEFI = DBItem(6, """
+        uefi
+
+        A UEFI boot loader image to be signed.
+        """)
+
 
 class PackageUploadStatus(DBEnumeratedType):
     """Distro Release Queue Status

=== modified file 'lib/lp/soyuz/interfaces/queue.py'
--- lib/lp/soyuz/interfaces/queue.py	2012-06-30 17:41:37 +0000
+++ lib/lp/soyuz/interfaces/queue.py	2012-07-03 12:54:32 +0000
@@ -207,9 +207,11 @@
     contains_translation = Attribute(
         "whether or not this upload contains translations")
     contains_upgrader = Attribute(
-        "wheter or not this upload contains upgrader images")
+        "whether or not this upload contains upgrader images")
     contains_ddtp = Attribute(
-        "wheter or not this upload contains DDTP images")
+        "whether or not this upload contains DDTP images")
+    contains_uefi = Attribute(
+        "whether or not this upload contains a signed UEFI boot loader image")
     isPPA = Attribute(
         "Return True if this PackageUpload is a PPA upload.")
     is_delayed_copy = Attribute(

=== modified file 'lib/lp/soyuz/model/queue.py'
--- lib/lp/soyuz/model/queue.py	2012-07-03 08:04:35 +0000
+++ lib/lp/soyuz/model/queue.py	2012-07-03 12:54:32 +0000
@@ -629,6 +629,12 @@
         return (PackageUploadCustomFormat.DDTP_TARBALL
                 in self._customFormats)
 
+    @cachedproperty
+    def contains_uefi(self):
+        """See `IPackageUpload`."""
+        return (PackageUploadCustomFormat.UEFI
+                in self._customFormats)
+
     @property
     def package_name(self):
         """See `IPackageUpload`."""
@@ -1270,8 +1276,7 @@
         try:
             # See the XXX near the import for getPubConfig.
             archive_config = getPubConfig(self.packageupload.archive)
-            action_method(
-                archive_config.archiveroot, temp_filename, suite)
+            action_method(archive_config, temp_filename, suite)
         finally:
             shutil.rmtree(os.path.dirname(temp_filename))
 
@@ -1376,6 +1381,14 @@
         self.libraryfilealias.open()
         copy_and_close(self.libraryfilealias, file_obj)
 
+    def publishUefi(self, logger=None):
+        """See `IPackageUploadCustom`."""
+        # XXX cprov 2005-03-03: We need to use the Zope Component Lookup
+        # to instantiate the object in question and avoid circular imports
+        from lp.archivepublisher.uefi import process_uefi
+
+        self._publishCustom(process_uefi)
+
     publisher_dispatch = {
         PackageUploadCustomFormat.DEBIAN_INSTALLER: publishDebianInstaller,
         PackageUploadCustomFormat.ROSETTA_TRANSLATIONS:
@@ -1385,6 +1398,7 @@
         PackageUploadCustomFormat.STATIC_TRANSLATIONS:
             publishStaticTranslations,
         PackageUploadCustomFormat.META_DATA: publishMetaData,
+        PackageUploadCustomFormat.UEFI: publishUefi,
         }
 
     # publisher_dispatch must have an entry for each value of

=== modified file 'lib/lp/soyuz/scripts/custom_uploads_copier.py'
--- lib/lp/soyuz/scripts/custom_uploads_copier.py	2012-06-22 17:26:53 +0000
+++ lib/lp/soyuz/scripts/custom_uploads_copier.py	2012-07-03 12:54:32 +0000
@@ -18,6 +18,7 @@
 
 from lp.archivepublisher.debian_installer import DebianInstallerUpload
 from lp.archivepublisher.dist_upgrader import DistUpgraderUpload
+from lp.archivepublisher.uefi import UefiUpload
 from lp.registry.interfaces.pocket import PackagePublishingPocket
 from lp.services.database.bulk import load_referencing
 from lp.soyuz.enums import PackageUploadCustomFormat
@@ -40,6 +41,7 @@
     copyable_types = {
         PackageUploadCustomFormat.DEBIAN_INSTALLER: DebianInstallerUpload,
         PackageUploadCustomFormat.DIST_UPGRADER: DistUpgraderUpload,
+        PackageUploadCustomFormat.UEFI: UefiUpload,
         }
 
     def __init__(self, target_series,