Merge lp:~mvo/launchpad-buildd/mvo into lp:launchpad-buildd

[Michael Vogt]

This branch takes the excellent buildlivefs and adds some tweaks to make it more useful outside the buildd environment.

If that branch lands, we can consider moving it outside of the launchpad-buildd branch and into somehthing more general like livecd-rootfs or ubuntu-dev-tools so that people who want to experiment with building a rootfs. It would also provide a canonical way of building the rootfs and could superseed tools like rootstock and BuildLiveCD.

A careful review for this is needed as I'm new to the world of livefs building/buildds and may miss important setup steps that are taken before buildlivefs is called (i.e. I'm not sure if the deboostrap step I added is sufficient).

Thanks for your consideration,
 Michael

[lp:~mvo/launchpad-buildd/mvo]

131. By Michael Vogt on 2014-07-24

add a default to build-id assuming that local use is mostly sequentical

[Colin Watson (cjwatson)]

review: Needs Fixing

[lp:~mvo/launchpad-buildd/mvo]

132. By Michael Vogt on 2014-07-24

fix review issues raised by Colin

[Michael Vogt (mvo)]

Thanks a lot for your review Colin. I addressed your points in r132. From reading your reply it seems like you disapprove of the idea to make this something that could live outside of launchpad-buildd. In which case the revert of the chroot creation and the build-id default make sense of course.

[Colin Watson (cjwatson)]

I think it would be odd to move buildlivefs outside of launchpad-buildd and not the corresponding helpers for other build types, and buildlivefs's interface is quite closely tied to launchpad-buildd so it would be nice not to end up overcomplicating our deployments, especially as buildds have historically been stuck on old Ubuntu releases (maybe this will change with scalingstack, but let's see). I don't mind making buildlivefs friendlier for occasional non-launchpad-buildd use; I'd just prefer this not to involve things that might be less resilient to bugs elsewhere in production.

Unmerged revisions

132. By Michael Vogt on 2014-07-24

fix review issues raised by Colin

131. By Michael Vogt on 2014-07-24

add a default to build-id assuming that local use is mostly sequentical

130. By Michael Vogt on 2014-07-24

refactor debootrap into _prepare_chroot() and add friendly message at the end of the build

129. By Michael Vogt on 2014-07-24

make buildlivefs more general purpose

128. By Michael Vogt on 2014-07-23

buildlivefs: if the chroot_path does not exits, bootstrap it

=== modified file 'buildlivefs'
--- buildlivefs	2014-06-24 14:59:08 +0000
+++ buildlivefs	2014-07-24 11:39:17 +0000
@@ -19,14 +19,14 @@
 RETCODE_FAILURE_BUILD = 201
 
 
-def get_build_path(build_id, *extra):
+def get_build_path(build_home, build_id, *extra):
     """Generate a path within the build directory.
 
     :param build_id: the build id to use.
     :param extra: the extra path segments within the build directory.
     :return: the generated path.
     """
-    return os.path.join(os.environ["HOME"], "build-" + build_id, *extra)
+    return os.path.join(build_home, "build-" + build_id, *extra)
 
 
 non_meta_re = re.compile(r'^[a-zA-Z0-9+,./:=@_-]+$')
@@ -64,13 +64,36 @@
     ]
 
 
+class MissingOptionError(Exception):
+    """A required option for the LiveFSBuilder is missing"""
+    pass
+
+
+class MissingChrootError(Exception):
+    """A chroot is missing"""
+    pass
+
+
 class LiveFSBuilder:
     """Builds a live file system."""
 
     def __init__(self, options):
+        self._check_options(options)
         self.options = options
         self.chroot_path = get_build_path(
-            self.options.build_id, 'chroot-autobuild')
+            options.build_home, options.build_id, 'chroot-autobuild')
+        # be gentle to people who run it outside the buildd environment
+        if not os.path.exists(self.chroot_path):
+            raise MissingChrootError(
+                "Missing chroot at '%s'. Please check chroot_url from "
+                "https://api.launchpad.net/devel/ubuntu/%s/%s" % (
+                    self.chroot_path, options.series, options.arch))
+
+    def _check_options(self, options):
+        for required_option in ("arch", "project", "series"):
+            if getattr(options, required_option) is None:
+                raise MissingOptionError(
+                    "Option '--%s' is required" % required_option)
 
     def chroot(self, args, echo=False):
         """Run a command in the chroot.
@@ -161,19 +184,23 @@
 
 def main():
     parser = OptionParser()
-    parser.add_option("--build-id", help="build identifier")
-    parser.add_option(
-        "--arch", metavar="ARCH", help="build for architecture ARCH")
+    parser.add_option(
+        "--build-id", help="build identifier (use buildlivefs if in doubt)")
+    parser.add_option(
+        "--arch", metavar="ARCH",
+        help="build for architecture ARCH (e.g. amd64)")
     parser.add_option(
         "--subarch", metavar="SUBARCH",
         help="build for subarchitecture SUBARCH")
     parser.add_option(
-        "--project", metavar="PROJECT", help="build for project PROJECT")
+        "--project", metavar="PROJECT",
+        help="build for project PROJECT (e.g. ubuntu-desktop)")
     parser.add_option(
         "--subproject", metavar="SUBPROJECT",
         help="build for subproject SUBPROJECT")
     parser.add_option(
-        "--series", metavar="SERIES", help="build for series SERIES")
+        "--series", metavar="SERIES",
+        help="build for series SERIES (e.g. trusty)")
     parser.add_option("--datestamp", help="date stamp")
     parser.add_option(
         "--image-format", metavar="FORMAT", help="produce an image in FORMAT")
@@ -186,6 +213,9 @@
     parser.add_option(
         "--extra-ppa", dest="extra_ppas", default=[], action="append",
         help="use this additional PPA")
+    parser.add_option(
+        "--build-home", metavar="DIRECTORY", default=os.environ["HOME"],
+        help="Root directory for the build results (default to $HOME)")
     options, _ = parser.parse_args()
 
     builder = LiveFSBuilder(options)
@@ -196,6 +226,8 @@
         return RETCODE_FAILURE_INSTALL
     try:
         builder.build()
+        print("Success. Build result is available at '%s'" % 
+              os.path.join(builder.chroot_path, "build"))
     except Exception:
         traceback.print_exc()
         return RETCODE_FAILURE_BUILD