Merge ~cjwatson/launchpad:livefs-build-pocket into launchpad:master

Proposed by Colin Watson
Status: Needs review
Proposed branch: ~cjwatson/launchpad:livefs-build-pocket
Merge into: launchpad:master
Diff against target: 59 lines (+20/-3)
2 files modified
lib/lp/soyuz/model/livefsbuildbehaviour.py (+8/-2)
lib/lp/soyuz/tests/test_livefsbuildbehaviour.py (+12/-1)
Reviewer Review Type Date Requested Status
Launchpad code reviewers Pending
Review via email: mp+373803@code.launchpad.net

Commit message

Allow livefs build metadata to override the default build pocket

Description of the change

This makes it possible, for example, to distinguish the case of building a livefs using tools from -updates but with contents from the release pocket from the case of building a livefs using tools from -updates and contents from -updates.

The use case here is that at the moment buildd images are built with tools from -updates but with contents from the release pocket. I'd like it to be possible to build buildd images with contents from -updates as well, since we could use those for builds against -updates and it would save a good deal of time at the start of each build.

We've ended up in a bit of a compatibility corner due to some unfortunate past decisions (mostly by me), and this is the least bad solution I can think of. launchpad-buildd and livecd-rootfs will also need some work to support this, but this design allows it to be done in a somewhat backwards-compatible way (that is, it wouldn't break existing livecd-rootfs code). The one compatibility issue is that existing livefses that intend to continue building with tools from -updates and contents from the release pocket (or similar) will need to have "pocket": "release" explicitly added to their metadata before landing the corresponding launchpad-buildd change which starts paying attention to the "pocket" argument; we'll need to go through the current database to check for that kind of thing.

This is essentially the same as https://code.launchpad.net/~cjwatson/launchpad/livefs-build-pocket/+merge/372444, converted to git and rebased on master.

To post a comment you must log in.

Unmerged commits

ed26842... by Colin Watson on 2019-09-06

Allow livefs build metadata to override the default build pocket.

This makes it possible, for example, to distinguish the case of building a
livefs using tools from -updates but with contents from the release pocket
from the case of building a livefs using tools from -updates and contents
from -updates.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/lib/lp/soyuz/model/livefsbuildbehaviour.py b/lib/lp/soyuz/model/livefsbuildbehaviour.py
2index bef72b0..ec5dbaf 100644
3--- a/lib/lp/soyuz/model/livefsbuildbehaviour.py
4+++ b/lib/lp/soyuz/model/livefsbuildbehaviour.py
5@@ -86,15 +86,21 @@ class LiveFSBuildBehaviour(BuildFarmJobBehaviourBase):
6 build = self.build
7 base_args = yield super(LiveFSBuildBehaviour, self).extraBuildArgs(
8 logger=logger)
9+ args = {}
10+ # Allow the metadata to override the default build pocket; this is
11+ # useful e.g. to build a livefs using tools from -updates but with
12+ # contents from the release pocket. Note that the "pocket" argument
13+ # here is only used for passing instructions to livecd-rootfs, not
14+ # for constructing the sources.list used for the build.
15+ args["pocket"] = build.pocket.name.lower()
16 # Non-trivial metadata values may have been security-wrapped, which
17 # is pointless here and just gets in the way of xmlrpclib
18 # serialisation.
19- args = dict(removeSecurityProxy(build.livefs.metadata))
20+ args.update(removeSecurityProxy(build.livefs.metadata))
21 if build.metadata_override is not None:
22 args.update(removeSecurityProxy(build.metadata_override))
23 # Everything else overrides anything in the metadata.
24 args.update(base_args)
25- args["pocket"] = build.pocket.name.lower()
26 args["datestamp"] = build.version
27 args["archives"], args["trusted_keys"] = (
28 yield get_sources_list_for_building(
29diff --git a/lib/lp/soyuz/tests/test_livefsbuildbehaviour.py b/lib/lp/soyuz/tests/test_livefsbuildbehaviour.py
30index 29e697e..bfafa1f 100644
31--- a/lib/lp/soyuz/tests/test_livefsbuildbehaviour.py
32+++ b/lib/lp/soyuz/tests/test_livefsbuildbehaviour.py
33@@ -261,7 +261,7 @@ class TestAsyncLiveFSBuildBehaviour(TestLiveFSBuildBehaviourBase):
34
35 @defer.inlineCallbacks
36 def test_extraBuildArgs_metadata_cannot_override_base(self):
37- # Items in the user-provided metadata cannot override the base
38+ # Most items in the user-provided metadata cannot override the base
39 # arguments.
40 job = self.makeJob(
41 metadata={"project": "distro", "arch_tag": "nonsense"},
42@@ -271,6 +271,17 @@ class TestAsyncLiveFSBuildBehaviour(TestLiveFSBuildBehaviourBase):
43 self.assertEqual("i386", args["arch_tag"])
44
45 @defer.inlineCallbacks
46+ def test_extraBuildArgs_metadata_pocket_overrides_base(self):
47+ # The "pocket" item in the user-provided metadata overrides the base
48+ # arguments, to allow for building a livefs with content whose
49+ # source differs from the tools used to build it.
50+ job = self.makeJob(
51+ pocket=PackagePublishingPocket.UPDATES,
52+ metadata={"pocket": "release"}, with_builder=True)
53+ args = yield job.extraBuildArgs()
54+ self.assertEqual("release", args["pocket"])
55+
56+ @defer.inlineCallbacks
57 def test_composeBuildRequest(self):
58 job = self.makeJob(with_builder=True)
59 lfa = self.factory.makeLibraryFileAlias(db_only=True)

Subscribers

People subscribed via source and target branches

to status/vote changes: