Merge ~twom/launchpad:concrete-oci-projects into launchpad:master

Proposed by Tom Wardill
Status: Merged
Approved by: Tom Wardill
Approved revision: c5e1d572e1ee61ffa33d73abb73cc87b0b67958a
Merge reported by: Otto Co-Pilot
Merged at revision: not available
Proposed branch: ~twom/launchpad:concrete-oci-projects
Merge into: launchpad:master
Prerequisite: ~twom/launchpad:oci-gitrepository
Diff against target: 1603 lines (+1329/-12)
24 files modified
lib/lp/_schema_circular_imports.py (+7/-0)
lib/lp/app/validators/path.py (+34/-0)
lib/lp/app/validators/tests/test_path.py (+51/-0)
lib/lp/buildmaster/enums.py (+6/-0)
lib/lp/code/model/tests/test_gitlookup.py (+3/-3)
lib/lp/configure.zcml (+1/-0)
lib/lp/oci/__init__.py (+0/-0)
lib/lp/oci/configure.zcml (+59/-0)
lib/lp/oci/interfaces/__init__.py (+0/-0)
lib/lp/oci/interfaces/ocirecipe.py (+242/-0)
lib/lp/oci/interfaces/ocirecipebuild.py (+63/-0)
lib/lp/oci/model/__init__.py (+0/-0)
lib/lp/oci/model/ocirecipe.py (+295/-0)
lib/lp/oci/model/ocirecipebuild.py (+194/-0)
lib/lp/oci/tests/__init__.py (+0/-0)
lib/lp/oci/tests/test_ocirecipe.py (+208/-0)
lib/lp/registry/interfaces/ociprojectseries.py (+1/-1)
lib/lp/registry/model/ociproject.py (+2/-2)
lib/lp/registry/model/ociprojectseries.py (+4/-4)
lib/lp/registry/personmerge.py (+22/-0)
lib/lp/registry/tests/test_ociprojectseries.py (+1/-1)
lib/lp/registry/tests/test_personmerge.py (+40/-0)
lib/lp/security.py (+41/-1)
lib/lp/testing/factory.py (+55/-0)
Reviewer Review Type Date Requested Status
Colin Watson (community) Approve
Review via email: mp+376132@code.launchpad.net

This proposal supersedes a proposal from 2019-11-28.

Commit message

OCIRecipe and skeleton of dependent models

To post a comment you must log in.
Revision history for this message
Colin Watson (cjwatson) :
Revision history for this message
Colin Watson (cjwatson) wrote :

Looks OK so far - just a few relatively minor comments.

review: Approve
7d660b0... by Tom Wardill

Traceback errors

b7d86c9... by Tom Wardill

Update for latest db patches

Revision history for this message
Colin Watson (cjwatson) wrote :

Generally good, though I have a bunch more comments that I'd like to see fixed before this lands.

It's probably obvious, but note that this can't land until the DB patch has been deployed and then merged into master.

review: Approve
6415899... by Tom Wardill

Path validator improvements

e77fc47... by Tom Wardill

Better named test

Revision history for this message
Colin Watson (cjwatson) wrote :

The DB patch has been deployed and I just merged it into master. However, before I forget, please note that you'll need to merge master (or rebase on it) and then implement personmerge properly for OCIRecipe.owner to replace the temporary stub I added for it. Compare https://git.launchpad.net/launchpad/commit?id=454dc1a00d36f984c7f0eb759f38df0fca0361df.

6821448... by Tom Wardill

Tidy up ocirecipe interface

14bfa69... by Tom Wardill

Tidy ocirecipebuild interface

704052f... by Tom Wardill

Use git_ref rather than separate git_repository and git_path variables

1a77f94... by Tom Wardill

Handle more build artifacts in destroySelf

6871c91... by Tom Wardill

displayname != display_name

229d750... by Tom Wardill

Add date_created as DEFAULT

d0463e7... by Tom Wardill

Correct uniqueness index

5ea7e97... by Tom Wardill

Implement required build duration methods

9670d0e... by Tom Wardill

Move to distro_arch_series rather than lower level variables

2771b2c... by Tom Wardill

Factory method simplifications

58703ee... by Tom Wardill

Add ocirecipe to personmerge

8c6f2ac... by Tom Wardill

Format imports

Revision history for this message
Colin Watson (cjwatson) :
review: Approve
88c112f... by Tom Wardill

Sentence case

c5e1d57... by Tom Wardill

Add basic virtualized tests for ocirecipebuild

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/lib/lp/_schema_circular_imports.py b/lib/lp/_schema_circular_imports.py
2index 372744f..70c5b72 100644
3--- a/lib/lp/_schema_circular_imports.py
4+++ b/lib/lp/_schema_circular_imports.py
5@@ -96,6 +96,8 @@ from lp.hardwaredb.interfaces.hwdb import (
6 IHWSubmissionDevice,
7 IHWVendorID,
8 )
9+from lp.oci.interfaces.ocirecipe import IOCIRecipe
10+from lp.oci.interfaces.ocirecipebuild import IOCIRecipeBuild
11 from lp.registry.interfaces.commercialsubscription import (
12 ICommercialSubscription,
13 )
14@@ -1090,3 +1092,8 @@ patch_operations_explicit_version(
15
16 # IWikiName
17 patch_entry_explicit_version(IWikiName, 'beta')
18+
19+# IOCIRecipe
20+patch_collection_property(IOCIRecipe, 'builds', IOCIRecipeBuild)
21+patch_collection_property(IOCIRecipe, 'completed_builds', IOCIRecipeBuild)
22+patch_collection_property(IOCIRecipe, 'pending_builds', IOCIRecipeBuild)
23diff --git a/lib/lp/app/validators/path.py b/lib/lp/app/validators/path.py
24new file mode 100644
25index 0000000..13e0b06
26--- /dev/null
27+++ b/lib/lp/app/validators/path.py
28@@ -0,0 +1,34 @@
29+# Copyright 2019 Canonical Ltd. This software is licensed under the
30+# GNU Affero General Public License version 3 (see the file LICENSE).
31+
32+"""Validators for paths and path functions."""
33+
34+from __future__ import absolute_import, print_function, unicode_literals
35+
36+__metaclass__ = type
37+__all__ = [
38+ 'path_does_not_escape'
39+]
40+
41+import os
42+
43+from lp.app.validators import LaunchpadValidationError
44+
45+
46+def path_does_not_escape(path):
47+ """First-pass validation that a given path does not escape a root.
48+
49+ This is only intended as a first defence, usage of this will also
50+ require checking for filesystem escapes (symlinks, etc).
51+ """
52+ # We're not working with complete paths, so we need to make them so
53+ fake_base_path = '/target'
54+ # Ensure that we start with a common base
55+ target_path = os.path.join(fake_base_path, path)
56+ # Resolve symlinks and such
57+ real_path = os.path.normpath(target_path)
58+ # If the paths don't have a common start anymore,
59+ # we are attempting an escape
60+ if not os.path.commonprefix((real_path, fake_base_path)) == fake_base_path:
61+ raise LaunchpadValidationError("Path would escape target directory")
62+ return True
63diff --git a/lib/lp/app/validators/tests/test_path.py b/lib/lp/app/validators/tests/test_path.py
64new file mode 100644
65index 0000000..24501a8
66--- /dev/null
67+++ b/lib/lp/app/validators/tests/test_path.py
68@@ -0,0 +1,51 @@
69+# Copyright 2020 Canonical Ltd. This software is licensed under the
70+# GNU Affero General Public License version 3 (see the file LICENSE).
71+
72+"""Tests for path validators."""
73+
74+from __future__ import absolute_import, print_function, unicode_literals
75+
76+__metaclass__ = type
77+
78+from lp.app.validators import LaunchpadValidationError
79+from lp.app.validators.path import path_does_not_escape
80+from lp.testing import TestCase
81+
82+
83+class TestPathDoesNotEscape(TestCase):
84+
85+ def test_valid_path(self):
86+ self.assertTrue(path_does_not_escape('Buildfile'))
87+
88+ def test_invalid_path_parent(self):
89+ self.assertRaises(
90+ LaunchpadValidationError,
91+ path_does_not_escape,
92+ '../Buildfile')
93+
94+ def test_invalid_path_elsewhere(self):
95+ self.assertRaises(
96+ LaunchpadValidationError,
97+ path_does_not_escape,
98+ '/var/foo/Buildfile')
99+
100+ def test_starts_with_target(self):
101+ self.assertRaises(
102+ LaunchpadValidationError,
103+ path_does_not_escape,
104+ '/target/../../../Buildfile')
105+
106+ def test_extra_dot_slash(self):
107+ self.assertRaises(
108+ LaunchpadValidationError,
109+ path_does_not_escape,
110+ '/foo/./../../bar/./Buildfile')
111+
112+ def test_starts_with_target_inclusive(self):
113+ self.assertRaises(
114+ LaunchpadValidationError,
115+ path_does_not_escape,
116+ '/targetfoo/../../../Buildfile')
117+
118+ def test_just_target(self):
119+ self.assertTrue(path_does_not_escape('/target'))
120diff --git a/lib/lp/buildmaster/enums.py b/lib/lp/buildmaster/enums.py
121index c6f0aac..a96bce0 100644
122--- a/lib/lp/buildmaster/enums.py
123+++ b/lib/lp/buildmaster/enums.py
124@@ -164,6 +164,12 @@ class BuildFarmJobType(DBEnumeratedType):
125 Build a snap package from a recipe.
126 """)
127
128+ OCIRECIPEBUILD = DBItem(7, """
129+ OCI image build
130+
131+ Build an OCI image from a recipe.
132+ """)
133+
134
135 class BuildQueueStatus(DBEnumeratedType):
136 """Build queue status.
137diff --git a/lib/lp/code/model/tests/test_gitlookup.py b/lib/lp/code/model/tests/test_gitlookup.py
138index 02da9e6..d527b06 100644
139--- a/lib/lp/code/model/tests/test_gitlookup.py
140+++ b/lib/lp/code/model/tests/test_gitlookup.py
141@@ -111,8 +111,8 @@ class TestGetByUniqueName(TestCaseWithFactory):
142 repository.unique_name + "-nonexistent"))
143
144 def test_ociproject(self):
145- ociproject = self.factory.makeOCIProject()
146- repository = self.factory.makeGitRepository(target=ociproject)
147+ oci_project = self.factory.makeOCIProject()
148+ repository = self.factory.makeGitRepository(target=oci_project)
149 self.assertEqual(
150 repository, self.lookup.getByUniqueName(repository.unique_name))
151 self.assertIsNone(self.lookup.getByUniqueName(
152@@ -168,7 +168,7 @@ class TestGetByPath(TestCaseWithFactory):
153 self.assertEqual(
154 (repository, ""), self.lookup.getByPath(repository.unique_name))
155
156- def test_ociproject_default(self):
157+ def test_ociproject_official(self):
158 oci_project = self.factory.makeOCIProject()
159 repository = self.factory.makeGitRepository(target=oci_project)
160 with person_logged_in(repository.target.distribution.owner):
161diff --git a/lib/lp/configure.zcml b/lib/lp/configure.zcml
162index 533d714..a3e7c08 100644
163--- a/lib/lp/configure.zcml
164+++ b/lib/lp/configure.zcml
165@@ -30,6 +30,7 @@
166 <include package="lp.code" />
167 <include package="lp.coop.answersbugs" />
168 <include package="lp.hardwaredb" />
169+ <include package="lp.oci" />
170 <include package="lp.snappy" />
171 <include package="lp.soyuz" />
172 <include package="lp.translations" />
173diff --git a/lib/lp/oci/__init__.py b/lib/lp/oci/__init__.py
174new file mode 100644
175index 0000000..e69de29
176--- /dev/null
177+++ b/lib/lp/oci/__init__.py
178diff --git a/lib/lp/oci/configure.zcml b/lib/lp/oci/configure.zcml
179new file mode 100644
180index 0000000..30212b9
181--- /dev/null
182+++ b/lib/lp/oci/configure.zcml
183@@ -0,0 +1,59 @@
184+<!-- Copyright 2015-2019 Canonical Ltd. This software is licensed under the
185+ GNU Affero General Public License version 3 (see the file LICENSE).
186+-->
187+<configure
188+ xmlns="http://namespaces.zope.org/zope"
189+ xmlns:i18n="http://namespaces.zope.org/i18n"
190+ xmlns:lp="http://namespaces.canonical.com/lp"
191+ i18n_domain="launchpad">
192+
193+ <!-- OCIRecipe -->
194+ <class
195+ class="lp.oci.model.ocirecipe.OCIRecipe">
196+ <require
197+ permission="launchpad.View"
198+ interface="lp.oci.interfaces.ocirecipe.IOCIRecipeView
199+ lp.oci.interfaces.ocirecipe.IOCIRecipeEditableAttributes
200+ lp.oci.interfaces.ocirecipe.IOCIRecipeAdminAttributes"/>
201+ <require
202+ permission="launchpad.Edit"
203+ interface="lp.oci.interfaces.ocirecipe.IOCIRecipeEdit"
204+ set_schema="lp.oci.interfaces.ocirecipe.IOCIRecipeEditableAttributes" />
205+ <require
206+ permission="launchpad.Admin"
207+ set_schema="lp.oci.interfaces.ocirecipe.IOCIRecipeAdminAttributes" />
208+ </class>
209+ <securedutility
210+ class="lp.oci.model.ocirecipe.OCIRecipeSet"
211+ provides="lp.oci.interfaces.ocirecipe.IOCIRecipeSet">
212+ <allow
213+ interface="lp.oci.interfaces.ocirecipe.IOCIRecipeSet"/>
214+ </securedutility>
215+
216+ <!-- OCIRecipeBuild -->
217+ <class class="lp.oci.model.ocirecipebuild.OCIRecipeBuild">
218+ <require
219+ permission="launchpad.View"
220+ interface="lp.oci.interfaces.ocirecipebuild.IOCIRecipeBuildView" />
221+ <require
222+ permission="launchpad.Edit"
223+ interface="lp.oci.interfaces.ocirecipebuild.IOCIRecipeBuildEdit" />
224+ <require
225+ permission="launchpad.Admin"
226+ interface="lp.oci.interfaces.ocirecipebuild.IOCIRecipeBuildAdmin" />
227+ </class>
228+
229+ <!-- OCIRecipeBuildSet -->
230+ <securedutility
231+ class="lp.oci.model.ocirecipebuild.OCIRecipeBuildSet"
232+ provides="lp.oci.interfaces.ocirecipebuild.IOCIRecipeBuildSet">
233+ <allow interface="lp.oci.interfaces.ocirecipebuild.IOCIRecipeBuildSet" />
234+ </securedutility>
235+ <securedutility
236+ class="lp.oci.model.ocirecipebuild.OCIRecipeBuildSet"
237+ provides="lp.buildmaster.interfaces.buildfarmjob.ISpecificBuildFarmJobSource"
238+ name="OCIRECIPEBUILD">
239+ <allow interface="lp.buildmaster.interfaces.buildfarmjob.ISpecificBuildFarmJobSource" />
240+ </securedutility>
241+
242+</configure>
243diff --git a/lib/lp/oci/interfaces/__init__.py b/lib/lp/oci/interfaces/__init__.py
244new file mode 100644
245index 0000000..e69de29
246--- /dev/null
247+++ b/lib/lp/oci/interfaces/__init__.py
248diff --git a/lib/lp/oci/interfaces/ocirecipe.py b/lib/lp/oci/interfaces/ocirecipe.py
249new file mode 100644
250index 0000000..340adfc
251--- /dev/null
252+++ b/lib/lp/oci/interfaces/ocirecipe.py
253@@ -0,0 +1,242 @@
254+# Copyright 2019 Canonical Ltd. This software is licensed under the
255+# GNU Affero General Public License version 3 (see the file LICENSE).
256+
257+"""Interfaces related to recipes for OCI Images."""
258+
259+from __future__ import absolute_import, print_function, unicode_literals
260+
261+__metaclass__ = type
262+__all__ = [
263+ 'DuplicateOCIRecipeName',
264+ 'IOCIRecipe',
265+ 'IOCIRecipeEdit',
266+ 'IOCIRecipeEditableAttributes',
267+ 'IOCIRecipeSet',
268+ 'IOCIRecipeView',
269+ 'NoSourceForOCIRecipe',
270+ 'NoSuchOCIRecipe',
271+ 'OCIRecipeBuildAlreadyPending',
272+ 'OCIRecipeNotOwner',
273+ ]
274+
275+from lazr.restful.declarations import error_status
276+from lazr.restful.fields import (
277+ CollectionField,
278+ Reference,
279+ ReferenceChoice,
280+ )
281+from six.moves import http_client
282+from zope.interface import Interface
283+from zope.schema import (
284+ Bool,
285+ Datetime,
286+ Int,
287+ Text,
288+ TextLine,
289+ )
290+from zope.security.interfaces import Unauthorized
291+
292+from lp import _
293+from lp.app.errors import NameLookupFailed
294+from lp.app.validators.name import name_validator
295+from lp.app.validators.path import path_does_not_escape
296+from lp.code.interfaces.gitref import IGitRef
297+from lp.code.interfaces.gitrepository import IGitRepository
298+from lp.registry.interfaces.ociproject import IOCIProject
299+from lp.registry.interfaces.role import IHasOwner
300+from lp.services.fields import (
301+ PersonChoice,
302+ PublicPersonChoice,
303+ )
304+
305+
306+@error_status(http_client.UNAUTHORIZED)
307+class OCIRecipeNotOwner(Unauthorized):
308+ """The registrant/requester is not the owner or a member of its team."""
309+
310+
311+@error_status(http_client.BAD_REQUEST)
312+class OCIRecipeBuildAlreadyPending(Exception):
313+ """A build was requested when an identical build was already pending."""
314+
315+ def __init__(self):
316+ super(OCIRecipeBuildAlreadyPending, self).__init__(
317+ "An identical build of this snap package is already pending.")
318+
319+
320+@error_status(http_client.BAD_REQUEST)
321+class DuplicateOCIRecipeName(Exception):
322+ """An OCI Recipe already exists with the same name."""
323+
324+
325+class NoSuchOCIRecipe(NameLookupFailed):
326+ """The requested OCI Recipe does not exist."""
327+ _message_prefix = "No such OCI recipe exists for this OCI project"
328+
329+
330+@error_status(http_client.BAD_REQUEST)
331+class NoSourceForOCIRecipe(Exception):
332+ """OCI Recipes must have a source and build file."""
333+
334+ def __init__(self):
335+ super(NoSourceForOCIRecipe, self).__init__(
336+ "New OCI recipes must have a git branch and build file.")
337+
338+
339+class IOCIRecipeView(Interface):
340+ """`IOCIRecipe` attributes that require launchpad.View permission."""
341+
342+ id = Int(title=_("ID"), required=True, readonly=True)
343+ date_created = Datetime(
344+ title=_("Date created"), required=True, readonly=True)
345+ date_last_modified = Datetime(
346+ title=_("Date last modified"), required=True, readonly=True)
347+
348+ registrant = PublicPersonChoice(
349+ title=_("Registrant"),
350+ description=_("The user who registered this recipe."),
351+ vocabulary='ValidPersonOrTeam', required=True, readonly=True)
352+
353+ builds = CollectionField(
354+ title=_("Completed builds of this OCI recipe."),
355+ description=_(
356+ "Completed builds of this OCI recipe, sorted in descending "
357+ "order of finishing."),
358+ # Really IOCIRecipeBuild, patched in _schema_circular_imports.
359+ value_type=Reference(schema=Interface),
360+ required=True, readonly=True)
361+
362+ completed_builds = CollectionField(
363+ title=_("Completed builds of this OCI recipe."),
364+ description=_(
365+ "Completed builds of this OCI recipe, sorted in descending "
366+ "order of finishing."),
367+ # Really IOCIRecipeBuild, patched in _schema_circular_imports.
368+ value_type=Reference(schema=Interface), readonly=True)
369+
370+ pending_builds = CollectionField(
371+ title=_("Pending builds of this OCI recipe."),
372+ description=_(
373+ "Pending builds of this OCI recipe, sorted in descending "
374+ "order of creation."),
375+ # Really IOCIRecipeBuild, patched in _schema_circular_imports.
376+ value_type=Reference(schema=Interface), readonly=True)
377+
378+ def requestBuild(requester, architecture):
379+ """Request that the OCI recipe is built.
380+
381+ :param requester: The person requesting the build.
382+ :param architecture: The architecture to build for.
383+ :return: `IOCIRecipeBuild`.
384+ """
385+
386+
387+class IOCIRecipeEdit(Interface):
388+ """`IOCIRecipe` methods that require launchpad.Edit permission."""
389+
390+ def destroySelf():
391+ """Delete this OCI recipe, provided that it has no builds."""
392+
393+
394+class IOCIRecipeEditableAttributes(IHasOwner):
395+ """`IOCIRecipe` attributes that can be edited.
396+
397+ These attributes need launchpad.View to see, and launchpad.Edit to change.
398+ """
399+
400+ name = TextLine(
401+ title=_("The name of this recipe."),
402+ constraint=name_validator,
403+ required=True,
404+ readonly=False)
405+
406+ owner = PersonChoice(
407+ title=_("Owner"),
408+ required=True,
409+ vocabulary="AllUserTeamsParticipationPlusSelf",
410+ description=_("The owner of this OCI recipe."),
411+ readonly=False)
412+
413+ oci_project = Reference(
414+ IOCIProject,
415+ title=_("The OCI project that this recipe is for."),
416+ required=True,
417+ readonly=True)
418+
419+ official = Bool(
420+ title=_("OCI project official"),
421+ required=True,
422+ default=False,
423+ description=_("True if this recipe is official for its OCI project."),
424+ readonly=False)
425+
426+ git_ref = Reference(
427+ IGitRef, title=_("Git branch"), required=False, readonly=False,
428+ description=_(
429+ "The Git branch containing a Dockerfile at the location "
430+ "defined by the build_file attribute."))
431+
432+ git_repository = ReferenceChoice(
433+ title=_("Git repository"),
434+ schema=IGitRepository, vocabulary="GitRepository",
435+ required=False, readonly=True,
436+ description=_(
437+ "A Git repository with a branch containing a Dockerfile "
438+ "at the location defined by the build_file attribute."))
439+
440+ git_path = TextLine(
441+ title=_("Git branch path"), required=False, readonly=False,
442+ description=_(
443+ "The path of the Git branch containing a Dockerfile "
444+ "at the location defined by the build_file attribute."))
445+
446+ description = Text(
447+ title=_("A short description of this recipe."),
448+ readonly=False)
449+
450+ build_file = TextLine(
451+ title=_("The relative path to the file within this recipe's "
452+ "branch that defines how to build the recipe."),
453+ constraint=path_does_not_escape,
454+ required=True,
455+ readonly=False)
456+
457+ build_daily = Bool(
458+ title=_("Build daily"),
459+ required=True,
460+ default=False,
461+ description=_("If True, this recipe should be built daily."),
462+ readonly=False)
463+
464+
465+class IOCIRecipeAdminAttributes(Interface):
466+ """`IOCIRecipe` attributes that can be edited by admins.
467+
468+ These attributes need launchpad.View to see, and launchpad.Admin to change.
469+ """
470+
471+ require_virtualized = Bool(
472+ title=_("Require virtualized builders"), required=True, readonly=False,
473+ description=_("Only build this OCI recipe on virtual builders."))
474+
475+
476+class IOCIRecipe(IOCIRecipeView, IOCIRecipeEdit, IOCIRecipeEditableAttributes,
477+ IOCIRecipeAdminAttributes):
478+ """A recipe for building Open Container Initiative images."""
479+
480+
481+class IOCIRecipeSet(Interface):
482+ """A utility to create and access OCI Recipes."""
483+
484+ def new(name, registrant, owner, oci_project, git_ref, description,
485+ official, require_virtualized, build_file, date_created):
486+ """Create an IOCIRecipe."""
487+
488+ def exists(owner, oci_project, name):
489+ """Check to see if an existing OCI Recipe exists."""
490+
491+ def getByName(owner, oci_project, name):
492+ """Return the appropriate `OCIRecipe` for the given objects."""
493+
494+ def findByOwner(owner):
495+ """Return all OCI Recipes with the given `owner`."""
496diff --git a/lib/lp/oci/interfaces/ocirecipebuild.py b/lib/lp/oci/interfaces/ocirecipebuild.py
497new file mode 100644
498index 0000000..5ee6b46
499--- /dev/null
500+++ b/lib/lp/oci/interfaces/ocirecipebuild.py
501@@ -0,0 +1,63 @@
502+# Copyright 2019 Canonical Ltd. This software is licensed under the
503+# GNU Affero General Public License version 3 (see the file LICENSE).
504+
505+"""Interfaces for a build record for OCI recipes."""
506+
507+from __future__ import absolute_import, print_function, unicode_literals
508+
509+__metaclass__ = type
510+__all__ = [
511+ 'IOCIRecipeBuild',
512+ 'IOCIRecipeBuildSet',
513+ ]
514+
515+from lazr.restful.fields import Reference
516+from zope.interface import Interface
517+from zope.schema import Text
518+
519+from lp import _
520+from lp.buildmaster.interfaces.buildfarmjob import ISpecificBuildFarmJobSource
521+from lp.buildmaster.interfaces.packagebuild import IPackageBuild
522+from lp.oci.interfaces.ocirecipe import IOCIRecipe
523+from lp.services.database.constants import DEFAULT
524+from lp.services.fields import PublicPersonChoice
525+
526+
527+class IOCIRecipeBuildEdit(Interface):
528+ # XXX twom 2020-02-10 This will probably need cancel() implementing
529+ pass
530+
531+
532+class IOCIRecipeBuildView(IPackageBuild):
533+
534+ requester = PublicPersonChoice(
535+ title=_("Requester"),
536+ description=_("The person who requested this OCI recipe build."),
537+ vocabulary='ValidPersonOrTeam', required=True, readonly=True)
538+
539+ recipe = Reference(
540+ IOCIRecipe,
541+ title=_("The OCI recipe to build."),
542+ required=True,
543+ readonly=True)
544+
545+
546+class IOCIRecipeBuildAdmin(Interface):
547+ # XXX twom 2020-02-10 This will probably need rescore() implementing
548+ pass
549+
550+
551+class IOCIRecipeBuild(IOCIRecipeBuildAdmin, IOCIRecipeBuildEdit,
552+ IOCIRecipeBuildView):
553+ """A build record for an OCI recipe."""
554+
555+
556+class IOCIRecipeBuildSet(ISpecificBuildFarmJobSource):
557+ """A utility to create and access OCIRecipeBuilds."""
558+
559+ def new(requester, recipe, distro_arch_series,
560+ date_created=DEFAULT):
561+ """Create an `IOCIRecipeBuild`."""
562+
563+ def preloadBuildsData(builds):
564+ """Load the data related to a list of OCI recipe builds."""
565diff --git a/lib/lp/oci/model/__init__.py b/lib/lp/oci/model/__init__.py
566new file mode 100644
567index 0000000..e69de29
568--- /dev/null
569+++ b/lib/lp/oci/model/__init__.py
570diff --git a/lib/lp/oci/model/ocirecipe.py b/lib/lp/oci/model/ocirecipe.py
571new file mode 100644
572index 0000000..0d4d20f
573--- /dev/null
574+++ b/lib/lp/oci/model/ocirecipe.py
575@@ -0,0 +1,295 @@
576+# Copyright 2019 Canonical Ltd. This software is licensed under the
577+# GNU Affero General Public License version 3 (see the file LICENSE).
578+
579+"""A recipe for building Open Container Initiative images."""
580+
581+from __future__ import absolute_import, print_function, unicode_literals
582+
583+__metaclass__ = type
584+__all__ = [
585+ 'OCIRecipe',
586+ 'OCIRecipeSet',
587+ ]
588+
589+
590+from lazr.lifecycle.event import ObjectCreatedEvent
591+import pytz
592+from storm.expr import (
593+ Desc,
594+ Not,
595+ )
596+from storm.locals import (
597+ Bool,
598+ DateTime,
599+ Int,
600+ Reference,
601+ Store,
602+ Storm,
603+ Unicode,
604+ )
605+from zope.component import getUtility
606+from zope.event import notify
607+from zope.interface import implementer
608+
609+from lp.buildmaster.enums import BuildStatus
610+from lp.buildmaster.interfaces.buildqueue import IBuildQueueSet
611+from lp.buildmaster.model.buildfarmjob import BuildFarmJob
612+from lp.buildmaster.model.buildqueue import BuildQueue
613+from lp.oci.interfaces.ocirecipe import (
614+ DuplicateOCIRecipeName,
615+ IOCIRecipe,
616+ IOCIRecipeSet,
617+ NoSourceForOCIRecipe,
618+ NoSuchOCIRecipe,
619+ OCIRecipeBuildAlreadyPending,
620+ OCIRecipeNotOwner,
621+ )
622+from lp.oci.interfaces.ocirecipebuild import IOCIRecipeBuildSet
623+from lp.oci.model.ocirecipebuild import OCIRecipeBuild
624+from lp.services.database.constants import DEFAULT
625+from lp.services.database.decoratedresultset import DecoratedResultSet
626+from lp.services.database.interfaces import (
627+ IMasterStore,
628+ IStore,
629+ )
630+from lp.services.database.stormexpr import (
631+ Greatest,
632+ NullsLast,
633+ )
634+
635+
636+@implementer(IOCIRecipe)
637+class OCIRecipe(Storm):
638+
639+ __storm_table__ = 'OCIRecipe'
640+
641+ id = Int(primary=True)
642+ date_created = DateTime(
643+ name="date_created", tzinfo=pytz.UTC, allow_none=False)
644+ date_last_modified = DateTime(
645+ name="date_last_modified", tzinfo=pytz.UTC, allow_none=False)
646+
647+ registrant_id = Int(name='registrant', allow_none=False)
648+ registrant = Reference(registrant_id, "Person.id")
649+
650+ owner_id = Int(name='owner', allow_none=False)
651+ owner = Reference(owner_id, 'Person.id')
652+
653+ oci_project_id = Int(name='oci_project', allow_none=False)
654+ oci_project = Reference(oci_project_id, "OCIProject.id")
655+
656+ name = Unicode(name="name", allow_none=False)
657+ description = Unicode(name="description")
658+
659+ official = Bool(name="official", default=False)
660+
661+ git_repository_id = Int(name="git_repository", allow_none=False)
662+ git_repository = Reference(git_repository_id, "GitRepository.id")
663+ git_path = Unicode(name="git_path", allow_none=False)
664+ build_file = Unicode(name="build_file", allow_none=False)
665+
666+ require_virtualized = Bool(name="require_virtualized", default=True,
667+ allow_none=False)
668+
669+ build_daily = Bool(name="build_daily", default=False)
670+
671+ def __init__(self, name, registrant, owner, oci_project, git_ref,
672+ description=None, official=False, require_virtualized=True,
673+ build_file=None, date_created=DEFAULT):
674+ super(OCIRecipe, self).__init__()
675+ self.name = name
676+ self.registrant = registrant
677+ self.owner = owner
678+ self.oci_project = oci_project
679+ self.description = description
680+ self.build_file = build_file
681+ self.official = official
682+ self.require_virtualized = require_virtualized
683+ self.date_created = date_created
684+ self.git_ref = git_ref
685+
686+ def destroySelf(self):
687+ """See `IOCIRecipe`."""
688+ # XXX twom 2019-11-26 This needs to expand as more build artifacts
689+ # are added
690+ store = IStore(OCIRecipe)
691+ buildqueue_records = store.find(
692+ BuildQueue,
693+ BuildQueue._build_farm_job_id == OCIRecipeBuild.build_farm_job_id,
694+ OCIRecipeBuild.recipe == self)
695+ for buildqueue_record in buildqueue_records:
696+ buildqueue_record.destroySelf()
697+ build_farm_job_ids = list(store.find(
698+ OCIRecipeBuild.build_farm_job_id, OCIRecipeBuild.recipe == self))
699+ store.find(OCIRecipeBuild, OCIRecipeBuild.recipe == self).remove()
700+ store.remove(self)
701+ store.find(
702+ BuildFarmJob, BuildFarmJob.id.is_in(build_farm_job_ids)).remove()
703+
704+ @property
705+ def git_ref(self):
706+ """See `IOCIRecipe`."""
707+ if self.git_repository is not None:
708+ return self.git_repository.getRefByPath(self.git_path)
709+ return None
710+
711+ @git_ref.setter
712+ def git_ref(self, value):
713+ """See `IOCIRecipe`."""
714+ if value is not None:
715+ self.git_path = value.path
716+ self.git_repository = value.repository
717+ else:
718+ self.git_repository = None
719+ self.git_path = None
720+
721+ def _checkRequestBuild(self, requester):
722+ if not requester.inTeam(self.owner):
723+ raise OCIRecipeNotOwner(
724+ "%s cannot create OCI image builds owned by %s." %
725+ (requester.display_name, self.owner.display_name))
726+
727+ def requestBuild(self, requester, distro_arch_series):
728+ self._checkRequestBuild(requester)
729+
730+ pending = IStore(self).find(
731+ OCIRecipeBuild,
732+ OCIRecipeBuild.recipe == self.id,
733+ OCIRecipeBuild.processor == distro_arch_series.processor,
734+ OCIRecipeBuild.status == BuildStatus.NEEDSBUILD)
735+ if pending.any() is not None:
736+ raise OCIRecipeBuildAlreadyPending
737+
738+ build = getUtility(IOCIRecipeBuildSet).new(
739+ requester, self, distro_arch_series)
740+ build.queueBuild()
741+ notify(ObjectCreatedEvent(build, user=requester))
742+ return build
743+
744+ @property
745+ def _pending_states(self):
746+ """All the build states we consider pending (non-final)."""
747+ return [
748+ BuildStatus.NEEDSBUILD,
749+ BuildStatus.BUILDING,
750+ BuildStatus.UPLOADING,
751+ BuildStatus.CANCELLING,
752+ ]
753+
754+ def _getBuilds(self, filter_term, order_by):
755+ """The actual query to get the builds."""
756+ query_args = [
757+ OCIRecipeBuild.recipe == self,
758+ ]
759+ if filter_term is not None:
760+ query_args.append(filter_term)
761+ result = Store.of(self).find(OCIRecipeBuild, *query_args)
762+ result.order_by(order_by)
763+
764+ def eager_load(rows):
765+ getUtility(IOCIRecipeBuildSet).preloadBuildsData(rows)
766+ getUtility(IBuildQueueSet).preloadForBuildFarmJobs(rows)
767+
768+ return DecoratedResultSet(result, pre_iter_hook=eager_load)
769+
770+ @property
771+ def builds(self):
772+ """See `IOCIRecipe`."""
773+ order_by = (
774+ NullsLast(Desc(Greatest(
775+ OCIRecipeBuild.date_started,
776+ OCIRecipeBuild.date_finished))),
777+ Desc(OCIRecipeBuild.date_created),
778+ Desc(OCIRecipeBuild.id))
779+ return self._getBuilds(None, order_by)
780+
781+ @property
782+ def completed_builds(self):
783+ """See `IOCIRecipe`."""
784+ filter_term = (Not(OCIRecipeBuild.status.is_in(self._pending_states)))
785+ order_by = (
786+ NullsLast(Desc(Greatest(
787+ OCIRecipeBuild.date_started,
788+ OCIRecipeBuild.date_finished))),
789+ Desc(OCIRecipeBuild.id))
790+ return self._getBuilds(filter_term, order_by)
791+
792+ @property
793+ def pending_builds(self):
794+ """See `IOCIRecipe`."""
795+ filter_term = (OCIRecipeBuild.status.is_in(self._pending_states))
796+ # We want to order by date_created but this is the same as ordering
797+ # by id (since id increases monotonically) and is less expensive.
798+ order_by = Desc(OCIRecipeBuild.id)
799+ return self._getBuilds(filter_term, order_by)
800+
801+
802+class OCIRecipeArch(Storm):
803+ """Link table to back `OCIRecipe.processors`."""
804+
805+ __storm_table__ = "OCIRecipeArch"
806+ __storm_primary__ = ("recipe_id", "processor_id")
807+
808+ recipe_id = Int(name="recipe", allow_none=False)
809+ recipe = Reference(recipe_id, "OCIRecipe.id")
810+
811+ processor_id = Int(name="processor", allow_none=False)
812+ processor = Reference(processor_id, "Processor.id")
813+
814+ def __init__(self, recipe, processor):
815+ self.recipe = recipe
816+ self.processor = processor
817+
818+
819+@implementer(IOCIRecipeSet)
820+class OCIRecipeSet:
821+
822+ def new(self, name, registrant, owner, oci_project, git_ref, build_file,
823+ description=None, official=False, require_virtualized=True,
824+ date_created=DEFAULT):
825+ """See `IOCIRecipeSet`."""
826+ if not registrant.inTeam(owner):
827+ if owner.is_team:
828+ raise OCIRecipeNotOwner(
829+ "%s is not a member of %s." %
830+ (registrant.displayname, owner.displayname))
831+ else:
832+ raise OCIRecipeNotOwner(
833+ "%s cannot create OCI images owned by %s." %
834+ (registrant.displayname, owner.displayname))
835+
836+ if not (git_ref and build_file):
837+ raise NoSourceForOCIRecipe
838+
839+ if self.exists(owner, oci_project, name):
840+ raise DuplicateOCIRecipeName
841+
842+ store = IMasterStore(OCIRecipe)
843+ oci_recipe = OCIRecipe(
844+ name, registrant, owner, oci_project, git_ref, description,
845+ official, require_virtualized, build_file, date_created)
846+ store.add(oci_recipe)
847+
848+ return oci_recipe
849+
850+ def _getByName(self, owner, oci_project, name):
851+ return IStore(OCIRecipe).find(
852+ OCIRecipe,
853+ OCIRecipe.owner == owner,
854+ OCIRecipe.name == name,
855+ OCIRecipe.oci_project == oci_project).one()
856+
857+ def exists(self, owner, oci_project, name):
858+ """See `IOCIRecipeSet`."""
859+ return self._getByName(owner, oci_project, name) is not None
860+
861+ def getByName(self, owner, oci_project, name):
862+ """See `IOCIRecipeSet`."""
863+ oci_recipe = self._getByName(owner, oci_project, name)
864+ if oci_recipe is None:
865+ raise NoSuchOCIRecipe(name)
866+ return oci_recipe
867+
868+ def findByOwner(self, owner):
869+ """See `IOCIRecipe`."""
870+ return IStore(OCIRecipe).find(OCIRecipe, OCIRecipe.owner == owner)
871diff --git a/lib/lp/oci/model/ocirecipebuild.py b/lib/lp/oci/model/ocirecipebuild.py
872new file mode 100644
873index 0000000..d069800
874--- /dev/null
875+++ b/lib/lp/oci/model/ocirecipebuild.py
876@@ -0,0 +1,194 @@
877+# Copyright 2019 Canonical Ltd. This software is licensed under the
878+# GNU Affero General Public License version 3 (see the file LICENSE).
879+
880+"""A build record for OCI Recipes."""
881+
882+from __future__ import absolute_import, print_function, unicode_literals
883+
884+__metaclass__ = type
885+__all__ = [
886+ 'OCIRecipeBuild',
887+ 'OCIRecipeBuildSet',
888+ ]
889+
890+from datetime import timedelta
891+
892+import pytz
893+from storm.locals import (
894+ Bool,
895+ DateTime,
896+ Desc,
897+ Int,
898+ Reference,
899+ Store,
900+ Storm,
901+ Unicode,
902+ )
903+from storm.store import EmptyResultSet
904+from zope.component import getUtility
905+from zope.interface import implementer
906+
907+from lp.buildmaster.enums import (
908+ BuildFarmJobType,
909+ BuildStatus,
910+ )
911+from lp.buildmaster.interfaces.buildfarmjob import IBuildFarmJobSource
912+from lp.buildmaster.model.buildfarmjob import SpecificBuildFarmJobSourceMixin
913+from lp.buildmaster.model.packagebuild import PackageBuildMixin
914+from lp.oci.interfaces.ocirecipebuild import (
915+ IOCIRecipeBuild,
916+ IOCIRecipeBuildSet,
917+ )
918+from lp.services.database.constants import DEFAULT
919+from lp.services.database.decoratedresultset import DecoratedResultSet
920+from lp.services.database.enumcol import DBEnum
921+from lp.services.database.interfaces import (
922+ IMasterStore,
923+ IStore,
924+ )
925+
926+
927+@implementer(IOCIRecipeBuild)
928+class OCIRecipeBuild(PackageBuildMixin, Storm):
929+
930+ __storm_table__ = 'OCIRecipeBuild'
931+
932+ job_type = BuildFarmJobType.OCIRECIPEBUILD
933+
934+ id = Int(name='id', primary=True)
935+
936+ requester_id = Int(name='requester', allow_none=False)
937+ requester = Reference(requester_id, 'Person.id')
938+
939+ recipe_id = Int(name='recipe', allow_none=False)
940+ recipe = Reference(recipe_id, 'OCIRecipe.id')
941+
942+ processor_id = Int(name='processor', allow_none=False)
943+ processor = Reference(processor_id, 'Processor.id')
944+
945+ virtualized = Bool(name='virtualized')
946+
947+ date_created = DateTime(
948+ name='date_created', tzinfo=pytz.UTC, allow_none=False)
949+ date_started = DateTime(name='date_started', tzinfo=pytz.UTC)
950+ date_finished = DateTime(name='date_finished', tzinfo=pytz.UTC)
951+ date_first_dispatched = DateTime(
952+ name='date_first_dispatched', tzinfo=pytz.UTC)
953+
954+ builder_id = Int(name='builder')
955+ builder = Reference(builder_id, 'Builder.id')
956+
957+ status = DBEnum(name='status', enum=BuildStatus, allow_none=False)
958+
959+ log_id = Int(name='log')
960+ log = Reference(log_id, 'LibraryFileAlias.id')
961+
962+ upload_log_id = Int(name='upload_log')
963+ upload_log = Reference(upload_log_id, 'LibraryFileAlias.id')
964+
965+ dependencies = Unicode(name='dependencies')
966+
967+ failure_count = Int(name='failure_count', allow_none=False)
968+
969+ build_farm_job_id = Int(name='build_farm_job', allow_none=False)
970+ build_farm_job = Reference(build_farm_job_id, 'BuildFarmJob.id')
971+
972+ def __init__(self, build_farm_job, requester, recipe,
973+ processor, virtualized, date_created):
974+
975+ self.requester = requester
976+ self.recipe = recipe
977+ self.processor = processor
978+ self.virtualized = virtualized
979+ self.date_created = date_created
980+ self.status = BuildStatus.NEEDSBUILD
981+ self.build_farm_job = build_farm_job
982+
983+ def calculateScore(self):
984+ # XXX twom 2020-02-11 - This might need an addition?
985+ return 2510
986+
987+ def estimateDuration(self):
988+ """See `IBuildFarmJob`."""
989+ median = self.getMedianBuildDuration()
990+ if median is not None:
991+ return median
992+ return timedelta(minutes=30)
993+
994+ def getMedianBuildDuration(self):
995+ """Return the median duration of our successful builds."""
996+ store = IStore(self)
997+ result = store.find(
998+ (OCIRecipeBuild.date_started, OCIRecipeBuild.date_finished),
999+ OCIRecipeBuild.recipe == self.recipe_id,
1000+ OCIRecipeBuild.processor == self.processor_id,
1001+ OCIRecipeBuild.status == BuildStatus.FULLYBUILT)
1002+ result.order_by(Desc(OCIRecipeBuild.date_finished))
1003+ durations = [row[1] - row[0] for row in result[:9]]
1004+ if len(durations) == 0:
1005+ return None
1006+ durations.sort()
1007+ return durations[len(durations) // 2]
1008+
1009+ @property
1010+ def archive(self):
1011+ # XXX twom 2019-12-05 This may need to change when an OCIProject
1012+ # pillar isn't just a distribution
1013+ return self.recipe.oci_project.distribution.main_archive
1014+
1015+ @property
1016+ def distribution(self):
1017+ # XXX twom 2019-12-05 This may need to change when an OCIProject
1018+ # pillar isn't just a distribution
1019+ return self.recipe.oci_project.distribution
1020+
1021+ # Stub attributes to match the IPackageBuild interface that we
1022+ # will not use in this implementation.
1023+ pocket = None
1024+ distro_series = None
1025+
1026+
1027+@implementer(IOCIRecipeBuildSet)
1028+class OCIRecipeBuildSet(SpecificBuildFarmJobSourceMixin):
1029+ """See `IOCIRecipeBuildSet`."""
1030+
1031+ def new(self, requester, recipe, distro_arch_series,
1032+ date_created=DEFAULT):
1033+ """See `IOCIRecipeBuildSet`."""
1034+
1035+ virtualized = (
1036+ not distro_arch_series.processor.supports_nonvirtualized
1037+ or recipe.require_virtualized)
1038+
1039+ store = IMasterStore(OCIRecipeBuild)
1040+ build_farm_job = getUtility(IBuildFarmJobSource).new(
1041+ OCIRecipeBuild.job_type, BuildStatus.NEEDSBUILD, date_created)
1042+ ocirecipebuild = OCIRecipeBuild(
1043+ build_farm_job, requester, recipe, distro_arch_series.processor,
1044+ virtualized, date_created)
1045+ store.add(ocirecipebuild)
1046+ return ocirecipebuild
1047+
1048+ def preloadBuildsData(self, builds):
1049+ """See `IOCIRecipeBuildSet`."""
1050+ # XXX twom 2019-12-02 Currently a no-op skeleton, to be filled in
1051+ return
1052+
1053+ def getByID(self, build_id):
1054+ """See `ISpecificBuildFarmJobSource`."""
1055+ store = IMasterStore(OCIRecipeBuild)
1056+ return store.get(OCIRecipeBuild, build_id)
1057+
1058+ def getByBuildFarmJob(self, build_farm_job):
1059+ """See `ISpecificBuildFarmJobSource`."""
1060+ return Store.of(build_farm_job).find(
1061+ OCIRecipeBuild, build_farm_job_id=build_farm_job.id).one()
1062+
1063+ def getByBuildFarmJobs(self, build_farm_jobs):
1064+ """See `ISpecificBuildFarmJobSource`."""
1065+ if len(build_farm_jobs) == 0:
1066+ return EmptyResultSet()
1067+ rows = Store.of(build_farm_jobs[0]).find(
1068+ OCIRecipeBuild, OCIRecipeBuild.build_farm_job_id.is_in(
1069+ bfj.id for bfj in build_farm_jobs))
1070+ return DecoratedResultSet(rows, pre_iter_hook=self.preloadBuildsData)
1071diff --git a/lib/lp/oci/tests/__init__.py b/lib/lp/oci/tests/__init__.py
1072new file mode 100644
1073index 0000000..e69de29
1074--- /dev/null
1075+++ b/lib/lp/oci/tests/__init__.py
1076diff --git a/lib/lp/oci/tests/test_ocirecipe.py b/lib/lp/oci/tests/test_ocirecipe.py
1077new file mode 100644
1078index 0000000..db71400
1079--- /dev/null
1080+++ b/lib/lp/oci/tests/test_ocirecipe.py
1081@@ -0,0 +1,208 @@
1082+# Copyright 2019 Canonical Ltd. This software is licensed under the
1083+# GNU Affero General Public License version 3 (see the file LICENSE).
1084+
1085+"""Tests for OCI image building recipe functionality."""
1086+
1087+from __future__ import absolute_import, print_function, unicode_literals
1088+
1089+from zope.component import getUtility
1090+from zope.security.proxy import removeSecurityProxy
1091+
1092+from lp.buildmaster.enums import BuildStatus
1093+from lp.oci.interfaces.ocirecipe import (
1094+ DuplicateOCIRecipeName,
1095+ IOCIRecipe,
1096+ IOCIRecipeSet,
1097+ NoSourceForOCIRecipe,
1098+ NoSuchOCIRecipe,
1099+ OCIRecipeBuildAlreadyPending,
1100+ OCIRecipeNotOwner,
1101+ )
1102+from lp.oci.interfaces.ocirecipebuild import IOCIRecipeBuildSet
1103+from lp.testing import (
1104+ admin_logged_in,
1105+ person_logged_in,
1106+ TestCaseWithFactory,
1107+ )
1108+from lp.testing.layers import DatabaseFunctionalLayer
1109+
1110+
1111+class TestOCIRecipe(TestCaseWithFactory):
1112+
1113+ layer = DatabaseFunctionalLayer
1114+
1115+ def test_implements_interface(self):
1116+ target = self.factory.makeOCIRecipe()
1117+ with admin_logged_in():
1118+ self.assertProvides(target, IOCIRecipe)
1119+
1120+ def test_checkRequestBuild(self):
1121+ ocirecipe = removeSecurityProxy(self.factory.makeOCIRecipe())
1122+ unrelated_person = self.factory.makePerson()
1123+ self.assertRaises(
1124+ OCIRecipeNotOwner,
1125+ ocirecipe._checkRequestBuild,
1126+ unrelated_person)
1127+
1128+ def test_requestBuild(self):
1129+ ocirecipe = self.factory.makeOCIRecipe()
1130+ oci_arch = self.factory.makeOCIRecipeArch(recipe=ocirecipe)
1131+ build = ocirecipe.requestBuild(ocirecipe.owner, oci_arch)
1132+ self.assertEqual(build.status, BuildStatus.NEEDSBUILD)
1133+
1134+ def test_requestBuild_already_exists(self):
1135+ ocirecipe = self.factory.makeOCIRecipe()
1136+ oci_arch = self.factory.makeOCIRecipeArch(recipe=ocirecipe)
1137+ ocirecipe.requestBuild(ocirecipe.owner, oci_arch)
1138+
1139+ self.assertRaises(
1140+ OCIRecipeBuildAlreadyPending,
1141+ ocirecipe.requestBuild,
1142+ ocirecipe.owner, oci_arch)
1143+
1144+ def test_destroySelf(self):
1145+ oci_recipe = self.factory.makeOCIRecipe()
1146+ build_ids = []
1147+ for x in range(3):
1148+ build_ids.append(
1149+ self.factory.makeOCIRecipeBuild(recipe=oci_recipe).id)
1150+
1151+ with person_logged_in(oci_recipe.owner):
1152+ oci_recipe.destroySelf()
1153+
1154+ for build_id in build_ids:
1155+ self.assertIsNone(getUtility(IOCIRecipeBuildSet).getByID(build_id))
1156+
1157+ def test_getBuilds(self):
1158+ # Test the various getBuilds methods.
1159+ oci_recipe = self.factory.makeOCIRecipe()
1160+ builds = [self.factory.makeOCIRecipeBuild(recipe=oci_recipe)
1161+ for x in range(3)]
1162+ # We want the latest builds first.
1163+ builds.reverse()
1164+
1165+ self.assertEqual(builds, list(oci_recipe.builds))
1166+ self.assertEqual([], list(oci_recipe.completed_builds))
1167+ self.assertEqual(builds, list(oci_recipe.pending_builds))
1168+
1169+ # Change the status of one of the builds and retest.
1170+ builds[0].updateStatus(BuildStatus.BUILDING)
1171+ builds[0].updateStatus(BuildStatus.FULLYBUILT)
1172+ self.assertEqual(builds, list(oci_recipe.builds))
1173+ self.assertEqual(builds[:1], list(oci_recipe.completed_builds))
1174+ self.assertEqual(builds[1:], list(oci_recipe.pending_builds))
1175+
1176+ def test_getBuilds_cancelled_never_started_last(self):
1177+ # A cancelled build that was never even started sorts to the end.
1178+ oci_recipe = self.factory.makeOCIRecipe()
1179+ fullybuilt = self.factory.makeOCIRecipeBuild(recipe=oci_recipe)
1180+ instacancelled = self.factory.makeOCIRecipeBuild(recipe=oci_recipe)
1181+ fullybuilt.updateStatus(BuildStatus.BUILDING)
1182+ fullybuilt.updateStatus(BuildStatus.FULLYBUILT)
1183+ instacancelled.updateStatus(BuildStatus.CANCELLED)
1184+ self.assertEqual([fullybuilt, instacancelled], list(oci_recipe.builds))
1185+ self.assertEqual(
1186+ [fullybuilt, instacancelled], list(oci_recipe.completed_builds))
1187+ self.assertEqual([], list(oci_recipe.pending_builds))
1188+
1189+
1190+class TestOCIRecipeSet(TestCaseWithFactory):
1191+
1192+ layer = DatabaseFunctionalLayer
1193+
1194+ def test_implements_interface(self):
1195+ target_set = getUtility(IOCIRecipeSet)
1196+ with admin_logged_in():
1197+ self.assertProvides(target_set, IOCIRecipeSet)
1198+
1199+ def test_new(self):
1200+ registrant = self.factory.makePerson()
1201+ owner = self.factory.makeTeam(members=[registrant])
1202+ oci_project = self.factory.makeOCIProject()
1203+ [git_ref] = self.factory.makeGitRefs()
1204+ target = getUtility(IOCIRecipeSet).new(
1205+ name='a name',
1206+ registrant=registrant,
1207+ owner=owner,
1208+ oci_project=oci_project,
1209+ git_ref=git_ref,
1210+ description='a description',
1211+ official=False,
1212+ require_virtualized=False,
1213+ build_file='build file')
1214+ self.assertEqual(target.registrant, registrant)
1215+ self.assertEqual(target.owner, owner)
1216+ self.assertEqual(target.oci_project, oci_project)
1217+ self.assertEqual(target.official, False)
1218+ self.assertEqual(target.require_virtualized, False)
1219+ self.assertEqual(target.git_ref, git_ref)
1220+
1221+ def test_already_exists(self):
1222+ owner = self.factory.makePerson()
1223+ oci_project = self.factory.makeOCIProject()
1224+ self.factory.makeOCIRecipe(
1225+ owner=owner, registrant=owner, name="already exists",
1226+ oci_project=oci_project)
1227+
1228+ self.assertRaises(
1229+ DuplicateOCIRecipeName,
1230+ self.factory.makeOCIRecipe,
1231+ owner=owner,
1232+ registrant=owner,
1233+ name="already exists",
1234+ oci_project=oci_project)
1235+
1236+ def test_no_source_git_ref(self):
1237+ owner = self.factory.makePerson()
1238+ oci_project = self.factory.makeOCIProject()
1239+ recipe_set = getUtility(IOCIRecipeSet)
1240+ self.assertRaises(
1241+ NoSourceForOCIRecipe,
1242+ recipe_set.new,
1243+ name="no source",
1244+ registrant=owner,
1245+ owner=owner,
1246+ oci_project=oci_project,
1247+ git_ref=None,
1248+ build_file='build_file')
1249+
1250+ def test_no_source_build_file(self):
1251+ owner = self.factory.makePerson()
1252+ oci_project = self.factory.makeOCIProject()
1253+ recipe_set = getUtility(IOCIRecipeSet)
1254+ [git_ref] = self.factory.makeGitRefs()
1255+ self.assertRaises(
1256+ NoSourceForOCIRecipe,
1257+ recipe_set.new,
1258+ name="no source",
1259+ registrant=owner,
1260+ owner=owner,
1261+ oci_project=oci_project,
1262+ git_ref=git_ref,
1263+ build_file=None)
1264+
1265+ def test_getByName(self):
1266+ owner = self.factory.makePerson()
1267+ name = "a test recipe"
1268+ oci_project = self.factory.makeOCIProject()
1269+ target = self.factory.makeOCIRecipe(
1270+ owner=owner, registrant=owner, name=name, oci_project=oci_project)
1271+
1272+ for _ in range(3):
1273+ self.factory.makeOCIRecipe(oci_project=oci_project)
1274+
1275+ result = getUtility(IOCIRecipeSet).getByName(owner, oci_project, name)
1276+ self.assertEqual(target, result)
1277+
1278+ def test_getByName_missing(self):
1279+ owner = self.factory.makePerson()
1280+ oci_project = self.factory.makeOCIProject()
1281+ for _ in range(3):
1282+ self.factory.makeOCIRecipe(
1283+ owner=owner, registrant=owner, oci_project=oci_project)
1284+ self.assertRaises(
1285+ NoSuchOCIRecipe,
1286+ getUtility(IOCIRecipeSet).getByName,
1287+ owner=owner,
1288+ oci_project=oci_project,
1289+ name="missing")
1290diff --git a/lib/lp/registry/interfaces/ociprojectseries.py b/lib/lp/registry/interfaces/ociprojectseries.py
1291index 8ef9705..3019c22 100644
1292--- a/lib/lp/registry/interfaces/ociprojectseries.py
1293+++ b/lib/lp/registry/interfaces/ociprojectseries.py
1294@@ -34,7 +34,7 @@ class IOCIProjectSeriesView(Interface):
1295
1296 id = Int(title=_("ID"), required=True, readonly=True)
1297
1298- ociproject = Reference(
1299+ oci_project = Reference(
1300 IOCIProject,
1301 title=_("The OCI project that this series belongs to."),
1302 required=True, readonly=True)
1303diff --git a/lib/lp/registry/model/ociproject.py b/lib/lp/registry/model/ociproject.py
1304index 7dcfdea..c13ee66 100644
1305--- a/lib/lp/registry/model/ociproject.py
1306+++ b/lib/lp/registry/model/ociproject.py
1307@@ -110,7 +110,7 @@ class OCIProject(BugTargetBase, StormBase):
1308 status=SeriesStatus.DEVELOPMENT, date_created=DEFAULT):
1309 """See `IOCIProject`."""
1310 series = OCIProjectSeries(
1311- ociproject=self,
1312+ oci_project=self,
1313 name=name,
1314 summary=summary,
1315 registrant=registrant,
1316@@ -123,7 +123,7 @@ class OCIProject(BugTargetBase, StormBase):
1317 """See `IOCIProject`."""
1318 ret = IStore(OCIProjectSeries).find(
1319 OCIProjectSeries,
1320- OCIProjectSeries.ociproject == self
1321+ OCIProjectSeries.oci_project == self
1322 ).order_by(OCIProjectSeries.date_created)
1323 return ret
1324
1325diff --git a/lib/lp/registry/model/ociprojectseries.py b/lib/lp/registry/model/ociprojectseries.py
1326index 04a81e9..31ccac3 100644
1327--- a/lib/lp/registry/model/ociprojectseries.py
1328+++ b/lib/lp/registry/model/ociprojectseries.py
1329@@ -36,8 +36,8 @@ class OCIProjectSeries(StormBase):
1330
1331 id = Int(primary=True)
1332
1333- ociproject_id = Int(name='ociproject', allow_none=False)
1334- ociproject = Reference(ociproject_id, "OCIProject.id")
1335+ oci_project_id = Int(name='ociproject', allow_none=False)
1336+ oci_project = Reference(oci_project_id, "OCIProject.id")
1337
1338 name = Unicode(name="name", allow_none=False)
1339
1340@@ -52,13 +52,13 @@ class OCIProjectSeries(StormBase):
1341 status = DBEnum(
1342 name='status', allow_none=False, enum=SeriesStatus)
1343
1344- def __init__(self, ociproject, name, summary,
1345+ def __init__(self, oci_project, name, summary,
1346 registrant, status, date_created=DEFAULT):
1347 if not valid_name(name):
1348 raise InvalidName(
1349 "%s is not a valid name for an OCI project series." % name)
1350 self.name = name
1351- self.ociproject = ociproject
1352+ self.oci_project = oci_project
1353 self.summary = summary
1354 self.registrant = registrant
1355 self.status = status
1356diff --git a/lib/lp/registry/personmerge.py b/lib/lp/registry/personmerge.py
1357index 323dc2e..e71d5ff 100644
1358--- a/lib/lp/registry/personmerge.py
1359+++ b/lib/lp/registry/personmerge.py
1360@@ -15,6 +15,7 @@ from lp.code.interfaces.branchcollection import (
1361 IBranchCollection,
1362 )
1363 from lp.code.interfaces.gitcollection import IGitCollection
1364+from lp.oci.interfaces.ocirecipe import IOCIRecipeSet
1365 from lp.registry.interfaces.mailinglist import (
1366 IMailingListSet,
1367 MailingListStatus,
1368@@ -671,6 +672,24 @@ def _mergeSnap(cur, from_person, to_person):
1369 IStore(snaps[0]).flush()
1370
1371
1372+def _mergeOCIRecipe(cur, from_person, to_person):
1373+ # This shouldn't use removeSecurityProxy
1374+ oci_recipes = getUtility(IOCIRecipeSet).findByOwner(from_person)
1375+ existing_names = [
1376+ r.name for r in getUtility(IOCIRecipeSet).findByOwner(to_person)]
1377+ for recipe in oci_recipes:
1378+ new_name = recipe.name
1379+ count = 1
1380+ while new_name in existing_names:
1381+ new_name = '%s-%s' % (recipe.name, count)
1382+ count += 1
1383+ naked_recipe = removeSecurityProxy(recipe)
1384+ naked_recipe.owner = to_person
1385+ naked_recipe.name = new_name
1386+ if not oci_recipes.is_empty():
1387+ IStore(oci_recipes[0]).flush()
1388+
1389+
1390 def _purgeUnmergableTeamArtifacts(from_team, to_team, reviewer):
1391 """Purge team artifacts that cannot be merged, but can be removed."""
1392 # A team cannot have more than one mailing list.
1393@@ -898,6 +917,9 @@ def merge_people(from_person, to_person, reviewer, delete=False):
1394 _mergeSnap(cur, from_person, to_person)
1395 skip.append(('snap', 'owner'))
1396
1397+ _mergeOCIRecipe(cur, from_person, to_person)
1398+ skip.append(('ocirecipe', 'owner'))
1399+
1400 # Sanity check. If we have a reference that participates in a
1401 # UNIQUE index, it must have already been handled by this point.
1402 # We can tell this by looking at the skip list.
1403diff --git a/lib/lp/registry/tests/test_ociprojectseries.py b/lib/lp/registry/tests/test_ociprojectseries.py
1404index 3d4e6cf..f3da75c 100644
1405--- a/lib/lp/registry/tests/test_ociprojectseries.py
1406+++ b/lib/lp/registry/tests/test_ociprojectseries.py
1407@@ -48,7 +48,7 @@ class TestOCIProjectSeries(TestCaseWithFactory):
1408 oci_project, name, summary, registrant, status, date_created)
1409 self.assertThat(
1410 project_series, MatchesStructure.byEquality(
1411- ociproject=project_series.ociproject,
1412+ oci_project=project_series.oci_project,
1413 name=project_series.name,
1414 summary=project_series.summary,
1415 registrant=project_series.registrant,
1416diff --git a/lib/lp/registry/tests/test_personmerge.py b/lib/lp/registry/tests/test_personmerge.py
1417index 5a6a78f..890b421 100644
1418--- a/lib/lp/registry/tests/test_personmerge.py
1419+++ b/lib/lp/registry/tests/test_personmerge.py
1420@@ -18,6 +18,7 @@ from zope.security.proxy import removeSecurityProxy
1421 from lp.app.enums import InformationType
1422 from lp.app.interfaces.launchpad import ILaunchpadCelebrities
1423 from lp.code.interfaces.gitrepository import IGitRepositorySet
1424+from lp.oci.interfaces.ocirecipe import IOCIRecipeSet
1425 from lp.registry.interfaces.accesspolicy import (
1426 IAccessArtifactGrantSource,
1427 IAccessPolicyGrantSource,
1428@@ -660,6 +661,45 @@ class TestMergePeople(TestCaseWithFactory, KarmaTestMixin):
1429 self.assertIsNone(snaps[1].git_path)
1430 self.assertEqual(u'foo-1', snaps[1].name)
1431
1432+ def test_merge_moves_oci_recipes(self):
1433+ # When person/teams are merged, oci recipes owned by the from
1434+ # person are moved.
1435+ duplicate = self.factory.makePerson()
1436+ mergee = self.factory.makePerson()
1437+ self.factory.makeOCIRecipe(registrant=duplicate, owner=duplicate)
1438+ self._do_premerge(duplicate, mergee)
1439+ login_person(mergee)
1440+ duplicate, mergee = self._do_merge(duplicate, mergee)
1441+ self.assertEqual(
1442+ 1, getUtility(IOCIRecipeSet).findByOwner(mergee).count())
1443+
1444+ def test_merge_with_duplicated_oci_recipes(self):
1445+ # If both the from and to people have oci recipes with the same
1446+ # name, merging renames the duplicate from the from person's side.
1447+ duplicate = self.factory.makePerson()
1448+ mergee = self.factory.makePerson()
1449+ [ref] = self.factory.makeGitRefs()
1450+ [ref2] = self.factory.makeGitRefs()
1451+ self.factory.makeOCIRecipe(
1452+ registrant=duplicate, owner=duplicate, name=u'foo', git_ref=ref)
1453+ self.factory.makeOCIRecipe(
1454+ registrant=mergee, owner=mergee, name=u'foo', git_ref=ref2)
1455+ self._do_premerge(duplicate, mergee)
1456+ login_person(mergee)
1457+ duplicate, mergee = self._do_merge(duplicate, mergee)
1458+ oci_recipes = sorted(
1459+ getUtility(IOCIRecipeSet).findByOwner(mergee),
1460+ key=attrgetter("name"))
1461+ self.assertEqual(2, len(oci_recipes))
1462+ self.assertEqual(ref2, oci_recipes[0].git_ref)
1463+ self.assertEqual(ref2.repository, oci_recipes[0].git_repository)
1464+ self.assertEqual(ref2.path, oci_recipes[0].git_path)
1465+ self.assertEqual(u'foo', oci_recipes[0].name)
1466+ self.assertEqual(ref, oci_recipes[1].git_ref)
1467+ self.assertEqual(ref.repository, oci_recipes[1].git_repository)
1468+ self.assertEqual(ref.path, oci_recipes[1].git_path)
1469+ self.assertEqual(u'foo-1', oci_recipes[1].name)
1470+
1471
1472 class TestMergeMailingListSubscriptions(TestCaseWithFactory):
1473
1474diff --git a/lib/lp/security.py b/lib/lp/security.py
1475index c2d45ac..e31c65c 100644
1476--- a/lib/lp/security.py
1477+++ b/lib/lp/security.py
1478@@ -112,6 +112,8 @@ from lp.hardwaredb.interfaces.hwdb import (
1479 IHWSubmissionDevice,
1480 IHWVendorID,
1481 )
1482+from lp.oci.interfaces.ocirecipe import IOCIRecipe
1483+from lp.oci.interfaces.ocirecipebuild import IOCIRecipeBuild
1484 from lp.registry.enums import PersonVisibility
1485 from lp.registry.interfaces.announcement import IAnnouncement
1486 from lp.registry.interfaces.distribution import IDistribution
1487@@ -3469,4 +3471,42 @@ class EditOCIProjectSeries(AuthorizationBase):
1488 def checkAuthenticated(self, user):
1489 """Maintainers, drivers, and admins can drive projects."""
1490 return (user.in_admin or
1491- user.isDriver(self.obj.ociproject.pillar))
1492+ user.isDriver(self.obj.oci_project.pillar))
1493+
1494+
1495+class ViewOCIRecipe(AnonymousAuthorization):
1496+ """Anyone can view an `IOCIRecipe`."""
1497+ usedfor = IOCIRecipe
1498+
1499+
1500+class EditOCIRecipe(AuthorizationBase):
1501+ permission = 'launchpad.Edit'
1502+ usedfor = IOCIRecipe
1503+
1504+ def checkAuthenticated(self, user):
1505+ return (
1506+ user.isOwner(self.obj) or
1507+ user.in_commercial_admin or user.in_admin)
1508+
1509+
1510+class AdminOCIRecipe(AuthorizationBase):
1511+ """Restrict changing build settings on OCI recipes.
1512+
1513+ The security of the non-virtualised build farm depends on these
1514+ settings, so they can only be changed by "PPA"/commercial admins, or by
1515+ "PPA" self admins on OCI recipes that they can already edit.
1516+ """
1517+ permission = 'launchpad.Admin'
1518+ usedfor = IOCIRecipe
1519+
1520+ def checkAuthenticated(self, user):
1521+ if user.in_ppa_admin or user.in_commercial_admin or user.in_admin:
1522+ return True
1523+ return (
1524+ user.in_ppa_self_admins
1525+ and EditSnap(self.obj).checkAuthenticated(user))
1526+
1527+
1528+class ViewOCIRecipeBuild(AnonymousAuthorization):
1529+ """Anyone can view an `IOCIRecipe`."""
1530+ usedfor = IOCIRecipeBuild
1531diff --git a/lib/lp/testing/factory.py b/lib/lp/testing/factory.py
1532index 4009403..d0ecd18 100644
1533--- a/lib/lp/testing/factory.py
1534+++ b/lib/lp/testing/factory.py
1535@@ -157,6 +157,9 @@ from lp.hardwaredb.interfaces.hwdb import (
1536 IHWSubmissionDeviceSet,
1537 IHWSubmissionSet,
1538 )
1539+from lp.oci.interfaces.ocirecipe import IOCIRecipeSet
1540+from lp.oci.interfaces.ocirecipebuild import IOCIRecipeBuildSet
1541+from lp.oci.model.ocirecipe import OCIRecipeArch
1542 from lp.registry.enums import (
1543 BranchSharingPolicy,
1544 BugSharingPolicy,
1545@@ -4936,6 +4939,58 @@ class BareLaunchpadObjectFactory(ObjectFactory):
1546 oci_project = self.makeOCIProject(**kwargs)
1547 return oci_project.newSeries(name, summary, registrant)
1548
1549+ def makeOCIRecipe(self, name=None, registrant=None, owner=None,
1550+ oci_project=None, git_ref=None, description=None,
1551+ official=False, require_virtualized=True,
1552+ build_file=None, date_created=DEFAULT):
1553+ """Make a new OCIRecipe."""
1554+ if name is None:
1555+ name = self.getUniqueString(u"oci-recipe-name")
1556+ if registrant is None:
1557+ registrant = self.makePerson()
1558+ if description is None:
1559+ description = self.getUniqueString(u"oci-recipe-description")
1560+ if owner is None:
1561+ owner = self.makeTeam(members=[registrant])
1562+ if oci_project is None:
1563+ oci_project = self.makeOCIProject()
1564+ if git_ref is None:
1565+ [git_ref] = self.makeGitRefs()
1566+ if build_file is None:
1567+ build_file = self.getUniqueUnicode(u"build_file_for")
1568+ return getUtility(IOCIRecipeSet).new(
1569+ name=name,
1570+ registrant=registrant,
1571+ owner=owner,
1572+ oci_project=oci_project,
1573+ git_ref=git_ref,
1574+ build_file=build_file,
1575+ description=description,
1576+ official=official,
1577+ require_virtualized=require_virtualized,
1578+ date_created=date_created)
1579+
1580+ def makeOCIRecipeArch(self, recipe=None, processor=None):
1581+ """Make a new OCIRecipeArch."""
1582+ if recipe is None:
1583+ recipe = self.makeOCIRecipe()
1584+ if processor is None:
1585+ processor = self.makeProcessor()
1586+ return OCIRecipeArch(recipe, processor)
1587+
1588+ def makeOCIRecipeBuild(self, requester=None, recipe=None,
1589+ distro_arch_series=None, date_created=DEFAULT):
1590+ """Make a new OCIRecipeBuild."""
1591+ if requester is None:
1592+ requester = self.makePerson()
1593+ if distro_arch_series is None:
1594+ distro_arch_series = self.makeDistroArchSeries()
1595+ if recipe is None:
1596+ recipe = self.makeOCIRecipe()
1597+
1598+ return getUtility(IOCIRecipeBuildSet).new(
1599+ requester, recipe, distro_arch_series, date_created)
1600+
1601
1602 # Some factory methods return simple Python types. We don't add
1603 # security wrappers for them, as well as for objects created by

Subscribers

People subscribed via source and target branches

to status/vote changes: