Merge lp:~sergiusens/snapcraft/ubuntu-core into lp:~snappy-dev/snapcraft/core

[Sergio Schvezov]

Replace Ubuntu plugin with a builtin solution

This completely removes the Ubuntu plugin and introduces stage-packages for
both plugins and parts in one big bundle.

I'm not sure the big bundle is the best solution, but if required a plugin author could use repo.Ubuntu directly and independently from what a part does.

This is missing complete feature parity as the get_snap_files equivalent is missing and coming with filtering implementation exposed in the part.

As you can see, this model invalidates the wget example because it is a 1:1 conversion and nothing else, not sure if this was the desired outcome from the proposal.

[lp:~sergiusens/snapcraft/ubuntu-core]

155. By Sergio Schvezov on 2015-09-02

Making PLUGIN_STAGE_PACKAGES private so isbaseclass in the plugin loader doesn't get all confused

156. By Sergio Schvezov on 2015-09-02

merging trunk

157. By Sergio Schvezov on 2015-09-02

stage-package for parts and plugins separation of concerns

[Ted Gould (ted)]

I've been playing with this for the Python PIP case, and I think it gets
a little confusing. The problem is that for the PIP case we need to have
the packages installed before we can use them to download the packages
from the PIP repository. So by specifying the packages we can't then use
PIP in the pull phase for the plugin. The reason that we need to be in
the pull phase is that we want to be the stage that everything is
downloaded in, which clearly PIP does a fair amount of.

I also think that we need to maintain the package list as a singleton in
the core of snapcraft as if two parts want to use the same Ubuntu
package, that should be fine, but we can't install both of them into the
snap. This gets complicated with multiple repositories (having one part
and set of deps that are built on one distro version and another on a
different version; complicated but something that'll probably happen
when people use the wiki entries) but I think the singleton can handle
this by installing them to different directories in that case.

I think that this architecture means that also we can effectively create
a virtual dependency for each plugin that uses Ubuntu packages, so that
it can run all of its phases after the Ubuntu plugin installs its
packages. In that way the other plugins can use the packages that are
installed to evaluate their own phases.

I'm a little concerned with the idea that we'd install the packages
before running the pull phases of the other plugins. I think that this
creates a scenario where we can't do all the downloads in one phase.
This is a traditional setup for the builders to limit exposure to user
delivered code, so they can turn off networking while things are
building. But, I don't see a way around that. I think we'll have to have
that conversation with the LP folks as it comes along. We may have to
provide a "virtual" ubuntu plugin for non-Ubuntu repositories so that
the only execution before pull are done with packages in the official
repos. But, I don't think we need to solve that problem today :-)

[lp:~sergiusens/snapcraft/ubuntu-core]

158. By Sergio Schvezov on 2015-09-04

pull stage packages before pulling the code

159. By Sergio Schvezov on 2015-09-04

cleanup

[Leo Arias (elopio)]

Just a couple of comments on the diff.

And a question: how would you build a deb into a snap?

[Sergio Schvezov (sergiusens)]

> And a question: how would you build a deb into a snap?

The simple answer, which may support the third paragraph in the description, is that the new design doesn't allow for a 1:1 translation, so snaps based out of pure ubuntu packages are not allowed but instead used to support greater snaps.

[lp:~sergiusens/snapcraft/ubuntu-core]

160. By Sergio Schvezov on 2015-09-07

minor elopio improvement

[Michael Vogt (mvo)]

Thanks, this looks good! I have some inline comments, but nothing that blocks this from landing, a mix of nitpick and ideas for improvement. I like the idea of running it before pull and to allow the plugins to override it.

[Sergio Schvezov (sergiusens)]

@mvo, thanks for the review, I'll look into addressing your comments

[lp:~sergiusens/snapcraft/ubuntu-core]

161. By Sergio Schvezov on 2015-09-07

Simplifying __init__ for plugins and stage packages

[lp:~sergiusens/snapcraft/ubuntu-core]

162. By Sergio Schvezov on 2015-09-07

making manifest.txt useful again

163. By Sergio Schvezov on 2015-09-08

Merging trunk

[Michael Vogt (mvo)]

Nice, thanks for addressing my comments.

review: Approve

=== modified file 'debian/control'
--- debian/control	2015-09-02 07:48:09 +0000
+++ debian/control	2015-09-08 12:03:39 +0000
@@ -31,7 +31,6 @@
 Depends: ${misc:Depends},
          ${python3:Depends},
          bzr,
-         devscripts,
          dpkg-dev,
          git,
          python3-apt,

=== modified file 'examples/webcam-webui-snap/snapcraft.yaml'
--- examples/webcam-webui-snap/snapcraft.yaml	2015-08-27 21:33:56 +0000
+++ examples/webcam-webui-snap/snapcraft.yaml	2015-09-08 12:03:39 +0000
@@ -13,9 +13,9 @@
     golang-static-http:
         plugin: go-project
         source: git://github.com/mikix/golang-static-http
-    fswebcam:
-        plugin: ubuntu
     glue:
         plugin: copy
+        stage-packages:
+          - fswebcam
         files:
             webcam-webui: bin/webcam-webui

=== removed directory 'examples/wget-deb'
=== removed file 'examples/wget-deb/icon.png'
Binary files examples/wget-deb/icon.png	2015-08-27 21:33:56 +0000 and examples/wget-deb/icon.png	1970-01-01 00:00:00 +0000 differ
=== removed file 'examples/wget-deb/snapcraft.yaml'
--- examples/wget-deb/snapcraft.yaml	2015-08-27 21:33:56 +0000
+++ examples/wget-deb/snapcraft.yaml	1970-01-01 00:00:00 +0000
@@ -1,12 +0,0 @@
-name: wget
-version: 0
-vendor: Michael Terry <[e-mail]>
-binaries:
-    - name: usr/bin/wget
-summary: wget for snappy
-description: a basic example from ubuntu packages
-icon: icon.png
-
-parts:
-  wget:
-    plugin: ubuntu

=== modified file 'integration-tests/data/assemble/snapcraft.yaml'
--- integration-tests/data/assemble/snapcraft.yaml	2015-08-27 21:33:56 +0000
+++ integration-tests/data/assemble/snapcraft.yaml	2015-09-08 12:03:39 +0000
@@ -16,4 +16,4 @@
 
 parts:
     make-project:
-        source: .
+      source: .

=== modified file 'plugins/ant-project.yaml'
--- plugins/ant-project.yaml	2015-09-02 08:56:06 +0000
+++ plugins/ant-project.yaml	2015-09-08 12:03:39 +0000
@@ -8,5 +8,6 @@
   source-type:
   source-tag:
   source-branch:
+  stage-packages:
 build-tools:
   - ant

=== modified file 'plugins/autotools-project.yaml'
--- plugins/autotools-project.yaml	2015-07-22 18:45:52 +0000
+++ plugins/autotools-project.yaml	2015-09-08 12:03:39 +0000
@@ -1,10 +1,11 @@
 module: autotools_project
 build-tools: [autoconf, automake, autopoint]
 options:
-    source:
-        required: true
-    source-type:
-    source-tag:
-    source-branch:
-    configflags:
-        required: false
+  source:
+    required: true
+  source-type:
+  source-tag:
+  source-branch:
+  configflags:
+    required: false
+  stage-packages:

=== modified file 'plugins/cmake-project.yaml'
--- plugins/cmake-project.yaml	2015-07-22 18:45:52 +0000
+++ plugins/cmake-project.yaml	2015-09-08 12:03:39 +0000
@@ -8,3 +8,4 @@
     source-branch:
     configflags:
         required: false
+    stage-packages:

=== modified file 'plugins/copy.yaml'
--- plugins/copy.yaml	2015-07-17 06:40:24 +0000
+++ plugins/copy.yaml	2015-09-08 12:03:39 +0000
@@ -1,4 +1,4 @@
 options:
   files:
     required: true
-    
+  stage-packages:

=== modified file 'plugins/go-project.yaml'
--- plugins/go-project.yaml	2015-07-31 15:16:47 +0000
+++ plugins/go-project.yaml	2015-09-08 12:03:39 +0000
@@ -6,3 +6,4 @@
     required: true
   configflags:
     required: false
+  stage-packages:

=== modified file 'plugins/make-project.yaml'
--- plugins/make-project.yaml	2015-07-22 18:45:52 +0000
+++ plugins/make-project.yaml	2015-09-08 12:03:39 +0000
@@ -6,3 +6,4 @@
     source-type:
     source-tag:
     source-branch:
+    stage-packages:

=== modified file 'plugins/maven-project.yaml'
--- plugins/maven-project.yaml	2015-08-19 17:42:25 +0000
+++ plugins/maven-project.yaml	2015-09-08 12:03:39 +0000
@@ -9,3 +9,4 @@
   source-type:
   source-tag:
   source-branch:
+  stage-packages:

=== modified file 'plugins/python2-project.yaml'
--- plugins/python2-project.yaml	2015-08-03 18:21:58 +0000
+++ plugins/python2-project.yaml	2015-09-08 12:03:39 +0000
@@ -7,3 +7,4 @@
   source-type:
   source-tag:
   source-branch:
+  stage-packages:

=== modified file 'plugins/python3-project.yaml'
--- plugins/python3-project.yaml	2015-08-03 18:19:07 +0000
+++ plugins/python3-project.yaml	2015-09-08 12:03:39 +0000
@@ -7,3 +7,4 @@
   source-type:
   source-tag:
   source-branch:
+  stage-packages:

=== modified file 'plugins/tar-content.yaml'
--- plugins/tar-content.yaml	2015-07-17 19:39:00 +0000
+++ plugins/tar-content.yaml	2015-09-08 12:03:39 +0000
@@ -2,3 +2,4 @@
 options:
   source:
     required: true
+  stage-packages:

=== removed file 'plugins/ubuntu.yaml'
--- plugins/ubuntu.yaml	2015-08-05 18:40:06 +0000
+++ plugins/ubuntu.yaml	1970-01-01 00:00:00 +0000
@@ -1,6 +0,0 @@
-build-tools: [devscripts]
-options:
-  packages:
-    required: false
-  recommends:
-    required: false

=== modified file 'schema/snapcraft.yaml'
--- schema/snapcraft.yaml	2015-08-27 22:04:07 +0000
+++ schema/snapcraft.yaml	2015-09-08 12:03:39 +0000
@@ -79,10 +79,16 @@
           source:
             type: string
             description: path to the sources
-        # TODO To be enabled once the Ubuntu plugin is removed
+          stage-packages:
+            type: array
+            description: Ubuntu packages used to support the part.
+            minItems: 1
+            uniqueItems: true
+            items:
+              type: string
         # required:
-          # - plugin
-          # - source
+        # - plugin
+        # - source
     addtionalProperties: false
 required:
   - name

=== modified file 'setup.py'
--- setup.py	2015-08-25 20:32:40 +0000
+++ setup.py	2015-09-08 12:03:39 +0000
@@ -34,7 +34,7 @@
       url="https://launchpad.net/snapcraft",
       packages=['snapcraft',
                 'snapcraft.plugins'],
-      package_data={'snapcraft.plugins': ['manifest.txt']},
+      package_data={'snapcraft': ['manifest.txt']},
       scripts=['bin/snapcraft'],
       data_files=[
           ('share/snapcraft/plugins', ['plugins/' + x for x in os.listdir('plugins')]),

=== modified file 'snapcraft/__init__.py'
--- snapcraft/__init__.py	2015-08-31 13:49:40 +0000
+++ snapcraft/__init__.py	2015-09-08 12:03:39 +0000
@@ -19,6 +19,7 @@
 
 import snapcraft.common
 import snapcraft.sources
+import snapcraft.repo
 
 
 logger = logging.getLogger(__name__)
@@ -26,11 +27,16 @@
 
 class BasePlugin:
 
+    @property
+    def PLUGIN_STAGE_PACKAGES(self):
+        return getattr(self, '_PLUGIN_STAGE_PACKAGES', [])
+
     def __init__(self, name, options):
         self.name = name
         self.options = options
         self.sourcedir = os.path.join(os.getcwd(), "parts", self.name, "src")
         self.builddir = os.path.join(os.getcwd(), "parts", self.name, "build")
+        self.ubuntudir = os.path.join(os.getcwd(), "parts", self.name, 'ubuntu')
         self.installdir = os.path.join(os.getcwd(), "parts", self.name, "install")
         self.stagedir = os.path.join(os.getcwd(), "stage")
         self.snapdir = os.path.join(os.getcwd(), "snap")
@@ -91,6 +97,13 @@
     def makedirs(self, d):
         os.makedirs(d, exist_ok=True)
 
+    def setup_stage_packages(self):
+        ubuntu = snapcraft.repo.Ubuntu(self.ubuntudir)
+        part_stage_packages = getattr(self.options, 'stage_packages', [])
+        if self.PLUGIN_STAGE_PACKAGES or part_stage_packages:
+            ubuntu.get(self.PLUGIN_STAGE_PACKAGES + part_stage_packages)
+            ubuntu.unpack(self.installdir)
+
 
 def _get_source_handler(source_type, source):
     if source_type is None:

=== renamed file 'snapcraft/plugins/manifest.txt' => 'snapcraft/manifest.txt'
=== modified file 'snapcraft/plugin.py'
--- snapcraft/plugin.py	2015-08-28 15:38:04 +0000
+++ snapcraft/plugin.py	2015-09-08 12:03:39 +0000
@@ -24,6 +24,7 @@
 
 import snapcraft
 from snapcraft import common
+from snapcraft import repo
 
 
 logger = logging.getLogger(__name__)
@@ -57,6 +58,7 @@
         parts_dir = os.path.join(os.getcwd(), 'parts')
         self.sourcedir = os.path.join(parts_dir, part_name, 'src')
         self.builddir = os.path.join(parts_dir, part_name, 'build')
+        self.ubuntudir = os.path.join(parts_dir, part_name, 'ubuntu')
         self.installdir = os.path.join(parts_dir, part_name, 'install')
         self.stagedir = os.path.join(os.getcwd(), 'stage')
         self.snapdir = os.path.join(os.getcwd(), 'snap')
@@ -127,7 +129,7 @@
     def makedirs(self):
         dirs = [
             self.sourcedir, self.builddir, self.installdir, self.stagedir,
-            self.snapdir
+            self.snapdir, self.ubuntudir
         ]
         for d in dirs:
             os.makedirs(d, exist_ok=True)
@@ -163,11 +165,25 @@
         if not self.should_stage_run('pull', force):
             return True
         self.makedirs()
-        if self.code and hasattr(self.code, 'pull'):
+
+        run_setup_stage_packages = self.code and hasattr(self.code, 'setup_stage_packages')
+        run_pull = self.code and hasattr(self.code, 'pull')
+
+        if run_setup_stage_packages or run_pull:
             self.notify_stage("Pulling")
+
+        if run_setup_stage_packages:
+            try:
+                self.code.setup_stage_packages()
+            except repo.PackageNotFoundError as e:
+                logger.error(e.message)
+                return False
+
+        if run_pull:
             if not getattr(self.code, 'pull')():
                 return False
-            self.mark_done('pull')
+
+        self.mark_done('pull')
         return True
 
     def build(self, force=False):
@@ -178,7 +194,8 @@
             self.notify_stage("Building")
             if not getattr(self.code, 'build')():
                 return False
-            self.mark_done('build')
+
+        self.mark_done('build')
         return True
 
     def stage(self, force=False):

=== modified file 'snapcraft/plugins/jdk.py'
--- snapcraft/plugins/jdk.py	2015-08-05 15:39:18 +0000
+++ snapcraft/plugins/jdk.py	2015-09-08 12:03:39 +0000
@@ -15,28 +15,18 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import snapcraft
-from snapcraft.plugins.ubuntu import UbuntuPlugin
 
 
 class JdkPlugin(snapcraft.BasePlugin):
 
-    def __init__(self, name, options):
-        super().__init__(name, options)
-
-        class UbuntuOptions:
-            packages = ["default-jdk"]
-        self.ubuntu = UbuntuPlugin(name, UbuntuOptions())
-
-    def pull(self):
-        return self.ubuntu.pull()
-
-    def build(self):
-        return self.ubuntu.build()
+    _PLUGIN_STAGE_PACKAGES = [
+        'default-jdk',
+    ]
 
     def env(self, root):
-        return self.ubuntu.env(root) + \
-            ['JAVA_HOME=%s/usr/lib/jvm/default-java' % root,
-             'PATH=%s/usr/lib/jvm/default-java/bin:%s/usr/lib/jvm/default-java/jre/bin:$PATH' % (root, root)]
+        return ['JAVA_HOME=%s/usr/lib/jvm/default-java' % root,
+                'PATH=%s/usr/lib/jvm/default-java/bin:'
+                '%s/usr/lib/jvm/default-java/jre/bin:$PATH' % (root, root)]
 
     def snap_files(self):
         # Cut out jdk bits (jre bits are in default-java/jre)

=== modified file 'snapcraft/plugins/python2.py'
--- snapcraft/plugins/python2.py	2015-08-05 15:39:18 +0000
+++ snapcraft/plugins/python2.py	2015-09-08 12:03:39 +0000
@@ -15,27 +15,14 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import snapcraft
-from snapcraft.plugins import ubuntu
 
 
 class Python2Plugin(snapcraft.BasePlugin):
 
-    def __init__(self, name, options):
-        super().__init__(name, options)
-
-        class UbuntuOptions:
-            packages = ["python-dev"]
-        self.ubuntu = ubuntu.UbuntuPlugin(name, UbuntuOptions())
+    _PLUGIN_STAGE_PACKAGES = [
+        'python-dev',
+    ]
 
     # note that we don't need to set PYTHONHOME here,
     # python discovers this automatically from it installed
     # location.  And PATH is automatically set by snapcraft.
-
-    def pull(self):
-        return self.ubuntu.pull()
-
-    def build(self):
-        return self.ubuntu.build()
-
-    def snap_files(self):
-        return self.ubuntu.snap_files()

=== modified file 'snapcraft/plugins/python3.py'
--- snapcraft/plugins/python3.py	2015-08-05 15:39:18 +0000
+++ snapcraft/plugins/python3.py	2015-09-08 12:03:39 +0000
@@ -15,29 +15,16 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import snapcraft
-from snapcraft.plugins.ubuntu import UbuntuPlugin
 
 
 class Python3Plugin(snapcraft.BasePlugin):
 
-    def __init__(self, name, options):
-        super().__init__(name, options)
-
-        class Py3Options:
-            packages = ["python3-dev"]
-        self.ubuntu = UbuntuPlugin(name, Py3Options())
+    _PLUGIN_STAGE_PACKAGES = [
+        'python3-dev',
+    ]
 
     # note that we don't need to set PYTHONHOME here,
     # python discovers this automatically from it installed
     # location, see https://code.launchpad.net/~mvo/snapcraft/python3-project/+merge/264521/comments/664308
     #
     # PATH is automatically set by snapcraft
-
-    def pull(self):
-        return self.ubuntu.pull()
-
-    def build(self):
-        return self.ubuntu.build()
-
-    def snap_files(self):
-        return self.ubuntu.snap_files()

=== modified file 'snapcraft/plugins/qml.py'
--- snapcraft/plugins/qml.py	2015-08-26 04:20:25 +0000
+++ snapcraft/plugins/qml.py	2015-09-08 12:03:39 +0000
@@ -15,61 +15,50 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
+import snapcraft
 import snapcraft.common
 
-from snapcraft.plugins.ubuntu import UbuntuPlugin
-
 
 class QmlPlugin(snapcraft.BasePlugin):
 
-    def __init__(self, name, options):
-        super().__init__(name, options)
-
-        class QmlPackageOptions:
-            packages = [
-                "qmlscene",
-                "qtdeclarative5-qtmir-plugin",
-                "mir-graphics-drivers-desktop",
-                "qtubuntu-desktop",
-                "ttf-ubuntu-font-family",
-                # if there's a metapackage for these, please swap it in here:
-                "qml-module-qt-labs-folderlistmodel",
-                "qml-module-qt-labs-settings",
-                "qml-module-qt-websockets",
-                "qml-module-qtfeedback",
-                "qml-module-qtgraphicaleffects",
-                "qml-module-qtlocation",
-                "qml-module-qtmultimedia",
-                "qml-module-qtorganizer",
-                "qml-module-qtpositioning",
-                "qml-module-qtqml-models2",
-                "qml-module-qtqml-statemachine",
-                "qml-module-qtquick-controls",
-                "qml-module-qtquick-dialogs",
-                "qml-module-qtquick-layouts",
-                "qml-module-qtquick-localstorage",
-                "qml-module-qtquick-particles2",
-                "qml-module-qtquick-privatewidgets",
-                "qml-module-qtquick-window2",
-                "qml-module-qtquick-xmllistmodel",
-                "qml-module-qtquick2",
-                "qml-module-qtsensors",
-                "qml-module-qtsysteminfo",
-                "qml-module-qttest",
-                "qml-module-qtwebkit",
-                "qml-module-ubuntu-connectivity",
-                "qml-module-ubuntu-onlineaccounts",
-                "qml-module-ubuntu-onlineaccounts-client",
-            ]
-            recommends = False
-
-        self.ubuntu = UbuntuPlugin(name, QmlPackageOptions())
-
-    def pull(self):
-        return self.ubuntu.pull()
+    _PLUGIN_STAGE_PACKAGES = [
+        "qmlscene",
+        "qtdeclarative5-qtmir-plugin",
+        "mir-graphics-drivers-desktop",
+        "qtubuntu-desktop",
+        "ttf-ubuntu-font-family",
+        # if there's a metapackage for these, please swap it in here:
+        "qml-module-qt-labs-folderlistmodel",
+        "qml-module-qt-labs-settings",
+        "qml-module-qt-websockets",
+        "qml-module-qtfeedback",
+        "qml-module-qtgraphicaleffects",
+        "qml-module-qtlocation",
+        "qml-module-qtmultimedia",
+        "qml-module-qtorganizer",
+        "qml-module-qtpositioning",
+        "qml-module-qtqml-models2",
+        "qml-module-qtqml-statemachine",
+        "qml-module-qtquick-controls",
+        "qml-module-qtquick-dialogs",
+        "qml-module-qtquick-layouts",
+        "qml-module-qtquick-localstorage",
+        "qml-module-qtquick-particles2",
+        "qml-module-qtquick-privatewidgets",
+        "qml-module-qtquick-window2",
+        "qml-module-qtquick-xmllistmodel",
+        "qml-module-qtquick2",
+        "qml-module-qtsensors",
+        "qml-module-qtsysteminfo",
+        "qml-module-qttest",
+        "qml-module-qtwebkit",
+        "qml-module-ubuntu-connectivity",
+        "qml-module-ubuntu-onlineaccounts",
+        "qml-module-ubuntu-onlineaccounts-client",
+    ]
 
     def snap_files(self):
-        include, exclude = self.ubuntu.snap_files()
+        include, exclude = ['*'], []
         include.append('./etc/xdg/qtchooser/snappy-qt5.conf')
         return (include, exclude)
 
@@ -84,12 +73,11 @@
         return True
 
     def build(self):
-        return self.ubuntu.build() and self.build_qt_config()
+        return self.build_qt_config()
 
     def env(self, root):
         arch = snapcraft.common.get_arch_triplet()
-        envs = self.ubuntu.env(root)
-        envs.extend([
+        return [
             "LD_LIBRARY_PATH=%s/usr/lib/%s:$LD_LIBRARY_PATH" % (root, arch),
             # Mir config
             "MIR_SOCKET=/run/mir_socket",
@@ -118,5 +106,4 @@
             # Font Config
             "FONTCONFIG_PATH=%s/etc/fonts/config.d" % root,
             "FONTCONFIG_FILE=%s/etc/fonts/fonts.conf" % root,
-        ])
-        return envs
+        ]

=== removed directory 'snapcraft/plugins/tests'
=== removed file 'snapcraft/plugins/tests/__init__.py'
=== removed file 'snapcraft/plugins/tests/test_ubuntu.py'
--- snapcraft/plugins/tests/test_ubuntu.py	2015-08-06 01:00:44 +0000
+++ snapcraft/plugins/tests/test_ubuntu.py	1970-01-01 00:00:00 +0000
@@ -1,61 +0,0 @@
-# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
-#
-# Copyright (C) 2015 Canonical Ltd
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import os
-import tempfile
-from unittest.mock import (
-    Mock,
-)
-
-from snapcraft.plugins.ubuntu import UbuntuPlugin
-
-from snapcraft.tests import TestCase
-
-
-class TestUbuntu(TestCase):
-
-    def test_fix_symlinks(self):
-        tempdirObj = tempfile.TemporaryDirectory()
-        self.addCleanup(tempdirObj.cleanup)
-        tempdir = tempdirObj.name
-
-        os.makedirs(tempdir + '/a')
-        open(tempdir + '/1', mode='w').close()
-
-        os.symlink('a', tempdir + '/rel-to-a')
-        os.symlink('/a', tempdir + '/abs-to-a')
-        os.symlink('/b', tempdir + '/abs-to-b')
-        os.symlink('1', tempdir + '/rel-to-1')
-        os.symlink('/1', tempdir + '/abs-to-1')
-
-        options = Mock()
-        options.packages = ['test']
-        ubuntu = UbuntuPlugin('ubuntu', options)
-        ubuntu.fix_symlinks(debdir=tempdir)
-
-        self.assertEqual(os.readlink(tempdir + '/rel-to-a'), 'a')
-        self.assertEqual(os.readlink(tempdir + '/abs-to-a'), 'a')
-        self.assertEqual(os.readlink(tempdir + '/abs-to-b'), '/b')
-        self.assertEqual(os.readlink(tempdir + '/rel-to-1'), '1')
-        self.assertEqual(os.readlink(tempdir + '/abs-to-1'), '1')
-
-    def test_recommends_ignored_properly(self):
-        class Options:
-            packages = ['my-excellent-package']
-        ubuntu = UbuntuPlugin('myplug', Options())
-
-        self.assertTrue('my-excellent-package' in ubuntu.included_packages)
-        self.assertEqual(ubuntu.recommends, None)

=== removed file 'snapcraft/plugins/ubuntu.py'
--- snapcraft/plugins/ubuntu.py	2015-08-26 08:52:09 +0000
+++ snapcraft/plugins/ubuntu.py	1970-01-01 00:00:00 +0000
@@ -1,132 +0,0 @@
-# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
-#
-# Copyright (C) 2015 Canonical Ltd
-#
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License version 3 as
-# published by the Free Software Foundation.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program.  If not, see <http://www.gnu.org/licenses/>.
-
-import itertools
-import logging
-import os
-import subprocess
-import sys
-
-import apt
-
-import snapcraft.common
-
-
-logger = logging.getLogger(__name__)
-
-
-class UbuntuPlugin(snapcraft.BasePlugin):
-
-    def __init__(self, name, options):
-        super().__init__(name, options)
-        self.downloadable_packages = []
-        self.included_packages = []
-        if options.packages:
-            self.included_packages.extend(options.packages)
-        else:
-            # User didn't specify a package, use the part name
-            if name == 'ubuntu':
-                logger.error('Part %s needs either a package option or a name', name)
-                sys.exit(1)
-            self.included_packages.append(name)
-
-        self.recommends = getattr(options, 'recommends', None)
-
-    def pull(self):
-        self.downloadable_packages = self.get_all_dep_packages(self.included_packages)
-        return self.download_debs(self.downloadable_packages)
-
-    def build(self):
-        if not self.downloadable_packages:
-            self.downloadable_packages = self.get_all_dep_packages(self.included_packages)
-        return self.unpack_debs(self.downloadable_packages, self.installdir) \
-            and self.fix_symlinks(debdir=self.installdir)
-
-    def snap_files(self):
-        return (['*'], ['usr/include', 'lib/*/*.a', 'usr/lib/*/*.a', 'usr/share/doc', 'usr/share/man', 'usr/share/lintian', 'usr/share/doc-base'])
-
-    def get_all_dep_packages(self, packages):
-        cache = apt.Cache()
-        alldeps = set()
-        manifestdeps = set()
-        skipped = set()
-
-        with open(os.path.abspath(os.path.join(__file__, '..', 'manifest.txt'))) as f:
-            for line in f:
-                pkg = line.strip()
-                if pkg in cache:
-                    manifestdeps.add(pkg)
-
-        def add_deps(pkgs):
-            for p in pkgs:
-                if p in alldeps:
-                    continue
-                if p in manifestdeps and p not in packages:
-                    skipped.add(p)
-                    continue
-                try:
-                    deps = set()
-                    candidatePkg = cache[p].candidate
-                    deps = candidatePkg.dependencies
-                    if self.recommends:
-                        deps += candidatePkg.recommends
-                    alldeps.add(p)
-                    add_deps([x[0].name for x in deps])
-                except:
-                    pass
-
-        add_deps(packages)
-
-        exit = False
-        for p in packages:
-            if p not in alldeps:
-                exit = True
-                logger.error("Package '%s' not recognized", p)
-        if exit:
-            sys.exit(1)
-
-        return sorted(alldeps)
-
-    def download_debs(self, pkgs, debdir=None):
-        debdir = debdir or self.builddir
-        if pkgs:
-            return self.run(['dget'] + pkgs, cwd=debdir, stdout=subprocess.DEVNULL)
-        else:
-            return True
-
-    def unpack_debs(self, pkgs, targetDir, debdir=None):
-        debdir = debdir or self.builddir
-        for p in pkgs:
-            if not self.run(['dpkg-deb', '--extract', p + '_*.deb', targetDir], cwd=debdir):
-                return False
-        return True
-
-    def fix_symlinks(self, debdir=None):
-        """Sometimes debs will contain absolute symlinks (e.g. if the relative
-           path would go all the way to root, they just do absolute).  We can't
-           have that, so instead clean those absolute symlinks."""
-        debdir = debdir or self.builddir
-        for root, dirs, files in os.walk(debdir):
-            # Symlinks to directories will be in dirs, while symlinks to
-            # non-directories will be in files.
-            for entry in itertools.chain(files, dirs):
-                path = os.path.join(root, entry)
-                if os.path.islink(path) and os.path.isabs(os.readlink(path)):
-                    target = os.path.join(debdir, os.readlink(path)[1:])
-                    if os.path.exists(target):
-                        os.remove(path)
-                        os.symlink(os.path.relpath(target, root), path)
-        return True

=== added file 'snapcraft/repo.py'
--- snapcraft/repo.py	1970-01-01 00:00:00 +0000
+++ snapcraft/repo.py	2015-09-08 12:03:39 +0000
@@ -0,0 +1,132 @@
+# -*- Mode:Python; indent-tabs-mode:nil; tab-width:4 -*-
+#
+# Copyright (C) 2015 Canonical Ltd
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 3 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+import apt
+import glob
+import itertools
+import os
+import subprocess
+
+
+class PackageNotFoundError(Exception):
+
+    @property
+    def message(self):
+        return 'The Ubuntu package \'%s\' was not found' % self.package_name
+
+    def __init__(self, package_name):
+        self.package_name = package_name
+
+
+class UnpackError(Exception):
+
+    @property
+    def message(self):
+        return 'Error while provisioning \'%s\'' % self.package_name
+
+    def __init__(self, package_name):
+        self.package_name = package_name
+
+
+class Ubuntu:
+
+    def __init__(self, download_dir, recommends=False):
+        self.apt_cache = apt.Cache()
+        self.manifest_dep_names = self._manifest_dep_names()
+        self.recommends = recommends
+        self.download_dir = download_dir
+
+    def get(self, package_names):
+        # TODO cleanup download_dir for clean gets and unpacks
+        self.all_dep_names = set()
+
+        all_package_names = self._compute_deps(package_names)
+
+        for pkg in all_package_names:
+            self.apt_cache[pkg].candidate.fetch_binary(destdir=self.download_dir)
+
+        return all_package_names
+
+    def unpack(self, root_dir):
+        pkgs_abs_path = glob.glob(os.path.join(self.download_dir, '*.deb'))
+        for pkg in pkgs_abs_path:
+            # TODO needs elegance and error control
+            try:
+                subprocess.check_call(['dpkg-deb', '--extract', pkg, root_dir])
+            except subprocess.CalledProcessError:
+                raise UnpackError(pkg)
+
+        _fix_symlinks(root_dir)
+
+    def _manifest_dep_names(self):
+        manifest_dep_names = set()
+
+        with open(os.path.abspath(os.path.join(__file__, '..', 'manifest.txt'))) as f:
+            for line in f:
+                pkg = line.strip()
+                if pkg in self.apt_cache:
+                    manifest_dep_names.add(pkg)
+
+        return manifest_dep_names
+
+    def _compute_deps(self, package_names):
+        self._add_deps(package_names)
+
+        for pkg in package_names:
+            if pkg not in self.all_dep_names:
+                raise PackageNotFoundError(pkg)
+
+        return sorted(self.all_dep_names)
+
+    def _add_deps(self, package_names):
+        def add_deps(packages):
+            for pkg in packages:
+                # Remove the :any in packages
+                # TODO support multiarch
+                pkg = pkg.rsplit(':', 1)[0]
+                if pkg in self.all_dep_names:
+                    continue
+                if pkg in self.manifest_dep_names and pkg not in package_names:
+                    continue
+                deps = set()
+                try:
+                    candidate_pkg = self.apt_cache[pkg].candidate
+                except KeyError:
+                    raise PackageNotFoundError(pkg)
+                deps = candidate_pkg.dependencies
+                if self.recommends:
+                    deps += candidate_pkg.recommends
+                self.all_dep_names.add(pkg)
+                add_deps([x[0].name for x in deps])
+        add_deps(package_names)
+
+
+def _fix_symlinks(debdir):
+    '''
+    Sometimes debs will contain absolute symlinks (e.g. if the relative
+    path would go all the way to root, they just do absolute).  We can't
+    have that, so instead clean those absolute symlinks.
+    '''
+    for root, dirs, files in os.walk(debdir):
+        # Symlinks to directories will be in dirs, while symlinks to
+        # non-directories will be in files.
+        for entry in itertools.chain(files, dirs):
+            path = os.path.join(root, entry)
+            if os.path.islink(path) and os.path.isabs(os.readlink(path)):
+                target = os.path.join(debdir, os.readlink(path)[1:])
+                if os.path.exists(target):
+                    os.remove(path)
+                    os.symlink(os.path.relpath(target, root), path)

=== renamed file 'snapcraft/tests/test_ubuntu_plugin.py' => 'snapcraft/tests/test_repo.py'
--- snapcraft/tests/test_ubuntu_plugin.py	2015-08-05 20:19:47 +0000
+++ snapcraft/tests/test_repo.py	2015-09-08 12:03:39 +0000
@@ -14,26 +14,42 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-import logging
-
-import fixtures
-
+import os
+import tempfile
+
+from snapcraft import repo
 from snapcraft import tests
-from snapcraft.plugins import ubuntu
-
-
-class UbuntuPluginTestCase(tests.TestCase):
-
-    def test_get_all_dep_packages_with_unrecognized_package(self):
-        fake_logger = fixtures.FakeLogger(level=logging.ERROR)
-        self.useFixture(fake_logger)
-
-        test_options = type('obj', (object,), {'packages': False, 'recommends': False})
-        plugin = ubuntu.UbuntuPlugin('test_plugin', test_options)
-
-        with self.assertRaises(SystemExit) as raised:
-            plugin.get_all_dep_packages(['test_package'])
-
-        self.assertEqual(raised.exception.code, 1, 'Wrong exit code returned.')
-        self.assertEqual(
-            "Package 'test_package' not recognized\n", fake_logger.output)
+
+
+class UbuntuTestCase(tests.TestCase):
+
+    def test_unrecognized_package_raises_exception(self):
+        ubuntu = repo.Ubuntu('download_dir')
+
+        with self.assertRaises(repo.PackageNotFoundError) as raised:
+            ubuntu.get(['test_package'])
+
+        expected_message = 'The Ubuntu package \'test_package\' was not found'
+        self.assertEqual(raised.exception.message, expected_message)
+
+    def test_fix_symlinks(self):
+        tempdirObj = tempfile.TemporaryDirectory()
+        self.addCleanup(tempdirObj.cleanup)
+        tempdir = tempdirObj.name
+
+        os.makedirs(tempdir + '/a')
+        open(tempdir + '/1', mode='w').close()
+
+        os.symlink('a', tempdir + '/rel-to-a')
+        os.symlink('/a', tempdir + '/abs-to-a')
+        os.symlink('/b', tempdir + '/abs-to-b')
+        os.symlink('1', tempdir + '/rel-to-1')
+        os.symlink('/1', tempdir + '/abs-to-1')
+
+        repo._fix_symlinks(debdir=tempdir)
+
+        self.assertEqual(os.readlink(tempdir + '/rel-to-a'), 'a')
+        self.assertEqual(os.readlink(tempdir + '/abs-to-a'), 'a')
+        self.assertEqual(os.readlink(tempdir + '/abs-to-b'), '/b')
+        self.assertEqual(os.readlink(tempdir + '/rel-to-1'), '1')
+        self.assertEqual(os.readlink(tempdir + '/abs-to-1'), '1')

=== modified file 'snapcraft/tests/test_yaml.py'
--- snapcraft/tests/test_yaml.py	2015-08-27 22:09:55 +0000
+++ snapcraft/tests/test_yaml.py	2015-09-08 12:03:39 +0000
@@ -57,12 +57,13 @@
 icon: my-icon.png
 
 parts:
-  ubuntu:
-    packages: [fswebcam]
+  part1:
+    plugin: go
+    stage-packages: [fswebcam]
 """)
         snapcraft.yaml.Config()
-        mock_loadPlugin.assert_called_with("ubuntu", "ubuntu", {
-            "packages": ["fswebcam"],
+        mock_loadPlugin.assert_called_with("part1", "go", {
+            "stage-packages": ["fswebcam"],
         })
 
     def test_config_raises_on_missing_snapcraft_yaml(self):
@@ -92,10 +93,10 @@
 
 parts:
   p1:
-    plugin: ubuntu
+    plugin: go
     after: [p2]
   p2:
-    plugin: ubuntu
+    plugin: go
     after: [p1]
 """)
         with self.assertRaises(SystemExit) as raised:
@@ -117,8 +118,9 @@
 icon: my-icon.png
 
 parts:
-  ubuntu:
-    packages: [fswebcam]
+  part1:
+    plugin: go
+    stage-packages: [fswebcam]
 """)
         with self.assertRaises(SystemExit) as raised:
             snapcraft.yaml.Config()
@@ -141,8 +143,9 @@
 icon: my-icon.png
 
 parts:
-  ubuntu:
-    packages: [fswebcam]
+  part1:
+    plugin: go
+    stage-packages: [fswebcam]
 """)
         with self.assertRaises(SystemExit) as raised:
             snapcraft.yaml.Config()
@@ -165,8 +168,9 @@
 icon: my-icon.png
 
 parts:
-  ubuntu:
-    packages: [fswebcam]
+  part1:
+    plugin: go
+    stage-packages: [fswebcam]
 """)
         with self.assertRaises(SystemExit) as raised:
             snapcraft.yaml.Config()
@@ -188,8 +192,9 @@
 icon: my-icon.png
 
 parts:
-  ubuntu:
-    packages: [fswebcam]
+  part1:
+    plugin: go
+    stage-packages: [fswebcam]
 """)
         with self.assertRaises(SystemExit) as raised:
             snapcraft.yaml.Config()
@@ -220,7 +225,7 @@
             'icon': 'my-icon.png',
             'parts': {
                 'part1': {
-                    'type': 'project',
+                    'plugin': 'project',
                 },
             },
         }