Merge lp:~mvo/aptdaemon/support-for-whitelisted-repositories into lp:aptdaemon

[Michael Vogt]

This branch implements the whitelist feature discussed in #1035207

It adds a new type of policykit priv "org.debian.apt.install-packages-from-whitelisted-repo"
that can be overriden by to allow passwordless installs. If its not
overriden it has the same requirements as install-remove-packages

The whitelist is defined via /etc/aptdaemon/repository-whitelist.cfg

[webapps test]
origin = Ubuntu
component = universe
pkgnames = 2vcard

[more whitelist]
origin = Ubuntu
component = main
pkgnames = ^unity.*

[lp:~mvo/aptdaemon/support-for-whitelisted-repositories]

861. By Michael Vogt on 2012-08-24

aptdaemon/worker.py: more paranoia

[Martin Pitt (pitti)]

As a general nitpick, I'd prefer to rename "whitelisted" in any public API/properties etc. to "passwordless" or something similarly meaningful, to point out what this list actually does. This especially applies to the conffile, as renaming it later is painful.

The test case does not actually verify that the expected PK privilege is being used, as it only goes until the simulation stage (or did I get that wrong?). You could tell the mock polkitd to permit org.debian.apt.install-packages-from-whitelisted-repo and nothing else, and verify that installation of silly-* works, but other-pkg fails due to a permission error. It would also be nice to then try to remove the package again and verify that for this case (i. e. any but "install" it requires the "full" privilege.

I also think read_repository_whitelist() should become more robust against errors in the conffile. ConfigParser defaults to "strict" and thus excepts out if there a duplicate keys, etc. We don't want aptdaemon to crash on a ConfigParser.* exception; I think it should just be logged and it should return an empty set then. (Also easy to testcase).

Otherwise this looks good to me. Thanks!

review: Needs Fixing

[lp:~mvo/aptdaemon/support-for-whitelisted-repositories]

862. By Michael Vogt on 2012-08-28

make config parser more robust against errors in the config file and add tests

863. By Michael Vogt on 2012-08-28

small cleanup/comment updates

864. By Michael Vogt on 2012-08-28

move the whitelsit releated tests into a single file

865. By Michael Vogt on 2012-08-28

test that the right polkit action is used for whitelisted packages

866. By Michael Vogt on 2012-08-28

test that non-whitelisted package fails to install with aptdaemon.policykit1.PK_ACTION_INSTALL_PACKAGES_FROM_WHITELISTED_REPO

867. By Michael Vogt on 2012-08-28

aptdaemon/test.py: make the debug in start_session_aptd option to have less noise

868. By Michael Vogt on 2012-08-28

documation updates and ensure that the is install-from-whitelist only applies to install

[Michael Vogt (mvo)]

On Mon, Aug 27, 2012 at 01:55:22PM -0000, Martin Pitt wrote:
> Review: Needs Fixing

Thanks for the review.

> As a general nitpick, I'd prefer to rename "whitelisted" in any public API/properties etc. to "passwordless" or something similarly meaningful, to point out what this list actually does. This especially applies to the conffile, as renaming it later is painful.

Right. I tend to agree but its not neseccarily passwordless, i.e. the
default implementation of this has auth_admin. Its just easier to
override this via e.g. the policykit-desktop-privileges or a special
webapps packages that does the same. Maybe a better name would be
"potentially_passordless" instead of "whitelisted"? But that gets a
bit long :/ Any hints appreciated.

> The test case does not actually verify that the expected PK privilege is being used, as it only goes until the simulation stage (or did I get that wrong?). You could tell the mock polkitd to permit org.debian.apt.install-packages-from-whitelisted-repo and nothing else, and verify that installation of silly-* works, but other-pkg fails due to a permission error. It would also be nice to then try to remove the package again and verify that for this case (i. e. any but "install" it requires the "full" privilege.

Indeed good points, I added tests for that now too and also
consolidated the whitelist test stuff into a single file.

> I also think read_repository_whitelist() should become more robust against errors in the conffile. ConfigParser defaults to "strict" and thus excepts out if there a duplicate keys, etc. We don't want aptdaemon to crash on a ConfigParser.* exception; I think it should just be logged and it should return an empty set then. (Also easy to testcase).

Thanks, good point. This is more robust now I added tests and logging
(and test for the logging).

This should be ready for re-review, except that we need to find a
better name for whitelisted.

Thanks,
 Michael

[Martin Pitt (pitti)]

The code/tests look good to me now, many thanks for the changes!

Some other ideas about the privilege name:

 - org.debian.apt.install-packages.relaxed-auth: connotes the "how"
 - org.debian.apt.install-packages.lightweight, org.debian.apt.install-packages.high-trust: connotes the "what for"

Perhaps you can also extend the comment to say what kind of packages this privilege is meant to be used for, i. e. webapps wrappers.

review: Approve

[lp:~mvo/aptdaemon/support-for-whitelisted-repositories]

869. By Michael Vogt on 2012-08-31

rename read_repository_whitelist to read_high_trust_repository_whitelist

870. By Michael Vogt on 2012-08-31

rename transaction_only_installs_packages_from_whitelisted_repos to transaction_only_installs_packages_from_high_trust_repos

871. By Michael Vogt on 2012-08-31

rename self._whitelisted_packages to self._high_trust_whitelisted_packages

872. By Michael Vogt on 2012-08-31

rename PK_ACTION_INSTALL_PACKAGES_FROM_WHITELISTED_REPO to PK_ACTION_INSTALL_PACKAGES_FROM_HIGH_TRUST_WHITELISTED_REPO

873. By Michael Vogt on 2012-08-31

rename /etc/aptdaemon/repository-whitelist.cfg to /etc/aptdaemon/high-trust-repository-whitelist.cfg

874. By Michael Vogt on 2012-08-31

read the high-trust repository whitelist from /etc/aptdaemon/high-trust-repository-whitelist.d by default and make it a .d directory to ensure packages like e.g. webapps can drop there whitelist in

[Michael Vogt (mvo)]

Thanks Martin! I like the "high-trust" one best, the current code calls it:
 "org.debian.apt.install-packages.high-trust-whitelisted-repo"

I also tried to make the naming clearer and improve the examples. A quick
double check would be very cool

review: Resubmit

[Martin Pitt (pitti)]

There are still a few places where this hasn't been renamed consistently:

+ def _get_whitelisted_packages(self):

+class WhiteListedRepositoryIntegrationTestCase(BaseWhitelistTestCase):
+ """ Test the whitelist feature inside the chroot """

and a few more.

Do we really need both "high trust" and "whitelisted" in the name? But anyway, that's bikeshedding at some point, at least it more self-explaining now.

Thanks!

[lp:~mvo/aptdaemon/support-for-whitelisted-repositories]

875. By Michael Vogt on 2012-08-31

rename AptWorker._whitelisted_repositories to _high_trust_repositories

[Michael Vogt (mvo)]

On Fri, Aug 31, 2012 at 08:38:20AM -0000, Martin Pitt wrote:
> There are still a few places where this hasn't been renamed consistently:
>
> + def _get_whitelisted_packages(self):
>
> +class WhiteListedRepositoryIntegrationTestCase(BaseWhitelistTestCase):
> + """ Test the whitelist feature inside the chroot """
>
> and a few more.

Thanks, I look over the diff once more and eliminiate the remaining
ones.

> Do we really need both "high trust" and "whitelisted" in the name? But anyway, that's bikeshedding at some point, at least it more self-explaining now.

I guess you are right about this, its a bit too much, so I will get
rid of more of the "whitelisted" strings.

[lp:~mvo/aptdaemon/support-for-whitelisted-repositories]

876. By Michael Vogt on 2012-08-31

-PK_ACTION_INSTALL_PACKAGES_FROM_HIGH_TRUST_WHITELISTED_REPO -> +PK_ACTION_INSTALL_PACKAGES_FROM_HIGH_TRUST_REPO

877. By Michael Vogt on 2012-08-31

trans.whitelisted_packages, -> trans.high_trust_packages

878. By Michael Vogt on 2012-08-31

more renaming

879. By Michael Vogt on 2012-09-03

data/org.debian.apt.policy.in: fix name to match org.debian.apt.install-packages.high-trust-repo

880. By Michael Vogt on 2012-09-03

some more renames

=== modified file 'aptdaemon/core.py'
--- aptdaemon/core.py	2012-06-06 13:53:40 +0000
+++ aptdaemon/core.py	2012-09-03 09:53:19 +0000
@@ -63,7 +63,11 @@
 from defer import inline_callbacks, return_value, Deferred
 from defer.utils import dbus_deferred_method
 from . import policykit1
-from .worker import AptWorker, DummyWorker
+from .worker import (
+    AptWorker,
+    DummyWorker,
+    transaction_only_installs_packages_from_high_trust_repos,
+    )
 from .loop import mainloop
 
 # Setup i18n
@@ -407,6 +411,8 @@
                                       for pkgs in packages],
                                      signature="asasasasasas")
         self._unauthenticated = dbus.Array([], signature=dbus.Signature('s'))
+        self._high_trust_packages = dbus.Array(
+            [], signature=dbus.Signature('s'))
         # Add a timeout which removes the transaction from the bus if it
         # hasn't been setup and run for the TRANSACTION_IDLE_TIMEOUT period
         self._idle_watch = GObject.timeout_add_seconds(
@@ -607,6 +613,24 @@
                                doc="Unauthenticated packages in this "
                                    "transaction")
 
+    # package that can have a different auth schema, useful for e.g.
+    # lightweight packages like unity-webapps or packages comming from
+    # a high trust repository (e.g. a internal company repo)
+    def _get_high_trust_packages(self):
+        return self._high_trust_packages
+
+    def _set_high_trust_packages(self, whitelisted_packages):
+        self._high_trust_packages = dbus.Array(
+            whitelisted_packages, signature="s")
+        self.PropertyChanged(
+            "HighTrustWhitelistedPackages", 
+            self._high_trust_packages)
+
+    high_trust_packages = property(
+        _get_high_trust_packages, 
+        _set_high_trust_packages,
+        doc="High trust packages in this transaction")
+
     def _get_depends(self):
         return self._depends
 
@@ -911,6 +935,12 @@
         action = self.ROLE_ACTION_MAP[self.role]
         if action is None:
             raise StopIteration
+        # Special case if InstallPackages only touches stuff from the 
+        # high trust whitelist
+        if (self.role in (enums.ROLE_INSTALL_PACKAGES, 
+                          enums.ROLE_COMMIT_PACKAGES) and
+            transaction_only_installs_packages_from_high_trust_repos(self)):
+            action = policykit1.PK_ACTION_INSTALL_PACKAGES_FROM_HIGH_TRUST_REPO
         # Special case if CommitPackages only upgrades
         if self.role == enums.ROLE_COMMIT_PACKAGES and \
            not self.packages[enums.PKGS_INSTALL] and \

=== modified file 'aptdaemon/policykit1.py'
--- aptdaemon/policykit1.py	2012-05-08 18:50:57 +0000
+++ aptdaemon/policykit1.py	2012-09-03 09:53:19 +0000
@@ -28,7 +28,8 @@
            "PK_ACTION_GET_TRUSTED_VENDOR_KEYS",
            "PK_ACTION_INSTALL_FILE",
            "PK_ACTION_INSTALL_OR_REMOVE_PACKAGES",
-           "PK_ACTION_INSTALL_PACKAGES_FROM_NRE_REPO",
+           "PK_ACTION_INSTALL_PACKAGES_FROM_NEW_REPO",
+           "PK_ACTION_INSTALL_PACKAGES_FROM_HIGH_TRUST_REPO",
            "PK_ACTION_INSTALL_PURCHASED_PACKAGES",
            "PK_ACTION_UPDATE_CACHE", "PK_ACTION_UPGRADE_PACKAGES",
            "PK_ACTION_SET_PROXY", "PK_ACTION_CLEAN")
@@ -44,6 +45,8 @@
         "org.debian.apt.install-purchased-packages"
 PK_ACTION_INSTALL_PACKAGES_FROM_NEW_REPO = \
         "org.debian.apt.install-packages-from-new-repo"
+PK_ACTION_INSTALL_PACKAGES_FROM_HIGH_TRUST_REPO = \
+        "org.debian.apt.install-packages.high-trust-repo"
 PK_ACTION_INSTALL_FILE = "org.debian.apt.install-file"
 PK_ACTION_UPGRADE_PACKAGES = "org.debian.apt.upgrade-packages"
 PK_ACTION_UPDATE_CACHE = "org.debian.apt.update-cache"

=== modified file 'aptdaemon/test.py'
--- aptdaemon/test.py	2012-06-13 07:57:12 +0000
+++ aptdaemon/test.py	2012-09-03 09:53:19 +0000
@@ -45,6 +45,14 @@
 PY3K = sys.version_info.major > 2
 
 
+class MockQueue(object):
+
+    """A fake TransactionQueue which only provides a limbo attribute."""
+
+    def __init__(self):
+        self.limbo = {}
+
+
 class Chroot(object):
 
     """Provides a chroot which can be used by APT."""
@@ -195,7 +203,7 @@
         self.addCleanup(self._kill_process, proc)
         return proc
 
-    def start_session_aptd(self, chroot="/"):
+    def start_session_aptd(self, chroot="/", debug=False):
         """Start an aptdaemon which listens on the session D-Bus.
 
         :param chroot: path to the chroot
@@ -210,10 +218,12 @@
             path = "/usr/sbin/aptd"
         else:
             path = os.path.join(dir, "../aptd")
-        proc = subprocess.Popen(["python3", path, "--debug",
-                                 "--disable-timeout", "--disable-plugins",
-                                 "--chroot", chroot],
-                                 env=env)
+        cmd = ["python3", path, 
+               "--disable-timeout", "--disable-plugins",
+               "--chroot", chroot]
+        if debug:
+            cmd.append("--debug")
+        proc = subprocess.Popen(cmd, env=env)
         self.addCleanup(self._kill_process, proc)
         return proc
 

=== modified file 'aptdaemon/utils.py'
--- aptdaemon/utils.py	2012-05-09 00:49:57 +0000
+++ aptdaemon/utils.py	2012-09-03 09:53:19 +0000
@@ -29,6 +29,7 @@
 import warnings
 from xml.etree import ElementTree
 
+
 def deprecated(func):
     """This is a decorator which can be used to mark functions
     as deprecated. It will result in a warning being emitted

=== modified file 'aptdaemon/worker.py'
--- aptdaemon/worker.py	2012-06-13 19:31:39 +0000
+++ aptdaemon/worker.py	2012-09-03 09:53:19 +0000
@@ -22,6 +22,7 @@
 __all__ = ("AptWorker", "DummyWorker")
 
 import contextlib
+import glob
 import logging
 import os
 import platform
@@ -32,9 +33,14 @@
 import time
 import traceback
 try:
-	from urllib.parse import urlsplit, urlunsplit
-except ImportError:
-	from urlparse import urlsplit, urlunsplit
+    from urllib.parse import urlsplit, urlunsplit
+except ImportError:
+    from urlparse import urlsplit, urlunsplit
+
+try:
+    from configparser import ConfigParser
+except ImportError:
+    from ConfigParser import ConfigParser
 
 import apt
 import apt.auth
@@ -52,15 +58,17 @@
 from .errors import *
 from . import lock
 from .loop import mainloop
-from .progress import DaemonOpenProgress, \
-                     DaemonInstallProgress, \
-                     DaemonAcquireProgress, \
-                     DaemonAcquireRepoProgress, \
-                     DaemonDpkgInstallProgress, \
-                     DaemonDpkgReconfigureProgress, \
-                     DaemonDpkgRecoverProgress, \
-                     DaemonLintianProgress, \
-                     DaemonForkProgress
+from .progress import (
+    DaemonOpenProgress,
+    DaemonInstallProgress,
+    DaemonAcquireProgress,
+    DaemonAcquireRepoProgress,
+    DaemonDpkgInstallProgress,
+    DaemonDpkgReconfigureProgress,
+    DaemonDpkgRecoverProgress,
+    DaemonLintianProgress,
+    DaemonForkProgress,
+    )
 
 log = logging.getLogger("AptDaemon.Worker")
 
@@ -69,6 +77,62 @@
 _ = lambda s: s
 
 
+def transaction_only_installs_packages_from_high_trust_repos(trans,
+                                                              whitelist=set()):
+    """ Return True if this transaction only touches packages in the 
+        aptdaemon repoisotry high trust repository whitelist
+    """
+    # the transaction *must* be simulated before
+    if not trans.simulated:
+        return False
+    # we never allow unauthenticated ones
+    if trans.unauthenticated:
+        return False
+    # paranoia: wrong role
+    if not trans.role in (ROLE_INSTALL_PACKAGES, ROLE_COMMIT_PACKAGES):
+        return False
+    # if there is anything touched that is not a install bail out
+    for enum in (PKGS_REINSTALL, PKGS_REMOVE, PKGS_PURGE, PKGS_DOWNGRADE, 
+                 PKGS_UPGRADE):
+        if trans.packages[enum]:
+            return False
+    # paranoia(2): we must want to install something
+    if not trans.packages[PKGS_INSTALL]:
+        return False
+    # if the install packages matches the whitelisted set we are good
+    return set(trans.packages[PKGS_INSTALL]) == set(trans.high_trust_packages)
+
+
+def read_high_trust_repository_dir(whitelist_cfg_d):
+    """Return a set of (origin, component, pkgname-regexp) from a 
+       high-trust-repository-whitelist.d directory
+    """
+    whitelist = set()
+    for path in glob.glob(os.path.join(whitelist_cfg_d, "*.cfg")):
+        whitelist |= _read_high_trust_repository_whitelist_file(path)
+    return whitelist
+
+
+def _read_high_trust_repository_whitelist_file(path):
+    """Read a individual high-trust-repository whitelist file and return
+       a set of tuples (origin, component, pkgname-regexp)
+    """
+    parser = ConfigParser()
+    whitelist = set()
+    try:
+        parser.read(path)
+    except Exception as e:
+        log.error("Failed to read repository whitelist '%s' (%s)" % (
+                path, e))
+        return whitelist
+    for section in parser.sections():
+        origin = parser.get(section, "origin")
+        component = parser.get(section, "component")
+        pkgnames = parser.get(section, "pkgnames")
+        whitelist.add( (origin, component, pkgnames) )
+    return whitelist
+
+
 class AptWorker(GObject.GObject):
 
     """Worker which processes transactions from the queue."""
@@ -92,7 +156,7 @@
         # Store the the tid of the transaction whose changes are currently
         # marked in the cache
         self.marked_tid = None
-
+        
         # Change to a given chroot
         if chroot:
             apt_conf_file = os.path.join(chroot, "etc/apt/apt.conf")
@@ -118,6 +182,16 @@
             lock.lists_lock.path = os.path.join(lists_dir, "lock")
             apt_pkg.init_system()
 
+        # a set of tuples of the type (origin, component, pkgname-regexp)
+        # that on install will trigger a different kind of polkit 
+        # authentication request (see LP: #1035207), useful for e.g. 
+        # webapps/company repos
+        self._high_trust_repositories = read_high_trust_repository_dir(
+            os.path.join(apt_pkg.config.find_dir("Dir"), 
+                         "etc/aptdaemon/high-trust-repository-whitelist.d"))
+        log.debug(
+            "using high-trust whitelist: '%s'" % self._high_trust_repositories)
+
         self._status_orig = apt_pkg.config.find_file("Dir::State::status")
         self._status_frozen = None
         self.plugins = {}
@@ -347,6 +421,25 @@
             version = release = None
         return name, version, release
 
+    def _get_high_trust_packages(self):
+        """ Return a list of packages that come from a high-trust repo """
+        def _in_high_trust_repository(pkgname, pkgorigin):
+            for origin, component, regexp in self._high_trust_repositories:
+                if (origin == pkgorigin.origin and
+                    component == pkgorigin.component and
+                    re.match(regexp, pkgname)):
+                    return True
+            return False
+        # loop
+        from_high_trust_repo = []
+        for pkg in self._iterate_packages():
+            if pkg.marked_install:
+                for origin in pkg.candidate.origins:
+                    if _in_high_trust_repository(pkg.name, origin):
+                        from_high_trust_repo.append(pkg.name)
+                        break
+        return from_high_trust_repo
+
     def _get_unauthenticated(self):
         """Return a list of unauthenticated package names """
         unauthenticated = []
@@ -1041,7 +1134,7 @@
     def _simulate(self, trans):
         try:
             trans.depends, trans.download, trans.space, \
-                    trans.unauthenticated = self.__simulate(trans)
+                    trans.unauthenticated, trans.high_trust_packages = self.__simulate(trans)
         except TransactionFailed as excep:
             trans.error = excep
             trans.exit = EXIT_FAILED
@@ -1068,6 +1161,7 @@
     def __simulate(self, trans):
         depends = [[], [], [], [], [], [], []]
         unauthenticated = []
+        high_trust_packages = []
         skip_pkgs = []
         size = 0
         installs = reinstalls = removals = purges = upgrades = upgradables = \
@@ -1079,7 +1173,7 @@
                               ROLE_UPGRADE_SYSTEM, ROLE_REMOVE_PACKAGES,
                               ROLE_COMMIT_PACKAGES, ROLE_INSTALL_FILE,
                               ROLE_FIX_BROKEN_DEPENDS]:
-            return depends, 0, 0, []
+            return depends, 0, 0, [], []
 
         # If a transaction is currently running use the former status file
         if self._status_frozen:
@@ -1157,6 +1251,7 @@
             changes_names.append(pkg.name)
         # get the unauthenticated packages
         unauthenticated = self._get_unauthenticated()
+        high_trust_packages = self._get_high_trust_packages()
         # Check for skipped upgrades
         for pkg in upgradables:
             if pkg.marked_keep:
@@ -1174,7 +1269,8 @@
 
         required_space = size + self._cache.required_space
 
-        return depends, required_download, required_space, unauthenticated
+        return depends, required_download, required_space, unauthenticated, \
+               high_trust_packages
 
     def _check_deb_file(self, trans, path, force):
         """Perform some basic checks for the Debian package.

=== modified file 'data/org.debian.apt.policy.in'
--- data/org.debian.apt.policy.in	2011-03-17 11:07:09 +0000
+++ data/org.debian.apt.policy.in	2012-09-03 09:53:19 +0000
@@ -106,6 +106,30 @@
     </defaults>
   </action>
 
+  <action id="org.debian.apt.install-packages.high-trust-repo">
+    <!-- This priviledge will be requested when installing a package
+         from a high trusted repository that can be explicitely whitelisted.
+         
+         The defaults for this action are the same as 
+         "org.debian.apt.install-or-remove-packages".
+
+         The admin can override them to e.g. allow passwordless installs for
+         leightweight applications like unity-webapps or for packages
+         comming from high trust repositories (like internal repositories)
+    -->
+    <_description>
+      Install software from a high-trust whitelisted repository.
+    </_description>
+    <_message>
+      To install software, you need to authenticate.
+    </_message>
+    <defaults>
+      <allow_any>auth_admin</allow_any>
+      <allow_inactive>auth_admin</allow_inactive>
+      <allow_active>auth_admin_keep</allow_active>
+    </defaults>
+  </action>
+
   <action id="org.debian.apt.install-packages-from-new-repo">
     <!-- This privilege allows to call AddRepository, UpdateCache(Partially)
 	 and InstallPackages in a row and only authenticating once.

=== added directory 'tests/data'
=== added file 'tests/data/high-trust-repository-whitelist-broken.cfg'
--- tests/data/high-trust-repository-whitelist-broken.cfg	1970-01-01 00:00:00 +0000
+++ tests/data/high-trust-repository-whitelist-broken.cfg	2012-09-03 09:53:19 +0000
@@ -0,0 +1,10 @@
+
+[repo name1]
+origin = Ubuntu
+component = main
+pkgnames = foo.*
+
+[repo name1]
+origin = Debian-Security
+component = non-free
+pkgnames = ^bar$
\ No newline at end of file

=== added file 'tests/data/high-trust-repository-whitelist.cfg'
--- tests/data/high-trust-repository-whitelist.cfg	1970-01-01 00:00:00 +0000
+++ tests/data/high-trust-repository-whitelist.cfg	2012-09-03 09:53:19 +0000
@@ -0,0 +1,10 @@
+
+[repo name1]
+origin = Ubuntu
+component = main
+pkgnames = foo.*
+
+[repo whitelist2]
+origin = Debian-Security
+component = non-free
+pkgnames = ^bar$
\ No newline at end of file

=== added directory 'tests/data/high-trust-repository-whitelist.d'
=== added symlink 'tests/data/high-trust-repository-whitelist.d/high-trust-repository-whitelist-broken.cfg'
=== target is u'../high-trust-repository-whitelist-broken.cfg'
=== added symlink 'tests/data/high-trust-repository-whitelist.d/high-trust-repository-whitelist.cfg'
=== target is u'../high-trust-repository-whitelist.cfg'
=== added directory 'tests/repo/whitelisted'
=== added file 'tests/repo/whitelisted/Packages'
--- tests/repo/whitelisted/Packages	1970-01-01 00:00:00 +0000
+++ tests/repo/whitelisted/Packages	2012-09-03 09:53:19 +0000
@@ -0,0 +1,34 @@
+Package: silly-base
+Priority: extra
+Section: admin
+Installed-Size: 44
+Maintainer: Sebastian Heinlein <[e-mail]>
+Architecture: all
+Source: silly-packages (0.1-0)
+Version: 0.1-0update1
+Filename: ./silly-base_0.1-0update1_all.deb
+Size: 1934
+MD5sum: 8e20af56a63a1e0cf40db3b0d07e7989
+SHA1: 7ce87423d9c7a734478c464021994944d07fbf1b
+SHA256: d3693c0e3e7a9519b2833fdf1301c7e03e0620edf15b95b4c7329d9eb0bee553
+SHA512: e9eded74e2449a98b02828539c55a83de85a762d2361cd8c929292eb9c5a6e5a9b8eb9b64c26c45d7b73280e12a280cd799a9b831126e484bcf55b56456d559f
+Description: working package
+ Silly packages is a set of packages which will break your package
+ management tool. They are created only for debugging purposes.
+ .
+ This package doesn't contain any files and should always be installable.
+
+Package: other-pkg
+Priority: extra
+Section: admin
+Installed-Size: 44
+Maintainer: Sebastian Heinlein <[e-mail]>
+Architecture: all
+Version: 2.0
+Filename: ./other-pkg_2.0_all.deb
+Size: 1934
+MD5sum: 8e20af56a63a1e0cf40db3b0d07e7989
+SHA1: 7ce87423d9c7a734478c464021994944d07fbf1b
+SHA256: d3693c0e3e7a9519b2833fdf1301c7e03e0620edf15b95b4c7329d9eb0bee553
+SHA512: e9eded74e2449a98b02828539c55a83de85a762d2361cd8c929292eb9c5a6e5a9b8eb9b64c26c45d7b73280e12a280cd799a9b831126e484bcf55b56456d559f
+Description: another working package

=== added file 'tests/repo/whitelisted/Packages.gpg'
--- tests/repo/whitelisted/Packages.gpg	1970-01-01 00:00:00 +0000
+++ tests/repo/whitelisted/Packages.gpg	2012-09-03 09:53:19 +0000
@@ -0,0 +1,34 @@
+-----BEGIN PGP SIGNED MESSAGE-----
+Hash: SHA256
+
+Package: silly-base
+Priority: extra
+Section: admin
+Installed-Size: 44
+Maintainer: Sebastian Heinlein <[e-mail]>
+Architecture: all
+Source: silly-packages (0.1-0)
+Version: 0.1-0update1
+Filename: ./silly-base_0.1-0update1_all.deb
+Size: 1934
+MD5sum: 8e20af56a63a1e0cf40db3b0d07e7989
+SHA1: 7ce87423d9c7a734478c464021994944d07fbf1b
+SHA256: d3693c0e3e7a9519b2833fdf1301c7e03e0620edf15b95b4c7329d9eb0bee553
+SHA512: e9eded74e2449a98b02828539c55a83de85a762d2361cd8c929292eb9c5a6e5a9b8eb9b64c26c45d7b73280e12a280cd799a9b831126e484bcf55b56456d559f
+Description: working package
+ Silly packages is a set of packages which will break your package
+ management tool. They are created only for debugging purposes.
+ .
+ This package doesn't contain any files and should always be installable.
+
+-----BEGIN PGP SIGNATURE-----
+Version: GnuPG v1.4.11 (GNU/Linux)
+
+iQEcBAEBCAAGBQJO3ZylAAoJEGg8U8fPmC0YooQH/inTQPInDCiN3pTDvzWfV16F
+Ea+UpwBN0vzuyS6f3xmhLRxLQpuz/yk8buc1H+f/XKn6eygydJRFwIEgtdWAN/Tk
+eG9I4c5zYiHzZnNWe4XNBhRdPVkIHPkbmbRs/RvDiM5Cq0LvXIe0X0RV+empJyrC
+EgKbt3PJxh8qpMfrf/OIF+GkSqAug4tq0i0n6QxLOi0raeb9PjfDwErmBpbLDSFg
+XyDnNvPET5BtWxjgupOwoFqs2QRkrLv10JBdGRz+7qG6WhH1BCAOfzYxxCtn++Ip
+kmwo8c/pmtOr1BzZyyNMWP8nvVtB728eb/M84WGYynIEyCObUQ+Q2HVsXcsPQLc=
+=j3Nx
+-----END PGP SIGNATURE-----

=== added file 'tests/repo/whitelisted/Release'
--- tests/repo/whitelisted/Release	1970-01-01 00:00:00 +0000
+++ tests/repo/whitelisted/Release	2012-09-03 09:53:19 +0000
@@ -0,0 +1,17 @@
+Origin: Ubuntu
+Label: Ubuntu-Whitelisted
+Components: main
+Suite: sid
+Date: Tue, 06 Dec 2011 04:40:53 UTC
+MD5Sum:
+ 0bd018b62f0031d1ee4993896f50ddd5              782 Packages
+ d41d8cd98f00b204e9800998ecf8427e                0 Release
+SHA1:
+ 42201c0ab580cb2d557ca22a7ed04717a68b2e6e              782 Packages
+ da39a3ee5e6b4b0d3255bfef95601890afd80709                0 Release
+SHA256:
+ 950233659ae7e86ac11ae886298963e57f4414691881af1357456c6a4038ef91              782 Packages
+ e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855                0 Release
+SHA512:
+ 984c9203b8e534b9b9cc486b21da94cf807428dc1d2daa56427a870f4d53111c58ce749c35509a5fcf7112433d7a403830378a0f482354fb382c0aa7a25bdbc2                              782 Packages
+ cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e                                0 Release

=== added file 'tests/repo/whitelisted/Release.gpg'
=== added symlink 'tests/repo/whitelisted/silly-base_0.1-0update1_all.deb'
=== target is u'../silly-base_0.1-0update1_all.deb'
=== modified file 'tests/test_client.py'
--- tests/test_client.py	2012-05-09 22:47:34 +0000
+++ tests/test_client.py	2012-09-03 09:53:19 +0000
@@ -111,6 +111,7 @@
                          aptdaemon.enums.ERROR_DEP_RESOLUTION_FAILED)
 
 
+
 if __name__ == "__main__":
     if DEBUG:
         logging.basicConfig(level=logging.DEBUG)

=== added file 'tests/test_high_trust_repository_whitelist.py'
--- tests/test_high_trust_repository_whitelist.py	1970-01-01 00:00:00 +0000
+++ tests/test_high_trust_repository_whitelist.py	2012-09-03 09:53:19 +0000
@@ -0,0 +1,201 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+"""Provides unit tests for the APTDAEMON high-trust-repo feature."""
+# Copyright (C) 2011 Sebastian Heinlein <[e-mail]>
+#
+# Licensed under the GNU General Public License Version 2
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
+# Licensed under the GNU General Public License Version 2
+
+__author__  = "Michael Vogt <[e-mail]>"
+
+import os
+import sys
+import time
+import unittest
+
+import dbus
+from gi.repository import GObject
+from mock import (
+    patch,
+    )
+
+import aptdaemon.client
+import aptdaemon.policykit1
+import aptdaemon.test
+
+from aptdaemon.worker import (
+    _read_high_trust_repository_whitelist_file,
+    read_high_trust_repository_dir,
+    transaction_only_installs_packages_from_high_trust_repos,
+    AptWorker,
+    )
+from aptdaemon.core import Transaction
+from aptdaemon import enums
+
+
+REPO_PATH = os.path.join(aptdaemon.test.get_tests_dir(), "repo")
+
+PY3K = sys.version_info.major > 2
+
+
+class BaseHighTrustTestCase(aptdaemon.test.AptDaemonTestCase):
+
+    def setUp(self):
+        self.chroot = aptdaemon.test.Chroot()
+        self.chroot.setup()
+        self.addCleanup(self.chroot.remove)
+        self.loop = GObject.MainLoop()
+
+
+class HighTrustRepositoryTestCase(BaseHighTrustTestCase):
+    """ Test the worker low-level bits of the high-trust repo implementation"""
+
+
+    def setUp(self):
+        super(HighTrustRepositoryTestCase, self).setUp()
+        self.queue = aptdaemon.test.MockQueue()
+        self.worker = AptWorker(chroot=self.chroot.path, load_plugins=False)
+        self.worker.connect("transaction-done", lambda w,t: self.loop.quit())
+        self.worker.connect("transaction-simulated",
+                            lambda w,t: self.loop.quit())
+
+    def test_read_high_trust_repository_whitelist_dir(self):
+        whitelist = read_high_trust_repository_dir(
+            os.path.join(aptdaemon.test.get_tests_dir(),
+                         "data/high-trust-repository-whitelist.d"))
+        self.assertEqual(
+            whitelist, set( [
+                ("Ubuntu", "main", "foo.*"), 
+                ("Debian-Security", "non-free", "^bar$"),
+                ]))
+
+    def test_read_high_trust_repository_whitelist(self):
+        whitelist = _read_high_trust_repository_whitelist_file(
+            os.path.join(aptdaemon.test.get_tests_dir(),
+                         "data/high-trust-repository-whitelist.cfg"))
+        self.assertEqual(
+            whitelist, set( [
+                ("Ubuntu", "main", "foo.*"), 
+                ("Debian-Security", "non-free", "^bar$"),
+                ]))
+
+    @patch("aptdaemon.worker.log")
+    def test_read_high_trust_repository_whitelist_broken(self, mock_log):
+        whitelist = _read_high_trust_repository_whitelist_file(
+            os.path.join(aptdaemon.test.get_tests_dir(),
+                         "data/high-trust-repository-whitelist-broken.cfg"))
+        self.assertEqual(whitelist, set())
+        # ensure we log a error if the config file is broken
+        mock_log.error.assert_called()
+    
+    @patch("aptdaemon.worker.log")
+    def test_read_high_trust_repository_whitelist_not_there(self, mock_log):
+        whitelist = _read_high_trust_repository_whitelist_file(
+            "lalalala-not-there-really.cfg")
+        self.assertEqual(whitelist, set())
+        # ensure we log no error if there is no config file
+        self.assertFalse(mock_log.called)
+
+    def test_high_trust_repository(self):
+        """Test if using a high_trust repo is working """
+        self.chroot.add_repository(os.path.join(aptdaemon.test.get_tests_dir(),
+                                                "repo/whitelisted"))
+        # setup a whitelist
+        self.worker._high_trust_repositories.add(
+            ("Ubuntu", "", "silly.*"))
+        # a high-trust whitelisted pkg and a non-whitelisted one
+        trans = Transaction(None, enums.ROLE_INSTALL_PACKAGES, self.queue,
+                            os.getpid(), os.getuid(), sys.argv[0],
+                            "org.debian.apt.test", connect=False,
+                            packages=[["silly-base", "other-pkg"],[],[],[],[], []])
+        self.worker.simulate(trans)
+        self.loop.run()
+        self.assertEqual(trans.high_trust_packages, ["silly-base"])
+        self.assertFalse(
+            transaction_only_installs_packages_from_high_trust_repos(
+                trans, self.worker._high_trust_repositories))
+        # whitelisted only
+        trans = Transaction(None, enums.ROLE_INSTALL_PACKAGES, self.queue,
+                            os.getpid(), os.getuid(), sys.argv[0],
+                            "org.debian.apt.test", connect=False,
+                            packages=[["silly-base"],[],[],[],[], []])
+        self.worker.simulate(trans)
+        self.loop.run()
+        self.assertTrue(
+            transaction_only_installs_packages_from_high_trust_repos(
+                trans, self.worker._high_trust_repositories))
+
+
+class HighTrustRepositoryIntegrationTestCase(BaseHighTrustTestCase):
+    """ Test the whitelist feature inside the chroot """
+
+    def setUp(self):
+        super(HighTrustRepositoryIntegrationTestCase, self).setUp()
+        # Start aptdaemon with the chroot on the session bus
+        self.start_dbus_daemon()
+        self.bus = dbus.bus.BusConnection(self.dbus_address)
+        # setup the environment first including the high-trust whitelist
+        self.chroot.add_repository(os.path.join(aptdaemon.test.get_tests_dir(),
+                                                "repo/whitelisted"))
+        whitelist_file = os.path.join(
+            self.chroot.path, "etc", "aptdaemon", 
+            "high-trust-repository-whitelist.d", "test.cfg")
+        os.makedirs(os.path.dirname(whitelist_file))
+
+        with open(whitelist_file, "w") as f:
+            f.write("""
+[test repo"]
+origin = Ubuntu
+component = 
+pkgnames = silly.*
+""")
+        # *after* that start the aptdaemon
+        self.start_session_aptd(self.chroot.path)
+        # start policykit and *only* allow from-whitelisted repo pk action
+        self.start_fake_polkitd(
+            aptdaemon.policykit1.PK_ACTION_INSTALL_PACKAGES_FROM_HIGH_TRUST_REPO)
+        time.sleep(1)
+        self.client = aptdaemon.client.AptClient(self.bus)
+
+    def test_high_trust_polkit_ok(self):
+        # test that the high trust whitelist works
+        trans = self.client.install_packages(["silly-base"])
+        trans.simulate()
+        trans.connect("finished", lambda a,b: self.loop.quit())
+        trans.run()
+        self.loop.run()
+        self.assertEqual(trans.exit, aptdaemon.enums.EXIT_SUCCESS)
+        # plus ensure removal will not work
+        trans = self.client.remove_packages(["silly-base"])
+        with self.assertRaises(aptdaemon.errors.NotAuthorizedError):
+            trans.run()
+
+    def test_high_trust_polkit_not_ok(self):
+        # ensure that non-whitelisted packages can not be installed
+        trans = self.client.install_packages(["other-pkg"])
+        trans.simulate()
+        trans.connect("finished", lambda a,b: self.loop.quit())
+        with self.assertRaises(aptdaemon.errors.NotAuthorizedError):
+            trans.run()
+        
+
+if __name__ == "__main__":
+    #import logging
+    #logging.basicConfig(level=logging.DEBUG)
+    unittest.main()
+
+# vim: ts=4 et sts=4

=== modified file 'tests/test_worker.py'
--- tests/test_worker.py	2012-08-10 13:59:10 +0000
+++ tests/test_worker.py	2012-09-03 09:53:19 +0000
@@ -31,26 +31,25 @@
 
 import apt_pkg
 from gi.repository import GObject
-import dbus
+from mock import (
+    Mock, 
+    patch,
+    )
 
 import aptdaemon.test
-from aptdaemon.worker import AptWorker
+from aptdaemon.worker import (
+    transaction_only_installs_packages_from_high_trust_repos,
+    AptWorker,
+    )
 from aptdaemon.core import Transaction
 from aptdaemon import enums, errors
 
+
 REPO_PATH = os.path.join(aptdaemon.test.get_tests_dir(), "repo")
 
 PY3K = sys.version_info.major > 2
 
 
-class MockQueue(object):
-
-    """A fake TransactionQueue which only provides a limbo attribute."""
-
-    def __init__(self):
-        self.limbo = {}
-
-
 class WorkerTestCase(aptdaemon.test.AptDaemonTestCase):
 
     """Test suite for the worker which performs the actual package
@@ -61,7 +60,7 @@
         self.chroot.setup()
         self.addCleanup(self.chroot.remove)
         self.loop = GObject.MainLoop()
-        self.queue = MockQueue()
+        self.queue = aptdaemon.test.MockQueue()
         self.worker = AptWorker(chroot=self.chroot.path, load_plugins=False)
         self.worker.connect("transaction-done", lambda w,t: self.loop.quit())
         self.worker.connect("transaction-simulated",
@@ -438,8 +437,6 @@
         """Test if credentials of repositories are store securely in a
         separate file.
         """
-        from mock import Mock
-
         source_file_name = "private_source.list"
         self.worker.add_repository(Mock(), "deb",
                                    "https://user:[e-mail]/path",
@@ -469,6 +466,8 @@
 
 
 if __name__ == "__main__":
+    #import logging
+    #logging.basicConfig(level=logging.DEBUG)
     unittest.main()
 
 # vim: ts=4 et sts=4