Merge lp:~kissiel/checkbox/confined-permission-control into lp:checkbox

[Maciej Kisielewski]

This MR brings a possibility of dropping specified permissions for confined tests in checkbox-converged.

bf3ffb4 checkbox-touch: fix missing test information for native jobs
eb2efd7 checkbox-touch: bump version in py backend
57cd0ed checkbox-touch: correct version in manifest to match PB format
0926cde checkbox-touch: add dropPersmission function
ac4d6a3 plainbox:data:plainbox-qml-modules: add 'clearedPermissions' field
4c7a6ff checkbox-touch: add logic for dropping permissions for confined tests
3367259 checkbox-touch: rename generate-confinement to make it importable
3656f0b checkbox-touch:confinement: make generate_confinement return the hook
c7a4f37 checkbox-touch:confinement: do not jsonize by default
7ddda80 checkbox-touch:confinement: fix PEP-8 issues
ccd2c6a checkbox-touch:build-me: add building confinement

[Zygmunt Krynicki (zyga)]

Most things are okay but I'd like to see something better for the database handling

review: Needs Fixing

[Maciej Kisielewski (kissiel)]

> Most things are okay but I'd like to see something better for the database
> handling

As for DBs being locked, if the timeout hits, this will get propagated to UI.
For proper cleanup, I addded try, finally.

Todo for XDG stuff added.

[lp:~kissiel/checkbox/confined-permission-control]

4119. By Maciej Kisielewski on 2015-11-25

checkbox-touch: add logic for dropping permissions for confined tests

This patch adds the logic that pulls information from the test's qml file about
which permissions to clear, and calls appropriate bits.
To extract this info we first instantiate the QmlJob object in a fake container
Item, so te field is initiated as test author specified, then immedietaly (and
synchronously) destroyed, so all resources are (not sure about that :)) freed.
Signed-off-by: Maciej Kisielewski <email address hidden>

4120. By Maciej Kisielewski on 2015-11-25

checkbox-touch: rename generate-confinement to make it importable

Signed-off-by: Maciej Kisielewski <email address hidden>

4121. By Maciej Kisielewski on 2015-11-25

checkbox-touch:confinement: make generate_confinement return the hook

Signed-off-by: Maciej Kisielewski <email address hidden>

4122. By Maciej Kisielewski on 2015-11-25

checkbox-touch:confinement: do not jsonize by default

This patch makes generate result hook object instead of strigified json. This
makes easier to handle marshalling everything together in a calling function.

Signed-off-by: Maciej Kisielewski <email address hidden>

4123. By Maciej Kisielewski on 2015-11-25

checkbox-touch:confinement: fix PEP-8 issues

Signed-off-by: Maciej Kisielewski <email address hidden>

4124. By Maciej Kisielewski on 2015-11-25

checkbox-touch:build-me: add building confinement

This patch runs the task of building files necessary for confined qml tests to
work while building click package.

From now on, after build-me successfully validates all the included providers,
it loads job units from them, checks if there's any qml confined job, and
generates appropriate files necessary to run those. It also overwrites the
manifest.json file, to include the new hooks.

Signed-off-by: Maciej Kisielewski <email address hidden>

[Zygmunt Krynicki (zyga)]

+1, looks good to me

review: Approve

=== modified file 'checkbox-touch/build-me'
--- checkbox-touch/build-me	2015-10-15 09:54:02 +0000
+++ checkbox-touch/build-me	2015-11-25 11:37:27 +0000
@@ -1,5 +1,6 @@
 #!/usr/bin/env python3
 import argparse
+from contextlib import contextmanager
 from distutils.dir_util import copy_tree
 from datetime import datetime
 import json
@@ -9,6 +10,9 @@
 import sys
 import textwrap
 
+from confinement.generate import generate_confinement
+from py.embedded_providers import EmbeddedProvider1PlugInCollection
+
 # DEFAULT_PROVIDERS is a list of providers directory names as listed in
 # ../providers. NOTE: those are directory names, not provider names as reported
 # by manage.py info
@@ -19,6 +23,24 @@
 ]
 
 
+@contextmanager
+def chdir(path):
+    previous_path = os.getcwd()
+    os.chdir(path)
+    yield
+    os.chdir(previous_path)
+
+
+def get_confined_jobs(path):
+    collection = EmbeddedProvider1PlugInCollection(path)
+    # this should be run within provider's directory, so there should be
+    # only one provider loaded
+    provider = collection.get_all_plugin_objects()[0]
+    for job in provider.job_list:
+        if job.plugin == 'qml' and 'confined' in job.get_flag_set():
+            yield job
+
+
 def check_libs_present():
     """
     Check if paths listed in NECESSARY_PATHS are present.
@@ -117,6 +139,16 @@
     if not validate_providers():
         sys.exit('Provider validation failed.')
 
+    current_manifest = json.loads(open('manifest.json', 'rt').read())
+    hooks = {
+        'checkbox-touch': current_manifest['hooks']['checkbox-touch']
+    }
+    for hook in build_confinement(current_manifest['version']):
+        hooks.update(hook)
+    new_manifest = current_manifest.copy()
+    new_manifest['hooks'] = hooks
+    with open('manifest.json', 'wt') as f:
+        f.write(json.dumps(new_manifest, sort_keys=True, indent=4))
     build_i18n()
 
     generate_desktop_file()
@@ -159,6 +191,21 @@
     return providers_valid
 
 
+def build_confinement(checkbox_version):
+    print("Generating confinement")
+    checkbox_name = "com.ubuntu.checkbox_checkbox-touch_" + checkbox_version
+    new_hooks = []
+    for provider_dir in os.listdir('providers'):
+        with chdir(os.path.join('providers', provider_dir)):
+            for job in get_confined_jobs('.'):
+                print(" job: {}".format(job.id))
+                qml_file = os.path.relpath(
+                    job.qml_file, 'data')
+                new_hooks.append(generate_confinement(
+                    provider_dir, job.partial_id, checkbox_name, qml_file))
+    return new_hooks
+
+
 def build_i18n():
     print("Building i18n")
     for provider_dir in os.listdir('providers'):
 
=== modified file 'checkbox-touch/checkbox-touch.qml'
--- checkbox-touch/checkbox-touch.qml	2015-11-18 09:00:54 +0000
+++ checkbox-touch/checkbox-touch.qml	2015-11-25 11:37:27 +0000
@@ -658,7 +658,7 @@
     }
     function performConfinedQmlTest(test) {
         runTestActivity(test, function(test) {
-            var qmlNativePage = createPage("components/QmlConfinedPage.qml");
+            var qmlNativePage = createPage("components/QmlConfinedPage.qml", test);
             qmlNativePage.applicationVersion = app.applicationVersion;
             qmlNativePage.testDone.connect(completeTest);
             pageStack.push(qmlNativePage);
 
=== modified file 'checkbox-touch/components/CheckboxTouchApplication.qml'
--- checkbox-touch/components/CheckboxTouchApplication.qml	2015-10-15 22:31:52 +0000
+++ checkbox-touch/components/CheckboxTouchApplication.qml	2015-11-25 11:37:27 +0000
@@ -169,6 +169,12 @@
             continuation_error(error);
         });
     }
+    function dropPermissions(appId, services, continuation, continuation_error) {
+        request("drop_permissions", [appId, services], continuation, function(error) {
+            console.error("Unable to remove permissions");
+            if (continuation_error) continuation_error(error);
+        });
+    }
 
     function rememberPassword(password, continuation) {
         // using low-level py.call() to 'silently' pass password string through pyotherside
 
=== modified file 'checkbox-touch/components/QmlConfinedPage.qml'
--- checkbox-touch/components/QmlConfinedPage.qml	2015-11-17 14:38:41 +0000
+++ checkbox-touch/components/QmlConfinedPage.qml	2015-11-25 11:37:27 +0000
@@ -107,11 +107,35 @@
             Layout.fillWidth: true
             text: i18n.tr("Start the test")
             onClicked: {
-                busyIndicatorGroup.visible = true;
-                var appName = "com.ubuntu.checkbox_" + test["partial_id"] + "_" + applicationVersion + ".desktop";
-                console.log("Launching " + appName);
-                Qt.openUrlExternally("application:///" + appName);
+                var appName = "com.ubuntu.checkbox_" + test["partial_id"];
+                // load the test qml file to check which permissions should be cleared
+                var testItemComponent = Qt.createComponent(Qt.resolvedUrl(test['qml_file']));
+                if (testItemComponent.status == Component.Error) {
+                    console.error("Error creating testItemComponent. Possible cause: Problem with job's qml file. Error:", testItemComponent.errorString());
+                    test['outcome'] = 'fail';
+                    testDone(test);
+                    return;
+                }
+                var testItem = testItemComponent.createObject(dummyContainer, {"testingShell": testingShell});
+                var clearedPermissions = testItem.clearedPermissions;
+                testItem.destroy();
+                var runConfinedTestApp = function() {
+                    busyIndicatorGroup.visible = true;
+                    Qt.openUrlExternally("application:///" + appName + "_" + applicationVersion+ ".desktop");
+                }
+                if (clearedPermissions) {
+                    app.dropPermissions(appName, clearedPermissions, function() {
+                        runConfinedTestApp();
+                    });
+                } else {
+                    runConfinedTestApp();
+                }
+
             }
         }
+        Item {
+            id: dummyContainer
+            visible: false
+        }
     }
 }
 
=== renamed file 'checkbox-touch/confinement/generate-confinement.py' => 'checkbox-touch/confinement/generate.py'
--- checkbox-touch/confinement/generate-confinement.py	2015-07-23 10:54:18 +0000
+++ checkbox-touch/confinement/generate.py	2015-11-25 11:37:27 +0000
@@ -17,6 +17,7 @@
 # You should have received a copy of the GNU General Public License
 # along with Checkbox.  If not, see <http://www.gnu.org/licenses/>.
 import argparse
+import json
 import os
 import string
 import sys
@@ -49,15 +50,9 @@
 X-Ubuntu-Touch=true
 """
 
-HOOK = """
-"${partial_id}": {
-      "apparmor": "providers/${provider_name}/data/confined/${partial_id}.apparmor",
-      "desktop": "providers/${provider_name}/data/confined/${partial_id}.desktop",
-      "content-hub": "providers/${provider_name}/data/confined/${partial_id}-ch.json"
-}
-"""
 
-def generate_confinement(provider_name, partial_id, full_checkbox_name, qml_file):
+def generate_confinement(provider_name, partial_id, full_checkbox_name,
+                         qml_file):
     # generate content-hub file
     target_dir = os.path.join('data', 'confined')
     if not os.path.exists(target_dir):
@@ -72,17 +67,24 @@
     with open(apparmor_path, "wt") as f:
         f.write(APPARMOR)
 
- 
     # generate desktop file
     desktop_path = os.path.join(target_dir, partial_id + '.desktop')
     template = string.Template(DESKTOP)
     with open(desktop_path, "wt") as f:
-        f.write(template.substitute(partial_id=partial_id, provider_name=provider_name, full_checkbox_name=full_checkbox_name, qml_file=qml_file))
+        f.write(template.substitute(
+            partial_id=partial_id, provider_name=provider_name,
+            full_checkbox_name=full_checkbox_name, qml_file=qml_file))
 
-def generate_hook(provider_name, partial_id):
-    return string.Template(HOOK).substitute(
+    base = 'providers/{provider_name}/data/confined/{partial_id}'.format(
         provider_name=provider_name, partial_id=partial_id)
-
+    hook = {
+        partial_id: {
+            'apparmor': base + '.apparmor',
+            'desktop': base + '.desktop',
+            'content-hub': base + '-ch.json',
+        }
+    }
+    return hook
 
 
 def main():
@@ -91,7 +93,8 @@
     parser.add_argument('--checkbox_version', action='store', type=str)
     parser.add_argument('job_ids', nargs='+')
     args = parser.parse_args()
-    checkbox_name = "com.ubuntu.checkbox_checkbox-touch_" + args.checkbox_version
+    checkbox_name = ("com.ubuntu.checkbox_checkbox-touch_" +
+                     args.checkbox_version)
 
     # check if current dir looks like a provider - very dumb heuristic
     if not os.path.exists('manage.py'):
@@ -100,8 +103,9 @@
 
     hooks = ''
     for job in args.job_ids:
-        generate_confinement(provider_name, job, checkbox_name, job + '.qml')
-        hooks += generate_hook(provider_name, job)
+        hook = generate_confinement(
+            provider_name, job, checkbox_name, job + '.qml')
+        hooks += json.dumps(hook, sort_keys=True, indent=4)[1:-1]
 
     print("Add following hooks to your checkbox-touch.manifest:")
     print(hooks)
 
=== modified file 'checkbox-touch/manifest.json'
--- checkbox-touch/manifest.json	2015-11-16 09:17:41 +0000
+++ checkbox-touch/manifest.json	2015-11-25 11:37:27 +0000
@@ -11,9 +11,9 @@
     "maintainer": "Zygmunt Krynicki <[e-mail]>",
     "name": "com.ubuntu.checkbox",
     "title": "Checkbox",
-    "version": "1.3.dev0",
+    "version": "1.3.dev",
     "x-source": {
         "vcs-bzr": "lp:checkbox",
-        "vcs-bzr-revno": "checkbox-touch-v1.3.dev0"
+        "vcs-bzr-revno": "checkbox-touch-v1.3.dev"
     }
 }
 
=== modified file 'checkbox-touch/py/checkbox_touch.py'
--- checkbox-touch/py/checkbox_touch.py	2015-11-16 10:20:39 +0000
+++ checkbox-touch/py/checkbox_touch.py	2015-11-25 11:37:27 +0000
@@ -36,6 +36,7 @@
 import logging
 import os
 import pyotherside
+import sqlite3
 import re
 import time
 import traceback
@@ -133,7 +134,7 @@
     response data to alter the user interface.
     """
 
-    __version__ = (1, 2, 1, 'final', 0)
+    __version__ = (1, 3, 0, 'dev', 0)
 
     def __init__(self):
         if plainbox.__version__ < (0, 22):
@@ -459,6 +460,28 @@
         return self.assistant.export_to_transport(
             '2013.com.canonical.plainbox::hexr', transport)
 
+    @view
+    def drop_permissions(self, app_id, services):
+        # TODO: use XDG once available
+        trust_dbs = {
+            'camera': '~/.local/share/CameraService/trust.db',
+            'audio': '~/.local/share/PulseAudio/trust.db',
+            'location': '~/.local/share/UbuntuLocationServices/trust.db',
+        }
+        sql = 'delete from requests where ApplicationId = (?);'
+
+        for service in services:
+            conn = None
+            try:
+                conn = sqlite3.connect(
+                    os.path.expanduser(trust_dbs[service]),
+                    isolation_level='EXCLUSIVE')
+                conn.execute(sql, (app_id,))
+                conn.commit()
+            finally:
+                if conn:
+                    conn.close()
+
     def _get_user_directory_documents(self):
         xdg_config_home = os.environ.get('XDG_CONFIG_HOME') or \
             os.path.expanduser('~/.config')
 
=== modified file 'plainbox/plainbox/data/plainbox-qml-modules/Plainbox/QmlJob.qml'
--- plainbox/plainbox/data/plainbox-qml-modules/Plainbox/QmlJob.qml	2015-07-07 08:32:21 +0000
+++ plainbox/plainbox/data/plainbox-qml-modules/Plainbox/QmlJob.qml	2015-11-25 11:37:27 +0000
@@ -24,4 +24,5 @@
 Item {
     signal testDone(var testResult)
     property var testingShell;
+    property var clearedPermissions: [];
 }