Merge lp:~kissiel/checkbox/cep-9 into lp:checkbox

[Maciej Kisielewski]

Well, this been for a while in the oven.
I've fixed all issues pointed out in the comments. The thing has been implemented last year, so I guess for docs' sake we should just land it :)

[Zygmunt Krynicki (zyga)]

+1

As we go on I'd like to see the same things people can put in their
manifest.json files supported. In reality the actual apparmor profile
is generated from a template. I'd like to mimic that.

On Fri, Jul 17, 2015 at 4:05 PM, Maciej Kisielewski
<email address hidden> wrote:
> Maciej Kisielewski has proposed merging lp:~kissiel/checkbox/cep-9 into lp:checkbox.
>
> Requested reviews:
> Checkbox Developers (checkbox-dev)
>
> For more details, see:
> https://code.launchpad.net/~kissiel/checkbox/cep-9/+merge/265107
>
> -- WIP --
>
> Sandboxed tests CEP
>
> Early feedback welcome!
> --
> You are subscribed to branch lp:checkbox.
>
> === added file 'cep/CEP-9.rst'
> --- cep/CEP-9.rst 1970-01-01 00:00:00 +0000
> +++ cep/CEP-9.rst 2015-07-17 13:43:21 +0000
> @@ -0,0 +1,119 @@
> +===================================================
> +Checkbox Enhancement Proposal 9: Confined QML jobs
> +===================================================
> +
> +Summary
> +=======
> +
> +This CEP defines 'confined' flag for qml-native jobs that makes
> +checkbox-converged run them as seperate apps.
> +
> +Rationale
> +=========
> +
> +Currently when qml-jobs are run by checkbox-converged, they are run with the
> +same apparmor profile as checkbox-converged. For some tests we want to change
> +the security profile the test will run in. This CEP proposes a new way of
> +running qml-native jobs in which the polciies are changed.
> +
> +
> +Example
> +=======
> +
> +We need to provide a test that checks if the device asks for permission, when
> +an app asks for location data. Let's call it "the Test"
> +This job cannot be an ordinary qml-job, as apparmor policy defined for
> +checkbox-converged would be used to carry the test out.
> +
> +Expected behaviour when running tests is as follows:
> +- checkbox-converged gets to the Test
> +- device runs the Test with different set of policies
> +- prompt asking whether to let the Test use location information
> +- testing concludes
> +- checkbox-converged switches to the next test
> +
> +Changing apparmor policies
> +===========================
> +
> +Confined tests will have additional syntax in their definition to allow
> +user-defined policy.
> +Flag ``confined`` will be used, to inform that the test should run in a
> +confined manner.
> +This field - ``apparmor`` - will point to the file that
> +should be used as apparmor profile. Should this file be missing, the test will
> +have no priviliges and will run fully confined.
> +
> +Impact
> +======
> +
> +Adding confined tests requires change in how providers are handled by
> +Checkbox-Converged. Instead of copying them as-is, they need to be built. This
> +*does not* require any change to exisitng providers and *does not* change
> +behaviour of the app.
> +For other Checkbox Applications (cli, gui), confined tests will be executed as
> +ordinary qml-native jobs, because apparmor policies don't affect platforms
> +that those front-ends run on.
> +
> +Implementation
> +==============
> +
> +Proposed flow
> +-------------
> +Checkbox-Converged launches an app containing the test and holds its execution.
> +Device switches to running the test-app.
> +T...

[Sylvain Pineau (sylvain-pineau)]

+1, I've made some comments only to continue the discussion about how to generate the consumer apps at build time.

[Po-Hsu Lin (cypressyew)]

Nitpicking for some typos.

[Sylvain Pineau (sylvain-pineau)]

The attempt to merge lp:~kissiel/checkbox/cep-9 into lp:checkbox failed. Below is the output from the failed tests.

[precise] starting container
[precise] (timing) 0.09user 0.10system 0:05.37elapsed 3%CPU (0avgtext+0avgdata 10252maxresident)k
[precise] (timing) 0inputs+32outputs (0major+8774minor)pagefaults 0swaps
[precise] provisioning container
[precise] (timing) 48.05user 19.53system 1:17.74elapsed 86%CPU (0avgtext+0avgdata 96456maxresident)k
[precise] (timing) 0inputs+19072outputs (0major+3207947minor)pagefaults 0swaps
[precise-testing] Starting tests...
Found a test script: ./checkbox-ng/requirements/container-tests-checkbox-ng-unit
[precise-testing] container-tests-checkbox-ng-unit: PASS
[precise-testing] (timing) 0.69user 0.15system 0:00.86elapsed 98%CPU (0avgtext+0avgdata 48332maxresident)k
[precise-testing] (timing) 0inputs+1384outputs (0major+22257minor)pagefaults 0swaps
Found a test script: ./checkbox-support/requirements/container-tests-checkbox-support
[precise-testing] container-tests-checkbox-support: PASS
[precise-testing] (timing) 32.69user 0.26system 0:33.00elapsed 99%CPU (0avgtext+0avgdata 150644maxresident)k
[precise-testing] (timing) 0inputs+1368outputs (0major+39230minor)pagefaults 0swaps
Found a test script: ./checkbox-touch/requirements/container-tests-touch-unit-tests
[precise-testing] container-tests-touch-unit-tests: PASS
[precise-testing] (timing) 0.01user 0.00system 0:00.02elapsed 69%CPU (0avgtext+0avgdata 2180maxresident)k
[precise-testing] (timing) 0inputs+8outputs (0major+2376minor)pagefaults 0swaps
Found a test script: ./plainbox/plainbox/impl/providers/categories/requirements/container-tests-provider-categories
[precise-testing] container-tests-provider-categories: PASS
[precise-testing] (timing) 0.99user 0.13system 0:01.14elapsed 98%CPU (0avgtext+0avgdata 46064maxresident)k
[precise-testing] (timing) 0inputs+64outputs (0major+14200minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/001-container-tests-plainbox-egg-info
[precise-testing] 001-container-tests-plainbox-egg-info: PASS
[precise-testing] (timing) 0.31user 0.07system 0:00.39elapsed 97%CPU (0avgtext+0avgdata 20260maxresident)k
[precise-testing] (timing) 0inputs+88outputs (0major+11874minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox
[precise-testing] container-tests-plainbox: PASS
[precise-testing] (timing) 53.69user 1.32system 0:55.18elapsed 99%CPU (0avgtext+0avgdata 198948maxresident)k
[precise-testing] (timing) 0inputs+3264outputs (0major+198801minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-documentation
[precise-testing] container-tests-plainbox-documentation: PASS
[precise-testing] (timing) 146.32user 0.73system 2:27.39elapsed 99%CPU (0avgtext+0avgdata 188728maxresident)k
[precise-testing] (timing) 0inputs+43208outputs (0major+57319minor)pagefaults 0swaps
Found a test script: ./plainbox/requirements/container-tests-plainbox-integration
[precise-testing] container-tests-plainbox-integration: PASS
[precise-testing] (timing) 0.83us...

[lp:~kissiel/checkbox/cep-9]

4403. By Maciej Kisielewski on 2016-06-20

add CEP-9

[Maciej Kisielewski (kissiel)]

> The attempt to merge lp:~kissiel/checkbox/cep-9 into lp:checkbox failed. Below
> is the output from the failed tests.
> (...)

The auto-branch-testing failed because the branch had test scripts from yesteryear. I fiddled with history and got it rebased. Should be good now.

=== added file 'cep/CEP-9.rst'
--- cep/CEP-9.rst	1970-01-01 00:00:00 +0000
+++ cep/CEP-9.rst	2016-06-20 21:14:57 +0000
@@ -0,0 +1,121 @@
+===================================================
+Checkbox Enhancement Proposal 9: Confined QML jobs
+===================================================
+
+Summary
+=======
+
+This CEP defines 'confined' flag for qml-native jobs that makes
+checkbox-converged run them as separate apps.
+
+Rationale
+=========
+
+Currently when qml-jobs are run by checkbox-converged, they are run with the
+same apparmor profile as checkbox-converged. For some tests we want to change
+the security profile the test will run in. This CEP proposes a new way of
+running qml-native jobs in which the policies are changed.
+
+
+Example
+=======
+
+We need to provide a test that checks if the device asks for permission, when
+an app asks for location data. Let's call it "the Test"
+This job cannot be an ordinary qml-job, as apparmor policy defined for
+checkbox-converged would be used to carry the test out.
+
+Expected behaviour when running tests is as follows:
+- checkbox-converged gets to the Test
+- device runs the Test with different set of policies
+- prompt asking whether to let the Test use location information
+- testing concludes
+- checkbox-converged switches to the next test
+
+Changing apparmor policies
+===========================
+
+Confined tests will have additional syntax in their definition to allow
+user-defined policy.
+Flag ``confined`` will be used, to inform that the test should run in a
+confined manner.
+This field - ``apparmor`` - will point to the file that
+should be used as apparmor profile. Should this file be missing, the test will
+have no privileges and will run fully confined.
+
+Impact
+======
+
+Adding confined tests requires change in how providers are handled by
+Checkbox-Converged. Instead of copying them as-is, they need to be built. This
+*does not* require any change to existing providers and *does not* change
+behaviour of the app.
+For other Checkbox Applications (cli, gui), confined tests will be executed as
+ordinary qml-native jobs, because apparmor policies don't affect platforms
+that those front-ends run on.
+
+Implementation
+==============
+
+Proposed flow
+-------------
+Checkbox-Converged launches an app containing the test and holds its execution.
+Device switches to running the test-app. ::
+
+- Test is performed.
+- Test-app prepares test result object.
+- Test-app pushes result object to ContentHub with Checkbox-Converged as the
+- destination requesting immediate consumption.
+- Test-app terminates.
+- Device switches back to Checkbox-Converged and processes the result object.
+- Checkbox-Converged carries on with normal execution.
+
+Multi-app
+---------
+Current architecture of click bundles lets us to bundle multiple apps in one
+click. Every app may have different apparmor policy so the easiest way is to
+provide means for checkbox-converged to be delivered with all confined tests
+packaged as seperate apps within the same click package. Proof of concept of
+that is here: https://code.launchpad.net/~kissiel/+git/multi-app/+ref/master
+
+
+Generating apps
+---------------
+The sandboxed test needs to communicate its result back to checkbox-converged;
+to achieve this goal Content Hub can be used. Normally the communication
+initiator (confined test) would have to ask user to pick the app which they
+would like to send the results to. This can be avoided by hardcoding qualified
+name of the destination. This qualified name requires precise information about
+current version of Checkbox-Converged. To make this part reliable, information
+about consumer and sender must be generated during creation of the click
+package.  The whole process can be governed by the ``build`` command of
+``manage.py`` script of the provider. Building a provider that has confined
+test would mean generating those applications and generating hook entry for the
+'global' click manifest to contain.
+
+
+Job definition examples
+=======================
+::
+
+    id: normal-qml-job
+    _summary: A QML job that runs with the default apparmor settings
+    plugin: qml
+    qml_file: foo.qml
+    flags: preserve-locale
+    estimated_duration: 10
+
+    id: confied-qml-job
+    _summary: A QML job that is fully confined
+    plugin: qml
+    qml_file: bar.qml
+    flags: confined preserve-locale
+    estimated_duration: 10
+
+    id: custom-confinement-qml-job
+    _summary: A QML job that is run using custom apparmor settings
+    plugin: qml
+    qml_file: baz.qml
+    flags: confined preserve-locale
+    apparmor: baz_apparmor.json
+    estimated_duration: 10