add directory allowing scopes and apps to share data

Assigning to Michi only so he can comment on the proposed path. Please assign back to me if you agree.

I'm agnostic as to the exact location. We just need to agree :-)

Currently, it is: ~/.local/share/unity-scopes/unconfined/@{APP_PKG_NAME} for unconfined scopes, and ~/.local/share/unity-scopes/leaf-net/@{APP_PKG_NAME} for confined scopes.

I can change this easily enough. However, currently, ~/.local/share/@{APP_PKGNAME} is used as the click *installation* directory. That is the directory where the click package installs the scope's .so and .ini file.

For data sharing between the application and the scope, I take it that we are talking about the *cache* directory? It doesn't seem a good idea to use the same directory for both installation files from the click package, and data files that are created by the scope and/or the application, so I think the install dir and the cache dir should be different locations (but the new location you suggest is the same as the current click installation location).

Is it OK to drop the leaf-net and unconfined components from the path? Currently, a confined scope relies on being able to create the final path component *itself* by calling mkdir(). So, assuming that ~/.local/share/unity-scopes/leaf-net exists, the scope will try to create ~/.local/share/unity-scopes/leaf-net/@{APP_PKG_NAME}.

The cache (~/.cache/) directory is not equivalent to the data directory (~/.local/share/), because it's meant for volatile data which could be deleted at any time, and which can be recreated later. So, you wouldn't use it for storing user-generated content such as drawings or documents, because they could get lost.
On the other hand, I like the suggestion of sharing the cache directory as well, because that could be used for image thumbnails, which could be shared between an app and a scope.

It may also be that the application and the scope want to share some configuration values, so I'd suggest sharing the config directory as well. :-)

> currently, ~/.local/share/@{APP_PKGNAME} is used as the click *installation* directory

There's something mixed up here I think. This is, for apps as well as scopes, where data is stored, wheras installations are in /opt/click.ubuntu.com/ .

Scopes installed from click packages are, as a matter of fact, installed in ~/.local/share/unity-scopes/@{APP_PKGNAME}

No scope, no matter where it comes from, is ever installed in /opt/click. If it is, it won't work because the registry won't look for it there.

Just to follow up on mardy's suggestion about sharing the cache directory. On the Reminders app we're integrating the scope and app, and the notes and their thumbnails are stored under the cache dir, which would be ideal to share. In this particular case, it might not apply, as we're shipping the scope and the app in the same click, though, so I understand they should both be able to access the cache data.

As I said, I don't really mind what particular directory is used as the cache directory. I just think it shouldn't be the same as the click scope installation directory.

Right now, when you call cache_directory(), you get a path that (as far as I know) should be shared with the app if the scope and the app are in the same click package. Is that actually the case? If not, either the app path or the scope path need to be changed, so they agree.

Ok, so, let's clear the terminology up a little. All click packages are installed in /opt (this is what Christian was talking about). However, those packages (whether they by apps or scopes) aren't available to the user until the user hooks are run, which creates symlinks in $HOME to the things in /opt (this is what Michi was talking about). Then there are the writable areas that apps and scopes have (this is what mardy and michi were talking about). Note, within the apparmor policy, @{APP_PKGNAME} refers to the 'name' field from the click manifest.

Scopes writable non-volatile areas (ie, not in /run) are:
  (unconfined) @{HOME}/.local/share/unity-scopes/unconfined/@{APP_PKGNAME}/*
  (confined network scope) @{HOME}/.local/share/unity-scopes/leaf-net/@{APP_PKGNAME}/*

Whereas apps writable non-volatile areas (excluding the QML cache directory) are:
  @{HOME}/.cache/@{APP_PKGNAME}/*
  @{HOME}/.config/@{APP_PKGNAME}/*
  @{HOME}/.local/share/@{APP_PKGNAME}/*

Also note, we already have a precedent for sharing between apps shipped in the same click: push helpers. Push helpers writable non-volatile areas are the same as for apps:
  @{HOME}/.cache/@{APP_PKGNAME}/*
  @{HOME}/.config/@{APP_PKGNAME}/*
  @{HOME}/.local/share/@{APP_PKGNAME}/*

Michi, my understanding was this would be perhaps a new API call, not something that you would change on your end, but that is an implementation detail and I can see arguments on either side.

Since Michi said he doesn't care where the directory is and we have a precedent for data sharing already and it seems to make the most sense for scopes to surface the data in an app rather than the other way around, I am going to add this to the ubuntu-scope-network template:
  # Allow scopes to share data with the app shipped in the same click
  owner @{HOME}/.local/share/@{APP_PKGNAME}/ rw,
  owner @{HOME}/.local/share/@{APP_PKGNAME}/** mrwkl,

"Is it OK to drop the leaf-net and unconfined components from the path? Currently, a confined scope relies on being able to create the final path component *itself* by calling mkdir(). So, assuming that ~/.local/share/unity-scopes/leaf-net exists, the scope will try to create ~/.local/share/unity-scopes/leaf-net/@{APP_PKG_NAME}."

No. This is important for isolation between scopes if we ever decide to support other scope templates to protect against certain types of attacks via differently versioned apps.

I've added a click-reviewers-tools task to handle this from the scopes confinement specification: "Because scopes share application data with apps shipped in the same click, reviewers must be careful if the click package contains any permissions that triggers a manual review (permissions that pass the automatic checks are considered safe). Eg, if the app has wide filesystem permissions but no networking and the scope has networking but no filesystem permissions, then the app can make data from the system available to the scope and so it could send it out over the network."

Actually, I think I'd prefer this, unless there is a really compelling otherwise (ie, read-only):
  # Allow scopes to share data with the app shipped in the same click
  owner @{HOME}/.local/share/@{APP_PKGNAME}/ r,
  owner @{HOME}/.local/share/@{APP_PKGNAME}/** mrkl,

> Actually, I think I'd prefer this, unless there is a really compelling otherwise (ie, read-only):
> # Allow scopes to share data with the app shipped in the same click
> owner @{HOME}/.local/share/@{APP_PKGNAME}/ r,
> owner @{HOME}/.local/share/@{APP_PKGNAME}/** mrkl,

Just double-checking here... Earlier, in comment #9, you said that we couldn't drop the leaf-net and unconfined directories, but your suggestion does drop these.

Michi,

this is what is currently in the scopes network template as a writable directory:
  @{HOME}/.local/share/unity-scopes/leaf-net/@{APP_PKGNAME}/*

this is the read-only directory I want to add to the scopes network template so scopes can surface data from the app:
  @{HOME}/.local/share/@{APP_PKGNAME}/*

As such, scopes have read access to *both*, but only write access to leaf-net.

Ah, OK, it seems the penny is finally dropping :-)

So, if I understand you correctly, you are saying that @{HOME}/.local/share/unity-scopes/leaf-net/@{APP_PKGNAME}/* is the writable cache directory for the scope (exactly as it is now), and @{HOME}/.local/share/@{APP_PKGNAME}/* is the readable directory where the scope can read data produced by the app?

I think we can do this. In practice, this would mean adding an app_directory() method to the scopes API that returns the path to the readable directory that a scope can use to pick up data from the app. There are some doc implications, but it's doable.

I take it that the readable directory would not be applicable to scopes that do *not* install together with an app in the same click package? (We would throw an exception if a scope doesn't have an associated app and tries to get the directory name.)

Now, if I have this picture correct, it implies that a scope can read from an app, but an app cannot read from a scope. Is that what we want? If so, I think I'm good with this. (I haven't implemented anything yet, so there are the usual caveats. But, off-hand, I don't see a show-stopper.)

"So, if I understand you correctly, you are saying that @{HOME}/.local/share/unity-scopes/leaf-net/@{APP_PKGNAME}/* is the writable cache directory for the scope (exactly as it is now), and @{HOME}/.local/share/@{APP_PKGNAME}/* is the readable directory where the scope can read data produced by the app?"

Yes

"I take it that the readable directory would not be applicable to scopes that do *not* install together with an app in the same click package? (We would throw an exception if a scope doesn't have an associated app and tries to get the directory name.)"

If I understand the question, APP_PKGNAME is unique to the scopes and apps installed by a particular click package. A differently named click package will have a different APP_PKGNAME set, so there is isolation between clicks. This bug only allows scopes and apps within the same click to coordinate with each other.

"Now, if I have this picture correct, it implies that a scope can read from an app, but an app cannot read from a scope. Is that what we want? If so, I think I'm good with this. (I haven't implemented anything yet, so there are the usual caveats. But, off-hand, I don't see a show-stopper.)"

This is correct and I think it is what we want. I'm still holding out for the possibility of a local content scope some time down the road and allowing writes to the app dir in a local content scope would allow an app to break out of confinement via this shared dir (of course, we could just not let that scope type coordinate with an app...). Let's go with what I suggested for now, and I will document the thinking in the policy so the thinking isn't lost.

Sorry for being so dense. I had the wrong mental model because, all along, I assumed that the cache_directory was to be used for sharing, so I was thinking that the proposal was to change the location of the existing cache dir, instead of adding a completely new one.

The linked branch should fix this.

This is in rtm silo 002.

This bug was fixed in the package apparmor-easyprof-ubuntu - 1.2.39

---------------
apparmor-easyprof-ubuntu (1.2.39) utopic; urgency=medium

  * ubuntu/{music,pictures,video}_files*: allow access to global SD card
    directories (LP: #1391930)
  * ubuntu/ubuntu-scope-network, pending/ubuntu-scope-local-content: allow
    scopes to read data from the apps data dir (LP: #1384286)
 -- Jamie Strandboge <email address hidden> Thu, 13 Nov 2014 09:54:18 -0600

This bug was fixed in the package apparmor-easyprof-ubuntu - 1.3.1

---------------
apparmor-easyprof-ubuntu (1.3.1) vivid; urgency=medium

  * ubuntu/ubuntu-sdk:
    - explicitly deny reads on ~/.cache/QML/Apps/ to silence noisy denials.
      Undo this when LP: 1381620 is fixed in qtdeclarative-opensource-src
    - explicitly deny dbus bind on name="org.freedesktop.Application" since
      it is noisy. Undo this when LP: 1378823 is fixed in ubuntu-ui-toolkit
  * ubuntu/1.3/ubuntu-sdk: drop html5-container policy. html5 apps should use
    webapp-container and specify the 'webview' policy group with 1.3 (15.04)
    policy (LP: #1392461)
  * ubuntu/ubuntu-scope-network, pending/ubuntu-scope-local-content: allow
    scopes to read data from the apps data dir (LP: #1384286)
  * adjust all dbus rules to use peer=(label=unconfined) to prevent
    coordinated communications between apps over DBus (LP: #1383824)
  * ubuntu/{music,pictures,video}_files*: allow access to global SD card
    directories (LP: #1391930)
  * debian/control: Depends on apparmor >= 2.8.98-0ubuntu2~ for the dbus peer
    changes (we need at least apparmor_parser 2.9.beta4 for these)
 -- Jamie Strandboge <email address hidden> Mon, 15 Dec 2014 15:53:32 +0000

What's still required from a click-reviewers-tools perspective?

I have a todo for what needs to happen with CRT. I plan to do an update this/next week that will include this change.