Merge lp:~ted/ubuntu-app-launch/install-root into lp:ubuntu-app-launch/16.10

[Ted Gould]

Make UAL relocatable for Snappy installs

For things that it makes sense to override we have an envvar to override it. For the Upstart jobs we're just relying on ${SNAP} to be set.

Also added docs for all the envvars that we use.

[unity-api-1-bot (unity-api-1-bot)]

PASSED: Continuous integration, rev:256
https://jenkins.canonical.com/unity-api-1/job/lp-ubuntu-app-launch-ci/99/
Executed test runs:
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build/647
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-0-fetch/653
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/469
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=vivid+overlay/469/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/469
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=xenial+overlay/469/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/469
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=amd64,release=yakkety/469/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/469
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=vivid+overlay/469/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/469
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=xenial+overlay/469/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/469
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=armhf,release=yakkety/469/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/469
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=vivid+overlay/469/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/469
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=xenial+overlay/469/artifact/output/*zip*/output.zip
    SUCCESS: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/469
        deb: https://jenkins.canonical.com/unity-api-1/job/build-2-binpkg/arch=i386,release=yakkety/469/artifact/output/*zip*/output.zip

Click here to trigger a rebuild:
https://jenkins.canonical.com/unity-api-1/job/lp-ubuntu-app-launch-ci/99/rebuild

review: Approve (continuous-integration)

[Marcus Tomlinson (marcustomlinson)]

Comforting to see that all code paths are covered in the tests. Also, your bug, your fix (extra comfort) :)

LGTM

review: Approve

[Ted Gould (ted)]

So further testing on real systems revealed an interesting flaw. Basically by including $SNAP in the exec line it doesn't work. Why? Because to resolve the environment variable Upstart switches from doing an exec directly, to instead using a shell which then does the exec. Not a big deal, except that there seems to be a bug in the way it works with AppArmor. So Upstart switches profiles on exec, but that is after the binary is read, so the exec-line-exec binary doesn't need to be in the Application's apparmor profile. But when the shell goes first the shell itself is exec'ing under the application's profile, and thus needs permission.

This is, at some level, a bug in Upstart. Not sure how it should be fixed. Will have to have conversations with security taking into account the maintenance level of the various packages involved.

review: Needs Fixing

[lp:~ted/ubuntu-app-launch/install-root]

258. By Ted Gould on 2016-11-10

Update to latest systemd branch

259. By Ted Gould on 2016-11-14

Pulling through updates

260. By Ted Gould on 2016-11-17

Adding an environment variable for XMir Helper

261. By Ted Gould on 2016-11-17

Pulling through trunk

262. By Ted Gould on 2016-11-17

Restructure exec line building so we can add more

263. By Ted Gould on 2016-11-17

Adding extra environment in the snap world

264. By Ted Gould on 2016-11-17

Make sure we get the other snap environment variables

265. By Ted Gould on 2016-11-18

Switch to calling legacy-exec

266. By Ted Gould on 2016-11-21

Starting utilities to put together an environment setup

267. By Ted Gould on 2016-11-21

Better exit codes

268. By Ted Gould on 2016-11-21

Fleshing out our utilities

269. By Ted Gould on 2016-11-21

Get the binaries into a deb

270. By Ted Gould on 2016-11-21

Make the command line reference the outside util

271. By Ted Gould on 2016-11-21

Call the snappy-xmir util unconfined

272. By Ted Gould on 2016-11-21

Handle write() returns

273. By Ted Gould on 2016-11-21

Disable for a moment

274. By Ted Gould on 2016-11-22

Remove the snap setting libertine launch

275. By Ted Gould on 2016-11-30

Fix command name

276. By Ted Gould on 2016-12-01

Make sure we sleep forever

277. By Ted Gould on 2016-12-01

Get negative errors as well

278. By Ted Gould on 2016-12-01

Make sure the FD stays open

279. By Ted Gould on 2016-12-01

Provide a way to test and put the fd number on the command line

280. By Ted Gould on 2016-12-01

Some basic test scripts

281. By Ted Gould on 2016-12-01

Switch to an abstract socket

282. By Ted Gould on 2016-12-02

Make sure we don't overrun sun_path

283. By Ted Gould on 2016-12-12

Clear out the MIR_* variables

284. By Ted Gould on 2016-12-12

Getting the snappy-xmir tests into the test suite

285. By Ted Gould on 2016-12-12

Check for MIR_ variables

286. By Ted Gould on 2016-12-12

Making sure we clean up the envvars utils

287. By Ted Gould on 2016-12-12

Increase output and debugging info in tests

288. By Ted Gould on 2016-12-12

Read everything in, and then start processing it into individual variables

289. By Ted Gould on 2016-12-12

Cleanup magic numbers

290. By Ted Gould on 2016-12-12

Explicitly set the DBus address as well

291. By Ted Gould on 2016-12-12

Make sure to pad with zeros

292. By Ted Gould on 2016-12-12

Test for long AppIDs

293. By Ted Gould on 2016-12-12

Removing snap variables from the Upstart jobs. If we're under a SNAP we'll be using systemd

294. By Ted Gould on 2016-12-12

Use the path in the current snap as the place to find the helper

295. By Ted Gould on 2016-12-12

Fix formatting

296. By Ted Gould on 2016-12-12

We don't need to care about the SNAP_ variables because they'll be reinserted when we come through the snap commands

297. By Ted Gould on 2016-12-12

Use a build variable to get the path

298. By Ted Gould on 2016-12-12

Switching back to setting the libertine-launch path in non-snap cases

299. By Ted Gould on 2016-12-12

Allow the legacy-exec util to be overridden

300. By Ted Gould on 2016-12-13

Merge error

301. By Ted Gould on 2016-12-13

More magic numbers brought down to Earth

302. By Ted Gould on 2016-12-13

Bigger libs warning. Like really people, read this.

303. By Ted Gould on 2016-12-13

Fix the libertine-launch case for non-snaps

304. By Ted Gould on 2017-01-03

Removing QT and XDG variables, don't think we need them

305. By Ted Gould on 2017-01-06

Removing more variables

306. By Ted Gould on 2017-01-06

Upstream changes

307. By Ted Gould on 2017-01-06

Make sure we don't have a Mir socket before checking that we don't

308. By Ted Gould on 2017-01-11

Merge upstream updates

309. By Ted Gould on 2017-01-19

Update to trunk

310. By Ted Gould on 2017-01-24

Update to current jobs-systemd

311. By Ted Gould on 2017-01-24

Grab QT_ and XDG_ for legacy apps on deb installs

312. By Ted Gould on 2017-01-24

Switching functions

313. By Ted Gould on 2017-01-26

Merge Trunk

314. By Ted Gould on 2017-01-27

Merge changes to jobs-systemd

315. By Ted Gould on 2017-01-30

Put the wrong environment var into the documentation

316. By Ted Gould on 2017-01-30

Getting these variables figured out

317. By Ted Gould on 2017-01-31

Fix docs to include default path for LEGACY_EXEC

318. By Ted Gould on 2017-01-31

Cache snap env var

319. By Ted Gould on 2017-01-31

Make sure we only use references in loops

320. By Ted Gould on 2017-01-31

Don't store if we have the variable

321. By Ted Gould on 2017-01-31

Return const pointer

322. By Ted Gould on 2017-01-31

Handle larger possible pids

323. By Ted Gould on 2017-01-31

Make sure we're gettin' data

324. By Ted Gould on 2017-01-31

Move debug outside of the print loop

325. By Ted Gould on 2017-02-01

A read failure is a helper failure

326. By Ted Gould on 2017-02-02

If we don't have a MIR_SOCKET use the default one

327. By Ted Gould on 2017-02-02

Special case legacy apps on Unity8 from debs

328. By Ted Gould on 2017-02-02

Put on the dunce cap and sit in the corner

Unmerged revisions

328. By Ted Gould on 2017-02-02

Put on the dunce cap and sit in the corner

327. By Ted Gould on 2017-02-02

Special case legacy apps on Unity8 from debs

326. By Ted Gould on 2017-02-02

If we don't have a MIR_SOCKET use the default one

325. By Ted Gould on 2017-02-01

A read failure is a helper failure

324. By Ted Gould on 2017-01-31

Move debug outside of the print loop

323. By Ted Gould on 2017-01-31

Make sure we're gettin' data

322. By Ted Gould on 2017-01-31

Handle larger possible pids

321. By Ted Gould on 2017-01-31

Return const pointer

320. By Ted Gould on 2017-01-31

Don't store if we have the variable

319. By Ted Gould on 2017-01-31

Make sure we only use references in loops

=== modified file 'docs/index.rst'
--- docs/index.rst	2016-11-10 21:59:11 +0000
+++ docs/index.rst	2016-11-10 21:59:12 +0000
@@ -18,6 +18,47 @@
 
 .. _Upstart: http://upstart.ubuntu.com/
 
+
+Environment Variables
+=====================
+
+There are a few environment variables that can effect the behavior of UAL while
+it is running.
+
+UBUNTU_APP_LAUNCH_CG_MANAGER_NAME
+  The DBus name that CG Manager registers under if it is on the session bus.
+
+UBUNTU_APP_LAUNCH_CG_MANAGER_SESSION_BUS
+  Tell UAL to look on the session bus for CG Manager.
+
+UBUNTU_APP_LAUNCH_DEMANGLER
+  Path to the UAL demangler tool that will get the Mir FD for trusted prompt session.
+
+UBUNTU_APP_LAUNCH_DISABLE_SNAPD_TIMEOUT
+  Wait as long as Snapd wants to return data instead of erroring after 100ms.
+
+UBUNTU_APP_LAUNCH_LEGACY_ROOT
+  Set the path that represents the root for legacy applications.
+
+UBUNTU_APP_LAUNCH_LIBERTINE_LAUNCH
+  Path to the libertine launch utility for setting up libertine containers and XMir based legacy apps.
+
+UBUNTU_APP_LAUNCH_LINK_FARM
+  Path to the link farm that is created by Click of all the installed Click applications.
+
+UBUNTU_APP_LAUNCH_OOM_HELPER
+  Path to the setuid helper that configures OOM values on application processes that we otherwise couldn't, mostly this is for Oxide.
+
+UBUNTU_APP_LAUNCH_OOM_PROC_PATH
+  Path to look for the files to set OOM values, defaults to /proc.
+
+UBUNTU_APP_LAUNCH_SNAP_BASEDIR
+  The place where snaps are installed in the system, /snap is the default.
+
+UBUNTU_APP_LAUNCH_SNAPD_SOCKET
+  Path to the snapd socket.
+
+
 API Documentation
 =================
 

=== modified file 'libubuntu-app-launch/registry-impl.cpp'
--- libubuntu-app-launch/registry-impl.cpp	2016-11-10 21:59:11 +0000
+++ libubuntu-app-launch/registry-impl.cpp	2016-11-10 21:59:12 +0000
@@ -48,6 +48,14 @@
         return std::shared_ptr<GDBusConnection>(g_bus_get_sync(G_BUS_TYPE_SESSION, cancel.get(), nullptr),
                                                 [](GDBusConnection* bus) { g_clear_object(&bus); });
     });
+
+    /* Determine where we're getting the helper from */
+    oomHelper_ = OOM_HELPER;
+    auto goomHelper = g_getenv("UBUNTU_APP_LAUNCH_OOM_HELPER");
+    if (goomHelper != nullptr)
+    {
+        oomHelper_ = goomHelper;
+    }
 }
 
 void Registry::Impl::initClick()

=== modified file 'libubuntu-app-launch/registry-impl.h'
--- libubuntu-app-launch/registry-impl.h	2016-11-10 21:59:11 +0000
+++ libubuntu-app-launch/registry-impl.h	2016-11-10 21:59:12 +0000
@@ -83,6 +83,11 @@
     static void watchingAppStarting(bool rWatching);
     static bool isWatchingAppStarting();
 
+    const std::string oomHelper()
+    {
+        return oomHelper_;
+    }
+
 private:
     Registry* _registry; /**< The Registry that we're spawned from */
 
@@ -97,6 +102,13 @@
     /** All of our icon finders based on the path that they're looking
         into */
     std::unordered_map<std::string, std::shared_ptr<IconFinder>> _iconFinders;
+
+    /** Getting the Upstart job path is relatively expensive in
+        that it requires a DBus call. Worth keeping a cache of. */
+    std::map<std::string, std::string> upstartJobPathCache_;
+
+    /** Path to the OOM Helper */
+    std::string oomHelper_;
 };
 
 }  // namespace app_launch

=== modified file 'libubuntu-app-launch/ubuntu-app-launch.cpp'
--- libubuntu-app-launch/ubuntu-app-launch.cpp	2016-11-10 21:59:11 +0000
+++ libubuntu-app-launch/ubuntu-app-launch.cpp	2016-11-10 21:59:12 +0000
@@ -1672,7 +1672,11 @@
 	/* The exec value */
 	gchar * envstr = NULL;
 	if (demangler) {
-		envstr = g_strdup_printf("APP_EXEC=%s %s", DEMANGLER_PATH, execline);
+		const gchar * demangler_path = g_getenv("UBUNTU_APP_LAUNCH_DEMANGLER");
+		if (demangler_path == nullptr) {
+			demangler_path = DEMANGLER_PATH;
+		}
+		envstr = g_strdup_printf("APP_EXEC=%s %s", demangler_path, execline);
 	} else {
 		envstr = g_strdup_printf("APP_EXEC=%s", execline);
 	}

=== modified file 'upstart-jobs/application-click.conf.in'
--- upstart-jobs/application-click.conf.in	2015-08-17 21:34:39 +0000
+++ upstart-jobs/application-click.conf.in	2016-11-10 21:59:12 +0000
@@ -14,7 +14,6 @@
 env APP_XMIR_ENABLE
 
 env UBUNTU_APP_LAUNCH_ARCH="@ubuntu_app_launch_arch@"
-export UBUNTU_APP_LAUNCH_ARCH
 
 apparmor switch ${APP_ID}
 cgroup freezer
@@ -23,12 +22,12 @@
 oom score 100
 
 # Remember, this is confined
-exec @pkglibexecdir@/exec-line-exec
+exec ${SNAP}/@pkglibexecdir@/exec-line-exec
 
-post-start exec @pkglibexecdir@/zg-report-app open
+post-start exec ${SNAP}/@pkglibexecdir@/zg-report-app open
 post-stop script
-	@pkglibexecdir@/zg-report-app close
-	@pkglibexecdir@/cgroup-reap-all
+	${SNAP}/@pkglibexecdir@/zg-report-app close
+	${SNAP}/@pkglibexecdir@/cgroup-reap-all
 
 	DEVELOPER_MODE=`gdbus call --system --dest com.canonical.PropertyService --object-path /com/canonical/PropertyService --method com.canonical.PropertyService.GetProperty adb`
 	if [ "$DEVELOPER_MODE" != "(true,)" ] ; then

=== modified file 'upstart-jobs/application-failed.conf.in'
--- upstart-jobs/application-failed.conf.in	2016-08-04 14:04:40 +0000
+++ upstart-jobs/application-failed.conf.in	2016-11-10 21:59:12 +0000
@@ -3,4 +3,4 @@
 start on stopped application-legacy RESULT=failed or stopped application-click RESULT=failed or stopped application-snap RESULT=failed
 task
 
-exec @pkglibexecdir@/application-failed
+exec ${SNAP}/@pkglibexecdir@/application-failed

=== modified file 'upstart-jobs/application-legacy.conf.in'
--- upstart-jobs/application-legacy.conf.in	2016-09-23 22:01:53 +0000
+++ upstart-jobs/application-legacy.conf.in	2016-11-10 21:59:12 +0000
@@ -22,12 +22,12 @@
 oom score 110
 
 # This could be confined
-exec @pkglibexecdir@/exec-line-exec
+exec ${SNAP}/@pkglibexecdir@/exec-line-exec
 
-post-start exec @pkglibexecdir@/zg-report-app open
+post-start exec ${SNAP}/@pkglibexecdir@/zg-report-app open
 post-stop script
-	@pkglibexecdir@/zg-report-app close
-	@pkglibexecdir@/cgroup-reap-all
+	${SNAP}/@pkglibexecdir@/zg-report-app close
+	${SNAP}/@pkglibexecdir@/cgroup-reap-all
 
 	DEVELOPER_MODE=`gdbus call --system --dest com.canonical.PropertyService --object-path /com/canonical/PropertyService --method com.canonical.PropertyService.GetProperty adb`
 	if [ "$DEVELOPER_MODE" != "(true,)" ] ; then

=== modified file 'upstart-jobs/application-snap.conf.in'
--- upstart-jobs/application-snap.conf.in	2016-09-08 15:05:06 +0000
+++ upstart-jobs/application-snap.conf.in	2016-11-10 21:59:12 +0000
@@ -15,7 +15,6 @@
 env INSTANCE_ID=""
 
 env UBUNTU_APP_LAUNCH_ARCH="@ubuntu_app_launch_arch@"
-export UBUNTU_APP_LAUNCH_ARCH
 
 # apparmor is taken care of by confine
 cgroup freezer
@@ -24,12 +23,12 @@
 oom score 100
 
 # Remember, this is confined
-exec @pkglibexecdir@/exec-line-exec
+exec ${SNAP}/@pkglibexecdir@/exec-line-exec
 
-post-start exec @pkglibexecdir@/zg-report-app open
+post-start exec ${SNAP}/@pkglibexecdir@/zg-report-app open
 post-stop script
-	@pkglibexecdir@/zg-report-app close
-	@pkglibexecdir@/cgroup-reap-all
+	${SNAP}/@pkglibexecdir@/zg-report-app close
+	${SNAP}/@pkglibexecdir@/cgroup-reap-all
 
 	DEVELOPER_MODE=`gdbus call --system --dest com.canonical.PropertyService --object-path /com/canonical/PropertyService --method com.canonical.PropertyService.GetProperty adb`
 	if [ "$DEVELOPER_MODE" != "(true,)" ] ; then

=== modified file 'upstart-jobs/application.conf.in'
--- upstart-jobs/application.conf.in	2014-02-05 20:54:29 +0000
+++ upstart-jobs/application.conf.in	2016-11-10 21:59:12 +0000
@@ -14,4 +14,4 @@
 env APP_URIS
 export APP_URIS
 
-exec @pkglibexecdir@/application-job
+exec ${SNAP}/@pkglibexecdir@/application-job

=== modified file 'upstart-jobs/untrusted-helper-type-end.conf.in'
--- upstart-jobs/untrusted-helper-type-end.conf.in	2014-01-22 17:35:10 +0000
+++ upstart-jobs/untrusted-helper-type-end.conf.in	2016-11-10 21:59:12 +0000
@@ -6,4 +6,4 @@
 
 emits untrusted-helper-end
 
-exec @pkglibexecdir@/untrusted-helper-type-end
+exec ${SNAP}/@pkglibexecdir@/untrusted-helper-type-end

=== modified file 'upstart-jobs/untrusted-helper.conf.in'
--- upstart-jobs/untrusted-helper.conf.in	2014-08-07 16:16:05 +0000
+++ upstart-jobs/untrusted-helper.conf.in	2016-11-10 21:59:12 +0000
@@ -12,7 +12,6 @@
 env APP_URIS
 
 env UBUNTU_APP_LAUNCH_ARCH="@ubuntu_app_launch_arch@"
-export UBUNTU_APP_LAUNCH_ARCH
 
 apparmor switch ${APP_ID}
 cgroup freezer
@@ -20,8 +19,8 @@
 
 # This is unconfined
 pre-start script
-	if [ -x "@pkglibexecdir@/${HELPER_TYPE}/exec-tool" ] ; then
-		@pkglibexecdir@/${HELPER_TYPE}/exec-tool
+	if [ -x "${SNAP}/@pkglibexecdir@/${HELPER_TYPE}/exec-tool" ] ; then
+		${SNAP}/@pkglibexecdir@/${HELPER_TYPE}/exec-tool
 	else
 		echo "Unable to find exec tool for ${HELPER_TYPE}"
 		exit -1
@@ -29,6 +28,6 @@
 end script
 
 # Remember, this is confined
-exec @pkglibexecdir@/exec-line-exec
+exec ${SNAP}/@pkglibexecdir@/exec-line-exec
 
-post-stop exec @pkglibexecdir@/cgroup-reap-all
+post-stop exec ${SNAP}/@pkglibexecdir@/cgroup-reap-all