apparmor should be allowed to start in containers

Bug #1628285 reported by Stéphane Graber
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
apparmor (Ubuntu)
Fix Released
High
Tyler Hicks
Trusty
Fix Released
High
Unassigned
Xenial
Fix Released
High
Tyler Hicks
upstart (Ubuntu)
Invalid
Undecided
Unassigned
Trusty
Won't Fix
High
Unassigned

Bug Description

=apparmor and upstart 14.04 SRU=
[Impact]
A recent 16.04 kernel (4.4.0-46.67) and the lxd (2.0.5-0ubuntu1~ubuntu16.04.1) allows us to enable stacked/namespaced AppArmor policy for 14.04 lxd containers. This means that the container can have an overall confinement profile while still allowing individual processes inside of the container to have individual confinement profiles. This bug is for the apparmor and upstart userspace changes needed to allow the container init to load apparmor profiles during the container boot procedure.

[Test Case]
Install the latest Xenial kernel and lxd. Reboot into the new kernel and set up a new 14.04 lxd container (MUST be an unprivileged container):

 $ lxc launch ubuntu-daily:14.04 t

Install apparmor from trusty-proposed (2.10.95-0ubuntu2.5~14.04.1) and upstart from trusty-proposed (1.12.1-0ubuntu4.3) inside of the container and reboot the container.

Verify that the container's dhclient is confined inside of an AppArmor namespace with a stacked profile that was loaded inside of the container:

$ ps auxZ | grep '^lxd-t_</var/lib/lxd>//&:lxd-t_<var-lib-lxd>:///sbin/dhclient'
lxd-t_</var/lib/lxd>//&:lxd-t_<var-lib-lxd>:///sbin/dhclient (enforce) 165536 3889 0.0 0.0 16120 860 ? Ss 03:55 0:00 /sbin/dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -I -df /var/lib/dhcp/dhclient6.eth0.leases eth0

Verify that aa-status works inside of the container:

$ lxc exec t -- aa-status
apparmor module is loaded.
4 profiles are loaded.
4 profiles are in enforce mode.
   /sbin/dhclient
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/connman/scripts/dhclient-script
   /usr/sbin/tcpdump
0 profiles are in complain mode.
1 processes have profiles defined.
1 processes are in enforce mode.
   /sbin/dhclient (518)
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

Now, examine the output of aa-status to verify that the /usr/sbin/tcpdump profile is loaded.

To validate the upstart change, use apparmor-profile-load to load a profile:

$ echo "profile lp1628285-test {} " | lxc exec t -- tee /etc/apparmor.d/lp1628285-test
$ lxc exec t -- /lib/init/apparmor-profile-load lp1628285-test
$ lxc exec t -- aa-status
apparmor module is loaded.
5 profiles are loaded.
5 profiles are in enforce mode.
   /sbin/dhclient
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/connman/scripts/dhclient-script
   /usr/sbin/tcpdump
   lp1628285-test
0 profiles are in complain mode.
1 processes have profiles defined.
1 processes are in enforce mode.
   /sbin/dhclient (518)
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.
$ lxc exec t -- ls /etc/apparmor.d/cache/lp1628285-test
/etc/apparmor.d/cache/lp1628285-test

Now, reboot and then run aa-status again to verify that the output is the same (except for the process ID numbers).

It is also a good test to install ntp and cups-daemon, use aa-status to verify that their profiles are in enforce mode and that their processes are confined. Then reboot and use aa-status to verify the same thing.

[Regression Potential]
The regression potential is relatively high because processes inside of Ubuntu containers can be confined with an additional profile that is loaded inside of the container. This feature was released in Ubuntu 16.10 and 16.04 with no known serious issues so far.

IMPORTANT: There is a known regression that may be seen by users of `lxc exec`. See bug #1641236 for details. Bug #1640868 is pre-existing, doesn't seem to have any negative side-effects, and is not caused by this SRU.

=apparmor 16.04 SRU=
[Impact]
The kernel in xenial-proposed (4.4.0-46.67) and the lxd that has recently migrated from xenial-proposed (2.0.5-0ubuntu1~ubuntu16.04.1) allows us to enable stacked/namespaced AppArmor policy for lxd containers. This means that the container can have an overall confinement profile while still allowing individual processes inside of the container to have individual confinement profiles. This bug is for the apparmor userspace changes needed to allow the container init to load apparmor profiles during the container boot procedure.

[Test Case]
Install the kernel from xenial-proposed (4.4.0-46.67). Reboot into the new kernel and set up a new xenial lxd container (MUST be an unprivileged container):

 $ lxc start ubuntu:16.04 x

Install apparmor from xenial-proposed (2.10.95-0ubuntu2.5) inside of the container and reboot the container.

Verify that the container's dhclient is confined inside of an AppArmor namespace with a stacked profile that was loaded inside of the container:

$ ps auxZ | grep '^lxd-x_</var/lib/lxd>//&:lxd-x_<var-lib-lxd>:///sbin/dhclient'
lxd-x_</var/lib/lxd>//&:lxd-x_<var-lib-lxd>:///sbin/dhclient (enforce) 165536 3889 0.0 0.0 16120 860 ? Ss 03:55 0:00 /sbin/dhclient -1 -v -pf /run/dhclient.eth0.pid -lf /var/lib/dhcp/dhclient.eth0.leases -I -df /var/lib/dhcp/dhclient6.eth0.leases eth0

[Regression Potential]
The regression potential is relatively high because processes inside of Ubuntu containers can be confined with an additional profile that is loaded inside of the container. However, this feature was released in Ubuntu 16.10 with no known issues so far.

=Original Description=

Now that we have support for apparmor namespacing and stacking, unprivileged containers can and should be allowed to load apparmor profiles.

The following changes are needed at least:
 - Change the systemd unit to remove the "!container" condition
 - Change the apparmor init script, replacing the current simple container check for something along the lines of:
    - If /proc/self/attr/current says "unconfined"
    - And /sys/kernel/security/apparmor/features/domain/stack contains "yes"
    - And /sys/kernel/security/apparmor/features/domain/version is 1.2 or higher
    - Then continue execing the script, otherwise exit 0

John suggested he could add a file which would provide a more reliable way to do this check ^

In either case, we need this change so that containers can behave more like normal systems as far as apparmor is concerned. That change should also be SRUed back to Xenial at the same time the kernel support for stacking is pushed.

This bug is effectively a blocker for snapd inside LXD as without this, snap-confine and snapd itself will not be confined after container restart.

Changed in apparmor (Ubuntu):
importance: Undecided → High
tags: added: lxd
Revision history for this message
John Johansen (jjohansen) wrote :

slight revision

/sys/kernel/security/apparmor/features/domain/ns_stacked contains yes/no if stacked across policy namespace

/sys/kernel/security/apparmor/features/domain/ns_name contains the name of the namespace

as long as lxc sets up a detectable namespace ns_name can be used to detect if it should load or not, as stacking, and stacking across namespaces will start to be used in other ways. So testing for just stack or ns_stack might not be enough

Revision history for this message
Tyler Hicks (tyhicks) wrote :

I'm willing to update the apparmor init script to fix this bug. What pattern should I check for when examining ns_name to decide if it is an LXC container?

Tyler Hicks (tyhicks)
Changed in apparmor (Ubuntu):
status: New → Incomplete
Revision history for this message
Stéphane Graber (stgraber) wrote :

You can check for "lxd-*" or "lxc-*", that should catch anything we do with LXC or LXD.

Changed in apparmor (Ubuntu):
status: Incomplete → New
Tyler Hicks (tyhicks)
Changed in apparmor (Ubuntu):
status: New → In Progress
assignee: nobody → Tyler Hicks (tyhicks)
Tyler Hicks (tyhicks)
Changed in apparmor (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apparmor - 2.10.95-4ubuntu5

---------------
apparmor (2.10.95-4ubuntu5) yakkety; urgency=medium

  * debian/lib/apparmor/functions, debian/apparmor.init,
    debian/apparmor.service, debian/apparmor.upstart,
    debian/lib/apparmor/profile-load: Adjust the checks that previously kept
    AppArmor policy from being loaded while booting a container. Now we
    attempt to load policy if we're in a LXD or LXC managed container that is
    using profile stacking inside of a policy namespace. (LP: #1628285)
  * Fix regression tests so that the kernel SRU process is not interrupted by
    failing tests
    - debian/patches/r3505-tests-fix-stacking-mode-checks.patch: Fix the
      stackonexec.sh and stackprofile.sh tests (LP: #1628295)
    - debian/patches/r3509-tests-fix-exec_stack-errors.patch: Fix the
      exec_stack.sh test (LP: #1628745)

 -- Tyler Hicks <email address hidden> Thu, 29 Sep 2016 00:38:47 -0500

Changed in apparmor (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Martin Pitt (pitti) wrote : Please test proposed package

Hello Stéphane, or anyone else affected,

Accepted apparmor into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apparmor/2.10.95-0ubuntu2.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apparmor (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed
Revision history for this message
Tyler Hicks (tyhicks) wrote :

I've completed the AppArmor test plan:

  https://wiki.ubuntu.com/Process/Merges/TestPlans/AppArmor

I've also manually verified the AppArmor portion of this SRU.

description: updated
Changed in apparmor (Ubuntu Xenial):
status: Fix Committed → In Progress
importance: Undecided → High
assignee: nobody → Tyler Hicks (tyhicks)
tags: added: verification-done
removed: verification-needed
Changed in apparmor (Ubuntu Xenial):
status: In Progress → Fix Committed
Tyler Hicks (tyhicks)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apparmor - 2.10.95-0ubuntu2.5

---------------
apparmor (2.10.95-0ubuntu2.5) xenial; urgency=medium

  * debian/lib/apparmor/functions, debian/apparmor.init,
    debian/apparmor.service, debian/apparmor.upstart,
    debian/lib/apparmor/profile-load: Adjust the checks that previously kept
    AppArmor policy from being loaded while booting a container. Now we
    attempt to load policy if we're in a LXD or LXC managed container that is
    using profile stacking inside of a policy namespace. (LP: #1628285)
  * Fix regression tests for stacking so that the kernel SRU process is not
    interrupted by failing tests whenever the AppArmor stacking features are
    backported from the 16.10 kernel or when the 16.04 LTS Enablement Stack
    receives a 4.8 or newer kernel
    - debian/patches/r3509-tests-fix-exec_stack-errors-1.patch: Fix the
      exec_stack.sh test when running on 4.8 or newer kernels (LP: #1628745)
    - debian/patches/r3558-tests-fix-exec_stack-errors-2.patch: Adjust the
      exec_stack.sh fix mentioned above to more accurately test kernels older
      than 4.8 (LP: #1630069)
    - debian/patches/allow-stacking-tests-to-use-system.patch: Apply this
      patch earlier in the series, as to match when it was committed upstream,
      so that the above two patches can be cherry-picked from lp:apparmor

 -- Tyler Hicks <email address hidden> Fri, 07 Oct 2016 05:21:44 +0000

Changed in apparmor (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for apparmor has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Tyler Hicks (tyhicks)
Changed in apparmor (Ubuntu Trusty):
assignee: nobody → Tyler Hicks (tyhicks)
importance: Undecided → High
status: New → In Progress
no longer affects: upstart (Ubuntu Xenial)
Changed in upstart (Ubuntu):
status: New → Invalid
Changed in upstart (Ubuntu Trusty):
status: New → In Progress
importance: Undecided → High
assignee: nobody → Tyler Hicks (tyhicks)
description: updated
Tyler Hicks (tyhicks)
description: updated
Revision history for this message
Steve Langasek (vorlon) wrote :

> IMPORTANT: There is a known regression that may be seen by
> users of `lxc exec`. See bug #1641243 for details.

I don't see any mention of an lxc exec regression in bug #1641243. Please explain here what the known regression is, and why this is thought to be acceptable in an SRU.

Please also elaborate why support for loading apparmor profiles in a 14.04 container on a 16.04 host is an appropriate rationale for an SRU. Is this related to supporting snappy inside a 14.04 container? I understand the argument for supporting snappy on a 14.04 host; I'm less clear on the rationale for users to want snappy support in a 14.04 lxd container, as opposed to simply spinning up a 16.04 lxd container to get snappy support.

Changed in apparmor (Ubuntu Trusty):
status: In Progress → Incomplete
Changed in upstart (Ubuntu Trusty):
status: In Progress → Incomplete
Tyler Hicks (tyhicks)
description: updated
Revision history for this message
Tyler Hicks (tyhicks) wrote : Re: [Bug 1628285] Re: apparmor should be allowed to start in containers

On 11/12/2016 12:36 PM, Steve Langasek wrote:
>> IMPORTANT: There is a known regression that may be seen by
>> users of `lxc exec`. See bug #1641243 for details.
>
> I don't see any mention of an lxc exec regression in bug #1641243.
> Please explain here what the known regression is, and why this is
> thought to be acceptable in an SRU.

That was a copy and paste error. I've updated the description to point
to the correct bug (bug #1641236).

It may not be acceptable for an SRU but it is low impact. I think we
need to weigh our options here. See below...

> Please also elaborate why support for loading apparmor profiles in a
> 14.04 container on a 16.04 host is an appropriate rationale for an SRU.
> Is this related to supporting snappy inside a 14.04 container? I
> understand the argument for supporting snappy on a 14.04 host; I'm less
> clear on the rationale for users to want snappy support in a 14.04 lxd
> container, as opposed to simply spinning up a 16.04 lxd container to get
> snappy support.

If we don't care to support snaps inside of a 14.04 container, then I
can back out the various apparmor changes that allow loading of policy
inside of lxd containers and I can also drop the upstart SRU.

I haven't heard of a hard requirement to support snaps inside of 14.04
LXD containers so I'll ask around to gauge the interest.

Revision history for this message
Tyler Hicks (tyhicks) wrote :

This bug will not be fixed in 14.04, meaning that AppArmor policy will not be loaded inside of 14.04 LXD containers and snaps will not work inside of 14.04 LXD containers. 16.04 LXD containers should be used in such use cases.

Changed in apparmor (Ubuntu Trusty):
status: Incomplete → Won't Fix
Changed in upstart (Ubuntu Trusty):
status: Incomplete → Won't Fix
assignee: Tyler Hicks (tyhicks) → nobody
Changed in apparmor (Ubuntu Trusty):
assignee: Tyler Hicks (tyhicks) → nobody
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (5.3 KiB)

This bug was fixed in the package apparmor - 2.10.95-0ubuntu2.5~14.04.1

---------------
apparmor (2.10.95-0ubuntu2.5~14.04.1) trusty; urgency=medium

  * Bring apparmor 2.10.95-0ubuntu2.5, from Ubuntu 16.04, to Ubuntu 14.04.
    - This allows for proper snap confinement on Ubuntu 14.04 when using the
      hardware enablement kernel (LP: #1641243)
  * Changes made on top of 2.10.95-0ubuntu2.5:
    - debian/apparmor.upstart: Remove the upstart job and continue using the
      init script in 14.04
    - debian/apparmor.postinst, debian/apparmor-profiles.postinst,
      debian/apparmor-profiles.postrm, debian/rules: Revert to using
      invoke-rc.d to load the profiles, rather than reloading them directly,
      since 14.04 will continue using the init script rather than the upstart
      job.
    - debian/apparmor.init, debian/lib/apparmor/functions,
      debian/apparmor.postinst, debian/apparmor.postrm: Remove functionality
      dealing with AppArmor policy in system image based environments since
      this 14.04 package will not need to handle such environments. This
      removes the handle_system_policy_package_updates(),
      compare_previous_version(), compare_and_save_debsums() functions and
      their callers.
    - debian/apparmor.init: Continue using running-in-container since
      systemd-detect-virt doesn't exist on 14.04
    - debian/lib/apparmor/functions, debian/apparmor.init: Remove the
      is_container_with_internal_policy() function and adjust its call sites
      in apparmor.init so that AppArmor policy is not loaded inside of 14.04
      LXD containers (avoids bug #1641236)
    - debian/lib/apparmor/profile-load, debian/apparmor.install: Remove
      profile-load as upstart's apparmor-profile-load is used in 14.04
    - debian/patches/libapparmor-mention-dbus-method-in-getcon-man.patch:
      Continue applying this patch since the dbus version in 14.04 isn't new
      enough to support fetching the AppArmor context from
      org.freedesktop.DBus.GetConnectionCredentials().
    - debian/patches/libapparmor-force-libtoolize-replacement.patch: Force
      libtoolize to replace existing files to fix a libapparmor FTBFS issue on
      14.04.
    - debian/control: Retain the original 14.04 Breaks and ignore the new
      Breaks from 2.10.95-0ubuntu2.5 since they were put in place as part of
      the enablement of UNIX domain socket mediation. They're not needed in
      this upload since UNIX domain socket mediation is disabled by default so
      updates to the profiles included in those packages are not needed.
    - Preserve the profiles and abstractions from 14.04's
      2.8.95~2430-0ubuntu5.3 apparmor package by recreating them in the
      top-level profiles-14.04/ directory of the source. They'll be installed
      to debian/tmp/etc/apparmor.d/ during the build process and then to
      /etc/apparmor.d/ on package install so that there are no changes to the
      shipped profiles or abstractions. The abstractions from
      2.10.95-0ubuntu2.5 will be installed into
      debian/tmp/snap/etc/apparmor.d/ during the build process and then into
      /etc/apparmor.d/snap/abstractions/ on package install for use wit...

Read more...

Changed in apparmor (Ubuntu Trusty):
status: Won't Fix → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.