Merge ~ahasenack/ubuntu/+source/rsyslog:lunar-rsyslog-enable-apparmor-dep8-take4-dot-d into ubuntu/+source/rsyslog:ubuntu/devel

[Andreas Hasenack]

Enable apparmor in enforce mode by default on fresh installs. UPDATE: upgrades from older versions (< $this-version) WILL change the confinement status.

PPA: https://launchpad.net/~ahasenack/+archive/ubuntu/rsyslog-apparmor-dep8-take4/

LP won't run the DEP8 tests because this package never had such tests before, so you will have to run them locally. Bileto also didn't work, they are queued up for days (and against an older version of this work, so don't bother): https://bileto.ubuntu.com/#/ticket/4980

This is take 4 of my experiments with enforcing apparmor on rsyslog. How it works is documented in the new d/README.apparmor file.

It's a lot of delta to add at first glance, but I think it's easy to maintain as it's mostly new files, and small touches in existing ones. More importantly, no dependency or packaging changes, and we have tests :)

I checked the whole archive (with apt-file find) for packages that install files in /etc/rsyslog.d, looking for the ones that would violate the current apparmor rules (I know this misses packages that generate those config files in postinst). I found two:

prometheus-postfix-exporter
This package adds a rsyslog config that tells it to log mail messages to /var/lib/prometheus/postfix-exporter/mail.log, which is not covered by the apparmor profile. I used this package as an example to show what maintainers can do in such a case, and pushed it to the same PPA, and this branch:

https://code.launchpad.net/~ahasenack/ubuntu/+source/prometheus-postfix-exporter/+git/prometheus-postfix-exporter/+ref/lunar-prometheus-postfix-exporter-rsyslog-apparmor

The change is simple (I think):

https://git.launchpad.net/~ahasenack/ubuntu/+source/prometheus-postfix-exporter/commit/?id=dac36355f663bdc1eaf8aac164865b9d19b028f2

UPDATE 2022-02-11: that is no longer needed, as debian's latest upload of this package gave up on the custom mail.log file and is now using the normal /var/log/mail.log one, which is covered by the normal rsyslog apparmor profile.

octopussy
This one I didn't bother updating, because a) it's removed from debian; b) it's broken even without these apparmor changes. But the change would be the same approach as prometheus-postfix-exporter. It is piping logs to a fifo, so we would need an extra apparmor rule for that.

I still have other bin:rsyslog-* packages (from src:rsyslog) to go over, but the approach is the same, and I think it's time to get this reviewed and hopefully uploaded.

# Sending to debian
Before sending any of this to debian, we need to see how it will work out in practice for us.

# Merge from debian
There is a new debian upload which needs merging, and I commit to that after this branch.

# Take4? What's in take1, 2, and 3?

Just as a summary of what I tried previously, in case you are wondering "why didn't you try X?". If you want more details about those attempts, ask.

## Triggers
Work beautifully, but run too late. I found out many packages (part of rsyslog, and third party ones) restart rsyslog in their postinst, even though there is a trigger for that (just place a file in /etc/rsyslog.d and the trigger will restart rsyslog at the end of the apt/dpkg transaction). This meant that for a while rsyslog would be running without the apparmor changes applied by the trigger, and that was enough to cause it to fail

## Config file analysis
This looked promising, and I can still use it in the future if I want, but as the sole method of adjusting the apparmor profile, it's complicated. I can easily enough get a full dump of the whole config file, with all the included files, but the problem is that rsyslog supports 3 config languages: very old syslog, old syslog, and current rainer script. The last two ones are hard to parse, as can be multi-line and have "if" statements and so on. "grep" only takes me so far. I can still use this for some cases, at least to warn the user that something might not work, like when the "omprog" plugin is used (or pipe), as this one can call out to any binary on the system, anywhere.

[Simon Déziel (sdeziel)]

Thanks for working on this, I'll happily take it for a test drive!

You cannot include <rsyslog/include.d/*.apparmor> but you can include a whole directory:

  include if exists <rsyslog/include.d/>

What do you think?

[Andreas Hasenack (ahasenack)]

> include if exists <rsyslog/include.d/>

I briefly looked at the manpage and didn't see this option mentioned, of including whole directories. I'll search again, but if you have a direct link to it, I'd appreciate it :)

I'm curious to see what types of files does this include, if it's all files regardless of extension (bad), or something else.

[Simon Déziel (sdeziel)]

I didn't find any mention in the man page but figured it with trial/error.

IMHO, having it include everything isn't that big of a deal. Putting a notice in commented out text in /etc/apparmor.d/rsyslog/include.d/README should make it clear to everyone. Something like this:

# Every file under /etc/apparmor.d/rsyslog/include.d/
# is included by the rsyslog Apparmor profile

[Andreas Hasenack (ahasenack)]

> IMHO, having it include everything isn't that big of a deal.

We can't have it include everything. Think of *.dpkg-new, *.conf~, *~ backup files, editor temp files, even this README, etc.

[Seth Arnold (seth-arnold)]

I think the parser already skips the *.dpkg-new etc files:

https://sources.debian.org/src/apparmor/3.0.8-2/parser/parser_main.c/#L1329
https://sources.debian.org/src/apparmor/3.0.8-2/libraries/libapparmor/src/private.c/#L127
https://sources.debian.org/src/apparmor/3.0.8-2/libraries/libapparmor/src/private.c/#L65

static struct ignored_suffix_t ignored_suffixes[] = {
 /* Debian packging files, which are in flux during install
           should be silently ignored. */
 { ".dpkg-new", 9, 1 },
 { ".dpkg-old", 9, 1 },
 { ".dpkg-dist", 10, 1 },
 { ".dpkg-bak", 9, 1 },
 { ".dpkg-remove", 12, 1 },
 /* Archlinux packaging files */
 { ".pacsave", 8, 1 },
 { ".pacnew", 7, 1 },
 /* RPM packaging files have traditionally not been silently
           ignored */
 { ".rpmnew", 7, 0 },
 { ".rpmsave", 8, 0 },
 /* patch file backups/conflicts */
 { ".orig", 5, 0 },
 { ".rej", 4, 0 },
 /* Backup files should be mentioned */
 { "~", 1, 0 },
 { NULL, 0, 0 }
};

[Andreas Hasenack (ahasenack)]

Any idea why this isn't documented? It was just forgotten, or is this an experimental feature, or a happy side effect of something else? I just don't want to rely on something that could go away all of a sudden, or change behavior unexpectedly, and then hear "you were relying on undocumented behavior, sorry" ;)

[Simon Déziel (sdeziel)]

If the `ignored_suffixes` thing is officially supported, including a README file containing a few lines starting with "# " shouldn't matter too much, wouldn't it?

[Sergio Durigan Junior (sergiodj)]

The code to ignore certain file extensions is also replicated in the Python code that sits under utils/apparmor/:

https://git.launchpad.net/ubuntu/+source/apparmor/tree/utils/apparmor/rule/include.py#n141
https://git.launchpad.net/ubuntu/+source/apparmor/tree/utils/apparmor/common.py#n174

The feature is undocumented, but seems to be something formally supported by the project.

Also, README files are ignored so you shouldn't need to prefix its lines with a comment.

[Andreas Hasenack (ahasenack)]

Thanks for these links, Sergio, and the discussion in our standup. I'll proceed with making the changes to use a single include directory.

[Andreas Hasenack (ahasenack)]

I force-pushed changes to use a include directory without the intermediate generated file. Opening up for review again.

[Simon Déziel (sdeziel)]

LGTM, my only small observation is that reload-apparmor-profile could well be using `/bin/sh` instead of `/bin/bash`.

review: Approve

[Andreas Hasenack (ahasenack)]

> LGTM, my only small observation is that reload-apparmor-profile could well be
> using `/bin/sh` instead of `/bin/bash`.

Indeed, it got even simpler now. Switched.

[Andreas Hasenack (ahasenack)]

After a brief exchange in ubuntu-devel, I added a commit that changes the upgrade behavior of the package.

If upgrading from a version older than this one, apparmor will be enabled. In this way, if someone decides to disable apparmor after this update, it will remain so.

[Athos Ribeiro (athos-ribeiro)]

Thanks, Andreas!

This LGTM. I do appreciate you took the time to write documentation in different places here.

I added a few inline comments, mostly regarding the DEP8 tests introduced here.

[Andreas Hasenack (ahasenack)]

Made the last changes, uploaded a new build to the ppa (ppa17).

[Andreas Hasenack (ahasenack)]

I ran the tests locally on amd64:
autopkgtest [15:19:42]: @@@@@@@@@@@@@@@@@@@@ summary
apparmor-include-mechanism PASS
simple-logger PASS
simple-mysql PASS
simple-pgsql PASS

real 2m27,433s

We will only be able to run them in the ppa, with all architectures, once the package with at least one DEP8 test is uploaded and migrates.

[Athos Ribeiro (athos-ribeiro)]

Thanks for the changes and verification, Andreas. LGTM.

review: Approve

[git-ubuntu bot (git-ubuntu-bot)]

Approvers: ahasenack, athos-ribeiro
Uploaders: ahasenack, athos-ribeiro
MP auto-approved

review: Approve

[Andreas Hasenack (ahasenack)]

Thanks, uploaded:
Uploading rsyslog_8.2210.0-3ubuntu2.dsc
Uploading rsyslog_8.2210.0-3ubuntu2.debian.tar.xz
Uploading rsyslog_8.2210.0-3ubuntu2_source.buildinfo
Uploading rsyslog_8.2210.0-3ubuntu2_source.changes

diff --git a/debian/NEWS b/debian/NEWS
index 35058fa..1421f59 100644
--- a/debian/NEWS
+++ b/debian/NEWS
@@ -1,3 +1,33 @@
+rsyslog (8.2210.0-3ubuntu2) lunar; urgency=medium
+
+  The apparmor profile of rsyslog now defaults to be enforced on a fresh
+  install and upgrades from an earlier version. Upgrades from this version
+  forward won't change the enforcement status.
+
+  Packages that add an rsyslog configuration that might be blocked by the
+  apparmor profile, can add an apparmor configuration snippet in
+
+  /etc/apparmor.d/rsyslog.d/
+
+  This file should preferably be named like <pkg>.apparmor, but only standard
+  backup extensions are excluded. See
+  https://sources.debian.org/src/apparmor/3.0.8-2/libraries/libapparmor/src/private.c/#L68
+  for a list.
+
+  When the rsyslog service is started, its systemd unit file first executes
+  the /usr/lib/rsyslog/reload-apparmor-profile script via ExecStartPre. That
+  script will reload the rsyslogd apparmor profile including the configuration
+  snippets in /etc/apparmor.d/rsyslogd.d/, if any.
+
+  The confinement status is not changed.
+
+  After this, the unit proceeds to start rsyslog as usual.
+
+  For more information, check the README.apparmor file in the documentation
+  directory of this package.
+
+ -- Andreas Hasenack <[e-mail]>  Sun, 05 Feb 2023 15:42:31 -0300
+
 rsyslog (5.8.1-1) unstable; urgency=low
 
   The way rsyslog processes SIGHUP has changed. It no longer does a reload
diff --git a/debian/README.apparmor b/debian/README.apparmor
new file mode 100644
index 0000000..ab5e706
--- /dev/null
+++ b/debian/README.apparmor
@@ -0,0 +1,132 @@
+# rsyslog and AppArmor
+
+Starting with version 8.2210.0-3ubuntu2, on fresh installs and upgrades from
+earlier versions, rsyslog will be confined by default with AppArmor in enforce
+mode.
+
+The AppArmor profile for rsyslog has a static component, and a dynamic one. It
+all starts with the main profile in `/etc/apparmor.d/usr.sbin.rsyslogd`. That
+profile has an include directive for the dynamic component in
+`/etc/apparmor.d/rsyslog.d`:
+
+    # apparmor snippets for rsyslog from other packages
+    include if exists <rsyslog.d>
+
+All files placed in `/etc/apparmor.d/rsyslog.d` will be included, with the
+exception of standard backup files like files ending in `~`, or with a suffix
+generated by `dpkg` when there was a config file prompt. The full list of
+exclusions is not really documented, but can be inspected in the source code at
+https://sources.debian.org/src/apparmor/3.0.8-2/libraries/libapparmor/src/private.c/#L65.
+A `README` file is also ignored.
+
+When `rsyslog` is started, it will reload the apparmor profile, including all
+the snippets that may exist in the `rsyslog.d` include directory. This is done
+via a `ExecStartPre` call in the systemd unit file:
+
+    [Service]
+    Type=notify
+    ExecStartPre=/usr/lib/rsyslog/reload-apparmor-profile
+    ExecStart=/usr/sbin/rsyslogd -n -iNONE
+    ...
+
+Packages (and users) can place apparmor profile config file snippets in
+`/etc/apparmor.d/rsyslog.d/`. It is suggested that the filename be in the form
+of `<pkg>.apparmor`.
+
+For example, the `rsyslog-pgsql` debian package installs this file
+`/etc/apparmor.d/rsyslog.d/rsyslog-pgsql.apparmor`:
+
+    # PostgreSQL local access
+    include <abstractions/openssl>
+    include <abstractions/ssl_certs>
+    /etc/gss/mech.d/ r,
+    /etc/gss/mech.d/* r,
+    /{,var/}run/postgresql/.s.PGSQL.*[0-9] rw,
+
+When `rsyslog` starts, the `reload-apparmor-profile` will run and
+reload the `rsyslogd` apparmor profile just before rsyslogd itself is
+started. Note that the enforcement status of the profile (enforce, complain) is
+not changed.
+
+
+# Troubleshooting
+
+When rsyslog gets something denied, particularly if it's in an output module,
+it will retry a few times and eventually give up. It usually won´t crash, so
+the only way to notice something is wrong is by inspecting the logs, or, well,
+by noticing something isn't working, like logging to a database.
+
+Here are the most useful troubleshooting tips.
+
+
+## Watch the logs
+
+Look for rsyslog errors in the logs, particularly `/var/log/syslog`, or via
+`journalctl -u rsyslog.service -f`. For example, when it can't connect to a
+local MySQL server, messages like these will appear:
+
+    Jan 31 17:27:15 sender rsyslogd[82257]: ommysql: db error (2002): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (13)  [v8.2210.0]
+    Jan 31 17:27:15 sender rsyslogd[82257]: action 'action-8-ommysql' suspended (module 'ommysql'), retry 0. There should be messages before this one giving the reason for suspe>
+
+
+## Inspect dmesg/apparmor
+
+If the rsyslog apparmor profile is interfering with rsyslog, there will be
+messages about it in the `dmesg` output (or in the audit log, if `auditd` is
+installed). For example, for the mysql case:
+
+    [Tue Feb  7 12:35:28 2023] audit: type=1400 audit(1675773329.453:84): apparmor="DENIED" operation="connect" class="file" profile="rsyslogd" name="/run/mysqld/mysqld.sock" pid=15495 comm=72733A6D61696E20513A526567 requested_mask="wr" denied_mask="wr" fsuid=101 ouid=107
+
+Since the rsyslog apparmor profile now may include multiple files from
+`/etc/apparmor.d/rsyslog.d`, it helps to visualize the whole profile as one
+file. The `apparmor_parser` command can be used for that with the `-p` option:
+
+    # apparmor_parser -p /etc/apparmor.d/usr.sbin.rsyslogd
+    ...
+    ##included <rsyslog.d/rsyslog-mysql.apparmor>
+    # MySQL local server access
+    ...
+
+This will show all included files, including abstractions.
+
+
+# Example
+
+Here is an example of what it would look like to adapt a package that ships a
+rsyslog configuration that needs the rsyslog apparmor profile to be adjusted.
+
+The `prometheus-postfix-exporter` adds an rsyslog config that has it write logs
+to `/var/lib/prometheus/postfix-exporter/mail.log`, which is not allowed by the
+base rsyslog apparmor profile.
+
+This is what the fix for this package would look like:
+
+```diff
+
+diff --git a/debian/dirs b/debian/dirs
+index 6d3533d..50d9ad8 100644
+--- a/debian/dirs
++++ b/debian/dirs
+@@ -1,3 +1,4 @@
+ etc/rsyslog.d
++etc/apparmor.d/rsyslog.d
+ var/lib/prometheus/postfix-exporter
+ var/log/prometheus
+diff --git a/debian/rsyslog-prometheus-postfix-exporter.apparmor b/debian/rsyslog-prometheus-postfix-exporter.apparmor
+new file mode 100644
+index 0000000..1b9f85f
+--- /dev/null
++++ b/debian/rsyslog-prometheus-postfix-exporter.apparmor
+@@ -0,0 +1 @@
++  /var/lib/prometheus/postfix-exporter/mail.log rw,
+diff --git a/debian/rules b/debian/rules
+index e8ce2f9..ffcf383 100755
+--- a/debian/rules
++++ b/debian/rules
+@@ -16,3 +16,5 @@ override_dh_auto_install:
+ 	dh_auto_install -- --no-source
+ 	install -m644 debian/rsyslog.conf \
+             debian/$(BINNAME)/etc/rsyslog.d/$(BINNAME).conf
++	install -m644 debian/rsyslog-prometheus-postfix-exporter.apparmor \
++			debian/$(BINNAME)/etc/apparmor.d/rsyslog.d
+```
diff --git a/debian/README.apparmor.rsyslog.d b/debian/README.apparmor.rsyslog.d
new file mode 100644
index 0000000..030b9b9
--- /dev/null
+++ b/debian/README.apparmor.rsyslog.d
@@ -0,0 +1,16 @@
+# This directory is meant to be used by packages that need to augment the
+# existing rsyslogd profile with extra rules.  All files in here will be
+# included by the /etc/apparmor.d/usr.sbin.rsyslogd profile, subject to the
+# exclusion rules defined in
+#
+# https://sources.debian.org/src/apparmor/3.0.8-2/libraries/libapparmor/src/private.c/#L65
+#
+# and
+#
+# https://sources.debian.org/src/apparmor/3.0.8-2/libraries/libapparmor/src/private.c/#L132
+#
+# Please check the README.apparmor file in the documentation directory of the
+# rsyslog package for more information.
+#
+# For the usual overrides and other additions by local administrators, please
+# use the /etc/apparmor.d/local/ mechanism.
diff --git a/debian/apparmor/rsyslog-gnutls.apparmor b/debian/apparmor/rsyslog-gnutls.apparmor
new file mode 100644
index 0000000..9d5147e
--- /dev/null
+++ b/debian/apparmor/rsyslog-gnutls.apparmor
@@ -0,0 +1,3 @@
+# GnuTLS library rules
+
+  /etc/gnutls/config r,
diff --git a/debian/apparmor/rsyslog-mysql.apparmor b/debian/apparmor/rsyslog-mysql.apparmor
new file mode 100644
index 0000000..0f9ad34
--- /dev/null
+++ b/debian/apparmor/rsyslog-mysql.apparmor
@@ -0,0 +1,20 @@
+# MySQL local server access
+
+include <abstractions/openssl>
+
+/etc/mysql/my.cnf r,
+/etc/mysql/mysql.cnf r,
+/etc/mysql/my.cnf.fallback r,
+
+/etc/mysql/conf.d/ r,
+/etc/mysql/conf.d/mysql.cnf r,
+/etc/mysql/conf.d/mysqldump.cnf r,
+
+/etc/mysql/mysql.conf.d/ r,
+/etc/mysql/mysql.conf.d/mysql.cnf r,
+/etc/mysql/mysql.conf.d/mysqld.cnf r,
+
+/usr/share/mysql/charsets/Index.xml r,
+
+/{,var/}run/mysqld/mysqld.sock rw,
+
diff --git a/debian/apparmor/rsyslog-openssl.apparmor b/debian/apparmor/rsyslog-openssl.apparmor
new file mode 100644
index 0000000..f561b40
--- /dev/null
+++ b/debian/apparmor/rsyslog-openssl.apparmor
@@ -0,0 +1,3 @@
+# OpenSSL library rules
+
+  /etc/ssl/openssl.cnf r,
diff --git a/debian/apparmor/rsyslog-pgsql.apparmor b/debian/apparmor/rsyslog-pgsql.apparmor
new file mode 100644
index 0000000..3111a70
--- /dev/null
+++ b/debian/apparmor/rsyslog-pgsql.apparmor
@@ -0,0 +1,9 @@
+# PostgreSQL local access
+
+include <abstractions/openssl>
+include <abstractions/ssl_certs>
+
+/etc/gss/mech.d/ r,
+/etc/gss/mech.d/* r,
+/{,var/}run/postgresql/.s.PGSQL.*[0-9] rw,
+
diff --git a/debian/changelog b/debian/changelog
index b470943..b782302 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,45 @@
+rsyslog (8.2210.0-3ubuntu2) lunar; urgency=medium
+
+  * Support apparmor profile snippets:
+    - d/usr.sbin.rsyslogd: add "include if exists" for the rsyslog.d
+      directory, and remove the now unnecessary  mysql and postgresql
+      sections
+    - d/rsyslog.preinst: don't disable the apparmor profile on install
+    - d/rsyslog.postinst: remove disabling of apparmor on upgrades if we
+      are upgrading from a version older than $now.
+    - d/rsyslog.dirs: install /etc/apparmor.d/rsyslog.d/
+    - d/{apparmor/rsyslog-mysql,rsyslog-mysql.install}: add apparmor
+      profile for mysql plugin
+    - d/{apparmor/rsyslog-pgsql,rsyslog-pgsql.install}: add apparmor
+      profile for postgresql plugin
+    - d/{apparmor/rsyslog-gnutls.apparmor,rsyslog-gnutls.install}: add
+      apparmor profile for the gnutls plugin
+    - d/{apparmor/rsyslog-openssl.apparmor,rsyslog-gnutls.install}: add
+      apparmor profile for the openssl plugin
+    - New script to reload apparmor profile:
+      + d/rsyslog.service: reload apparmor profile in ExecStartPre and
+        set StandardError to journal so we can see errors from the
+        script
+      + d/rsyslog.install: install reload-apparmor-profile
+      + d/reload-apparmor-profile: script to reload the
+        rsyslogd apparmor profile
+    - d/NEWS: add info about apparmor changes in the Ubuntu packaging
+    - d/rsyslog.docs, d/README.apparmor: explains how the dynamic
+      component of the rsyslog apparmor profile is applied
+    - d/README.apparmor.rsyslog.d, d/rsyslog.install: install a specific
+      README file in the apparmor include directory for rsyslog
+  * Add DEP8 tests (LP: #1906333):
+    - d/t/control, d/t/simple-logger: simple logger test
+    - d/t/utils: common function(s)
+    - d/t/control, d/t/simple-mysql: DEP8 test using rsyslog with a
+      MySQL server
+    - d/t/control, d/t/simple-pgsql: DEP8 test using rsyslog with a
+      PostgreSQL server
+    - d/t/apparmor-include-mechanism: DEP8 test for the rsyslog.d
+      include mechanism used by the rsyslog apparmor profile
+
+ -- Andreas Hasenack <[e-mail]>  Fri, 17 Feb 2023 14:22:27 -0300
+
 rsyslog (8.2210.0-3ubuntu1) lunar; urgency=low
 
   * Merge from Debian unstable. Remaining changes:
diff --git a/debian/reload-apparmor-profile b/debian/reload-apparmor-profile
new file mode 100755
index 0000000..25c39e3
--- /dev/null
+++ b/debian/reload-apparmor-profile
@@ -0,0 +1,14 @@
+#!/bin/sh
+
+apparmor_profile="/etc/apparmor.d/usr.sbin.rsyslogd"
+include_dir="/etc/apparmor.d/rsyslog.d"
+
+[ -f "${apparmor_profile}" ] || exit 0
+[ -d "${include_dir}" ] || exit 0
+aa-status --enabled 2>/dev/null || exit 0
+
+apparmor_parser -r -W -T "${apparmor_profile}" || {
+    echo "Failed to reload the ${apparmor_profile} apparmor profile, continuing anyway" >&2
+}
+
+exit 0
diff --git a/debian/rsyslog-gnutls.install b/debian/rsyslog-gnutls.install
index c5784ce..4579469 100644
--- a/debian/rsyslog-gnutls.install
+++ b/debian/rsyslog-gnutls.install
@@ -1 +1,2 @@
 usr/lib/${DEB_HOST_MULTIARCH}/rsyslog/lmnsd_gtls.so
+debian/apparmor/rsyslog-gnutls.apparmor etc/apparmor.d/rsyslog.d/
diff --git a/debian/rsyslog-mysql.install b/debian/rsyslog-mysql.install
index 9de8064..fedff5c 100644
--- a/debian/rsyslog-mysql.install
+++ b/debian/rsyslog-mysql.install
@@ -1,2 +1,3 @@
 usr/lib/${DEB_HOST_MULTIARCH}/rsyslog/ommysql.so
 debian/rsyslog-mysql.conf.template usr/share/rsyslog-mysql/
+debian/apparmor/rsyslog-mysql.apparmor etc/apparmor.d/rsyslog.d/
diff --git a/debian/rsyslog-openssl.install b/debian/rsyslog-openssl.install
index 492defb..bc5358e 100644
--- a/debian/rsyslog-openssl.install
+++ b/debian/rsyslog-openssl.install
@@ -1 +1,2 @@
 usr/lib/${DEB_HOST_MULTIARCH}/rsyslog/lmnsd_ossl.so
+debian/apparmor/rsyslog-openssl.apparmor etc/apparmor.d/rsyslog.d/
diff --git a/debian/rsyslog-pgsql.install b/debian/rsyslog-pgsql.install
index 3fb57d3..946fe3f 100644
--- a/debian/rsyslog-pgsql.install
+++ b/debian/rsyslog-pgsql.install
@@ -1,2 +1,3 @@
 usr/lib/${DEB_HOST_MULTIARCH}/rsyslog/ompgsql.so
 debian/rsyslog-pgsql.conf.template usr/share/rsyslog-pgsql/
+debian/apparmor/rsyslog-pgsql.apparmor etc/apparmor.d/rsyslog.d/
diff --git a/debian/rsyslog.dirs b/debian/rsyslog.dirs
index 93f7b6a..1fdfad8 100644
--- a/debian/rsyslog.dirs
+++ b/debian/rsyslog.dirs
@@ -3,4 +3,4 @@
 /etc/apparmor.d/force-complain/
 /etc/apparmor.d/disable/
 /etc/apparmor.d/local/
-
+/etc/apparmor.d/rsyslog.d/
diff --git a/debian/rsyslog.docs b/debian/rsyslog.docs
index 62deb04..336a33e 100644
--- a/debian/rsyslog.docs
+++ b/debian/rsyslog.docs
@@ -1 +1,2 @@
 AUTHORS
+debian/README.apparmor
diff --git a/debian/rsyslog.install b/debian/rsyslog.install
index 5455991..66cd7b1 100755
--- a/debian/rsyslog.install
+++ b/debian/rsyslog.install
@@ -3,6 +3,7 @@ debian/rsyslog.conf  etc/
 debian/00rsyslog.conf  usr/lib/tmpfiles.d/
 debian/50-default.conf /usr/share/rsyslog
 debian/rsyslog-rotate usr/lib/rsyslog/
+debian/reload-apparmor-profile usr/lib/rsyslog/
 usr/sbin/
 usr/share/man/man5/
 usr/share/man/man8/
@@ -39,4 +40,5 @@ usr/lib/${DEB_HOST_MULTIARCH}/rsyslog/pm*.so
 [linux-any] usr/lib/${DEB_HOST_MULTIARCH}/rsyslog/imjournal.so
 [linux-any] usr/lib/${DEB_HOST_MULTIARCH}/rsyslog/omjournal.so
 debian/usr.sbin.rsyslogd etc/apparmor.d/
+debian/README.apparmor.rsyslog.d => etc/apparmor.d/rsyslog.d/README
 debian/dmesg.service lib/systemd/system
diff --git a/debian/rsyslog.postinst b/debian/rsyslog.postinst
index efc6d08..ea93fbf 100644
--- a/debian/rsyslog.postinst
+++ b/debian/rsyslog.postinst
@@ -44,6 +44,13 @@ case "$1" in
 	if dpkg --compare-versions "$2" lt-nl "8.2110.0-2"; then
 		update-rc.d -f rsyslog remove || true
 	fi
+
+	if dpkg --compare-versions "$2" lt-nl "8.2210.0-3ubuntu2~"; then
+	    # In this version we removed the disabling of the rsyslog apparmor
+	    # profile, i.e., it's enabled by default.  Gate on it to avoid
+	    # re-enabling it if the user has explicitly disabled it afterwards.
+	    rm -f /etc/apparmor.d/disable/usr.sbin.rsyslogd
+	fi
     ;;
 
     triggered)
diff --git a/debian/rsyslog.preinst b/debian/rsyslog.preinst
index 7e86bb8..2f726ab 100644
--- a/debian/rsyslog.preinst
+++ b/debian/rsyslog.preinst
@@ -6,19 +6,4 @@ if [ "$1" = "install" ] && [ -n "$2" ] ; then
 	[ -f /etc/logrotate.d/rsyslog.disabled ] && mv -f /etc/logrotate.d/rsyslog.disabled /etc/logrotate.d/rsyslog
 fi
 
-disable_profile() {
-    APP_CONFFILE="/etc/apparmor.d/usr.sbin.rsyslogd"
-    APP_DISABLE="/etc/apparmor.d/disable/usr.sbin.rsyslogd"
-    # Create a symlink to the yet-to-be-unpacked profile
-    if [ ! -e "$APP_CONFFILE" ]; then
-        mkdir -p `dirname $APP_DISABLE` 2>/dev/null || true
-        ln -sf $APP_CONFFILE $APP_DISABLE
-    fi
-}
-
-if [ "$1" = "install" ]; then
-    # Disable AppArmor profile on install
-    disable_profile
-fi
-
 #DEBHELPER#
diff --git a/debian/rsyslog.service b/debian/rsyslog.service
index c64a9ea..0d24b10 100644
--- a/debian/rsyslog.service
+++ b/debian/rsyslog.service
@@ -7,8 +7,10 @@ Documentation=https://www.rsyslog.com/doc/
 
 [Service]
 Type=notify
+ExecStartPre=/usr/lib/rsyslog/reload-apparmor-profile
 ExecStart=/usr/sbin/rsyslogd -n -iNONE
 StandardOutput=null
+StandardError=journal
 Restart=on-failure
 
 # Increase the default a bit in order to allow many simultaneous
diff --git a/debian/tests/apparmor-include-mechanism b/debian/tests/apparmor-include-mechanism
new file mode 100755
index 0000000..728d17f
--- /dev/null
+++ b/debian/tests/apparmor-include-mechanism
@@ -0,0 +1,92 @@
+#!/bin/bash
+
+set -e
+set -o pipefail
+
+include_dir="/etc/apparmor.d/rsyslog.d"
+apparmor_profile="/etc/apparmor.d/usr.sbin.rsyslogd"
+declare -i ret
+ret=0
+
+cleanup() {
+    rm -f "${include_dir}"/do-not-include*
+    rm -f "${include_dir}"/README
+    rm -f "${include_dir}"/pkg1.apparmor
+    rm -f "${include_dir}"/randomfile
+}
+
+trap cleanup EXIT
+
+standard_backup_files_are_not_included() {
+    local -a ignored_suffixes
+    local -a exclusions
+    local -a inclusions
+    local -i lines=0
+    local fname
+    local suffix
+    local full_profile
+
+    cleanup
+
+    # taken from https://sources.debian.org/src/apparmor/3.0.8-2/libraries/libapparmor/src/private.c/#L65
+    # and https://sources.debian.org/src/apparmor/3.0.8-2/libraries/libapparmor/src/private.c/#L133
+    ignored_suffixes=(.dpkg-new .dpkg-old .dpkg-dist .dpkg-bak .dpkg-remove .pacsave .pacnew .rpmnew .rpmsave .orig .rej \~)
+    exclusions+=("README" ".somedotfile")
+    for suffix in "${ignored_suffixes[@]}"; do
+        exclusions+=("do-not-include${suffix}")
+    done
+
+    echo "## Files with known backup extensions, that start with a dot, and a README file, are not included. Testing with:"
+    echo "${exclusions[*]}"
+    echo
+
+    for fname in "${exclusions[@]}"; do
+        echo "# BUG this should not be included: ${fname}" > "${include_dir}/${fname}"
+    done
+
+    # just a few, for a sanity check
+    inclusions=(pkg1.apparmor randomfile)
+    echo "## These, however, should be included: ${inclusions[*]}"
+    for fname in "${inclusions[@]}"; do
+        echo "# must be included: ${fname}" > "${include_dir}/${fname}"
+    done
+
+    echo "## Generated test files:"
+    ls -la /etc/apparmor.d/rsyslog.d/
+
+    full_profile=$(apparmor_parser -p "${apparmor_profile}")
+
+    echo "## Verifying that none of the excluded files were included in the apparmor profile:"
+    if echo "${full_profile}" | grep -F "BUG this should not be included"; then
+        return 1 # the caller will print ## FAIL
+    else
+        echo "## OK"
+    fi
+
+    echo "## Verifying that all the allowed files were included:"
+    lines=$(echo "${full_profile}" | grep -F "must be included" | wc -l)
+    if [ ${lines} -ne ${#inclusions[@]} ]; then
+        echo "## Found ${lines} inclusions, expected ${#inclusions[@]}"
+        return 1
+    fi
+}
+
+
+for t in \
+    standard_backup_files_are_not_included; do
+
+    echo
+    if "${t}"; then
+        echo "## OK"
+    else
+        ret=1
+        echo "## FAIL"
+    fi
+done
+
+echo
+if [ ${ret} -ne 0 ]; then
+    echo "## One or more tests FAILED"
+fi
+
+exit ${ret}
diff --git a/debian/tests/control b/debian/tests/control
new file mode 100644
index 0000000..89703c0
--- /dev/null
+++ b/debian/tests/control
@@ -0,0 +1,19 @@
+Tests: apparmor-include-mechanism
+Depends: rsyslog, apparmor
+Restrictions: needs-root allow-stderr
+
+Tests: simple-logger
+Depends: rsyslog, bsdutils, apparmor-utils, uuid-runtime
+Restrictions: needs-root
+
+Tests: simple-mysql
+# rsyslog-mysql has to be installed by the test script, because it pulls in dbconfig-common
+# which will be setup *before* mysql-server is configured and running, which fails.
+Depends: bsdutils, apparmor-utils, mysql-server, uuid-runtime
+Restrictions: needs-root
+
+Tests: simple-pgsql
+# rsyslog-pgsql has to be installed by the test script, because it pulls in dbconfig-common
+# which will be setup *before* postgresql is configured and running, which fails.
+Depends: bsdutils, apparmor-utils, postgresql, postgresql-client, uuid-runtime
+Restrictions: needs-root
diff --git a/debian/tests/simple-logger b/debian/tests/simple-logger
new file mode 100755
index 0000000..1c38834
--- /dev/null
+++ b/debian/tests/simple-logger
@@ -0,0 +1,36 @@
+#!/bin/bash
+
+set -e
+set -o pipefail
+
+# make sure we are confined and in enforce mode for this test, if supported
+apparmor_profile="/etc/apparmor.d/usr.sbin.rsyslogd"
+if [ ! -d /etc/apparmor.d/rsyslog.d ]; then
+    echo "No /etc/apparmor.d/rsyslog directory, not touching apparmor status"
+
+elif [ ! -f "${apparmor_profile}" ]; then
+    echo "No ${apparmor_profile} file, not touching apparmor status"
+
+elif ! aa-status --enabled 2>/dev/null; then
+    echo "Apparmor disabled (aa-status)"
+
+else
+    echo "Enforcing the ${apparmor_profile} apparmor profile"
+    aa-enforce "${apparmor_profile}"
+fi
+
+message="logger-test-value=$(uuidgen)"
+
+echo "Logging message: ${message}"
+logger --id=$$ --priority user.notice "${message}"
+
+logs=$(tail -n 10 /var/log/syslog)
+if echo "${logs}" | grep -qE "${message}"; then
+    echo "Message correctly found in system logs"
+else
+    echo "Failed to find message \"${message}\" in /var/log/syslog"
+    echo "Last 5 lines are:"
+    echo "${logs}"
+    exit 1
+fi
+
diff --git a/debian/tests/simple-mysql b/debian/tests/simple-mysql
new file mode 100755
index 0000000..036f1af
--- /dev/null
+++ b/debian/tests/simple-mysql
@@ -0,0 +1,42 @@
+#!/bin/bash
+
+set -e
+set -o pipefail
+
+source debian/tests/utils
+
+# make sure we are confined and in enforce mode for this test, if supported
+apparmor_profile="/etc/apparmor.d/usr.sbin.rsyslogd"
+if [ ! -d /etc/apparmor.d/rsyslog.d ]; then
+    echo "No /etc/apparmor.d/rsyslog directory, not touching apparmor status"
+
+elif [ ! -f "${apparmor_profile}" ]; then
+    echo "No ${apparmor_profile} file, not touching apparmor status"
+
+elif ! aa-status --enabled 2>/dev/null; then
+    echo "Apparmor disabled (aa-status)"
+
+else
+    echo "Enforcing the ${apparmor_profile} apparmor profile"
+    aa-enforce "${apparmor_profile}"
+fi
+
+# Installing rsyslog-mysql without having a mysql DB already configured on
+# localhost fails, because the dbconfig-common postinst runs before the mysql
+# postinst. A Depends cannot be used in the packaging because the database
+# might be remote.
+# Therefore we add mysql-server to the DEP8 dependency list in d/t/control, and
+# install rsyslog-mysql from inside the test. In this way, mysql is already
+# configured when we get here.
+DEBIAN_FRONTEND=noninteractive apt-get install -y rsyslog-mysql
+
+# Values from a default install of rsyslog-mysql
+DBNAME="Syslog"
+TABLE="SystemEvents"
+
+message="logger-test-value=$(uuidgen)"
+
+echo "Logging message: ${message}"
+logger --id=$$ --priority user.notice "${message}"
+
+check_db_for_message mysql "${message}"
diff --git a/debian/tests/simple-pgsql b/debian/tests/simple-pgsql
new file mode 100755
index 0000000..9f4c163
--- /dev/null
+++ b/debian/tests/simple-pgsql
@@ -0,0 +1,38 @@
+#!/bin/bash
+
+set -e
+set -o pipefail
+
+source debian/tests/utils
+
+# make sure we are confined and in enforce mode for this test, if supported
+apparmor_profile="/etc/apparmor.d/usr.sbin.rsyslogd"
+if [ ! -d /etc/apparmor.d/rsyslog.d ]; then
+    echo "No /etc/apparmor.d/rsyslog directory, not touching apparmor status"
+
+elif [ ! -f "${apparmor_profile}" ]; then
+    echo "No ${apparmor_profile} file, not touching apparmor status"
+
+elif ! aa-status --enabled 2>/dev/null; then
+    echo "Apparmor disabled (aa-status)"
+
+else
+    echo "Enforcing the ${apparmor_profile} apparmor profile"
+    aa-enforce "${apparmor_profile}"
+fi
+
+# Installing rsyslog-pgsql without having a postgresq DB already configured on
+# localhost fails, because the dbconfig-common postinst runs before the
+# postgresql postinst. A Depends cannot be used in the packaging because the
+# database might be remote.
+# Therefore we add postgresql to the DEP8 dependency list in d/t/control, and
+# install rsyslog-pgsql from inside the test. In this way, postgresql is
+# already configured when we get here.
+DEBIAN_FRONTEND=noninteractive apt-get install -y rsyslog-pgsql
+
+message="logger-test-value=$(uuidgen)"
+
+echo "Logging message: ${message}"
+logger --id=$$ --priority user.notice "${message}"
+
+check_db_for_message postgresql "${message}"
diff --git a/debian/tests/utils b/debian/tests/utils
new file mode 100644
index 0000000..5d503ef
--- /dev/null
+++ b/debian/tests/utils
@@ -0,0 +1,41 @@
+check_db_for_message() {
+    local db="${1}"
+    local message="${2}"
+    local -i counter=10
+
+    case "${db}" in
+        mysql)
+            dbname="Syslog"
+            table="SystemEvents"
+            cmd="mysql -uroot ${dbname} --batch -N -e \"SELECT COUNT(*) FROM ${table} WHERE trim(Message) = \\\"${message}\\\";\""
+            ;;
+        postgresql)
+            dbname="Syslog"
+            table="systemevents"
+            cmd="sudo -u postgres -i  psql -At -d ${dbname} -c \"SELECT COUNT(*) FROM ${table} WHERE trim(message) = '${message}';\""
+            ;;
+        *)
+            echo "Unrecognized db: ${db}"
+            return 1
+            ;;
+    esac
+    echo -n "Checking ${db} for the message (${counter} attempts): "
+    while [ ${counter} -gt 0 ]; do
+        count=$(eval "${cmd}")
+        if [ ${count} -eq 1 ]; then
+            echo
+            echo "Message correctly found in the ${db} ${dbname}.${table} table"
+            break
+        else
+            echo -n "."
+            counter=$((counter-1))
+            sleep 1s
+            continue
+        fi
+    done
+    if [ ${counter} -eq 0 ]; then
+        echo
+        echo "Failed to find message \"${message}\" in the ${db} ${dbname}.${table} table"
+        return 1
+    fi
+}
diff --git a/debian/usr.sbin.rsyslogd b/debian/usr.sbin.rsyslogd
index ab130da..3d80417 100644
--- a/debian/usr.sbin.rsyslogd
+++ b/debian/usr.sbin.rsyslogd
@@ -45,14 +45,8 @@ profile rsyslogd /usr/sbin/rsyslogd {
   # 'r' is needed when using imfile
   /var/log/**                   rw,
 
-  # Add these for mysql support
-  #/etc/mysql/my.cnf r,
-  #/{,var/}run/mysqld/mysqld.sock rw,
-
-  # Add thes for postgresql support
-  ##include <abstractions/openssl>
-  ##include <abstractions/ssl_certs>
-  #/{,var/}run/postgresql/.s.PGSQL.*[0-9] rw,
+  # apparmor snippets for rsyslog from other packages
+  include if exists <rsyslog.d>
 
   # Site-specific additions and overrides. See local/README for details.
   #include <local/usr.sbin.rsyslogd>