Merge ~pguimaraes/charm-rsyslog-forwarder-ha/+git/rsyslog-forwarder-ha-charm:master into ~rsyslog-charmers/charm-rsyslog-forwarder-ha:master

[Pedro Guimarães]

* update Rsyslog to work with Bionic
* add TLS support to communicate with rsyslog servers
* add option to import certificate: juju config rsyslog-forwarder-ha cert=$(base64 -w0 CERT_FILE_AND_PATH)
* updated unit tests accordingly
* added py36 support on tox for unit tests
LP: #1807124, #1796973, #1797162

* update Rsyslog to work with Bionic
* add TLS support to communicate with rsyslog servers
* add option to import certificate: juju config rsyslog-forwarder-ha cert=$(base64 -w0 CERT_FILE_AND_PATH)
* updated unit tests accordingly
* added py36 support on tox for unit tests
LP: #1807124, #1796973, #1797162

[Edward Hope-Morley (hopem)]

Hi thanks for this patch. There are multiple things being fixed/added here so typically it should be split into separate patches but since the cert change is pretty isolated i thinks its ok land in one. I would however like to hear more about what testing has been done specifically for Bionic since iiuc the rsyslog and rsyslog-forwarder-ha have never has formal Bionic testing. Also I see that you have added rsyslog-gnutls as a dep but on my Xenial system that is not installed by default.

review: Needs Fixing

[Pedro Guimarães (pguimaraes)]

> Hi thanks for this patch. There are multiple things being fixed/added here so
> typically it should be split into separate patches but since the cert change
> is pretty isolated i thinks its ok land in one. I would however like to hear
> more about what testing has been done specifically for Bionic since iiuc the
> rsyslog and rsyslog-forwarder-ha have never has formal Bionic testing. Also I
> see that you have added rsyslog-gnutls as a dep but on my Xenial system that
> is not installed by default.

Yes, there were multiple small fixes but my main targets are Bionic and TLS support. So, the rsyslog-gnutls does not come installed by default and it is setup during install hook (line 198/199 on hooks/hooks.py). That package enables "gtls" option on rsyslog configs.
To test everything, I've strictly followed this step-by-step: https://www.rsyslog.com/doc/master/tutorials/tls.html?highlight=defaultnetstreamdrivercafile and used logger to send messages from client-side.
The main bundle for this test was: https://pastebin.canonical.com/p/Ksf5Jx9D8G/

[Edward Hope-Morley (hopem)]

Excellent thanks for that info Pedro. I'm deploying your bundle now to check it for myself and will +1 once complete.

[Edward Hope-Morley (hopem)]

lgtm +1

review: Approve

diff --git a/config.yaml b/config.yaml
index ab55f59..6fd48cb 100644
--- a/config.yaml
+++ b/config.yaml
@@ -1,4 +1,8 @@
 options:
+  cert:
+    type: string
+    default: ""
+    description: TLS certificate in base64 format to access rsyslog server
   log-locally:
     type: boolean
     default: false
diff --git a/hooks/hooks.py b/hooks/hooks.py
index 0069817..16aa879 100755
--- a/hooks/hooks.py
+++ b/hooks/hooks.py
@@ -42,11 +42,15 @@ from model import Server, session
 from utils import get_template_dir, get_template
 from utils import __die as die
 
+import base64
+import subprocess
+
 required_packages = [
     'rsyslog',
     'rsyslog-relp',
     'python-jinja2',
-    'python-sqlalchemy'
+    'python-sqlalchemy',
+    'rsyslog-gnutls'
 ]
 
 
@@ -54,11 +58,35 @@ IMFILE_FILE = '/etc/rsyslog.d/40-rsyslog-imfile.conf'
 LOGS_TEMPLATE = 'keep_local.template'
 LOGS_SYSTEM_FILE = '/etc/rsyslog.d/50-default.conf'
 REPLICATION_FILE = '/etc/rsyslog.d/45-rsyslog-replication.conf'
-
+CERT_FILE = '/etc/rsyslog.d/42-cert-rsyslog.conf'
 
 hooks = Hooks()
 
 
+def update_certfile():
+    # Check:
+    # https://www.rsyslog.com/doc/master/tutorials/tls.html?highlight=defaultnetstreamdrivercafile
+    if config_get('cert'):
+        if len(config_get('cert')) > 0:
+            subprocess.check_output(['mkdir','-pv','/etc/rsyslog.d/keys/ca.d'])
+            encoded_cert=config_get('cert')
+            cert=base64.b64decode(encoded_cert)
+            with open("/etc/rsyslog.d/keys/ca.d/rsyslog.crt","w") as c:
+                c.write(cert)
+                c.close()
+
+    if not config_get('cert'):
+        if os.path.exists(CERT_FILE):
+            os.remove(CERT_FILE)
+        return
+    if len(config_get('cert')) == 0:
+        if os.path.exists(CERT_FILE):
+            os.remove(CERT_FILE)
+        return
+
+    with open(CERT_FILE, 'w') as fd:
+        fd.write(get_template('certificate').render())
+
 def update_imfile(imfiles):
     if imfiles == []:
         if os.path.exists(IMFILE_FILE):
@@ -169,6 +197,7 @@ def install():
     for package in required_packages:
         apt_install(package, fatal=True)
 
+    update_certfile()
     update_local_logs(
         keep=config_get('log-locally'))
 
@@ -224,6 +253,7 @@ def upgrade_charm():
 def config_changed():
     update_local_logs(config_get("log-locally"))
     update_imfile(config_get("watch-files").split())
+    update_certfile()
     update_replication()
     update_nrpe_config()
 
diff --git a/metadata.yaml b/metadata.yaml
index b1bc49c..ecef969 100644
--- a/metadata.yaml
+++ b/metadata.yaml
@@ -8,6 +8,7 @@ tags:
   - audits
   - ops
 series:
+  - bionic
   - xenial
   - trusty
 provides:
diff --git a/templates/certificate.template b/templates/certificate.template
new file mode 100644
index 0000000..460159b
--- /dev/null
+++ b/templates/certificate.template
@@ -0,0 +1,4 @@
+$DefaultNetstreamDriverCAFile /etc/rsyslog.d/keys/ca.d/rsyslog.crt
+$DefaultNetstreamDriver gtls
+$ActionSendStreamDriverMode 1
+$ActionSendStreamDriverAuthMode anon
diff --git a/tests/unit/test_basic.py b/tests/unit/test_basic.py
index bc07bd7..85991f8 100644
--- a/tests/unit/test_basic.py
+++ b/tests/unit/test_basic.py
@@ -70,6 +70,7 @@ class HooksTestCase(unittest.TestCase):
             mock.call("rsyslog-relp", fatal=True),
             mock.call("python-jinja2", fatal=True),
             mock.call("python-sqlalchemy", fatal=True),
+            mock.call("rsyslog-gnutls", fatal=True),
         ]
 
         self.assertEquals(sorted(self.apt_install.call_args_list),
diff --git a/tox.ini b/tox.ini
index ac45fdc..401fdb2 100644
--- a/tox.ini
+++ b/tox.ini
@@ -1,6 +1,6 @@
 [tox]
 skipsdist=True
-envlist = py34, py35
+envlist = py34, py35, py36
 skip_missing_interpreters = True
 
 [testenv]