Merge ~freyes/charm-grafana:bug/1872682 into charm-grafana:master

Proposed by Felipe Reyes
Status: Superseded
Proposed branch: ~freyes/charm-grafana:bug/1872682
Merge into: charm-grafana:master
Diff against target: 982 lines (+454/-40) (has conflicts)
24 files modified
src/actions/create-user (+14/-6)
src/files/dashboards_backup (+4/-1)
src/layer.yaml (+8/-1)
src/lib/charms/layer/grafana.py (+33/-3)
src/reactive/grafana.py (+137/-15)
src/requirements.txt (+1/-0)
src/templates/grafana.ini.j2 (+7/-1)
src/templates/juju-dashboards-backup.j2 (+1/-1)
src/templates/sync-grafana-snap (+7/-0)
src/tests/functional/requirements.txt (+6/-0)
src/tests/functional/tests/bundles/bionic-snap-tls.yaml (+1/-0)
src/tests/functional/tests/bundles/bionic-tls.yaml (+1/-0)
src/tests/functional/tests/bundles/focal-snap-tls.yaml (+1/-0)
src/tests/functional/tests/bundles/focal-tls.yaml (+1/-0)
src/tests/functional/tests/bundles/overlays/bionic-snap-tls.yaml.j2 (+11/-0)
src/tests/functional/tests/bundles/overlays/bionic-tls.yaml.j2 (+10/-0)
src/tests/functional/tests/bundles/overlays/focal-snap-tls.yaml.j2 (+32/-0)
src/tests/functional/tests/bundles/overlays/focal-tls.yaml.j2 (+31/-0)
src/tests/functional/tests/bundles/overlays/xenial-tls.yaml.j2 (+12/-0)
src/tests/functional/tests/bundles/xenial-tls.yaml (+1/-0)
src/tests/functional/tests/test_grafana.py (+76/-12)
src/tests/functional/tests/tests.yaml (+7/-0)
src/tests/unit/test_grafana.py (+48/-0)
src/tox.ini (+4/-0)
Conflict in src/tests/functional/requirements.txt
Conflict in src/tox.ini
Reviewer Review Type Date Requested Status
Liam Young (community) Needs Fixing
Alvaro Uria (community) Needs Fixing
BootStack Reviewers Pending
BootStack Reviewers Pending
Review via email: mp+397611@code.launchpad.net

This proposal has been superseded by a proposal from 2021-08-06.

Commit message

Add tls-client layer to support HTTPS

To post a comment you must log in.
Revision history for this message
🤖 Canonical IS Merge Bot (canonical-is-mergebot) wrote :

This merge proposal is being monitored by mergebot. Change the status to Approved to merge.

Revision history for this message
Felipe Reyes (freyes) wrote :

I've been trying to run the full functional tests (make functional) without success, I'm getting consistently the following error now:

unit-prometheus-0: 15:56:11 WARNING unit.prometheus/0.install subprocess.CalledProcessError: Command '['snap', 'refresh', '--amend', '--channel=stable', 'core']' returned non-zero exit status 1.

a few hours ago the charmstore was giving "error 500" for certain charms (e.g. easyrsa). I will post the results here once I get to full successful run.

Revision history for this message
Drew Freiberger (afreiberger) wrote :

we very much have had that problem throughout the release cycle. it's just a poke and hope game until snapstore does the right thing.

Revision history for this message
Felipe Reyes (freyes) wrote :

I was running the functional test suite and got into this race condition:

2021-02-11 20:32:54 INFO juju-log certificates:21: Restarting grafana-server
2021-02-11 20:32:54 INFO juju-log certificates:21: Invoking reactive handler: reactive/grafana.py:625:configure_sources
2021-02-11 20:32:54 INFO juju-log certificates:21: Found datasource: {'service_name': 'prometheus', 'type': 'prometheus', 'url': 'http://10.5.0.99:9090', 'description': 'Juju generated source
'}
2021-02-11 20:32:54 INFO juju-log certificates:21: Datasource already exist, updating: prometheus - Juju generated source
2021-02-11 20:32:54 ERROR juju-log certificates:21: Hook error:
...
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f5e101bcca0>: Failed to establish a new connection: [Errno 111] Connection refused
...
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='127.0.0.1', port=3000): Max retries exceeded with url: /api/search?type=dash-db (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5e101bcca0>: Failed to establish a new connection: [Errno 111] Connection refused'))

So it seems that the restart of the service is not blocking until it's available to serve requests. I will propose a different patch to improve this.

Revision history for this message
Alvaro Uria (aluria) wrote :

Please find a comment inline (re: tls_client.certs.changed decorated handler). Other than that, code review looks good. I'll run all tests and share the output here.

Revision history for this message
Alvaro Uria (aluria) wrote :

* make lint failed with:
./tests/unit/test_grafana.py:161:1: D102 Missing docstring in public method

* make unittests failed with:
tests/unit/test_grafana.py::GrafanaTestCase::test_request_certificate FAILED

See output at: https://pastebin.ubuntu.com/p/KsMPg5gHMm/

* make functional failed on the first bundle (tests/bundles/focal.yaml)
2021-03-19 07:31:39 [ERROR] unit-grafana-0.log: 2021-03-19 07:31:37 WARNING install Exception: port 3000 is closed

Revision history for this message
Alvaro Uria (aluria) wrote :

Added comment inline re: retry_on_exception not having a delay

review: Needs Fixing
Revision history for this message
Felipe Reyes (freyes) wrote :

On Fri, 2021-03-19 at 08:29 +0000, Alvaro Uria wrote:
> * make lint failed with:
> ./tests/unit/test_grafana.py:161:1: D102 Missing docstring in public
> method
>
> * make unittests failed with:
> tests/unit/test_grafana.py::GrafanaTestCase::test_request_certificate
> FAILED
>
> See output at: https://pastebin.ubuntu.com/p/KsMPg5gHMm/
>
> * make functional failed on the first bundle
> (tests/bundles/focal.yaml)
> 2021-03-19 07:31:39 [ERROR] unit-grafana-0.log: 2021-03-19 07:31:37
> WARNING install Exception: port 3000 is closed

This failure happens because there was no base_delay as you pointed out
in the retry_on_exception().

>
--
Felipe Reyes
Software Sustaining Engineer @ Canonical
# Email: <email address hidden> (GPG:0x9B1FFF39)
# Launchpad: ~freyes | IRC: freyes

Revision history for this message
Felipe Reyes (freyes) wrote :

Hi Alvaro, I just pushed a new version of the patch. make lint/unittests are OK, I'm running make functional now, but it will take a while :-) . Thanks for reviewing my patch.

Revision history for this message
Felipe Reyes (freyes) wrote :

Has anyone been able to run a successful run of grafana functional testing recently?, not necessarily with this patch, but in general, I get different issues all the time (snap store, or the charm store, or stsstack, or just juju timing out)

Revision history for this message
Xav Paice (xavpaice) wrote :

Functest run from the candidate/21.04 branch (a copy of master at the time of writing) https://pastebin.canonical.com/p/j8VxbzYNvc/ -> passed for the first bundle. Because I'm using Openstack as a cloud for the tests, latest versions of Zaza require python-openstackclient in order to complete the teardown of the model, and that means my tests only run for the first bundle. I'll look at a separate fix for this (for any of the charms running zaza) and then we can re-test here.

Tests on this branch also passed functests on my environment for the first bundle, however there are a LOT of test runs to complete for this and without automation of the tests this will cause a delay since we're locked into a release cycle right now.

Once this is reviewed and merged to master, it'll be available on cs:~llama-charmers-next/openstack-service-checks in any case.

Revision history for this message
Xav Paice (xavpaice) wrote :

Functest failed with:

2021-04-13 10:53:55 [ERROR] {'model_apt_install': 'zaza-2010f9813ac7'}
2021-04-13 10:53:55 [ERROR] Model model_apt_install (zaza-2010f9813ac7)
Traceback (most recent call last):
  File "/home/xav/charms/charm-grafana/src/.tox/func/bin/functest-run-suite", line 8, in <module>
    sys.exit(main())
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/zaza/charm_lifecycle/func_test_runner.py", line 308, in main
    func_test_runner(
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/zaza/charm_lifecycle/func_test_runner.py", line 242, in func_test_runner
    run_env_deployment(env_deployment, keep_model=preserve_model,
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/zaza/charm_lifecycle/func_test_runner.py", line 122, in run_env_deployment
    deploy.deploy(
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/zaza/charm_lifecycle/deploy.py", line 385, in deploy
    zaza.model.wait_for_application_states(
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/zaza/__init__.py", line 77, in _wrapper
    return run(_run_it())
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/zaza/__init__.py", line 62, in run
    return task.result()
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/zaza/__init__.py", line 76, in _run_it
    return await f(*args, **kwargs)
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/zaza/model.py", line 1167, in async_wait_for_application_states
    await model.block_until(
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/juju/model.py", line 727, in block_until
    await utils.block_until(done,
  File "/home/xav/charms/charm-grafana/src/.tox/func/lib/python3.8/site-packages/juju/utils.py", line 87, in block_until
    await asyncio.wait_for(_block(), timeout, loop=loop)
  File "/usr/lib/python3.8/asyncio/tasks.py", line 490, in wait_for
    raise exceptions.TimeoutError()
asyncio.exceptions.TimeoutError

Bundle it was deploying: src/tests/functional/tests/bundles/focal-tls.yaml

Revision history for this message
Liam Young (gnuoy) wrote :

I've raised a PR against freyes's branch which includes fixes to the TimeoutError mentioned in the previous comment and the dependency on OpenStack clients mentioned before that.

https://code.launchpad.net/~gnuoy/charm-grafana/+git/charm-grafana/+merge/404961

Revision history for this message
James Troup (elmo) wrote :

Marking this as rejected to get it out of the review queue. The work done on this branch landed with Liam's MP. Thanks Felipe and Liam!

Revision history for this message
Felipe Reyes (freyes) wrote :

Hi James, if by "Liam's MP" you mean https://code.launchpad.net/~gnuoy/charm-grafana/+git/charm-grafana/+merge/404961 , that one was proposed for merge in my branch[0] and it's already merged. My branch was just rebased and pushed it for review.

Is there some other MP in flight or pending to be created and I'm confusing things?

Best,

PS: I'm running "make functional"at the moment, still running, but not failures so far[1]

[0]Proposed branch: ~gnuoy/charm-grafana:bug/1872682-fixes
Merge into: ~freyes/charm-grafana:bug/1872682

[1] $ grep "Deploying bundle" functional.log
2021-08-04 23:03:50 [INFO] Deploying bundle '/home/freyes/Projects/charms/grafana-charm/src/tests/functional/tests/bundles/focal.yaml' on to 'zaza-985edaed9dda' model
2021-08-04 23:27:32 [INFO] Deploying bundle '/home/freyes/Projects/charms/grafana-charm/src/tests/functional/tests/bundles/focal-tls.yaml' on to 'zaza-4fa38806714d' model
2021-08-04 23:51:24 [INFO] Deploying bundle '/home/freyes/Projects/charms/grafana-charm/src/tests/functional/tests/bundles/bionic.yaml' on to 'zaza-7c7752252403' model
2021-08-05 00:14:42 [INFO] Deploying bundle '/home/freyes/Projects/charms/grafana-charm/src/tests/functional/tests/bundles/bionic-tls.yaml' on to 'zaza-763edc3c044f' model
2021-08-05 00:35:21 [INFO] Deploying bundle '/home/freyes/Projects/charms/grafana-charm/src/tests/functional/tests/bundles/xenial.yaml' on to 'zaza-4ed0f8e6798e' model
2021-08-05 00:57:14 [INFO] Deploying bundle '/home/freyes/Projects/charms/grafana-charm/src/tests/functional/tests/bundles/xenial-tls.yaml' on to 'zaza-e26f68ffd00e' model

Revision history for this message
Liam Young (gnuoy) wrote :

Looks like src/tox.ini needs fixing

review: Needs Fixing
Revision history for this message
Felipe Reyes (freyes) wrote :

Weird, I'm not seeing that merge conflict locally, will give it a try in a fresh git clone.

re: functional tests, they were running fine, but due to a networking issue the full run couldn't complete - https://paste.ubuntu.com/p/3wvHsZsg8f/ - running again from a different node to avoid relying on the VPN.

Revision history for this message
Felipe Reyes (freyes) wrote :
Download full text (3.8 KiB)

This is merge of my branch into master -> https://paste.ubuntu.com/p/QHqcTykB8t/ , I don't see changes to src/tox.ini (nor merge conflicts), the change that is showing Launchpad existed in a previous version of the patch (before the rebase I did yesterday), so I wonder if there is a caching issue since this Merge Proposal is in "rejected" state.

ubuntu@freyes-bastion:~$ git clone -b master https://git.launchpad.net/charm-grafana
Cloning into 'charm-grafana'...
remote: Enumerating objects: 1757, done.
remote: Counting objects: 100% (1757/1757), done.
remote: Compressing objects: 100% (1009/1009), done.
remote: Total 1757 (delta 882), reused 902 (delta 469)
Receiving objects: 100% (1757/1757), 309.80 KiB | 433.00 KiB/s, done.
Resolving deltas: 100% (882/882), done.
ubuntu@freyes-bastion:~$ cd charm-grafana/
ubuntu@freyes-bastion:~/charm-grafana$ git remote -v
origin https://git.launchpad.net/charm-grafana (fetch)
origin https://git.launchpad.net/charm-grafana (push)
ubuntu@freyes-bastion:~/charm-grafana$ git remote add freyes https://git.launchpad.net/~freyes/charm-grafana
ubuntu@freyes-bastion:~/charm-grafana$ git fetch freyes
remote: Enumerating objects: 84, done.
remote: Counting objects: 100% (84/84), done.
 * [new branch] bug/1872682 -> freyes/bug/1872682
 * [new branch] bug/1893137 -> freyes/bug/1893137
ubuntu@freyes-bastion:~/charm-grafana$ git checkout -b bug/1872682 --track freyes/bug/1872682
Branch 'bug/1872682' set up to track remote branch 'bug/1872682' from 'freyes'.
Switched to a new branch 'bug/1872682'
ubuntu@freyes-bastion:~/charm-grafana$ git branch
* bug/1872682
  master
ubuntu@freyes-bastion:~/charm-grafana$ git merge --no-ff --no-commit bug/1872682
Already up to date.
ubuntu@freyes-bastion:~/charm-grafana$ git status
On branch bug/1872682
Your branch is up to date with 'freyes/bug/1872682'.

nothing to commit, working tree clean
ubuntu@freyes-bastion:~/charm-grafana$ git checkout master
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
ubuntu@freyes-bastion:~/charm-grafana$ git merge --no-ff --no-commit bug/1872682
Automatic merge went well; stopped before committing as requested
ubuntu@freyes-bastion:~/charm-grafana$ git status
On branch master
Your branch is up to date with 'origin/master'.

All conflicts fixed but you are still merging.
  (use "git commit" to conclude merge)

Changes to be committed:
        modified: src/actions/create-user
        modified: src/files/dashboards_backup
        modified: src/layer.yaml
        modified: src/lib/charms/layer/grafana.py
        modified: src/reactive/grafana.py
        modified: src/requirements.txt
        modified: src/templates/grafana.ini.j2
        modified: src/templates/juju-dashboards-backup.j2
        new file: src/templates/sync-grafana-snap
        new file: src/tests/functional/tests/bundles/bionic-snap-tls.yaml
        new file: src/tests/functional/tests/bundles/bionic-tls.yaml
        new file: src/tests/functional/tests/bundles/focal-snap-tls.yaml
        new file: src/tests/functional/tests/bundles/focal-tls.yaml
        new file: src/tests/functional/tests/bundles/overlays/bionic-snap...

Read more...

Revision history for this message
Felipe Reyes (freyes) wrote :

I could get a full run of "make functional" succeed, to achieve this I had to fix a race condition on the test_13_grafana_dashboard_backup()[0] test.

summary of the tests execution:

2021-08-06 02:37:18 [INFO] Events:
  Deploy Bundle:
    Start: 1628230469.9741626
    Finish: 1628230580.8524628
    Elapsed Time: 110.87830018997192
    PCT Of Run Time: 2
  Prepare Environment:
    Start: 1628230458.4041638
    Finish: 1628230469.9739351
    Elapsed Time: 11.56977128982544
    PCT Of Run Time: 1
  Test tests.test_grafana.AptGrafanaTest:
    Start: 1628231751.6462507
    Finish: 1628231776.0118518
    Elapsed Time: 24.365601062774658
    PCT Of Run Time: 1
  Test tests.test_grafana.CharmOperationTest:
    Start: 1628231214.4372096
    Finish: 1628231751.6461709
    Elapsed Time: 537.2089612483978
    PCT Of Run Time: 6
  Test tests.test_grafana.SnappedGrafanaTest:
    Start: 1628222783.1681652
    Finish: 1628222887.3098197
    Elapsed Time: 104.14165449142456
    PCT Of Run Time: 2
  Wait for Deployment:
    Start: 1628230580.8525686
    Finish: 1628231212.1852653
    Elapsed Time: 631.3326966762543
    PCT Of Run Time: 8
Metadata: {}

Full output can be found at https://paste.ubuntu.com/p/79nwNB6TKr/

[0] https://git.launchpad.net/~freyes/charm-grafana/commit/?id=229263cbaf7ddc85d1f84a34c15dfed1c5e547a4

Revision history for this message
Felipe Reyes (freyes) wrote :

I will re-submit my proposal, the commit ids shown in the "unmerged commits" are invalid

Unmerged commits in Launchpad

6e52a25... by Liam Young on 2021-06-30 Use easyrsa on bionic for xenial tests
5ca8ed8... by Liam Young on 2021-06-30 Bug fixes for enabling TLS
4c7b77b... by Felipe Reyes on 2021-02-05 Add tls-client layer to support HTTPS

versus

$ git log --oneline
229263c (HEAD -> bug/1872682, freyes/bug/1872682) Wait until /etc/cron.d/juju-dashboards-backup appears
817ed45 Replace assertTrue with assertIn, assertNotIn and assertEqual
7108c49 Use easyrsa on bionic for xenial tests
ef9711e Bug fixes for enabling TLS
c9d57f8 Add tls-client layer to support HTTPS

Unmerged commits

6e52a25... by Liam Young

Use easyrsa on bionic for xenial tests

Use easyrsa on bionic for xenial tests as easyrsa is currently
failing to install on xenial.

5ca8ed8... by Liam Young

Bug fixes for enabling TLS

* Update create-user action to work with https
* Update backup to work with https
* Update functional test requirements to work-around zaza bug *1
* Update tests.yaml to expect easyrsa to come up with a workload
  status message of 'Certificate Authority connected.'
* Do not use f strings as the charm needs to run on xenial.

*1 https://github.com/openstack-charmers/zaza/issues/452

4c7b77b... by Felipe Reyes

Add tls-client layer to support HTTPS

This patch adds a new interface provided by the tls-client later. When related
to a certificates provider (e.g. EasyRSA) it will configure the daemon to
serve over HTTPS.

Fixes-Bug: #1893137

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/src/actions/create-user b/src/actions/create-user
2index ab9f96c..916534c 100755
3--- a/src/actions/create-user
4+++ b/src/actions/create-user
5@@ -12,18 +12,24 @@ from charmhelpers.core.hookenv import (
6 log,
7 )
8
9-from charms.layer.grafana import get_admin_password
10+from charms.layer.grafana import (
11+ get_admin_password,
12+ get_protocol,
13+)
14
15 action = "create-user"
16
17 admin_passwd = get_admin_password()
18
19+# Talking to ourselves on localhost so don't worry about CAs
20+verify_ca = False
21+
22 if admin_passwd is None:
23 action_fail('Unable to retrieve password.')
24 sys.exit(0)
25
26 port = config('port')
27-grafana = "http://localhost:%s" % (port)
28+grafana = "{}://localhost:{}".format(get_protocol(), port)
29 api_auth = ('admin', admin_passwd)
30
31 # http://docs.grafana.org/http_api/admin/#global-users
32@@ -57,13 +63,13 @@ user_data = {
33 headers = {'Content-Type': 'application/json'}
34
35 grafana_api_create_user_url = grafana + api_create_user_url
36-
37 if requests.utils.urlparse(grafana_api_create_user_url).scheme:
38 r_create = requests.post(
39 grafana_api_create_user_url,
40 auth=api_auth,
41 headers=headers,
42- data=json.dumps(user_data)
43+ data=json.dumps(user_data),
44+ verify=verify_ca,
45 )
46 else:
47 action_fail("Grafana url %s failed to parse!" % (grafana_api_create_user_url))
48@@ -90,7 +96,8 @@ grafana_api_org_url = grafana + api_org_url
49 if requests.utils.urlparse(grafana_api_org_url).scheme:
50 r_org = requests.get(
51 grafana_api_org_url,
52- auth=api_auth
53+ auth=api_auth,
54+ verify=verify_ca
55 )
56 else:
57 action_fail("Grafana url %s failed to parse" % (grafana_api_org_url))
58@@ -108,7 +115,8 @@ if r_org.status_code == 200:
59 grafana_api_org_user_url,
60 auth=api_auth,
61 headers=headers,
62- data=json.dumps(org_user_data)
63+ data=json.dumps(org_user_data),
64+ verify=verify_ca
65 )
66 else:
67 action_fail("Grafana url %s failed to parse" % (grafana_api_org_user_url))
68diff --git a/src/files/dashboards_backup b/src/files/dashboards_backup
69index 30299a2..88c6310 100755
70--- a/src/files/dashboards_backup
71+++ b/src/files/dashboards_backup
72@@ -18,7 +18,7 @@ def get_backup_filename(directory, org_name, uri):
73
74
75 def main(args):
76- base_url = 'http://localhost:{port}/api/'.format(port=args.port)
77+ base_url = '{scheme}://localhost:{port}/api/'.format(scheme=args.scheme, port=args.port)
78 for key in args.api_keys:
79 headers = {'Authorization': 'Bearer {}'.format(key)}
80 org_name = requests.get(base_url+'org', headers=headers).json()['name']
81@@ -38,6 +38,9 @@ if __name__ == '__main__':
82 parser.add_argument('-d', '--directory', help='Directory where to store backups',
83 default='/srv/backups')
84 parser.add_argument('-p', '--port', help='Port to access grafana API', default='3000')
85+ parser.add_argument('-s', '--scheme',
86+ help='Scheme to use to access grafana API e.g. http or https',
87+ default='http')
88 parser.add_argument('api_keys', help='List of API keys to use for backups', nargs='+')
89 args = parser.parse_args()
90 main(args)
91diff --git a/src/layer.yaml b/src/layer.yaml
92index 6e51d38..3ddcaa7 100644
93--- a/src/layer.yaml
94+++ b/src/layer.yaml
95@@ -1,4 +1,11 @@
96-includes: ['layer:basic', 'layer:snap', 'interface:nrpe-external-master', 'interface:grafana-source', 'interface:http', 'interface:grafana-dashboard']
97+includes:
98+ - 'layer:basic'
99+ - 'layer:snap'
100+ - 'layer:tls-client'
101+ - 'interface:nrpe-external-master'
102+ - 'interface:grafana-source'
103+ - 'interface:http'
104+ - 'interface:grafana-dashboard'
105 ignore: ['.*.swp' ]
106 options:
107 basic:
108diff --git a/src/lib/charms/layer/grafana.py b/src/lib/charms/layer/grafana.py
109index d8fc7b8..0acb980 100644
110--- a/src/lib/charms/layer/grafana.py
111+++ b/src/lib/charms/layer/grafana.py
112@@ -15,10 +15,19 @@ from charmhelpers.core.hookenv import (
113 )
114
115 from charms.layer import snap
116+from charms.reactive.flags import is_flag_set
117
118 import requests
119
120
121+# When using 'certifi' from the virtualenv, the system-wide certificates store
122+# is not used, so installed certificates won't be used to validate hosts.
123+# Adding the system CA bundle
124+# https://git.launchpad.net/ubuntu/+source/python-certifi/tree/debian/patches/0001-Use-Debian-provided-etc-ssl-certs-ca-certificates.cr.patch
125+SYSTEM_CA_BUNDLE = "/etc/ssl/certs/ca-certificates.crt"
126+CA_CERT_PATH = "/var/snap/grafana/common/ssl/ca-certificates.crt"
127+
128+
129 class ChangeStatus(enum.Enum):
130 """Model Snap channel states."""
131
132@@ -42,6 +51,22 @@ def get_admin_password():
133 return config("admin_password") or unitdata.kv().get("grafana.admin_password")
134
135
136+def get_protocol():
137+ """Check if the SSL certificates were configured and return http or https."""
138+ if is_flag_set("grafana.certificates.configured"):
139+ return "https"
140+ else:
141+ return "http"
142+
143+
144+def get_ca_cert_path():
145+ """Return the path where the CA certificates store is."""
146+ if config("install_method") == "snap":
147+ return CA_CERT_PATH
148+ else:
149+ return SYSTEM_CA_BUNDLE
150+
151+
152 def compute_dash_title(title, remote_app=None):
153 """Compute title for dashboards.
154
155@@ -80,9 +105,10 @@ def compute_dash_title(title, remote_app=None):
156 def get_folders():
157 """Retrieve all folders."""
158 r = requests.get(
159- "http://localhost:{}/api/folders".format(config("port")),
160+ "{}://localhost:{}/api/folders".format(get_protocol(), config("port")),
161 auth=("admin", get_admin_password()),
162 headers={"Accept": "application/json"},
163+ verify=get_ca_cert_path(),
164 )
165 r.raise_for_status()
166 folders = r.json()
167@@ -93,10 +119,11 @@ def get_folders():
168 def create_folder(folder_name):
169 """Create a folder through Grafana API."""
170 r = requests.post(
171- "http://localhost:{}/api/folders".format(config("port")),
172+ "{}://localhost:{}/api/folders".format(get_protocol(), config("port")),
173 auth=("admin", get_admin_password()),
174 headers={"Accept": "application/json"},
175 data={"title": folder_name},
176+ verify=get_ca_cert_path(),
177 )
178 r.raise_for_status()
179 folder = r.json()
180@@ -129,7 +156,9 @@ def ensure_and_get_dash_folder(remote_model):
181 def post_dashboard(name, dashboard):
182 """Upload a dashboard to Grafana."""
183 headers = {"Content-Type": "application/json"}
184- import_url = "http://localhost:{}/api/dashboards/db".format(config("port"))
185+ import_url = "{}://localhost:{}/api/dashboards/db".format(
186+ get_protocol(), config("port")
187+ )
188 passwd = get_admin_password()
189 if passwd is None:
190 return (False, "Unable to retrieve grafana password.")
191@@ -139,6 +168,7 @@ def post_dashboard(name, dashboard):
192 auth=api_auth,
193 headers=headers,
194 data=json.dumps(dashboard),
195+ verify=get_ca_cert_path(),
196 )
197 if r.status_code == 200:
198 return (True, None)
199diff --git a/src/reactive/grafana.py b/src/reactive/grafana.py
200index 37958d3..10a1656 100644
201--- a/src/reactive/grafana.py
202+++ b/src/reactive/grafana.py
203@@ -68,10 +68,13 @@ wipe_nrpe_checks (no nrpe-external-master.available)
204 import base64
205 import datetime
206 import glob
207+import grp
208 import json
209 import os
210+import pwd
211 import re
212 import shutil
213+import socket
214 import subprocess
215 import time
216
217@@ -80,18 +83,23 @@ from charmhelpers.contrib.charmsupport import nrpe
218 from charmhelpers.core import (
219 hookenv,
220 host,
221+ templating,
222 unitdata,
223 )
224+from charmhelpers.core.decorators import retry_on_exception
225 from charmhelpers.core.templating import render
226
227-from charms.layer import snap
228+from charms.layer import snap, tls_client
229 from charms.layer.grafana import (
230+ CA_CERT_PATH,
231 ChangeStatus,
232 check_snap_channel,
233 download_file,
234 get_admin_password,
235+ get_ca_cert_path,
236 get_deb_package_version,
237 get_installed_package_version,
238+ get_protocol,
239 import_dashboard,
240 )
241 from charms.reactive import (
242@@ -102,6 +110,7 @@ from charms.reactive import (
243 when_any,
244 when_not,
245 )
246+from charms.reactive.flags import is_flag_set
247 from charms.reactive.helpers import (
248 any_file_changed,
249 is_state,
250@@ -119,8 +128,16 @@ import six
251 SVCNAME = {"snap": "snap.grafana.grafana", "apt": "grafana-server"}
252 SNAP_NAME = "grafana"
253 SNAP_DATA = "/var/snap/{}/current".format(SNAP_NAME)
254-SNAP_COMMON = "/var/snap/{}/common/data".format(SNAP_NAME)
255-
256+SNAP_COMMON_DIR = "/var/snap/{}/common".format(SNAP_NAME)
257+SNAP_COMMON_DATA = "{}/data".format(SNAP_COMMON_DIR)
258+CERT_PATH = {
259+ "snap": "{}/ssl/server.crt".format(SNAP_COMMON_DIR),
260+ "apt": "/etc/grafana/ssl/server.crt",
261+}
262+CERT_KEY_PATH = {
263+ "snap": "{}/ssl/server.key".format(SNAP_COMMON_DIR),
264+ "apt": "/etc/grafana/ssl/server.key",
265+}
266 GRAFANA_INI = {
267 "snap": "{}/conf/grafana.ini".format(SNAP_DATA),
268 "apt": "/etc/grafana/grafana.ini",
269@@ -129,6 +146,7 @@ GRAFANA_INI_TMPL = "grafana.ini.j2"
270 GRAFANA_DEPS = ["libfontconfig1"]
271 DASHBOARDS_BACKUP_CRON = "/etc/cron.d/juju-dashboards-backup"
272 DASHBOARDS_BACKUP_CRON_TMPL = "juju-dashboards-backup.j2"
273+CA_CERTIFICATES_HOOK = "/etc/ca-certificates/update.d/sync-grafana-snap"
274
275
276 try:
277@@ -233,7 +251,7 @@ def install_packages():
278
279 def data_path():
280 """Retrieve data store depending on install method."""
281- data_dir = {"snap": SNAP_COMMON, "apt": "/var/lib/grafana"}
282+ data_dir = {"snap": SNAP_COMMON_DATA, "apt": "/var/lib/grafana"}
283 source = get_install_source()
284 if not source:
285 remove_state("grafana.installed")
286@@ -460,7 +478,13 @@ def setup_grafana():
287 grafana_ini = GRAFANA_INI[source]
288 hookenv.status_set("maintenance", "Configuring grafana")
289 config = hookenv.config()
290- settings = {"config": config}
291+ settings = {
292+ "config": config,
293+ "protocol": get_protocol(),
294+ "cert_file": CERT_PATH[config["install_method"]],
295+ "cert_key": CERT_KEY_PATH[config["install_method"]],
296+ }
297+
298 smtp_auth = config.get("smtp_auth", False)
299 if smtp_auth and len(smtp_auth.split(":")) == 2:
300 settings["smtp_user"] = smtp_auth.split(":")[0]
301@@ -501,6 +525,7 @@ def setup_backup_shedule():
302 "directory": config.get("dashboards_backup_dir"),
303 "port": config.get("port"),
304 "backup_keys": " ".join(add_backup_api_keys()),
305+ "scheme": get_protocol(),
306 }
307 render(
308 source=DASHBOARDS_BACKUP_CRON_TMPL,
309@@ -536,6 +561,8 @@ def restart_grafana():
310 hookenv.log(msg)
311 hookenv.status_set("maintenance", msg)
312 host.service_restart(svcname)
313+
314+ _block_until_port_open()
315 hookenv.status_set("active", "Ready")
316 set_state("grafana.started")
317 hookenv.status_set("active", "Started {}".format(svcname))
318@@ -562,11 +589,18 @@ def update_nrpe_config():
319 nrpe.add_init_service_checks(nrpe_setup, [svcname], current_unit)
320
321 # write the http check
322- nrpe_setup.add_check(
323- "grafana_http",
324- "Grafana HTTP check",
325- "check_http -I 127.0.0.1 -p {} -u /login".format(config["port"]),
326- )
327+ if get_protocol() == "https":
328+ nrpe_setup.add_check(
329+ "grafana_http",
330+ "Grafana HTTPS check",
331+ "check_http -S -I 127.0.0.1 -p {} -u /login".format(config["port"]),
332+ )
333+ else:
334+ nrpe_setup.add_check(
335+ "grafana_http",
336+ "Grafana HTTP check",
337+ "check_http -I 127.0.0.1 -p {} -u /login".format(config["port"]),
338+ )
339
340 nrpe_setup.write()
341 set_state("grafana.nagios-setup.completed")
342@@ -747,8 +781,9 @@ def get_current_dashboards(port, passwd):
343 https://grafana.com/docs/grafana/latest/http_api/folder_dashboard_search/
344 """
345 dash_req = requests.get(
346- "http://127.0.0.1:{}/api/search?type=dash-db".format(port),
347+ "{}://127.0.0.1:{}/api/search?type=dash-db".format(get_protocol(), port),
348 auth=("admin", passwd),
349+ verify=get_ca_cert_path(),
350 )
351 return dash_req.json() if dash_req.status_code == 200 else []
352
353@@ -766,8 +801,9 @@ def get_current_dashboard_json(uid, port, passwd):
354 return default_dashboard
355
356 dash_req = requests.get(
357- "http://127.0.0.1:{}/api/dashboards/uid/{}".format(port, uid),
358+ "{}://127.0.0.1:{}/api/dashboards/uid/{}".format(get_protocol(), port, uid),
359 auth=("admin", passwd),
360+ verify=get_ca_cert_path(),
361 )
362 if dash_req.status_code != 200:
363 return default_dashboard
364@@ -856,8 +892,13 @@ def check_and_add_dashboard(
365 )
366
367 hookenv.log("Using Dashboard Template: {}".format(filename))
368- post_req = "http://127.0.0.1:{}/api/dashboards/db".format(port)
369- r = requests.post(post_req, json=dashboard_json, auth=("admin", gf_adminpasswd))
370+ post_req = "{}://127.0.0.1:{}/api/dashboards/db".format(get_protocol(), port)
371+ r = requests.post(
372+ post_req,
373+ json=dashboard_json,
374+ auth=("admin", gf_adminpasswd),
375+ verify=get_ca_cert_path(),
376+ )
377
378 if r.status_code != 200:
379 hookenv.log(
380@@ -1075,8 +1116,9 @@ def get_orgs(port, passwd):
381 https://grafana.com/docs/grafana/latest/http_api/org/
382 """
383 req = requests.get(
384- "http://127.0.0.1:{}/api/orgs".format(port),
385+ "{}://127.0.0.1:{}/api/orgs".format(get_protocol(), port),
386 auth=("admin", passwd),
387+ verify=get_ca_cert_path(),
388 )
389 return req.json() if req.status_code == 200 else []
390
391@@ -1255,3 +1297,83 @@ def import_dashboards(dashboards):
392 req.respond(success, reason)
393
394 kv.set("dash_digests", dashboard_digests)
395+
396+
397+@when("certificates.available")
398+@when_not("grafana.certificates.configured")
399+@when_not("grafana.certificates.requested")
400+def tls_request_certificate(tls):
401+ """Create a server certificate for this server."""
402+ # Use the public ip of this unit as the Common Name for the certificate.
403+ common_name = hookenv.unit_public_ip()
404+ # Get a list of Subject Alt Names for the certificate.
405+ sans = []
406+ sans.append(hookenv.unit_public_ip())
407+ sans.append(hookenv.unit_private_ip())
408+ sans.append(socket.gethostname())
409+ sans.append("127.0.0.1")
410+ sans.append("localhost")
411+ hookenv.log(
412+ "Requesting certificate with CN: {common_name}".format(common_name=common_name)
413+ )
414+ tls_client.request_server_cert(
415+ common_name,
416+ sans,
417+ crt_path=CERT_PATH[hookenv.config("install_method")],
418+ key_path=CERT_KEY_PATH[hookenv.config("install_method")],
419+ )
420+ _maybe_install_ca_certificates_hook()
421+ set_state("grafana.certificates.requested")
422+
423+
424+@when("tls_client.certs.changed")
425+@when("grafana.certificates.requested")
426+def tls_certificate_changed():
427+ """Request to reconfigure grafana after the certificates were written."""
428+ # the certificates are written in owned by root and 0o770 permissions
429+ # which prevents grafana-server process to read them when using the debian
430+ # package, so the parent directory needs to allow the access by 'grafana'
431+ # group.
432+ if hookenv.config("install_method") == "apt":
433+ uid = pwd.getpwnam("root").pw_uid
434+ gid = grp.getgrnam("grafana").gr_gid
435+ ssl_dir = os.path.dirname(CERT_PATH[hookenv.config("install_method")])
436+ os.chown(ssl_dir, uid, gid)
437+
438+ # making sure the hook to update the certificate inside the snap's common
439+ # dir is executed if needed.
440+ subprocess.check_call(["update-ca-certificates"])
441+
442+ set_state("grafana.certificates.configured")
443+ if is_flag_set("grafana.configured"):
444+ remove_state("grafana.configured")
445+
446+
447+def _maybe_install_ca_certificates_hook():
448+ # When grafana is running within a snap won't have access to the system's
449+ # certificates store, so a hook script is installed to rsync it on any
450+ # execution of update-ca-certificates
451+ if hookenv.config("install_method") != "snap":
452+ # when running from the deb package there is no need to install the
453+ # hook, the ca-certificates will take care of everything for us.
454+ return
455+
456+ templating.render(
457+ "sync-grafana-snap",
458+ CA_CERTIFICATES_HOOK,
459+ context={"CA_CERT_PATH": CA_CERT_PATH},
460+ perms=0o755,
461+ )
462+ subprocess.check_call(["update-ca-certificates"])
463+
464+
465+# Block for a not so small period of time, because in hyperconverged
466+# environments, specially while a bundle is being deployed, the amount of IOPS
467+# in flight may prevent restarts from happening fast enough.
468+@retry_on_exception(10, base_delay=2)
469+def _block_until_port_open():
470+ a_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
471+ port = hookenv.config("port")
472+ location = ("127.0.0.1", int(port))
473+ if a_socket.connect_ex(location) != 0:
474+ raise Exception("port %s is closed" % port)
475diff --git a/src/requirements.txt b/src/requirements.txt
476index 8462291..236ad65 100644
477--- a/src/requirements.txt
478+++ b/src/requirements.txt
479@@ -1 +1,2 @@
480 # Include python requirements here
481+rpdb
482diff --git a/src/templates/grafana.ini.j2 b/src/templates/grafana.ini.j2
483index 06278fc..77d4673 100644
484--- a/src/templates/grafana.ini.j2
485+++ b/src/templates/grafana.ini.j2
486@@ -8,7 +8,13 @@ instance_name = {{ config.instance_name }}
487
488 [server]
489 # Protocol (http or https)
490-;protocol = http
491+protocol = {{ protocol }}
492+
493+{%- if protocol == "https" %}
494+# https certs & key file
495+cert_file = {{ cert_file }}
496+cert_key = {{ cert_key }}
497+{%- endif %}
498
499 # The ip address to bind to, empty will bind to all interfaces
500 http_addr = 0.0.0.0
501diff --git a/src/templates/juju-dashboards-backup.j2 b/src/templates/juju-dashboards-backup.j2
502index 00f0737..993c48a 100644
503--- a/src/templates/juju-dashboards-backup.j2
504+++ b/src/templates/juju-dashboards-backup.j2
505@@ -1,5 +1,5 @@
506 #
507 # Please do not edit. Juju will overwrite it.
508 #
509-{{ schedule }} root /usr/local/bin/dashboards_backup -d {{ directory }} -p {{ port }} {{ backup_keys }}
510+{{ schedule }} root /usr/local/bin/dashboards_backup -d {{ directory }} -p {{ port }} -s {{ scheme }} {{ backup_keys }}
511
512diff --git a/src/templates/sync-grafana-snap b/src/templates/sync-grafana-snap
513new file mode 100644
514index 0000000..7bf28b2
515--- /dev/null
516+++ b/src/templates/sync-grafana-snap
517@@ -0,0 +1,7 @@
518+#!/bin/sh
519+#
520+# This file is managed by juju.
521+#
522+
523+set -e
524+rsync /etc/ssl/certs/ca-certificates.crt {{ CA_CERT_PATH }}
525diff --git a/src/tests/functional/requirements.txt b/src/tests/functional/requirements.txt
526index bbe8435..a9af82d 100644
527--- a/src/tests/functional/requirements.txt
528+++ b/src/tests/functional/requirements.txt
529@@ -1,2 +1,8 @@
530 git+https://github.com/openstack-charmers/zaza.git#egg=zaza
531+<<<<<<< src/tests/functional/requirements.txt
532 python-openstackclient
533+=======
534+# OpenStack tools current;y required when using OpenStack provider.
535+python-keystoneclient
536+python-novaclient
537+>>>>>>> src/tests/functional/requirements.txt
538diff --git a/src/tests/functional/tests/bundles/bionic-snap-tls.yaml b/src/tests/functional/tests/bundles/bionic-snap-tls.yaml
539new file mode 120000
540index 0000000..f81f6ff
541--- /dev/null
542+++ b/src/tests/functional/tests/bundles/bionic-snap-tls.yaml
543@@ -0,0 +1 @@
544+base.yaml
545\ No newline at end of file
546diff --git a/src/tests/functional/tests/bundles/bionic-tls.yaml b/src/tests/functional/tests/bundles/bionic-tls.yaml
547new file mode 120000
548index 0000000..f81f6ff
549--- /dev/null
550+++ b/src/tests/functional/tests/bundles/bionic-tls.yaml
551@@ -0,0 +1 @@
552+base.yaml
553\ No newline at end of file
554diff --git a/src/tests/functional/tests/bundles/focal-snap-tls.yaml b/src/tests/functional/tests/bundles/focal-snap-tls.yaml
555new file mode 120000
556index 0000000..f81f6ff
557--- /dev/null
558+++ b/src/tests/functional/tests/bundles/focal-snap-tls.yaml
559@@ -0,0 +1 @@
560+base.yaml
561\ No newline at end of file
562diff --git a/src/tests/functional/tests/bundles/focal-tls.yaml b/src/tests/functional/tests/bundles/focal-tls.yaml
563new file mode 120000
564index 0000000..f81f6ff
565--- /dev/null
566+++ b/src/tests/functional/tests/bundles/focal-tls.yaml
567@@ -0,0 +1 @@
568+base.yaml
569\ No newline at end of file
570diff --git a/src/tests/functional/tests/bundles/overlays/bionic-snap-tls.yaml.j2 b/src/tests/functional/tests/bundles/overlays/bionic-snap-tls.yaml.j2
571new file mode 100644
572index 0000000..e302099
573--- /dev/null
574+++ b/src/tests/functional/tests/bundles/overlays/bionic-snap-tls.yaml.j2
575@@ -0,0 +1,11 @@
576+series: bionic
577+applications:
578+ grafana:
579+ options:
580+ install_method: snap
581+ snap_channel: 6/stable
582+ easyrsa:
583+ charm: cs:~containers/easyrsa
584+ num_units: 1
585+relations:
586+ - [ grafana:certificates, easyrsa ]
587diff --git a/src/tests/functional/tests/bundles/overlays/bionic-tls.yaml.j2 b/src/tests/functional/tests/bundles/overlays/bionic-tls.yaml.j2
588new file mode 100644
589index 0000000..e728037
590--- /dev/null
591+++ b/src/tests/functional/tests/bundles/overlays/bionic-tls.yaml.j2
592@@ -0,0 +1,10 @@
593+series: bionic
594+applications:
595+ grafana:
596+ options:
597+ install_method: apt
598+ easyrsa:
599+ charm: cs:~containers/easyrsa
600+ num_units: 1
601+relations:
602+ - [ grafana:certificates, easyrsa ]
603diff --git a/src/tests/functional/tests/bundles/overlays/focal-snap-tls.yaml.j2 b/src/tests/functional/tests/bundles/overlays/focal-snap-tls.yaml.j2
604new file mode 100644
605index 0000000..259f3c5
606--- /dev/null
607+++ b/src/tests/functional/tests/bundles/overlays/focal-snap-tls.yaml.j2
608@@ -0,0 +1,32 @@
609+series: focal
610+applications:
611+ grafana:
612+ num_units: 1
613+ options:
614+ install_method: snap
615+ snap_channel: 6/stable
616+ ceph-mon:
617+ series: bionic
618+ ceph-osd:
619+ series: bionic
620+ glance:
621+ series: bionic
622+ keystone:
623+ series: bionic
624+ mysql:
625+ series: bionic
626+ rabbitmq-server:
627+ series: bionic
628+ prometheus:
629+ series: bionic
630+ prometheus-ceph-exporter:
631+ series: bionic
632+ prometheus-libvirt-exporter:
633+ series: bionic
634+ nagios:
635+ series: bionic
636+ easyrsa:
637+ charm: cs:~containers/easyrsa
638+ num_units: 1
639+relations:
640+ - [ grafana:certificates, easyrsa ]
641diff --git a/src/tests/functional/tests/bundles/overlays/focal-tls.yaml.j2 b/src/tests/functional/tests/bundles/overlays/focal-tls.yaml.j2
642new file mode 100644
643index 0000000..b6283dc
644--- /dev/null
645+++ b/src/tests/functional/tests/bundles/overlays/focal-tls.yaml.j2
646@@ -0,0 +1,31 @@
647+series: focal
648+applications:
649+ grafana:
650+ num_units: 1
651+ options:
652+ install_method: apt
653+ ceph-mon:
654+ series: bionic
655+ ceph-osd:
656+ series: bionic
657+ glance:
658+ series: bionic
659+ keystone:
660+ series: bionic
661+ mysql:
662+ series: bionic
663+ rabbitmq-server:
664+ series: bionic
665+ prometheus:
666+ series: bionic
667+ prometheus-ceph-exporter:
668+ series: bionic
669+ prometheus-libvirt-exporter:
670+ series: bionic
671+ nagios:
672+ series: bionic
673+ easyrsa:
674+ charm: cs:~containers/easyrsa
675+ num_units: 1
676+relations:
677+ - [ grafana:certificates, easyrsa ]
678diff --git a/src/tests/functional/tests/bundles/overlays/xenial-tls.yaml.j2 b/src/tests/functional/tests/bundles/overlays/xenial-tls.yaml.j2
679new file mode 100644
680index 0000000..86f7ebf
681--- /dev/null
682+++ b/src/tests/functional/tests/bundles/overlays/xenial-tls.yaml.j2
683@@ -0,0 +1,12 @@
684+series: xenial
685+applications:
686+ grafana:
687+ options:
688+ install_method: apt
689+ easyrsa:
690+ # Use bionic version as easyrsa is failing to install on xenial
691+ series: bionic
692+ charm: cs:~containers/easyrsa
693+ num_units: 1
694+relations:
695+ - [ grafana:certificates, easyrsa ]
696diff --git a/src/tests/functional/tests/bundles/xenial-tls.yaml b/src/tests/functional/tests/bundles/xenial-tls.yaml
697new file mode 120000
698index 0000000..f81f6ff
699--- /dev/null
700+++ b/src/tests/functional/tests/bundles/xenial-tls.yaml
701@@ -0,0 +1 @@
702+base.yaml
703\ No newline at end of file
704diff --git a/src/tests/functional/tests/test_grafana.py b/src/tests/functional/tests/test_grafana.py
705index d0e54d4..03fe575 100644
706--- a/src/tests/functional/tests/test_grafana.py
707+++ b/src/tests/functional/tests/test_grafana.py
708@@ -1,6 +1,8 @@
709 """Encapsulate prometheus-openstack-exporter testing."""
710 import json
711 import logging
712+import os
713+import tempfile
714 import time
715 import unittest
716
717@@ -16,7 +18,10 @@ DEFAULT_BACKUP_DIRECTORY = "/srv/backups"
718
719
720 class BaseGrafanaTest(unittest.TestCase):
721- """Base for Prometheus-openstack-exporter charm tests."""
722+ """Base for Prometheus-openstack-exporter charm tests.
723+
724+ - Get the CA from easyrsa (when available) and write it on disk.
725+ """
726
727 _admin_pass = None
728
729@@ -32,11 +37,45 @@ class BaseGrafanaTest(unittest.TestCase):
730 cls.units = model.get_units(cls.application_name, model_name=cls.model_name)
731 cls.grafana_ip = model.get_app_ips(cls.application_name)[0]
732
733+ # get the CA certificate from the relation between easy easyrsa and
734+ # grafana.
735+ if cls.get_protocol() == "https":
736+ rel_id = model.get_relation_id(
737+ "grafana", "easyrsa", remote_interface_name="client"
738+ )
739+ easyrsa_unit = model.get_units("easyrsa")[0]
740+ cmd = "relation-get -r client:{} ca {}".format(
741+ rel_id, easyrsa_unit.entity_id
742+ )
743+ logging.info(cmd)
744+ result = model.run_on_unit(easyrsa_unit.entity_id, cmd)
745+
746+ with tempfile.NamedTemporaryFile(mode="w", delete=False) as f:
747+ f.write(result["Stdout"])
748+ f.flush()
749+ cls.ca_path = f.name
750+ else:
751+ cls.ca_path = None
752+
753+ @classmethod
754+ def tearDownClass(cls):
755+ """Remove the CA file that was written to disk during the setUp."""
756+ if cls.ca_path:
757+ os.remove(cls.ca_path)
758+
759 def get_unit_status(self, unit):
760 """Get grafana unit workload status."""
761 u = model.get_unit_from_name(unit)
762 return u.workload_status_message
763
764+ @classmethod
765+ def get_protocol(cls):
766+ """Get protocol configured to serve Grafana."""
767+ if model.get_relation_id("grafana", "easyrsa"):
768+ return "https"
769+ else:
770+ return "http"
771+
772 @property
773 def admin_password(self):
774 """Get grafana admin password."""
775@@ -58,11 +97,14 @@ class BaseGrafanaTest(unittest.TestCase):
776 dashboards = []
777 while True:
778 req = requests.get(
779- "http://{host}:{port}"
780+ "{protocol}://{host}:{port}"
781 "/api/search?dashboardIds".format(
782- host=self.grafana_ip, port=DEFAULT_API_PORT
783+ protocol=self.get_protocol(),
784+ host=self.grafana_ip,
785+ port=DEFAULT_API_PORT,
786 ),
787 auth=("admin", self.admin_password),
788+ verify=self.ca_path,
789 )
790 self.assertTrue(req.status_code == 200)
791 dashboards = json.loads(req.text)
792@@ -113,11 +155,19 @@ class CharmOperationTest(BaseGrafanaTest):
793 Curl the api endpoint.
794 We'll retry until the TEST_TIMEOUT.
795 """
796+ if self.get_protocol() == "https":
797+ # -S instructs the check_http plugin to check the SSL port which
798+ # defaults to 443.
799+ cmd = "/usr/lib/nagios/plugins/check_http -S"
800+ else:
801+ cmd = "/usr/lib/nagios/plugins/check_http"
802+
803 test_command = "{} -I 127.0.0.1 -p {} -u {}".format(
804- "/usr/lib/nagios/plugins/check_http",
805+ cmd,
806 DEFAULT_API_PORT,
807 DEFAULT_API_URL,
808 )
809+ logging.debug("test api ready command: %s" % test_command)
810 timeout = time.time() + TEST_TIMEOUT
811 while time.time() < timeout:
812 response = model.run_on_unit(self.lead_unit_name, test_command)
813@@ -130,9 +180,11 @@ class CharmOperationTest(BaseGrafanaTest):
814
815 # we didn't get rc=0 in the allowed time, fail the test
816 self.fail(
817- "http port didn't respond to the command \n"
818+ "{protocol} port didn't respond to the command \n"
819 "'{test_command}' as expected.\n"
820- "Result: {result}".format(test_command=test_command, result=response)
821+ "Result: {result}".format(
822+ protocol=self.get_protocol(), test_command=test_command, result=response
823+ )
824 )
825
826 def test_02_nrpe_http_check(self):
827@@ -140,10 +192,19 @@ class CharmOperationTest(BaseGrafanaTest):
828 logging.debug(
829 "Verify the nrpe check is created and has the required content..."
830 )
831- expected_nrpe_check = (
832- "command[check_grafana_http]=/usr/lib/nagios/plugins/check_http "
833- "-I 127.0.0.1 -p 3000 -u /login"
834- )
835+ if self.get_protocol() == "https":
836+ expected_nrpe_check = (
837+ "command[check_grafana_http]="
838+ "/usr/lib/nagios/plugins/check_http "
839+ "-S -I 127.0.0.1 -p 3000 -u /login"
840+ )
841+ else:
842+ expected_nrpe_check = (
843+ "command[check_grafana_http]="
844+ "/usr/lib/nagios/plugins/check_http "
845+ "-I 127.0.0.1 -p 3000 -u /login"
846+ )
847+
848 cmd = "cat /etc/nagios/nrpe.d/check_grafana_http.cfg"
849 result = model.run_on_unit(self.lead_unit_name, cmd)
850 code = result.get("Code")
851@@ -213,10 +274,13 @@ class CharmOperationTest(BaseGrafanaTest):
852 self.assertTrue(action.data["results"]["Code"] == "0")
853 time.sleep(30) # Dirty hack to overcome race condition
854 req = requests.get(
855- "http://{host}:{port}/api/org/".format(
856- host=self.grafana_ip, port=DEFAULT_API_PORT
857+ "{protocol}://{host}:{port}/api/org/".format(
858+ protocol=self.get_protocol(),
859+ host=self.grafana_ip,
860+ port=DEFAULT_API_PORT,
861 ),
862 auth=("foouser", "sikkrit"),
863+ verify=self.ca_path,
864 )
865 self.assertTrue(req.status_code == 200)
866 self.assertTrue("name" in req.json())
867diff --git a/src/tests/functional/tests/tests.yaml b/src/tests/functional/tests/tests.yaml
868index 909b776..736607f 100644
869--- a/src/tests/functional/tests/tests.yaml
870+++ b/src/tests/functional/tests/tests.yaml
871@@ -1,10 +1,15 @@
872 charm_name: grafana
873 gate_bundles:
874 - model_apt_install: focal
875+ - model_apt_install: focal-tls
876 - model_apt_install: bionic
877+ - model_apt_install: bionic-tls
878 - model_apt_install: xenial
879+ - model_apt_install: xenial-tls
880 - model_snap_install: bionic-snap
881+ - model_snap_install: bionic-snap-tls
882 - model_snap_install: focal-snap
883+ - model_snap_install: focal-snap-tls
884 smoke_bundles:
885 - model_snap_install: bionic-snap
886 dev_bundles:
887@@ -25,3 +30,5 @@ target_deploy_status:
888 workload-status-message: Monitoring
889 prometheus-libvirt-exporter:
890 workload-status-message: "Exporter installed and connected to libvirt slot"
891+ easyrsa:
892+ workload-status-message: Certificate Authority connected.
893diff --git a/src/tests/unit/test_grafana.py b/src/tests/unit/test_grafana.py
894index 663fa17..2e460ef 100644
895--- a/src/tests/unit/test_grafana.py
896+++ b/src/tests/unit/test_grafana.py
897@@ -1,4 +1,6 @@
898 """Unit tests module."""
899+import os
900+import socket
901 import sys
902 import unittest
903 from os.path import isfile
904@@ -7,6 +9,8 @@ from unittest.mock import call
905
906 from charmhelpers.core import unitdata
907
908+tls_client_mock = mock.Mock()
909+sys.modules["charms.layer.tls_client"] = tls_client_mock
910 sys.modules["charms.layer.snap"] = mock.Mock()
911
912
913@@ -148,3 +152,47 @@ class GrafanaTestCase(unittest.TestCase):
914 "dash-name",
915 {"dashboard": {"title": "[juju-foo-app] test-title"}, "folderId": 0},
916 )
917+
918+ @mock.patch("charmhelpers.core.hookenv.charm_dir", auto_spec=True)
919+ @mock.patch("reactive.grafana.templating.render", auto_spec=True)
920+ @mock.patch("reactive.grafana.hookenv.config", auto_spec=True)
921+ @mock.patch("reactive.grafana.hookenv.unit_public_ip", auto_spec=True)
922+ @mock.patch("reactive.grafana.hookenv.unit_private_ip", auto_spec=True)
923+ @mock.patch("subprocess.check_call", auto_spec=True)
924+ def test_request_certificate(
925+ self,
926+ mock_check_call,
927+ mock_private_ip,
928+ mock_public_ip,
929+ mock_config,
930+ mock_render,
931+ mock_charm_dir,
932+ ):
933+ """Test request certificate."""
934+ charm_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "../../")
935+ mock_charm_dir.return_value = charm_dir
936+
937+ config = {"install_method": "snap"}
938+
939+ def fake_config(key):
940+ return config[key]
941+
942+ mock_config.side_effect = fake_config
943+ mock_public_ip.return_value = "1.2.3.4"
944+ mock_private_ip.return_value = "5.6.7.8"
945+
946+ mock_tls = mock.Mock()
947+ grafana_reactive.tls_request_certificate(mock_tls)
948+ tls_client_mock.request_server_cert.assert_called_with(
949+ "1.2.3.4",
950+ ["1.2.3.4", "5.6.7.8", socket.gethostname(), "127.0.0.1", "localhost"],
951+ crt_path=grafana_reactive.CERT_PATH["snap"],
952+ key_path=grafana_reactive.CERT_KEY_PATH["snap"],
953+ )
954+
955+ mock_render.assert_called_with(
956+ "sync-grafana-snap",
957+ grafana_reactive.CA_CERTIFICATES_HOOK,
958+ context={"CA_CERT_PATH": grafana_reactive.CA_CERT_PATH},
959+ perms=0o755,
960+ )
961diff --git a/src/tox.ini b/src/tox.ini
962index 5845e03..72d33e7 100644
963--- a/src/tox.ini
964+++ b/src/tox.ini
965@@ -21,6 +21,7 @@ passenv =
966 NO_PROXY
967 SNAP_HTTP_PROXY
968 SNAP_HTTPS_PROXY
969+<<<<<<< src/tox.ini
970 OS_REGION_NAME
971 OS_AUTH_VERSION
972 OS_AUTH_URL
973@@ -31,6 +32,9 @@ passenv =
974 OS_USER_DOMAIN_NAME
975 OS_PROJECT_NAME
976 OS_IDENTITY_API_VERSION
977+=======
978+ OS_*
979+>>>>>>> src/tox.ini
980
981 [testenv:lint]
982 commands =

Subscribers

People subscribed via source and target branches

to all changes: