bootstack-ops

Merge bootstack-ops:jjo-p-o-e-creds into bootstack-ops:master

  1. Git
  2. lp:bootstack-ops
  3. jjo-p-o-e-creds
  4. Merge into master
Proposed by JuanJo Ciarlante
Status: Merged
Merged at revision: 656040d256b6bd788187f7c0ac331d4718fd57ba
Proposed branch: bootstack-ops:jjo-p-o-e-creds
Merge into: bootstack-ops:master
Diff against target: 253 lines (+117/-20)
7 files modified
ops-bundle/devel/opsdeploy-prometheus.yaml (+8/-7)
ops-bundle/devel/prometheus.codetree (+4/-5)
ops-bundle/manifest-prometheus (+3/-1)
ops-bundle/production/opsdeploy-prometheus.yaml (+21/-2)
ops-bundle/production/prometheus.codetree (+4/-5)
ops-bundle/scripts/juju_bundle_export.sh (+1/-0)
ops-bundle/scripts/prometheus_exporter_creds.py (+76/-0)
Reviewer Review Type Date Requested Status
Legacy - Canonical WTFB Pending
Review via email: mp+322262@code.launchpad.net

Commit message

[jjo] add p-o-e charm instrumentation for its os-credentials settings + other fixes

Description of the change

[jjo] add p-o-e charm instrumentation for its os-credentials settings + other fixes

To post a comment you must log in.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/ops-bundle/devel/opsdeploy-prometheus.yaml b/ops-bundle/devel/opsdeploy-prometheus.yaml
index fef0e9e..485400c 100644
--- a/ops-bundle/devel/opsdeploy-prometheus.yaml
+++ b/ops-bundle/devel/opsdeploy-prometheus.yaml
@@ -5,7 +5,7 @@ nil:
5 services: {}5 services: {}
66
7ops-bundle-prometheus:7ops-bundle-prometheus:
8 inherits: [ secrets, bsdeploy ]8 inherits: [ secrets, bsdeploy, poe-secrets ]
9 series: xenial9 series: xenial
10 overrides:10 overrides:
11 snap_proxy: "http://10.76.12.1:8000"11 snap_proxy: "http://10.76.12.1:8000"
@@ -29,7 +29,7 @@ ops-bundle-prometheus:
29 telegraf:29 telegraf:
30 charm: telegraf30 charm: telegraf
31 options:31 options:
32 hostname: '{unit}'32 hostname: '{host}'
33 prometheus_output_port: default33 prometheus_output_port: default
3434
35# WIP35# WIP
@@ -39,9 +39,9 @@ ops-bundle-prometheus:
39# to: *PROMSTACK_TO39# to: *PROMSTACK_TO
40#40#
41# # snap -> needs to be at metal or juju2 lxd41# # snap -> needs to be at metal or juju2 lxd
42# prometheus-openstack-exporter:42 prometheus-openstack-exporter:
43# charm: prometheus-openstack-exporter43 charm: prometheus-openstack-exporter
44# to: *PROMSTACK_TO44 to: *PROMSTACK_TO
4545
46 relations:46 relations:
47# Connect the grafana -> prometheus -> telegraf stack47# Connect the grafana -> prometheus -> telegraf stack
@@ -49,7 +49,8 @@ ops-bundle-prometheus:
49 - ["telegraf:prometheus-client", "prometheus:target"]49 - ["telegraf:prometheus-client", "prometheus:target"]
50# Deploy telegraf on : os-hosts xor nova-compute, etc50# Deploy telegraf on : os-hosts xor nova-compute, etc
51 - ["telegraf:juju-info", "os-hosts"]51 - ["telegraf:juju-info", "os-hosts"]
52# WIP52# WIP: alertmanager
53# - ["prometheus:alertmanager-service", "prometheus-alertmanager:alertmanager-service"]53# - ["prometheus:alertmanager-service", "prometheus-alertmanager:alertmanager-service"]
54# WIP: prometheus-openstack-exporter
54# - ["prometheus-openstack-exporter", "keystone"]55# - ["prometheus-openstack-exporter", "keystone"]
55# - ["prometheus-openstack-exporter", "prometheus:target"]56 - ["prometheus-openstack-exporter", "prometheus:target"]
diff --git a/ops-bundle/devel/prometheus.codetree b/ops-bundle/devel/prometheus.codetree
index 455ea24..aafc6e1 100644
--- a/ops-bundle/devel/prometheus.codetree
+++ b/ops-bundle/devel/prometheus.codetree
@@ -1,7 +1,6 @@
1# Reporting1# Reporting
2prometheus git+https://git.launchpad.net/prometheus-charm2prometheus cs:~prometheus-charmers/prometheus
3grafana git+https://git.launchpad.net/grafana-charm3grafana cs:~prometheus-charmers/grafana
4prometheus-alertmanager git+https://git.launchpad.net/prometheus-alertmanager-charm4telegraf cs:telegraf
5telegraf git+https://git.launchpad.net/telegraf-charm
6#prometheus-openstack-exporter cs:~jjo/prometheus-openstack-exporter
7prometheus-openstack-exporter git+https://git.launchpad.net/~jjo/bootstack-ops/+git/prometheus-openstack-exporter-charm5prometheus-openstack-exporter git+https://git.launchpad.net/~jjo/bootstack-ops/+git/prometheus-openstack-exporter-charm
6#prometheus-openstack-exporter cs:~jjo/prometheus-openstack-exporter
diff --git a/ops-bundle/manifest-prometheus b/ops-bundle/manifest-prometheus
index 9bd0bcb..b044a49 100644
--- a/ops-bundle/manifest-prometheus
+++ b/ops-bundle/manifest-prometheus
@@ -3,12 +3,14 @@
33
4# Create bundle and codetree from running env4# Create bundle and codetree from running env
5script config=scripts/juju_bundle_export.sh OUT_BUNDLE=bsdeploy.yaml OUT_CODETREE=bsdeploy.codetree NAME=bsdeploy5script config=scripts/juju_bundle_export.sh OUT_BUNDLE=bsdeploy.yaml OUT_CODETREE=bsdeploy.codetree NAME=bsdeploy
6script config=scripts/prometheus_exporter_creds.py DEPLOY_BUNDLE=bsdeploy.yaml DEPLOY_NAME=bsdeploy OUT_FILE=poe-secrets.yaml
67
7# Collect Charms for running env8# Collect Charms for running env
8collect config=bsdeploy.codetree9collect config=bsdeploy.codetree
910
10# Collect Charms for added services11# Collect Charms for added services
11collect config=prometheus.codetree12collect config=prometheus.codetree
13# mojo collect --options config=prometheus.codetree
1214
13# Some hints for peeking diffs15# Some hints for peeking diffs
14# mojo deploy-diff --options local=secrets,bsdeploy.yaml config=opsdeploy-prometheus.yaml target=ops-bundle-prometheus16# mojo deploy-diff --options local=secrets,bsdeploy.yaml config=opsdeploy-prometheus.yaml target=ops-bundle-prometheus
@@ -18,4 +20,4 @@ collect config=prometheus.codetree
18# Deploy services20# Deploy services
19# "nil" deploy just inherits from running, should not deploy anything21# "nil" deploy just inherits from running, should not deploy anything
20#deploy local=secrets,bsdeploy.yaml config=opsdeploy-prometheus.yaml target=nil22#deploy local=secrets,bsdeploy.yaml config=opsdeploy-prometheus.yaml target=nil
21deploy local=secrets,bsdeploy.yaml config=opsdeploy-prometheus.yaml target=ops-bundle-prometheus23deploy local=secrets,poe-secrets.yaml,bsdeploy.yaml config=opsdeploy-prometheus.yaml target=ops-bundle-prometheus
diff --git a/ops-bundle/production/opsdeploy-prometheus.yaml b/ops-bundle/production/opsdeploy-prometheus.yaml
index f9fd133..485400c 100644
--- a/ops-bundle/production/opsdeploy-prometheus.yaml
+++ b/ops-bundle/production/opsdeploy-prometheus.yaml
@@ -5,7 +5,7 @@ nil:
5 services: {}5 services: {}
66
7ops-bundle-prometheus:7ops-bundle-prometheus:
8 inherits: [ secrets, bsdeploy ]8 inherits: [ secrets, bsdeploy, poe-secrets ]
9 series: xenial9 series: xenial
10 overrides:10 overrides:
11 snap_proxy: "http://10.76.12.1:8000"11 snap_proxy: "http://10.76.12.1:8000"
@@ -15,23 +15,42 @@ ops-bundle-prometheus:
15 charm: prometheus15 charm: prometheus
16 to: *PROMSTACK_TO16 to: *PROMSTACK_TO
17 options:17 options:
18
18 # non snap -> could be LXC'd19 # non snap -> could be LXC'd
19 grafana:20 grafana:
20 charm: grafana21 charm: grafana
21 to: *PROMSTACK_TO22 to: *PROMSTACK_TO
22 options:23 options:
24 ## XXX: must go to secrets.yaml
25 admin_password: admin
23 install_file: https://grafanarel.s3.amazonaws.com/builds/grafana_4.1.2-1486989747_amd64.deb26 install_file: https://grafanarel.s3.amazonaws.com/builds/grafana_4.1.2-1486989747_amd64.deb
2427
25 # subord -> will land on relation to ppal services (os-hosts or nova-compute, etc)28 # subord -> will land on relation to ppal services (os-hosts or nova-compute, etc)
26 telegraf:29 telegraf:
27 charm: telegraf30 charm: telegraf
28 options:31 options:
29 hostname: '{unit}'32 hostname: '{host}'
30 prometheus_output_port: default33 prometheus_output_port: default
3134
35# WIP
36# # snap -> needs to be at metal or juju2 lxd
37# prometheus-alertmanager:
38# charm: prometheus-alertmanager
39# to: *PROMSTACK_TO
40#
41# # snap -> needs to be at metal or juju2 lxd
42 prometheus-openstack-exporter:
43 charm: prometheus-openstack-exporter
44 to: *PROMSTACK_TO
45
32 relations:46 relations:
33# Connect the grafana -> prometheus -> telegraf stack47# Connect the grafana -> prometheus -> telegraf stack
34 - ["prometheus:grafana-source", "grafana:grafana-source"]48 - ["prometheus:grafana-source", "grafana:grafana-source"]
35 - ["telegraf:prometheus-client", "prometheus:target"]49 - ["telegraf:prometheus-client", "prometheus:target"]
36# Deploy telegraf on : os-hosts xor nova-compute, etc50# Deploy telegraf on : os-hosts xor nova-compute, etc
37 - ["telegraf:juju-info", "os-hosts"]51 - ["telegraf:juju-info", "os-hosts"]
52# WIP: alertmanager
53# - ["prometheus:alertmanager-service", "prometheus-alertmanager:alertmanager-service"]
54# WIP: prometheus-openstack-exporter
55# - ["prometheus-openstack-exporter", "keystone"]
56 - ["prometheus-openstack-exporter", "prometheus:target"]
diff --git a/ops-bundle/production/prometheus.codetree b/ops-bundle/production/prometheus.codetree
index cd999eb..aafc6e1 100644
--- a/ops-bundle/production/prometheus.codetree
+++ b/ops-bundle/production/prometheus.codetree
@@ -1,7 +1,6 @@
1# Reporting1# Reporting
2prometheus git+https://git.launchpad.net/prometheus-charm2prometheus cs:~prometheus-charmers/prometheus
3grafana git+https://git.launchpad.net/grafana-charm3grafana cs:~prometheus-charmers/grafana
4#prometheus-alertmanager git+https://git.launchpad.net/prometheus-alertmanager-charm4telegraf cs:telegraf
5telegraf git+https://git.launchpad.net/telegraf-charm5prometheus-openstack-exporter git+https://git.launchpad.net/~jjo/bootstack-ops/+git/prometheus-openstack-exporter-charm
6#prometheus-openstack-exporter cs:~jjo/prometheus-openstack-exporter6#prometheus-openstack-exporter cs:~jjo/prometheus-openstack-exporter
7#prometheus-openstack-exporter git+https://git.launchpad.net/~jjo/bootstack-ops/+git/prometheus-openstack-exporter-charm
diff --git a/ops-bundle/scripts/juju_bundle_export.sh b/ops-bundle/scripts/juju_bundle_export.sh
index 56e68ef..f0c8e93 100755
--- a/ops-bundle/scripts/juju_bundle_export.sh
+++ b/ops-bundle/scripts/juju_bundle_export.sh
@@ -8,5 +8,6 @@ PY_SCRIPT=${0%.sh}.py
8(set -x8(set -x
9$PY_SCRIPT -o ${MOJO_LOCAL_DIR}/${OUT_BUNDLE} -c ${MOJO_SPEC_DIR}/${MOJO_STAGE}/${OUT_CODETREE} --name ${NAME}9$PY_SCRIPT -o ${MOJO_LOCAL_DIR}/${OUT_BUNDLE} -c ${MOJO_SPEC_DIR}/${MOJO_STAGE}/${OUT_CODETREE} --name ${NAME}
10# HACK: above script will create xenial/CHARM while mojo likes ./CHARM 10# HACK: above script will create xenial/CHARM while mojo likes ./CHARM
11rm -rf ${MOJO_BUILD_DIR:?}/*
11ln -sfn . ${MOJO_BUILD_DIR}/xenial12ln -sfn . ${MOJO_BUILD_DIR}/xenial
12)13)
diff --git a/ops-bundle/scripts/prometheus_exporter_creds.py b/ops-bundle/scripts/prometheus_exporter_creds.py
13new file mode 10075514new file mode 100755
index 0000000..8ccfb6e
--- /dev/null
+++ b/ops-bundle/scripts/prometheus_exporter_creds.py
@@ -0,0 +1,76 @@
1#!/usr/bin/env python3
2# XXX(jjo) HACK to ease os-credentials settings for p-o-e charm,
3# crafted from bundle's keystone settings as saved from running env
4# NOTE this is very mojo specific, as it not only relies on MOJO_LOCAL_DIR
5# but also wants args passed as env vars (mojo manifest line)
6import os
7import yaml
8import logging
9
10logging.basicConfig()
11logger = logging.getLogger()
12
13# The bundle file where running env was saved
14deploy_bundle = "{}/{}".format(
15 os.environ['MOJO_LOCAL_DIR'],
16 os.environ['DEPLOY_BUNDLE'])
17
18# Then deployment name inside the bundle, e.g. 'bsdeploy'
19deploy_name = os.environ['DEPLOY_NAME']
20
21# Output secrets.yaml file for p-o-e charm, it'll have
22# 'os-credentials' and 'ssl_ca' settings
23out_file = os.environ.get('OUT_FILE')
24if out_file:
25 out_file = "{}/{}".format(os.environ['MOJO_LOCAL_DIR'], out_file)
26
27# Output deployment name which contains p-o-e charm secrets
28out_name = os.environ.get('OUT_NAME', 'poe-secrets')
29auth_url = os.environ.get('OS_AUTH_URL')
30
31# Load DEPLOY_BUNDLE[DEPLOY_NAME]'s keystone config
32deploy = yaml.load(open(deploy_bundle).read())[deploy_name]
33ks_config = deploy['services']['keystone']['options']
34
35# Having OS_AUTH_URL from the running shell env would be great,
36# else try some educated guess
37if not auth_url:
38 logging.warning("No 'OS_AUTH_URL' in environment, guessing from keystone")
39 hostname = ks_config.get('os-public-hostname', ks_config.get('vip'))
40 proto = 'https' if ks_config.get('https-service-endpoints') else 'http'
41 auth_url = '{}://{}:5000/v2.0'.format(proto, hostname)
42
43logging.info(
44 "out_file: {}, out_name: {}, deploy_name: {}, auth_url: {}".format(
45 out_file, out_name, deploy_name, auth_url))
46
47# os-credentials must be a string'd YAML, i.e. can't directly be a python dict
48# (as yaml.dump would save it as a yaml dict itself, while juju only supports
49# "flat" non-structured settings)
50poe_settings = {
51 'os-credentials': '{{username: {}, password: {}, region_name: {}, auth_url: "{}" }}'.format( # noqa
52 'Admin', ks_config['admin-password'],
53 ks_config['region'], auth_url)
54}
55
56# Only save ssl_ca if present in keystone
57ssl_ca = ks_config.get('ssl_ca')
58if ssl_ca:
59 poe_settings.update({'ssl_ca': ssl_ca})
60
61# Output mini-bundle with just p-o-e secrets, meant to be used
62# as part of local= mojo manifest line, then inherited from
63# prometheus.yaml main bundle
64out_content = {
65 out_name: {
66 'services': {
67 'prometheus-openstack-exporter': {
68 'options': poe_settings
69 }
70 }
71 }
72}
73
74# print to stdout if no OUT_FILE
75print_kw = {'file': open(out_file, 'w')} if out_file else {}
76print(yaml.dump(out_content, default_flow_style=False), **print_kw)

Subscribers

People subscribed via source and target branches

to all changes: