Merge lp:~afreiberger/landscape-client-charm/add-juju-annotations into lp:landscape-client-charm

Proposed by Drew Freiberger
Status: Rejected
Rejected by: Haw Loeung
Proposed branch: lp:~afreiberger/landscape-client-charm/add-juju-annotations
Merge into: lp:landscape-client-charm
Diff against target: 193 lines (+119/-3)
4 files modified
config.yaml (+13/-2)
hooks/hooks.py (+8/-1)
hooks/install.py (+34/-0)
scripts/write_juju_annotations.py (+64/-0)
To merge this branch: bzr merge lp:~afreiberger/landscape-client-charm/add-juju-annotations
Reviewer Review Type Date Requested Status
Landscape Builder test results Needs Fixing
Simon Poirier (community) Needs Fixing
Review via email: mp+413315@code.launchpad.net

Commit message

Add cronjob and charm config for registering juju agents as Landscape annotations.

Description of the change

Based on previous efforts (which are then obsolete) by:
* @xavpaice https://code.launchpad.net/~xavpaice/landscape-client-charm/add_tags/+merge/407578
* @exsdev https://code.launchpad.net/~exsdev/landscape-client-charm/juju-app-annotator
* @stephanpampel https://code.launchpad.net/~stephanpampel/landscape-client-charm/landscape-client-charm/+merge/411409

Add hook and cronjob for registering juju apps as Landscape annotations
Can be enabled via config flag juju-annotations-enabled (default=false)

Details 'juju-annotations-enabled':
        Periodically updates landscape annotations (key:value) for each juju agent
        found on the machine and the juju availability-zone (AZ) of the machine.
        Key is 'juju-${application}' OR 'juju-az-${AZ}' if an AZ is set.
        Value is 'deployed=true' OR 'deployed=false' if the agent got removed.
        Example: juju-landscape-client:deployed=true
        Cronjob runs every 5 min, update frequency depends on exchange-interval.

To post a comment you must log in.
Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
75. By Drew Freiberger

Address import issues

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
76. By Drew Freiberger

py3 updates for write-juju-annotations.py

77. By Drew Freiberger

Rename annotations script

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
78. By Drew Freiberger

fix data-path for juju annotations cronjob

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
79. By Drew Freiberger

Fixed path to annotations script

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
80. By Drew Freiberger

Annotations - Detect multiple dashes in unit names and add unit number along with app name

Revision history for this message
Drew Freiberger (afreiberger) wrote :

I've completed manual functional testing and am satisfied with this feature implementation now.

Ready for review by Landscape team.

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
81. By Drew Freiberger

Add clarification to config.yaml for annotations

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
82. By Drew Freiberger

Fix ordering of config-changed for juju-annotations cronjob

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
83. By Drew Freiberger

Fix regex for annotations

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
Revision history for this message
Drew Freiberger (afreiberger) wrote :

Update from landscape team is that this is going to be superceded by work being done to allow client-side tag manipulation directly. This branch is being held available as the underlying branch published at cs:~afreiberger/landscape-client.

Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
Revision history for this message
Simon Poirier (simpoir) wrote :

First, this changes needs some test coverage (for both hooks and the externally-called `write_juju_annotations.py` script), and fixing broken tests (which might be mostly covered by my comments below).

review: Needs Fixing
Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)
Revision history for this message
Landscape Builder (landscape-builder) :
review: Abstain (executing tests)
Revision history for this message
Landscape Builder (landscape-builder) wrote :
review: Needs Fixing (test results)

Unmerged revisions

83. By Drew Freiberger

Fix regex for annotations

82. By Drew Freiberger

Fix ordering of config-changed for juju-annotations cronjob

81. By Drew Freiberger

Add clarification to config.yaml for annotations

80. By Drew Freiberger

Annotations - Detect multiple dashes in unit names and add unit number along with app name

79. By Drew Freiberger

Fixed path to annotations script

78. By Drew Freiberger

fix data-path for juju annotations cronjob

77. By Drew Freiberger

Rename annotations script

76. By Drew Freiberger

py3 updates for write-juju-annotations.py

75. By Drew Freiberger

Address import issues

74. By Drew Freiberger

Update juju-annotations to be an external root cron

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'config.yaml'
2--- config.yaml 2019-02-03 14:50:36 +0000
3+++ config.yaml 2021-12-16 20:35:21 +0000
4@@ -21,7 +21,7 @@
5 origin:
6 description: |
7 *DEPRECATED*. See install_sources and install_keys
8-
9+
10 Origin of ppa or private deb repository from which to install
11 landscape-client. May be one of the following:
12 distro (default), ppa:somecustom/ppa or a full APT url source
13@@ -131,4 +131,15 @@
14 regardless of the setting created by the unattended-upgrades package.
15 type: boolean
16 default: False
17-
18+ juju-annotations-enabled:
19+ description: |
20+ Periodically updates landscape annotations (key:value) for each juju agent
21+ found on the machine and the juju availability-zone (AZ) of the machine.
22+ Key is 'juju-${application}' OR 'juju-az-${AZ}' if an AZ is set.
23+ Additionally, annotations for 'juju-${application}-${unit_number}' will
24+ be set. Values are 'deployed=true' OR 'deployed=false' denoting presence
25+ of these agents.
26+ Example: juju-landscape-client:deployed=true
27+ Cronjob runs every 5 min, update frequency depends on exchange-interval.
28+ type: boolean
29+ default: False
30
31=== modified file 'hooks/hooks.py'
32--- hooks/hooks.py 2019-04-04 16:07:48 +0000
33+++ hooks/hooks.py 2021-12-16 20:35:21 +0000
34@@ -18,7 +18,8 @@
35 from ceph import (
36 get_ceph_client_path, write_ceph_client_keyring, write_ceph_client_config)
37
38-from install import install
39+from install import (
40+ install, create_annotations_cronjob, remove_annotations_cronjob)
41
42 CERT_FILE = "/etc/ssl/certs/landscape_server_ca.crt"
43
44@@ -62,6 +63,12 @@
45 else:
46 set_config[key] = value
47
48+ # Manage toggling of juju-annotations cronjob
49+ if service_config["juju-annotations-enabled"]:
50+ create_annotations_cronjob()
51+ else:
52+ remove_annotations_cronjob()
53+
54 relation_data.update(set_config)
55 if service_config.get("disable-unattended-upgrades"):
56 Cnf = apt_pkg.Configuration()
57
58=== modified file 'hooks/install.py'
59--- hooks/install.py 2019-02-03 14:50:36 +0000
60+++ hooks/install.py 2021-12-16 20:35:21 +0000
61@@ -7,14 +7,19 @@
62
63 import subprocess
64 import sys
65+from os import environ, remove
66
67 from charmhelpers import fetch
68 import charmhelpers.core.hookenv as hookenv
69+import charmhelpers.core.host as host
70
71
72 hooks = hookenv.Hooks()
73
74
75+ANNOTATIONS_CRON_PATH = '/etc/cron.d/landscape-client-juju-annotations'
76+
77+
78 @hooks.hook("install")
79 def install(fetch=fetch, hookenv=hookenv):
80 charm_config = hookenv.config()
81@@ -34,6 +39,11 @@
82 "install", "-o", "landscape", "-g", "root", "-m", "755", "-d",
83 data_path])
84
85+ if charm_config["juju-annotations-enabled"]:
86+ create_annotations_cronjob()
87+ else:
88+ remove_annotations_cronjob()
89+
90
91 def add_apt_source(url, fetch=fetch):
92 """Add an apt source entry, with passed in key.
93@@ -54,6 +64,30 @@
94 return fetch.add_source(url, key)
95
96
97+def create_annotations_cronjob():
98+ hookenv.log('Create juju-annotations cronjob', hookenv.DEBUG)
99+ unit_name = hookenv.local_unit()
100+ agent_name = 'unit-' + unit_name.replace('/', '-')
101+ zone_env = ""
102+ if environ['JUJU_AVAILABILITY_ZONE']:
103+ az = environ['JUJU_AVAILABILITY_ZONE']
104+ zone_env = "JUJU_AVAILABILITY_ZONE=%s" % az
105+ data_path = hookenv.config().get("data-path")
106+ if not data_path:
107+ data_path = "/var/lib/landscape/client"
108+ cronjob = (
109+ "*/5 * * * * root %s "
110+ "/var/lib/juju/agents/%s/charm/scripts/write_juju_annotations.py "
111+ "%s\n"
112+ ) % (zone_env, agent_name, data_path)
113+ host.write_file(ANNOTATIONS_CRON_PATH, cronjob)
114+
115+
116+def remove_annotations_cronjob():
117+ hookenv.log('Remove juju-annotations cronjob', hookenv.DEBUG)
118+ remove(ANNOTATIONS_CRON_PATH)
119+
120+
121 if __name__ == '__main__':
122 try:
123 sys.exit(hooks.execute(sys.argv))
124
125=== added directory 'scripts'
126=== added file 'scripts/write_juju_annotations.py'
127--- scripts/write_juju_annotations.py 1970-01-01 00:00:00 +0000
128+++ scripts/write_juju_annotations.py 2021-12-16 20:35:21 +0000
129@@ -0,0 +1,64 @@
130+#!/usr/bin/env python3
131+"""Update landscape-client juju info annotations.
132+
133+usage: [JUJU_AVAILABILITY_ZONE="zone1"] write-juju-annotations [lds-data-path]
134+"""
135+
136+import sys
137+import os
138+import re
139+
140+
141+JUJU_AGENT_DIR = "/var/lib/juju/agents"
142+LDS_CLIENT_DATA_PATH = "/var/lib/landscape/client"
143+
144+
145+def write_juju_annotations(argv):
146+ """Write annotation files in the '$DATA_PATH/annotations.d' folder.
147+
148+ For each currently deployed juju unit and one annotation for the juju
149+ availability zone (if JUJU_AVAILABILITY_ZONE is set). The
150+ annotation-value/file-content is 'deployed=true'.
151+
152+ For previously-deployed/removed units change the annotation value to
153+ 'deployed=false'.
154+
155+ This is used in conjunction with lp:charm-landscape-tagger.
156+ """
157+ if len(argv) == 2:
158+ data_path = argv[1]
159+ else:
160+ data_path = LDS_CLIENT_DATA_PATH
161+ annotations_folder = os.path.join(data_path, 'annotations.d')
162+ unit_regex = re.compile(r"^unit-(((\w|-)+)-\d+)")
163+ prefix = 'juju-'
164+
165+ target_annotations = []
166+ for agent in os.listdir(JUJU_AGENT_DIR):
167+ result = unit_regex.match(agent)
168+ if result:
169+ unit = result[1]
170+ app = result[2]
171+ target_annotations.append(prefix + unit)
172+ target_annotations.append(prefix + app)
173+
174+ current_annotations = \
175+ [a for a in os.listdir(annotations_folder) if a.startswith(prefix)]
176+
177+ if "JUJU_AVAILABILITY_ZONE" in os.environ:
178+ target_annotations.append(
179+ f"juju-az-{os.environ['JUJU_AVAILABILITY_ZONE']}"
180+ )
181+
182+ all_annotations = list(set(target_annotations + current_annotations))
183+ for annotation in all_annotations:
184+ annotation_file = os.path.join(annotations_folder, annotation)
185+ with open(annotation_file, "w") as f:
186+ if annotation in target_annotations:
187+ f.write("deployed=true")
188+ else:
189+ f.write("deployed=false")
190+
191+
192+if __name__ == '__main__':
193+ write_juju_annotations(sys.argv)

Subscribers

People subscribed via source and target branches

to all changes: