Merge lp:~cprov/adt-continuous-deployer/better-monitor into lp:adt-continuous-deployer
- better-monitor
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Celso Providelo |
Approved revision: | 48 |
Merged at revision: | 43 |
Proposed branch: | lp:~cprov/adt-continuous-deployer/better-monitor |
Merge into: | lp:adt-continuous-deployer |
Diff against target: |
643 lines (+312/-229) 6 files modified
ci_automation/nova.py (+92/-0) ci_automation/tests/test_branch.py (+6/-6) ci_automation/tests/test_nova.py (+90/-81) ci_automation/utils.py (+30/-0) list.py (+56/-0) monitor.py (+38/-142) |
To merge this branch: | bzr merge lp:~cprov/adt-continuous-deployer/better-monitor |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Francis Ginther | Approve | ||
Para Siva (community) | Approve | ||
Evan (community) | Needs Information | ||
Paul Larson | Approve | ||
Review via email: mp+257315@code.launchpad.net |
Commit message
Refactoring monitor.py and also adding support for basic auto-deployed service listing with list.py script.
Description of the change
Refactoring monitor.py and also adding support for basic service listing via list.py:
{{{
stg-ue-
* mojo-ue-
11a8f85b: Thu Apr 16 22:05:46 2015 (4 units)
* mojo-ue-
e240a00e: Thu Apr 23 18:50:44 2015 (7 units)
* mojo-ue-
c1ec6cc8: Wed Apr 15 19:20:48 2015 (4 units)
* mojo-ue-
13d54727: Wed Apr 15 21:35:30 2015 (2 units)
* mojo-ue-
324e91c7: Thu Apr 23 04:15:42 2015 (2 units)
* mojo-ue-
057b67aa: Thu Apr 23 19:20:44 2015 (2 units)
* mojo-ue-
1aeb45ff: Thu Apr 23 02:10:52 2015 (2 units)
* mojo-ue-
428911cd: Tue Apr 14 15:40:40 2015 (2 units)
}}}
Removal of obsolete deployments still supported by `./monitor.py mojo-ue-
Also fixing some broken and ignored tests from the shortening-
- 45. By Celso Providelo
-
Sorted service list
Does pretty-printing the deployments and purging old deployments really belong in the same program, especially as the algorithm to decide what old environments grows in complexity?
See inline comment.
Para Siva (psivaa) wrote : | # |
An inline comment about a typo for logstash env variable.
Also tested the 'list' option to be working.
- 46. By Celso Providelo
-
Fixing typo.
Celso Providelo (cprov) wrote : | # |
Psivaa, nice catch, typo fixed.
I still have to address Ev's comment/suggestion for preserving the old monitor.py behavior and creating a list.py script for the pretty-print task.
- 47. By Celso Providelo
-
Dedicated script for listing auto-deployed services (list.py), monitor.py preserves previous behaviour.
Celso Providelo (cprov) wrote : | # |
Ev and others,
I've restored monitor.py behaviour (refactored on top of the common nova functions) and introduced list.py for listing auto-deployed services nicely.
Let me know what do you think.
Para Siva (psivaa) wrote : | # |
LGTM now, approving.
A docstring for list.py would be a bonus.
- 48. By Celso Providelo
-
Docstring fixing.
Preview Diff
1 | === added file 'ci_automation/nova.py' |
2 | --- ci_automation/nova.py 1970-01-01 00:00:00 +0000 |
3 | +++ ci_automation/nova.py 2015-04-27 18:16:41 +0000 |
4 | @@ -0,0 +1,92 @@ |
5 | +# Copyright (C) 2015 Canonical |
6 | +# |
7 | +# This program is free software: you can redistribute it and/or modify |
8 | +# it under the terms of the GNU General Public License as published by |
9 | +# the Free Software Foundation, either version 3 of the License, or |
10 | +# (at your option) any later version. |
11 | +# |
12 | +# This program is distributed in the hope that it will be useful, |
13 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
14 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
15 | +# GNU General Public License for more details. |
16 | +# |
17 | +# You should have received a copy of the GNU General Public License |
18 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
19 | +# |
20 | + |
21 | +"""CI Automation: nova helpers.""" |
22 | + |
23 | +import io |
24 | +import os |
25 | +import stat |
26 | +import subprocess |
27 | + |
28 | + |
29 | +def nova_list(): |
30 | + """Default 'nova list' accessor.""" |
31 | + args = [ |
32 | + 'nova', |
33 | + 'list', |
34 | + '--minimal', |
35 | + ] |
36 | + |
37 | + output = subprocess.check_output( |
38 | + args, stderr=subprocess.DEVNULL).decode('utf-8').strip() |
39 | + |
40 | + return output |
41 | + |
42 | + |
43 | +def get_services_map(base_dir, nova_output=None): |
44 | + """Return a services map dictionary. |
45 | + |
46 | + 'base_dir' is the path where deployment identifier can be found. |
47 | + |
48 | + If 'nova_output' is not provided `nova_list` accessor is used to get |
49 | + this information. |
50 | + |
51 | + Returns a dictionary key-ed by service name (middle term of the mojo |
52 | + stage used in the deployment, e.g. 'mojo-ue-core-image-watcher'): |
53 | + |
54 | + { |
55 | + 'mojo-ue-core-image-watcher': { |
56 | + 'DEADBEEF': { |
57 | + 'path': <identifier_path>, |
58 | + 'epoch': <identifier_ST_MTIME>, |
59 | + 'units': [<unit_name>, <unit_name>, ...], |
60 | + }, |
61 | + 'ABCD1234': { |
62 | + 'path': <identifier_path>, |
63 | + 'epoch': <identifier_ST_MTIME>, |
64 | + 'units': [<unit_name>, <unit_name>, ...], |
65 | + }, |
66 | + }, |
67 | + } |
68 | + |
69 | + Manually deployed environments, i.e. no identifiers on disk, are ignored. |
70 | + """ |
71 | + if nova_output is None: |
72 | + nova_output = nova_list() |
73 | + |
74 | + services = {} |
75 | + |
76 | + for line in io.StringIO(nova_output).readlines()[3:-1]: |
77 | + unit_name = line.split('|')[2].strip() |
78 | + service_name = '-'.join(unit_name.split('-')[1:-3]) |
79 | + identifier = unit_name.split('-')[-3] |
80 | + |
81 | + path = os.path.join(base_dir, identifier) |
82 | + if not os.path.exists(path): |
83 | + continue |
84 | + |
85 | + service = services.setdefault(service_name, {}) |
86 | + deployment = service.setdefault(identifier, {}) |
87 | + units = deployment.setdefault('units', []) |
88 | + units.append(unit_name) |
89 | + |
90 | + if deployment.get('path') is None: |
91 | + deployment.update({ |
92 | + 'path': path, |
93 | + 'epoch': os.stat(path)[stat.ST_MTIME], |
94 | + }) |
95 | + |
96 | + return services |
97 | |
98 | === modified file 'ci_automation/tests/test_branch.py' |
99 | --- ci_automation/tests/test_branch.py 2015-04-02 19:29:27 +0000 |
100 | +++ ci_automation/tests/test_branch.py 2015-04-27 18:16:41 +0000 |
101 | @@ -99,13 +99,13 @@ |
102 | self.assertEqual( |
103 | ('\n'.join(['bob/devel\t1', |
104 | 'lp:foo;revno=1\t1']), |
105 | - '90945bed4b6e27d500ae1ae05c7bba1c'), |
106 | + '7ebe5386'), |
107 | get_identifier(self.branch, stage)) |
108 | self.branch_add_stage('the-builder/devel') |
109 | self.assertEqual( |
110 | ('\n'.join(['bob/devel\t1', |
111 | 'lp:foo;revno=1\t1']), |
112 | - '90945bed4b6e27d500ae1ae05c7bba1c'), |
113 | + '7ebe5386'), |
114 | get_identifier(self.branch, stage)) |
115 | |
116 | def test_get_identifier_spec_changes(self): |
117 | @@ -115,7 +115,7 @@ |
118 | self.assertEqual( |
119 | ('\n'.join(['bob/devel\t1', |
120 | 'lp:foo;revno=1\t1']), |
121 | - '90945bed4b6e27d500ae1ae05c7bba1c'), |
122 | + '7ebe5386'), |
123 | get_identifier(self.branch, stage)) |
124 | repo_path = os.path.join(self.branch, stage, 'repo') |
125 | with open(repo_path, 'w'): |
126 | @@ -125,7 +125,7 @@ |
127 | self.assertEqual( |
128 | ('\n'.join(['bob/devel\t2', |
129 | 'lp:foo;revno=1\t1']), |
130 | - 'dce1f2c4a1a1634f49a4572fa8630765'), |
131 | + 'fd2e0853'), |
132 | get_identifier(self.branch, stage)) |
133 | |
134 | def test_get_identifier_collect_changes(self): |
135 | @@ -136,7 +136,7 @@ |
136 | self.assertEqual( |
137 | ('\n'.join(['bob/devel\t2', |
138 | 'lp:foo;revno=7\t7']), |
139 | - 'f4bdf7cb82d6f1a9fc725d1bf70eef75'), |
140 | + '77ff4c59'), |
141 | get_identifier(self.branch, stage)) |
142 | self.branch_update_collect( |
143 | stage, '\n'.join(['foo lp:~cprov/foo;revno=7', |
144 | @@ -145,7 +145,7 @@ |
145 | ('\n'.join(['bob/devel\t3', |
146 | 'lp:~cprov/foo;revno=7\t7', |
147 | 'lp:bar;revno=5\t5']), |
148 | - '261a6c824fa28680e7e0869194a12152'), |
149 | + '7a1b03fa'), |
150 | get_identifier(self.branch, stage)) |
151 | |
152 | |
153 | |
154 | === renamed file 'test_monitor.py' => 'ci_automation/tests/test_nova.py' |
155 | --- test_monitor.py 2015-04-08 17:59:46 +0000 |
156 | +++ ci_automation/tests/test_nova.py 2015-04-27 18:16:41 +0000 |
157 | @@ -1,93 +1,102 @@ |
158 | -import logging |
159 | +#!/usr/bin/env python3 |
160 | +# |
161 | +# Copyright (C) 2015 Canonical |
162 | +# |
163 | +# This program is free software: you can redistribute it and/or modify |
164 | +# it under the terms of the GNU General Public License as published by |
165 | +# the Free Software Foundation, either version 3 of the License, or |
166 | +# (at your option) any later version. |
167 | +# |
168 | +# This program is distributed in the hope that it will be useful, |
169 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
170 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
171 | +# GNU General Public License for more details. |
172 | +# |
173 | +# You should have received a copy of the GNU General Public License |
174 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
175 | +# |
176 | + |
177 | +"""CI Automation tests for nova helpers.""" |
178 | + |
179 | import os |
180 | import shutil |
181 | +import sys |
182 | import tempfile |
183 | import time |
184 | import unittest |
185 | |
186 | -from monitor import ( |
187 | - get_service_deployments, |
188 | - get_service_info, |
189 | - parse_nova_output, |
190 | -) |
191 | - |
192 | - |
193 | -class TestServiceWatcher(unittest.TestCase): |
194 | + |
195 | +HERE = os.path.abspath(os.path.dirname(__file__)) |
196 | +sys.path.append(os.path.join(HERE, '../..')) |
197 | + |
198 | +from ci_automation.nova import get_services_map |
199 | + |
200 | + |
201 | +class TestNova(unittest.TestCase): |
202 | + |
203 | + identifiers = ( |
204 | + '12345', |
205 | + 'abcdef', |
206 | + '56789', |
207 | + ) |
208 | + service_names = ( |
209 | + 'mojo-ue-core-result-watcher', |
210 | + 'mojo-ue-core-image-watcher', |
211 | + 'mojo-ue-core-image-publisher', |
212 | + 'mojo-ue-core-image-tester', |
213 | + ) |
214 | + |
215 | def setUp(self): |
216 | - self.prefix = 'juju-mojo-ue' |
217 | - self.identifiers = [ |
218 | - '12345', |
219 | - 'abcdef', |
220 | - '56789', |
221 | - ] |
222 | - self.service_names = [ |
223 | - 'core-result-watcher', |
224 | - 'core-image-watcher', |
225 | - 'core-image-publisher', |
226 | - 'core-image-tester', |
227 | - ] |
228 | self.base_dir = tempfile.mkdtemp() |
229 | self.addCleanup(shutil.rmtree, self.base_dir) |
230 | - self.unit_name_pattern = '{}-(.*)-(.*)-machine-\d+'.format(self.prefix) |
231 | - |
232 | - # generate some fake 'nova list' output |
233 | + |
234 | + def generate_identifiers(self): |
235 | + """Generate deployment identifiers on disk.""" |
236 | + for ident in self.identifiers: |
237 | + # get unique timestamps |
238 | + time.sleep(.1) |
239 | + os.makedirs(os.path.join(self.base_dir, ident)) |
240 | + |
241 | + def create_output(self): |
242 | + """Return some fake 'nova list' output.""" |
243 | count = 0 |
244 | - self.output = "" |
245 | + output = "" |
246 | for ident in self.identifiers: |
247 | count += 1 |
248 | - |
249 | - for service_name in self.service_names: |
250 | - self.output = '{}| id-{} | {}-{}-{}-machine-0 |\n'.format( |
251 | - self.output, count, self.prefix, service_name, ident) |
252 | - |
253 | - #self._create_tmp_files() |
254 | - |
255 | - def _create_tmp_files(self): |
256 | - for ident in self.identifiers: |
257 | - # get unique timestamps |
258 | - time.sleep(2) |
259 | - os.makedirs(os.path.join(self.base_dir, ident)) |
260 | - |
261 | - def test_parse_nova_output_valid(self): |
262 | - |
263 | - units = parse_nova_output(self.output, self.service_names[0]) |
264 | - logging.warn("JOE: units: %s", units) |
265 | - |
266 | - self.assertEqual(len(self.identifiers), len(units)) |
267 | - |
268 | - def test_get_service_info_valid(self): |
269 | - unit_name = '{}-{}-{}-machine-0'.format( |
270 | - self.prefix, |
271 | - self.service_names[0], |
272 | - self.identifiers[0], |
273 | - ) |
274 | - |
275 | - service_name, identifier = get_service_info(unit_name, |
276 | - self.unit_name_pattern) |
277 | - |
278 | - self.assertEqual(self.service_names[0], service_name) |
279 | - self.assertEqual(self.identifiers[0], identifier) |
280 | - |
281 | - def test_get_service_info_invalid(self): |
282 | - unit_name = '{}-{}-{}-machine-0'.format( |
283 | - self.prefix, |
284 | - self.service_names[0], |
285 | - self.identifiers[0], |
286 | - ) |
287 | - |
288 | - service_name, identifier = get_service_info("junk-%s" % unit_name, |
289 | - '') |
290 | - |
291 | - self.assertEqual(None, service_name) |
292 | - self.assertEqual(None, identifier) |
293 | - |
294 | - def test_get_service_deployments(self): |
295 | - service_deployments = get_service_deployments(self.service_names[0], |
296 | - self.output, |
297 | - self.unit_name_pattern) |
298 | - |
299 | - self.assertEqual(1, len(service_deployments)) |
300 | - for key in service_deployments: |
301 | - self.assertEqual(len(self.identifiers), |
302 | - len(service_deployments[key]), |
303 | - "mismatch for '{}'".format(key)) |
304 | + for name in self.service_names: |
305 | + output += '| id-{} | juju-{}-{}-machine-0 |\n'.format( |
306 | + count, name, ident) |
307 | + return output |
308 | + |
309 | + def test_service_map_ignores_missing_identifiers(self): |
310 | + # Deployments with no identifiers on disk are ignored (they were |
311 | + # probably deployed manually with mojo.py, not with cd.py via cron). |
312 | + service_map = get_services_map( |
313 | + self.base_dir, nova_output=self.create_output()) |
314 | + self.assertEqual({}, service_map) |
315 | + |
316 | + def test_service_map(self): |
317 | + self.generate_identifiers() |
318 | + service_map = get_services_map( |
319 | + self.base_dir, nova_output=self.create_output()) |
320 | + |
321 | + # service_map is key-ed by services names. |
322 | + self.assertEqual( |
323 | + sorted(self.service_names), sorted(service_map.keys())) |
324 | + |
325 | + # Each service points to a dictionary key-ed by the deployment |
326 | + # identifier. |
327 | + service_contents = service_map['mojo-ue-core-image-watcher'] |
328 | + self.assertEqual( |
329 | + ['56789', 'abcdef'], sorted(service_contents.keys())) |
330 | + |
331 | + # Each deployment contains 'path' (identifier file on disk), 'epoch' |
332 | + # (identifier file mtime) and 'units' (list of full unit names, |
333 | + # including the bootstrap node). |
334 | + deployment_contents = service_contents['abcdef'] |
335 | + self.assertEqual( |
336 | + ['epoch', 'path', 'units'], sorted(deployment_contents.keys())) |
337 | + |
338 | + |
339 | +if __name__ == '__main__': |
340 | + unittest.main() |
341 | |
342 | === modified file 'ci_automation/utils.py' |
343 | --- ci_automation/utils.py 2015-04-02 19:29:27 +0000 |
344 | +++ ci_automation/utils.py 2015-04-27 18:16:41 +0000 |
345 | @@ -16,6 +16,8 @@ |
346 | |
347 | """CI Automation: miscelaneous utilities.""" |
348 | |
349 | +import logging |
350 | +import os |
351 | import subprocess |
352 | import textwrap |
353 | |
354 | @@ -61,3 +63,31 @@ |
355 | {}""" |
356 | ''').format(stdout.decode(), stderr.decode()) |
357 | ) |
358 | + |
359 | + |
360 | +def setup_logger(): |
361 | + """Return a configured logger instance. |
362 | + |
363 | + Defaults to INFO level and if 'CI_LOGSTASH_HOST' environment variable |
364 | + is set and python-logstash is reachable, adds a corresponding 'logstash' |
365 | + handler. |
366 | + """ |
367 | + logging.basicConfig( |
368 | + format='%(asctime)s %(name)s %(levelname)s: %(message)s') |
369 | + logger = logging.getLogger() |
370 | + logger.setLevel(logging.INFO) |
371 | + |
372 | + # Optional remote logging via logstash. |
373 | + logstash_host = os.environ.get('CI_LOGSTASH_HOST') |
374 | + if logstash_host is not None: |
375 | + try: |
376 | + import logstash |
377 | + except ImportError: |
378 | + print('Follow the README instructions for installing ' |
379 | + 'python-logstash') |
380 | + return 1 |
381 | + else: |
382 | + logger.addHandler( |
383 | + logstash.LogstashHandler(logstash_host, 5959, 1)) |
384 | + |
385 | + return logger |
386 | |
387 | === added file 'list.py' |
388 | --- list.py 1970-01-01 00:00:00 +0000 |
389 | +++ list.py 2015-04-27 18:16:41 +0000 |
390 | @@ -0,0 +1,56 @@ |
391 | +#!/usr/bin/env python3 |
392 | +# |
393 | +# Copyright (C) 2015 Canonical |
394 | +# |
395 | +# This program is free software: you can redistribute it and/or modify |
396 | +# it under the terms of the GNU General Public License as published by |
397 | +# the Free Software Foundation, either version 3 of the License, or |
398 | +# (at your option) any later version. |
399 | +# |
400 | +# This program is distributed in the hope that it will be useful, |
401 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
402 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
403 | +# GNU General Public License for more details. |
404 | +# |
405 | +# You should have received a copy of the GNU General Public License |
406 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
407 | +# |
408 | +"""CI Automation: script for listing auto-deployed services.""" |
409 | + |
410 | +import argparse |
411 | +import datetime |
412 | +import os |
413 | +import sys |
414 | + |
415 | + |
416 | +from ci_automation.nova import get_services_map |
417 | +from ci_automation.utils import setup_logger |
418 | + |
419 | + |
420 | +def main(): |
421 | + parser = argparse.ArgumentParser( |
422 | + description="Check the number of deployments for a service") |
423 | + parser.add_argument('--base-dir', '-b', |
424 | + default=os.path.expanduser("~/ci-cd-identifiers"), |
425 | + help="The directory where identifiers are stored " |
426 | + " (eg. ~/ci-cd-identifiers)") |
427 | + args = parser.parse_args() |
428 | + logger = setup_logger() |
429 | + |
430 | + services = get_services_map(args.base_dir) |
431 | + sorted_services = sorted(services.items(), key=lambda s: s[0]) |
432 | + for srv, deployments in sorted_services: |
433 | + print('* {}:'.format(srv)) |
434 | + sorted_deployments = sorted( |
435 | + deployments.items(), key=lambda d: d[1]['epoch'], reverse=True) |
436 | + for ident, contents in sorted_deployments: |
437 | + date_created = datetime.datetime.utcfromtimestamp( |
438 | + contents['epoch']).ctime() |
439 | + print('\t{}: {} ({} units)'.format( |
440 | + ident, date_created, len(contents['units']))) |
441 | + |
442 | + return 0 |
443 | + |
444 | + |
445 | +if __name__ == "__main__": |
446 | + sys.exit(main()) |
447 | |
448 | === modified file 'monitor.py' |
449 | --- monitor.py 2015-04-10 20:20:17 +0000 |
450 | +++ monitor.py 2015-04-27 18:16:41 +0000 |
451 | @@ -15,155 +15,51 @@ |
452 | # You should have received a copy of the GNU General Public License |
453 | # along with this program. If not, see <http://www.gnu.org/licenses/>. |
454 | # |
455 | +"""CI Automation: monitoring (purging obsolete deployments) services.""" |
456 | |
457 | import argparse |
458 | -import subprocess |
459 | -import io |
460 | -import logging |
461 | -import operator |
462 | import os |
463 | -import re |
464 | import sys |
465 | |
466 | -from stat import ST_MTIME |
467 | - |
468 | - |
469 | -def parse_args(): |
470 | - desc = "Check the number of deployments for a service" |
471 | - parser = argparse.ArgumentParser(description=desc) |
472 | - parser.add_argument('--base-dir', '-b', |
473 | - default=os.path.expanduser("~/ci-cd-identifiers"), |
474 | - help="The directory where identifiers are stored " |
475 | - " (eg. ~/ci-cd-identifiers)") |
476 | - parser.add_argument('--max-deployments', '-m', type=int, default=2, |
477 | - help="The maximum number of deployments to keep.") |
478 | - parser.add_argument('stage_dirname', |
479 | - help="the mojo stage directory name (eg. " |
480 | - "mojo-ue-core-image-tester") |
481 | + |
482 | +from ci_automation.nova import get_services_map |
483 | +from ci_automation.utils import setup_logger |
484 | + |
485 | + |
486 | +def main(): |
487 | + parser = argparse.ArgumentParser( |
488 | + description="Check the number of deployments for a service") |
489 | + parser.add_argument( |
490 | + '--base-dir', '-b', |
491 | + default=os.path.expanduser("~/ci-cd-identifiers"), |
492 | + help="The directory where identifiers are stored " |
493 | + " (eg. ~/ci-cd-identifiers)") |
494 | + parser.add_argument( |
495 | + '--max-deployments', '-m', type=int, default=2, |
496 | + help="The maximum number of deployments to keep.") |
497 | + parser.add_argument( |
498 | + 'stage_dirname', |
499 | + help="the mojo stage directory name (eg. mojo-ue-core-image-tester") |
500 | |
501 | args = parser.parse_args() |
502 | - |
503 | - return args |
504 | - |
505 | - |
506 | -def get_service_info(unit_name, unit_name_pattern): |
507 | - |
508 | - unit_name_regex = re.compile(unit_name_pattern) |
509 | - match = unit_name_regex.match(unit_name) |
510 | - stage_dirname = None |
511 | - identifier = None |
512 | - |
513 | - if match and len(match.groups()) == 2: |
514 | - stage_dirname = match.group(1) |
515 | - identifier = match.group(2) |
516 | - |
517 | - return (stage_dirname, identifier) |
518 | - |
519 | - |
520 | -def nova_list(): |
521 | - args = [ |
522 | - 'nova', |
523 | - 'list', |
524 | - '--minimal', |
525 | - ] |
526 | - |
527 | - output = subprocess.check_output( |
528 | - args, stderr=subprocess.DEVNULL).decode('utf-8').strip() |
529 | - |
530 | - return output |
531 | - |
532 | - |
533 | -def parse_nova_output(output, stage_dirname): |
534 | - units = [] |
535 | - buf = io.StringIO(output) |
536 | - for line in buf.readlines(): |
537 | - # skip everything but the instance entries |
538 | - if stage_dirname not in line: |
539 | - continue |
540 | - data = line.split('|') |
541 | - instance_id = data[1].strip() |
542 | - unit_name = data[2].strip() |
543 | - units.append({'id': instance_id, 'name': unit_name}) |
544 | - |
545 | - return units |
546 | - |
547 | - |
548 | -def check_deployments(service_deployments, max_deployments, base_dir, logger): |
549 | - """ |
550 | - :param service_deployments: dictionary of the form: |
551 | - {'stage_dirname': ['identifier_1', 'identifier_2', ...]} |
552 | - """ |
553 | - for key in service_deployments: |
554 | - if len(service_deployments[key]) > max_deployments: |
555 | - data = [] |
556 | - for identifier in service_deployments[key]: |
557 | - path = os.path.join(base_dir, identifier) |
558 | - # Manually deployed environments (i.e. no identifiers |
559 | - # recorded on disk) are ignored and left alone. |
560 | - if not os.path.exists(path): |
561 | - continue |
562 | - data.append((identifier, os.stat(path)[ST_MTIME], path)) |
563 | - data = sorted(data, key=operator.itemgetter(1), reverse=True) |
564 | - for identifier, _, _ in data[max_deployments:]: |
565 | - extra = { |
566 | - "service": "ci/cd", |
567 | - "deployment_status": "DESTROYED", |
568 | - "deployment_name": key, |
569 | - "deployment_identifier": identifier, |
570 | - } |
571 | - logger.info( |
572 | - "Destroying deployment %s", identifier, extra=extra) |
573 | - |
574 | - |
575 | -def get_service_deployments(stage_dirname, output, unit_name_pattern): |
576 | - service_deployments = {stage_dirname: []} |
577 | - |
578 | - units = parse_nova_output(output, stage_dirname) |
579 | - |
580 | - for unit in units: |
581 | - service_name, identifier = get_service_info(unit['name'], |
582 | - unit_name_pattern) |
583 | - # skip unparsable unit names |
584 | - if service_name is None or identifier is None: |
585 | - continue |
586 | - |
587 | - service_deployments[stage_dirname].append(identifier) |
588 | - |
589 | - return service_deployments |
590 | - |
591 | - |
592 | -def main(): |
593 | - args = parse_args() |
594 | - |
595 | - base_dir = args.base_dir |
596 | - max_deployments = args.max_deployments |
597 | - stage_dirname = args.stage_dirname |
598 | - |
599 | - unit_name_pattern = 'juju-({})-(.*)-machine-\d+'.format(stage_dirname) |
600 | - |
601 | - logging.basicConfig( |
602 | - format='%(asctime)s %(name)s %(levelname)s: %(message)s') |
603 | - logger = logging.getLogger() |
604 | - logger.setLevel(logging.INFO) |
605 | - |
606 | - # Optional remote logging via logstash. |
607 | - logstash_host = os.environ.get('CI_LOGSTASH_HOST') |
608 | - if logstash_host is not None: |
609 | - try: |
610 | - import logstash |
611 | - except ImportError: |
612 | - print('Follow the README instructions for installing ' |
613 | - 'python-logstash') |
614 | - return 1 |
615 | - else: |
616 | - logger.addHandler( |
617 | - logstash.LogstashHandler(logstash_host, 5959, 1)) |
618 | - |
619 | - output = nova_list() |
620 | - service_deployments = get_service_deployments( |
621 | - stage_dirname, output, unit_name_pattern) |
622 | - check_deployments( |
623 | - service_deployments, max_deployments, base_dir, logger) |
624 | + logger = setup_logger() |
625 | + |
626 | + deployments = get_services_map(args.base_dir).get(args.stage_dirname) |
627 | + if deployments is None: |
628 | + print('No deployments found!') |
629 | + return |
630 | + |
631 | + sorted_deployments = sorted( |
632 | + deployments.items(), key=lambda d: d[1]['epoch'], reverse=True) |
633 | + for identifier, _ in sorted_deployments[args.max_deployments:]: |
634 | + extra = { |
635 | + "service": "ci/cd", |
636 | + "deployment_status": "DESTROYED", |
637 | + "deployment_name": args.stage_dirname, |
638 | + "deployment_identifier": identifier, |
639 | + } |
640 | + logger.info( |
641 | + "Destroying deployment %s", identifier, extra=extra) |
642 | |
643 | return 0 |
644 |
Nice, this will be really useful