Merge lp:~pwlars/snappy-proposed-selftest-agent/rename-agent-code into lp:snappy-proposed-selftest-agent
- rename-agent-code
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Paul Larson |
Approved revision: | 10 |
Merged at revision: | 7 |
Proposed branch: | lp:~pwlars/snappy-proposed-selftest-agent/rename-agent-code |
Merge into: | lp:snappy-proposed-selftest-agent |
Diff against target: |
733 lines (+308/-307) 15 files modified
README.rst (+5/-5) called-by-tarmac.py (+2/-2) core-selftest-agent.py (+0/-7) core_selftest_agent/__init__.py (+0/-116) core_selftest_agent/constants.py (+0/-49) core_selftest_agent/tests/__init__.py (+0/-16) core_selftest_agent/tests/test_worker.py (+0/-53) core_selftest_agent/worker.py (+0/-52) setup.py (+7/-7) snappy-proposed-selftest-agent.py (+7/-0) snappy_proposed_selftest_agent/__init__.py (+117/-0) snappy_proposed_selftest_agent/constants.py (+49/-0) snappy_proposed_selftest_agent/tests/__init__.py (+16/-0) snappy_proposed_selftest_agent/tests/test_worker.py (+53/-0) snappy_proposed_selftest_agent/worker.py (+52/-0) |
To merge this branch: | bzr merge lp:~pwlars/snappy-proposed-selftest-agent/rename-agent-code |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Para Siva (community) | Approve | ||
Paul Larson | Needs Resubmitting | ||
Review via email: mp+260295@code.launchpad.net |
Commit message
Rename core-selftest-agent to snappy-
Description of the change
Rename core-selftest-agent to snappy-
- 8. By Paul Larson
-
add back snappy-
proposed- selftest- agent.py and snappy_ proposed_ selftest_ agent/
Paul Larson (pwlars) wrote : | # |
argh, I forgot to do bzr before the mv when renaming the module and the cli. I've just added them back, which makes this a bit long, but it's just a rename and s/core-
Para Siva (psivaa) wrote : | # |
Thanks for adding the worker code, Paul. I've got some inline comments about the queue names etc, since we're renaming. Let me know if they dont make sense.
- 9. By Paul Larson
-
change conf file name
- 10. By Paul Larson
-
Add .v1 to exchanges as discussed
Paul Larson (pwlars) wrote : | # |
I was going to handle the exchange .v1 addition we discussed this morning in a separate mp, but no good reason not to do it here. Also change the .conf file name as is being done elsewhere.
Preview Diff
1 | === modified file 'README.rst' |
2 | --- README.rst 2015-05-20 22:00:00 +0000 |
3 | +++ README.rst 2015-05-27 17:01:56 +0000 |
4 | @@ -1,10 +1,10 @@ |
5 | -Core Selftest Agent |
6 | -################### |
7 | +Snappy Proposed Selftest Agent |
8 | +############################## |
9 | |
10 | A micro-service that watches for britney-announcer events of new |
11 | packages to test, checks if they are in the manifest for core images in |
12 | -the channel we want to test, and triggers core-image-builder to start |
13 | -building an image with the proposed changes. This service is only |
14 | +the channel we want to test, and triggers snappy-proposed-image-builder |
15 | +to start building an image with the proposed changes. This service is only |
16 | responsible for triggering test runs of the snappy selftests. |
17 | |
18 | Get the Source |
19 | @@ -12,7 +12,7 @@ |
20 | |
21 | Branch the code:: |
22 | |
23 | - $ bzr branch lp:core-selftest-agent |
24 | + $ bzr branch lp:snappy-proposed-selftest-agent |
25 | |
26 | Install the Service |
27 | =================== |
28 | |
29 | === modified file 'called-by-tarmac.py' |
30 | --- called-by-tarmac.py 2015-05-22 16:26:11 +0000 |
31 | +++ called-by-tarmac.py 2015-05-27 17:01:56 +0000 |
32 | @@ -1,6 +1,6 @@ |
33 | #!/usr/bin/env python3 |
34 | # |
35 | -# core-selftest-agent |
36 | +# snappy-proposed-selftest-agent |
37 | # Copyright (C) 2015 Canonical |
38 | # |
39 | # This program is free software: you can redistribute it and/or modify |
40 | @@ -33,7 +33,7 @@ |
41 | import sys |
42 | import tempfile |
43 | |
44 | -SERVICE_NAME = 'core-selftest-agent' |
45 | +SERVICE_NAME = 'snappy-proposed-selftest-agent' |
46 | VENV_DIR = 'venv-{}'.format(SERVICE_NAME) |
47 | PIP_DIR = 'pip-cache-'.format(SERVICE_NAME) |
48 | PIP_CACHE_BRANCH = 'lp:~canonical-ci-engineering/{}/' \ |
49 | |
50 | === removed file 'core-selftest-agent.py' |
51 | --- core-selftest-agent.py 2015-05-20 17:20:50 +0000 |
52 | +++ core-selftest-agent.py 1970-01-01 00:00:00 +0000 |
53 | @@ -1,7 +0,0 @@ |
54 | -#!/usr/bin/env python3 |
55 | - |
56 | -from core_selftest_agent import main |
57 | - |
58 | - |
59 | -if __name__ == '__main__': |
60 | - main() |
61 | |
62 | === removed directory 'core_selftest_agent' |
63 | === removed file 'core_selftest_agent/__init__.py' |
64 | --- core_selftest_agent/__init__.py 2015-05-22 19:49:10 +0000 |
65 | +++ core_selftest_agent/__init__.py 1970-01-01 00:00:00 +0000 |
66 | @@ -1,116 +0,0 @@ |
67 | -# core-selftest-agent |
68 | -# Copyright (C) 2015 Canonical |
69 | -# |
70 | -# This program is free software: you can redistribute it and/or modify |
71 | -# it under the terms of the GNU General Public License as published by |
72 | -# the Free Software Foundation, either version 3 of the License, or |
73 | -# (at your option) any later version. |
74 | -# |
75 | -# This program is distributed in the hope that it will be useful, |
76 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
77 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
78 | -# GNU General Public License for more details. |
79 | -# |
80 | -# You should have received a copy of the GNU General Public License |
81 | -# along with this program. If not, see <http://www.gnu.org/licenses/>. |
82 | -# |
83 | - |
84 | -import argparse |
85 | -import configparser |
86 | -import kombu |
87 | -import logging |
88 | -import os |
89 | - |
90 | -from uservice_utils.logging import ( |
91 | - configure_service_logging, |
92 | - ExtraLogger, |
93 | -) |
94 | - |
95 | -from core_selftest_agent import constants |
96 | -from core_selftest_agent.worker import CoreSelftestAgentWorker |
97 | - |
98 | - |
99 | -class CoreImageResultPublisher(object): |
100 | - """A callable that can notify a results exchange that the test is running |
101 | - |
102 | - This will be a fanout exchange so multiple consumers can pick up these |
103 | - messages and decide whether to take action or ignore them. |
104 | - """ |
105 | - |
106 | - def __init__(self, connection): |
107 | - self.connection = connection |
108 | - exchange = kombu.Exchange(constants.RESULTS_EXCHANGE, |
109 | - type='fanout', durable=False) |
110 | - self.queue = kombu.Queue(constants.RESULTS_QUEUE, exchange) |
111 | - |
112 | - def __call__(self, payload): |
113 | - """Take 'payload' and enqueue it on the rabbit exchange.""" |
114 | - queue = self.connection.SimpleQueue(self.queue) |
115 | - queue.put(payload) |
116 | - queue.close() |
117 | - |
118 | - |
119 | -class CoreImageBuilder(object): |
120 | - """A callable that knows how to queue message for the image builder |
121 | - |
122 | - For now we're using a simple queue, but this can easily be extended to use |
123 | - a full-blown topic exchange in the future. |
124 | - """ |
125 | - |
126 | - def __init__(self, connection): |
127 | - self.connection = connection |
128 | - |
129 | - def __call__(self, payload): |
130 | - """Take 'payload' and enqueue it on the rabbit queue.""" |
131 | - queue = self.connection.SimpleQueue(constants.IMAGE_BUILD_QUEUE) |
132 | - queue.put(payload) |
133 | - queue.close() |
134 | - |
135 | - |
136 | -def read_config(): |
137 | - parser = argparse.ArgumentParser(description='Core Selftest Agent.') |
138 | - parser.add_argument( |
139 | - '-c', |
140 | - '--conf', |
141 | - default='core-service.conf', |
142 | - help='Configuration file path' |
143 | - ) |
144 | - args = parser.parse_args() |
145 | - |
146 | - # Load configuration options. |
147 | - config = configparser.ConfigParser() |
148 | - config.read(args.conf) |
149 | - return config |
150 | - |
151 | - |
152 | -def get_logger(name, extra={}): |
153 | - if logging.getLoggerClass() != ExtraLogger: |
154 | - logging.setLoggerClass(ExtraLogger) |
155 | - logger = logging.getLogger(name) |
156 | - e = constants.LOGGING_EXTRA.copy() |
157 | - e.update(extra) |
158 | - logger.set_extra_args(e) |
159 | - return logger |
160 | - |
161 | - |
162 | -def main(): |
163 | - config = read_config() |
164 | - log_path = os.path.abspath( |
165 | - os.path.join(__file__, '../../logs/core-selftest-agent.log')) |
166 | - configure_service_logging( |
167 | - log_path, |
168 | - config['logstash'] if 'logstash' in config else None |
169 | - ) |
170 | - |
171 | - ampq_uris = config.get('amqp', 'uris').split() |
172 | - try: |
173 | - with kombu.Connection(ampq_uris) as connection: |
174 | - monitor = CoreSelftestAgentWorker( |
175 | - connection, |
176 | - constants, |
177 | - CoreImageBuilder(connection), |
178 | - CoreImageResultPublisher(connection), |
179 | - ) |
180 | - monitor.run() |
181 | - except KeyboardInterrupt: |
182 | - print("Bye!") |
183 | |
184 | === removed file 'core_selftest_agent/constants.py' |
185 | --- core_selftest_agent/constants.py 2015-05-26 14:57:41 +0000 |
186 | +++ core_selftest_agent/constants.py 1970-01-01 00:00:00 +0000 |
187 | @@ -1,49 +0,0 @@ |
188 | -# core-selftest-agent |
189 | -# Copyright (C) 2015 Canonical |
190 | -# |
191 | -# This program is free software: you can redistribute it and/or modify |
192 | -# it under the terms of the GNU General Public License as published by |
193 | -# the Free Software Foundation, either version 3 of the License, or |
194 | -# (at your option) any later version. |
195 | -# |
196 | -# This program is distributed in the hope that it will be useful, |
197 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
198 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
199 | -# GNU General Public License for more details. |
200 | -# |
201 | -# You should have received a copy of the GNU General Public License |
202 | -# along with this program. If not, see <http://www.gnu.org/licenses/>. |
203 | -# |
204 | - |
205 | -"""Constants for this service.""" |
206 | -import socket |
207 | - |
208 | -# Queue names are constant, rather than being defined in the config. |
209 | - |
210 | -# We listen to a fanout exchange for candidate packages to test |
211 | -INPUT_EXCHANGE = "candidates.exchange" |
212 | - |
213 | -# The queue needs to be unique to selftest agents |
214 | -INPUT_QUEUE = "core.selftest.candidates" |
215 | - |
216 | -# The queue we send events on to request image build and test |
217 | -IMAGE_BUILD_QUEUE = "snappy-proposed-migration.package.v1" |
218 | - |
219 | -# We need to notify the results exchange that the test is in progress |
220 | -RESULTS_EXCHANGE = "results.exchange" |
221 | -RESULTS_QUEUE = "core.selftest.results" |
222 | - |
223 | -# The queue we put fatally error'd payloads into: |
224 | -DEAD_LETTER_QUEUE = "core.deadletters.v1" |
225 | - |
226 | -SOLUTION_NAME = "core-image-testing" |
227 | - |
228 | -SERVICE_NAME = "core-selftest-agent" |
229 | - |
230 | -HOSTNAME = socket.gethostname() |
231 | - |
232 | -LOGGING_EXTRA = { |
233 | - 'solution': SOLUTION_NAME, |
234 | - 'service': SERVICE_NAME, |
235 | - 'hostname': HOSTNAME, |
236 | -} |
237 | |
238 | === removed directory 'core_selftest_agent/tests' |
239 | === removed file 'core_selftest_agent/tests/__init__.py' |
240 | --- core_selftest_agent/tests/__init__.py 2015-05-22 19:49:10 +0000 |
241 | +++ core_selftest_agent/tests/__init__.py 1970-01-01 00:00:00 +0000 |
242 | @@ -1,16 +0,0 @@ |
243 | -# core-selftest-agent |
244 | -# Copyright (C) 2015 Canonical |
245 | -# |
246 | -# This program is free software: you can redistribute it and/or modify |
247 | -# it under the terms of the GNU General Public License as published by |
248 | -# the Free Software Foundation, either version 3 of the License, or |
249 | -# (at your option) any later version. |
250 | -# |
251 | -# This program is distributed in the hope that it will be useful, |
252 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
253 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
254 | -# GNU General Public License for more details. |
255 | -# |
256 | -# You should have received a copy of the GNU General Public License |
257 | -# along with this program. If not, see <http://www.gnu.org/licenses/>. |
258 | -# |
259 | |
260 | === removed file 'core_selftest_agent/tests/test_worker.py' |
261 | --- core_selftest_agent/tests/test_worker.py 2015-05-26 14:57:41 +0000 |
262 | +++ core_selftest_agent/tests/test_worker.py 1970-01-01 00:00:00 +0000 |
263 | @@ -1,53 +0,0 @@ |
264 | -# core-selftest-agent |
265 | -# Copyright (C) 2015 Canonical |
266 | -# |
267 | -# This program is free software: you can redistribute it and/or modify |
268 | -# it under the terms of the GNU General Public License as published by |
269 | -# the Free Software Foundation, either version 3 of the License, or |
270 | -# (at your option) any later version. |
271 | -# |
272 | -# This program is distributed in the hope that it will be useful, |
273 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
274 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
275 | -# GNU General Public License for more details. |
276 | -# |
277 | -# You should have received a copy of the GNU General Public License |
278 | -# along with this program. If not, see <http://www.gnu.org/licenses/>. |
279 | -# |
280 | - |
281 | -import kombu |
282 | -from testtools import TestCase |
283 | - |
284 | -from core_selftest_agent import constants |
285 | -from core_selftest_agent.worker import CoreSelftestAgentWorker |
286 | - |
287 | - |
288 | -class WorkerTests(TestCase): |
289 | - def test_can_read_message(self): |
290 | - conn = kombu.Connection('memory:///') |
291 | - queue_message(conn, {'test': 'value'}) |
292 | - |
293 | - build_worker = LoggingConsumer() |
294 | - result_worker = LoggingConsumer() |
295 | - q = CoreSelftestAgentWorker(conn, constants, |
296 | - build_worker, result_worker) |
297 | - next(q.consume(limit=1, timeout=5.0)) |
298 | - self.assertEqual(build_worker.messages_seen, [dict(test='value')]) |
299 | - self.assertEqual(result_worker.messages_seen, [dict(test='value')]) |
300 | - |
301 | - |
302 | -class LoggingConsumer(object): |
303 | - """A consumer callback object that acks and logs all received payloads.""" |
304 | - def __init__(self): |
305 | - self.messages_seen = [] |
306 | - |
307 | - def __call__(self, message): |
308 | - self.messages_seen.append(message) |
309 | - |
310 | - |
311 | -def queue_message(conn, message): |
312 | - exchange = kombu.Exchange(constants.INPUT_EXCHANGE, |
313 | - type='fanout', durable=False) |
314 | - queue = kombu.Queue(constants.INPUT_QUEUE, exchange) |
315 | - with conn.SimpleQueue(queue) as q: |
316 | - q.put(message) |
317 | |
318 | === removed file 'core_selftest_agent/worker.py' |
319 | --- core_selftest_agent/worker.py 2015-05-22 19:49:10 +0000 |
320 | +++ core_selftest_agent/worker.py 1970-01-01 00:00:00 +0000 |
321 | @@ -1,52 +0,0 @@ |
322 | -# core-selftest-agent |
323 | -# Copyright (C) 2015 Canonical |
324 | -# |
325 | -# This program is free software: you can redistribute it and/or modify |
326 | -# it under the terms of the GNU General Public License as published by |
327 | -# the Free Software Foundation, either version 3 of the License, or |
328 | -# (at your option) any later version. |
329 | -# |
330 | -# This program is distributed in the hope that it will be useful, |
331 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
332 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
333 | -# GNU General Public License for more details. |
334 | -# |
335 | -# You should have received a copy of the GNU General Public License |
336 | -# along with this program. If not, see <http://www.gnu.org/licenses/>. |
337 | -# |
338 | - |
339 | -import kombu |
340 | - |
341 | -from kombu.mixins import ConsumerMixin |
342 | - |
343 | - |
344 | -class CoreSelftestAgentWorker(ConsumerMixin): |
345 | - |
346 | - def __init__(self, connection, config, build_worker, result_worker): |
347 | - self.config = config |
348 | - self.connection = connection |
349 | - self.build_worker = build_worker |
350 | - self.result_worker = result_worker |
351 | - |
352 | - exchange = kombu.Exchange(self.config.INPUT_EXCHANGE, |
353 | - type='fanout', durable=False) |
354 | - self.queue = kombu.Queue(self.config.INPUT_QUEUE, exchange) |
355 | - |
356 | - def get_consumers(self, Consumer, channel): |
357 | - """Return consumers instances for all configured queues.""" |
358 | - queues = [self.queue] |
359 | - return [kombu.Consumer( |
360 | - channel, queues=queues, callbacks=[self.process])] |
361 | - |
362 | - def process(self, body, message): |
363 | - """ Process incoming messages about candidate packages |
364 | - |
365 | - TODO: Check if we care about this package, if so, send a message |
366 | - to the image builder to create the image and start the tests, and |
367 | - a message to the results exchange to mark it as in progress. |
368 | - |
369 | - For now, just blindly post messages for these |
370 | - """ |
371 | - |
372 | - self.build_worker(body) |
373 | - self.result_worker(body) |
374 | |
375 | === modified file 'setup.py' |
376 | --- setup.py 2015-05-22 19:49:10 +0000 |
377 | +++ setup.py 2015-05-27 17:01:56 +0000 |
378 | @@ -1,6 +1,6 @@ |
379 | #!/usr/bin/env python3 |
380 | |
381 | -# core-selftest-agent |
382 | +# snappy-proposed-selftest-agent |
383 | # Copyright (C) 2015 Canonical |
384 | # |
385 | # This program is free software: you can redistribute it and/or modify |
386 | @@ -27,15 +27,15 @@ |
387 | |
388 | |
389 | setup( |
390 | - name='core-selftest-agent', |
391 | + name='snappy-proposed-selftest-agent', |
392 | version=VERSION, |
393 | - description=('A microservice for triggering snappy selftest on changes to ' |
394 | - 'core packages.'), |
395 | + description=('A microservice for triggering snappy selftest on changes ' |
396 | + 'to core packages.'), |
397 | author='Canonical CI Engineering Team', |
398 | author_email='canonical-ci-engineering@lists.launchpad.net', |
399 | - url='https://launchpad.net/core-selftest-agent', |
400 | + url='https://launchpad.net/snappy-proposed-selftest-agent', |
401 | license='GPLv3', |
402 | packages=find_packages(), |
403 | - scripts=['core-selftest-agent.py'], |
404 | - test_suite='core_selftest_agent.tests', |
405 | + scripts=['snappy-proposed-selftest-agent.py'], |
406 | + test_suite='snappy_proposed_selftest_agent.tests', |
407 | ) |
408 | |
409 | === added file 'snappy-proposed-selftest-agent.py' |
410 | --- snappy-proposed-selftest-agent.py 1970-01-01 00:00:00 +0000 |
411 | +++ snappy-proposed-selftest-agent.py 2015-05-27 17:01:56 +0000 |
412 | @@ -0,0 +1,7 @@ |
413 | +#!/usr/bin/env python3 |
414 | + |
415 | +from snappy_proposed_selftest_agent import main |
416 | + |
417 | + |
418 | +if __name__ == '__main__': |
419 | + main() |
420 | |
421 | === added directory 'snappy_proposed_selftest_agent' |
422 | === added file 'snappy_proposed_selftest_agent/__init__.py' |
423 | --- snappy_proposed_selftest_agent/__init__.py 1970-01-01 00:00:00 +0000 |
424 | +++ snappy_proposed_selftest_agent/__init__.py 2015-05-27 17:01:56 +0000 |
425 | @@ -0,0 +1,117 @@ |
426 | +# snappy-proposed-selftest-agent |
427 | +# Copyright (C) 2015 Canonical |
428 | +# |
429 | +# This program is free software: you can redistribute it and/or modify |
430 | +# it under the terms of the GNU General Public License as published by |
431 | +# the Free Software Foundation, either version 3 of the License, or |
432 | +# (at your option) any later version. |
433 | +# |
434 | +# This program is distributed in the hope that it will be useful, |
435 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
436 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
437 | +# GNU General Public License for more details. |
438 | +# |
439 | +# You should have received a copy of the GNU General Public License |
440 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
441 | +# |
442 | + |
443 | +import argparse |
444 | +import configparser |
445 | +import kombu |
446 | +import logging |
447 | +import os |
448 | + |
449 | +from uservice_utils.logging import ( |
450 | + configure_service_logging, |
451 | + ExtraLogger, |
452 | +) |
453 | + |
454 | +from snappy_proposed_selftest_agent import constants |
455 | +from snappy_proposed_selftest_agent.worker import CoreSelftestAgentWorker |
456 | + |
457 | + |
458 | +class CoreImageResultPublisher(object): |
459 | + """A callable that can notify a results exchange that the test is running |
460 | + |
461 | + This will be a fanout exchange so multiple consumers can pick up these |
462 | + messages and decide whether to take action or ignore them. |
463 | + """ |
464 | + |
465 | + def __init__(self, connection): |
466 | + self.connection = connection |
467 | + exchange = kombu.Exchange(constants.RESULTS_EXCHANGE, |
468 | + type='fanout', durable=False) |
469 | + self.queue = kombu.Queue(constants.RESULTS_QUEUE, exchange) |
470 | + |
471 | + def __call__(self, payload): |
472 | + """Take 'payload' and enqueue it on the rabbit exchange.""" |
473 | + queue = self.connection.SimpleQueue(self.queue) |
474 | + queue.put(payload) |
475 | + queue.close() |
476 | + |
477 | + |
478 | +class CoreImageBuilder(object): |
479 | + """A callable that knows how to queue message for the image builder |
480 | + |
481 | + For now we're using a simple queue, but this can easily be extended to use |
482 | + a full-blown topic exchange in the future. |
483 | + """ |
484 | + |
485 | + def __init__(self, connection): |
486 | + self.connection = connection |
487 | + |
488 | + def __call__(self, payload): |
489 | + """Take 'payload' and enqueue it on the rabbit queue.""" |
490 | + queue = self.connection.SimpleQueue(constants.IMAGE_BUILD_QUEUE) |
491 | + queue.put(payload) |
492 | + queue.close() |
493 | + |
494 | + |
495 | +def read_config(): |
496 | + parser = argparse.ArgumentParser(description='Core Selftest Agent.') |
497 | + parser.add_argument( |
498 | + '-c', |
499 | + '--conf', |
500 | + default='snappy-proposed-service.conf', |
501 | + help='Configuration file path' |
502 | + ) |
503 | + args = parser.parse_args() |
504 | + |
505 | + # Load configuration options. |
506 | + config = configparser.ConfigParser() |
507 | + config.read(args.conf) |
508 | + return config |
509 | + |
510 | + |
511 | +def get_logger(name, extra={}): |
512 | + if logging.getLoggerClass() != ExtraLogger: |
513 | + logging.setLoggerClass(ExtraLogger) |
514 | + logger = logging.getLogger(name) |
515 | + e = constants.LOGGING_EXTRA.copy() |
516 | + e.update(extra) |
517 | + logger.set_extra_args(e) |
518 | + return logger |
519 | + |
520 | + |
521 | +def main(): |
522 | + config = read_config() |
523 | + log_path = os.path.abspath( |
524 | + os.path.join(__file__, |
525 | + '../../logs/snappy-proposed-selftest-agent.log')) |
526 | + configure_service_logging( |
527 | + log_path, |
528 | + config['logstash'] if 'logstash' in config else None |
529 | + ) |
530 | + |
531 | + ampq_uris = config.get('amqp', 'uris').split() |
532 | + try: |
533 | + with kombu.Connection(ampq_uris) as connection: |
534 | + monitor = CoreSelftestAgentWorker( |
535 | + connection, |
536 | + constants, |
537 | + CoreImageBuilder(connection), |
538 | + CoreImageResultPublisher(connection), |
539 | + ) |
540 | + monitor.run() |
541 | + except KeyboardInterrupt: |
542 | + print("Bye!") |
543 | |
544 | === added file 'snappy_proposed_selftest_agent/constants.py' |
545 | --- snappy_proposed_selftest_agent/constants.py 1970-01-01 00:00:00 +0000 |
546 | +++ snappy_proposed_selftest_agent/constants.py 2015-05-27 17:01:56 +0000 |
547 | @@ -0,0 +1,49 @@ |
548 | +# snappy-proposed-selftest-agent |
549 | +# Copyright (C) 2015 Canonical |
550 | +# |
551 | +# This program is free software: you can redistribute it and/or modify |
552 | +# it under the terms of the GNU General Public License as published by |
553 | +# the Free Software Foundation, either version 3 of the License, or |
554 | +# (at your option) any later version. |
555 | +# |
556 | +# This program is distributed in the hope that it will be useful, |
557 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
558 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
559 | +# GNU General Public License for more details. |
560 | +# |
561 | +# You should have received a copy of the GNU General Public License |
562 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
563 | +# |
564 | + |
565 | +"""Constants for this service.""" |
566 | +import socket |
567 | + |
568 | +# Queue names are constant, rather than being defined in the config. |
569 | + |
570 | +# We listen to a fanout exchange for candidate packages to test |
571 | +INPUT_EXCHANGE = "candidates.exchange.v1" |
572 | + |
573 | +# The queue needs to be unique to selftest agents |
574 | +INPUT_QUEUE = "core.selftest.candidates" |
575 | + |
576 | +# The queue we send events on to request image build and test |
577 | +IMAGE_BUILD_QUEUE = "snappy-proposed-migration.package.v1" |
578 | + |
579 | +# We need to notify the results exchange that the test is in progress |
580 | +RESULTS_EXCHANGE = "results.exchange.v1" |
581 | +RESULTS_QUEUE = "core.selftest.results" |
582 | + |
583 | +# The queue we put fatally error'd payloads into: |
584 | +DEAD_LETTER_QUEUE = "core.deadletters.v1" |
585 | + |
586 | +SOLUTION_NAME = "snappy-proposed-image-testing" |
587 | + |
588 | +SERVICE_NAME = "snappy-proposed-selftest-agent" |
589 | + |
590 | +HOSTNAME = socket.gethostname() |
591 | + |
592 | +LOGGING_EXTRA = { |
593 | + 'solution': SOLUTION_NAME, |
594 | + 'service': SERVICE_NAME, |
595 | + 'hostname': HOSTNAME, |
596 | +} |
597 | |
598 | === added directory 'snappy_proposed_selftest_agent/tests' |
599 | === added file 'snappy_proposed_selftest_agent/tests/__init__.py' |
600 | --- snappy_proposed_selftest_agent/tests/__init__.py 1970-01-01 00:00:00 +0000 |
601 | +++ snappy_proposed_selftest_agent/tests/__init__.py 2015-05-27 17:01:56 +0000 |
602 | @@ -0,0 +1,16 @@ |
603 | +# snappy-proposed-selftest-agent |
604 | +# Copyright (C) 2015 Canonical |
605 | +# |
606 | +# This program is free software: you can redistribute it and/or modify |
607 | +# it under the terms of the GNU General Public License as published by |
608 | +# the Free Software Foundation, either version 3 of the License, or |
609 | +# (at your option) any later version. |
610 | +# |
611 | +# This program is distributed in the hope that it will be useful, |
612 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
613 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
614 | +# GNU General Public License for more details. |
615 | +# |
616 | +# You should have received a copy of the GNU General Public License |
617 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
618 | +# |
619 | |
620 | === added file 'snappy_proposed_selftest_agent/tests/test_worker.py' |
621 | --- snappy_proposed_selftest_agent/tests/test_worker.py 1970-01-01 00:00:00 +0000 |
622 | +++ snappy_proposed_selftest_agent/tests/test_worker.py 2015-05-27 17:01:56 +0000 |
623 | @@ -0,0 +1,53 @@ |
624 | +# snappy-proposed-selftest-agent |
625 | +# Copyright (C) 2015 Canonical |
626 | +# |
627 | +# This program is free software: you can redistribute it and/or modify |
628 | +# it under the terms of the GNU General Public License as published by |
629 | +# the Free Software Foundation, either version 3 of the License, or |
630 | +# (at your option) any later version. |
631 | +# |
632 | +# This program is distributed in the hope that it will be useful, |
633 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
634 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
635 | +# GNU General Public License for more details. |
636 | +# |
637 | +# You should have received a copy of the GNU General Public License |
638 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
639 | +# |
640 | + |
641 | +import kombu |
642 | +from testtools import TestCase |
643 | + |
644 | +from snappy_proposed_selftest_agent import constants |
645 | +from snappy_proposed_selftest_agent.worker import CoreSelftestAgentWorker |
646 | + |
647 | + |
648 | +class WorkerTests(TestCase): |
649 | + def test_can_read_message(self): |
650 | + conn = kombu.Connection('memory:///') |
651 | + queue_message(conn, {'test': 'value'}) |
652 | + |
653 | + build_worker = LoggingConsumer() |
654 | + result_worker = LoggingConsumer() |
655 | + q = CoreSelftestAgentWorker(conn, constants, |
656 | + build_worker, result_worker) |
657 | + next(q.consume(limit=1, timeout=5.0)) |
658 | + self.assertEqual(build_worker.messages_seen, [dict(test='value')]) |
659 | + self.assertEqual(result_worker.messages_seen, [dict(test='value')]) |
660 | + |
661 | + |
662 | +class LoggingConsumer(object): |
663 | + """A consumer callback object that acks and logs all received payloads.""" |
664 | + def __init__(self): |
665 | + self.messages_seen = [] |
666 | + |
667 | + def __call__(self, message): |
668 | + self.messages_seen.append(message) |
669 | + |
670 | + |
671 | +def queue_message(conn, message): |
672 | + exchange = kombu.Exchange(constants.INPUT_EXCHANGE, |
673 | + type='fanout', durable=False) |
674 | + queue = kombu.Queue(constants.INPUT_QUEUE, exchange) |
675 | + with conn.SimpleQueue(queue) as q: |
676 | + q.put(message) |
677 | |
678 | === added file 'snappy_proposed_selftest_agent/worker.py' |
679 | --- snappy_proposed_selftest_agent/worker.py 1970-01-01 00:00:00 +0000 |
680 | +++ snappy_proposed_selftest_agent/worker.py 2015-05-27 17:01:56 +0000 |
681 | @@ -0,0 +1,52 @@ |
682 | +# snappy-proposed-selftest-agent |
683 | +# Copyright (C) 2015 Canonical |
684 | +# |
685 | +# This program is free software: you can redistribute it and/or modify |
686 | +# it under the terms of the GNU General Public License as published by |
687 | +# the Free Software Foundation, either version 3 of the License, or |
688 | +# (at your option) any later version. |
689 | +# |
690 | +# This program is distributed in the hope that it will be useful, |
691 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
692 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
693 | +# GNU General Public License for more details. |
694 | +# |
695 | +# You should have received a copy of the GNU General Public License |
696 | +# along with this program. If not, see <http://www.gnu.org/licenses/>. |
697 | +# |
698 | + |
699 | +import kombu |
700 | + |
701 | +from kombu.mixins import ConsumerMixin |
702 | + |
703 | + |
704 | +class CoreSelftestAgentWorker(ConsumerMixin): |
705 | + |
706 | + def __init__(self, connection, config, build_worker, result_worker): |
707 | + self.config = config |
708 | + self.connection = connection |
709 | + self.build_worker = build_worker |
710 | + self.result_worker = result_worker |
711 | + |
712 | + exchange = kombu.Exchange(self.config.INPUT_EXCHANGE, |
713 | + type='fanout', durable=False) |
714 | + self.queue = kombu.Queue(self.config.INPUT_QUEUE, exchange) |
715 | + |
716 | + def get_consumers(self, Consumer, channel): |
717 | + """Return consumers instances for all configured queues.""" |
718 | + queues = [self.queue] |
719 | + return [kombu.Consumer( |
720 | + channel, queues=queues, callbacks=[self.process])] |
721 | + |
722 | + def process(self, body, message): |
723 | + """ Process incoming messages about candidate packages |
724 | + |
725 | + TODO: Check if we care about this package, if so, send a message |
726 | + to the image builder to create the image and start the tests, and |
727 | + a message to the results exchange to mark it as in progress. |
728 | + |
729 | + For now, just blindly post messages for these |
730 | + """ |
731 | + |
732 | + self.build_worker(body) |
733 | + self.result_worker(body) |
+1, Thanks for the comment about the worker code as incoming MPs