Merge lp:~pwlars/core-selftest-agent/agent-config into lp:core-selftest-agent

Proposed by Paul Larson
Status: Merged
Approved by: Paul Larson
Approved revision: 9
Merged at revision: 5
Proposed branch: lp:~pwlars/core-selftest-agent/agent-config
Merge into: lp:core-selftest-agent
Diff against target: 126 lines (+95/-1)
4 files modified
called-by-tarmac.py (+6/-0)
core_selftest_agent/__init__.py (+45/-1)
core_selftest_agent/constants.py (+42/-0)
requirements.txt (+2/-0)
To merge this branch: bzr merge lp:~pwlars/core-selftest-agent/agent-config
Reviewer Review Type Date Requested Status
Para Siva (community) Approve
Paul Larson Needs Resubmitting
Francis Ginther Approve
Review via email: mp+259799@code.launchpad.net

Commit message

Add config bits

Description of the change

Add config bits

To post a comment you must log in.
Revision history for this message
Francis Ginther (fginther) wrote :

A have a couple of inline comments. I think we're now safe to remove the modified hostname methods.

review: Needs Fixing
Revision history for this message
Paul Larson (pwlars) wrote :

Actually, I don't know that I'll even need a retry queue so I took it out for now. It may be necessary to retry internally for a failed manifest download or something, but I don't think this service has a concept of an infrastructure failure outside of its control that would require requeueing. I also replaced the hostname bit with socket.gethostname()

review: Needs Resubmitting
Revision history for this message
Francis Ginther (fginther) wrote :

I'm happy with this. As mentioned over IRC, there really is no reason for the agent to retry messages. The input is a fan-out exchange so every consumer will already have a copy, no need to throw it back to see if another consumer can make use of it. I also doubt we'll need deadletter, but am happy to wait and see on this.

I'm curious as to why you kept ".replace('-machine', '')" on the hostname. I don't think this is hurting anything. But I'm not going to block on this either.

review: Approve
Revision history for this message
Ubuntu CI Bot (uci-bot) wrote :
Download full text (3.4 KiB)

The attempt to merge lp:~pwlars/core-selftest-agent/agent-config into lp:core-selftest-agent failed. Below is the output from the failed tests.

Using base prefix '/usr'
New python executable in /tmp/venv-core-selftest-agentwtd8oky_/bin/python3
Also creating executable in /tmp/venv-core-selftest-agentwtd8oky_/bin/python
Installing setuptools, pip...done.
Running virtualenv with interpreter /usr/bin/python3
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking python-logstash==0.4.2 (from -r requirements.txt (line 1))
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/commands/install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/req.py", line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/index.py", line 252, in find_requirement
    for page in self._get_pages(locations, req):
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/index.py", line 387, in _get_pages
    page = self._get_page(location, req)
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/index.py", line 568, in _get_page
    session=self.session,
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/index.py", line 670, in get_page
    resp = session.get(url, headers={"Accept": "text/html"})
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/_vendor/requests/sessions.py", line 395, in get
    return self.request('GET', url, **kwargs)
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/download.py", line 237, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/_vendor/requests/sessions.py", line 349, in request
    prep = self.prepare_request(req)
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/_vendor/requests/sessions.py", line 287, in prepare_request
    hooks=merge_hooks(request.hooks, self.hooks),
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/_vendor/requests/models.py", line 287, in prepare
    self.prepare_url(url, params)
  File "/tmp/venv-core-selftest-agentwtd8oky_/lib/python3.4/site-packages/pip/_vendor/requests/models.py", line 338, in prepare_url
    "Perhaps you meant http://{0}?".format(url))
pip._vendor.requests.exceptions.MissingSchema: Invalid URL '/tmp/tmpgxqm1lttpip-cache-/cache': No schema supplied. Perhaps you meant http:///tmp/tmpgxqm1lttpip-cache-/cache?

Storing debug log for failure in /home/tarmac/.pip/pip.log
INFO: Executing: virtualenv -p python3 /tmp/venv-core-selftest-agentwtd8oky_
INFO: Executing: /tmp/venv-core-selftest-agentwtd8...

Read more...

Revision history for this message
Ubuntu CI Bot (uci-bot) wrote :

There are additional revisions which have not been approved in review. Please seek review and approval of these new revisions.

Revision history for this message
Paul Larson (pwlars) :
review: Needs Resubmitting
Revision history for this message
Para Siva (psivaa) wrote :

I see core-service.conf and SOLUTION_NAME=core-image-testing as potentially confusing but could notice some other services in the same solution are using this.

review: Approve
Revision history for this message
Paul Larson (pwlars) wrote :

Yeah, that's just a placeholder for now. A lot of that is going to need to change.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'called-by-tarmac.py'
--- called-by-tarmac.py 2015-05-20 22:06:09 +0000
+++ called-by-tarmac.py 2015-05-22 16:26:21 +0000
@@ -59,6 +59,12 @@
59 pip_cache = os.path.join(pip_dir, 'cache')59 pip_cache = os.path.join(pip_dir, 'cache')
60 all_cmds = (60 all_cmds = (
61 [61 [
62 'bzr',
63 'branch',
64 PIP_CACHE_BRANCH,
65 pip_cache,
66 ],
67 [
62 'virtualenv',68 'virtualenv',
63 '-p',69 '-p',
64 'python3',70 'python3',
6571
=== modified file 'core_selftest_agent/__init__.py'
--- core_selftest_agent/__init__.py 2015-05-20 17:30:07 +0000
+++ core_selftest_agent/__init__.py 2015-05-22 16:26:21 +0000
@@ -15,6 +15,50 @@
15# along with this program. If not, see <http://www.gnu.org/licenses/>.15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16#16#
1717
18import argparse
19import configparser
20import logging
21import os
22
23from uservice_utils.logging import (
24 configure_service_logging,
25 ExtraLogger,
26)
27
28from core_selftest_agent import constants
29
30
31def read_config():
32 parser = argparse.ArgumentParser(description='Core Selftest Agent.')
33 parser.add_argument(
34 '-c',
35 '--conf',
36 default='core-service.conf',
37 help='Configuration file path'
38 )
39 args = parser.parse_args()
40
41 # Load configuration options.
42 config = configparser.ConfigParser()
43 config.read(args.conf)
44 return config
45
46
47def get_logger(name, extra={}):
48 if logging.getLoggerClass() != ExtraLogger:
49 logging.setLoggerClass(ExtraLogger)
50 logger = logging.getLogger(name)
51 e = constants.LOGGING_EXTRA.copy()
52 e.update(extra)
53 logger.set_extra_args(e)
54 return logger
55
1856
19def main():57def main():
20 pass58 config = read_config()
59 log_path = os.path.abspath(
60 os.path.join(__file__, '../../logs/core-selftest-agent.log'))
61 configure_service_logging(
62 log_path,
63 config['logstash'] if 'logstash' in config else None
64 )
2165
=== added file 'core_selftest_agent/constants.py'
--- core_selftest_agent/constants.py 1970-01-01 00:00:00 +0000
+++ core_selftest_agent/constants.py 2015-05-22 16:26:21 +0000
@@ -0,0 +1,42 @@
1# core-selftest-agent
2# Copyright (C) 2015 Canonical
3#
4# This program is free software: you can redistribute it and/or modify
5# it under the terms of the GNU General Public License as published by
6# the Free Software Foundation, either version 3 of the License, or
7# (at your option) any later version.
8#
9# This program is distributed in the hope that it will be useful,
10# but WITHOUT ANY WARRANTY; without even the implied warranty of
11# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12# GNU General Public License for more details.
13#
14# You should have received a copy of the GNU General Public License
15# along with this program. If not, see <http://www.gnu.org/licenses/>.
16#
17
18"""Constants for this service."""
19import socket
20
21# Queue names are constant, rather than being defined in the config.
22
23# The queue we listen to for new payloads to check:
24INPUT_QUEUE = "pm.candidates"
25
26# The queue we send events on to request image build and test
27OUTPUT_QUEUE = "core.package.v1"
28
29# The queue we put fatally error'd payloads into:
30DEAD_LETTER_QUEUE = "core.deadletters.v1"
31
32SOLUTION_NAME = "core-image-testing"
33
34SERVICE_NAME = "core-selftest-agent"
35
36HOSTNAME = socket.gethostname().replace('-machine', '')
37
38LOGGING_EXTRA = {
39 'solution': SOLUTION_NAME,
40 'service': SERVICE_NAME,
41 'hostname': HOSTNAME,
42}
043
=== modified file 'requirements.txt'
--- requirements.txt 2015-05-20 17:10:57 +0000
+++ requirements.txt 2015-05-22 16:26:21 +0000
@@ -0,0 +1,2 @@
1python-logstash==0.4.2
2uservice-utils==1.0.2.1

Subscribers

People subscribed via source and target branches