Offspring

Merge lp:~salgado/offspring/path-independence into lp:offspring

  1. path-independence
  2. Merge into trunk
Proposed by Guilherme Salgado
Status: Superseded
Proposed branch: lp:~salgado/offspring/path-independence
Merge into: lp:offspring
Prerequisite: lp:~salgado/offspring/all-python-config-plus
Diff against target: 287 lines (+57/-23)
12 files modified
.bzrignore (+2/-0)
Makefile (+1/-1)
config/offspring.cfg (+11/-3)
lib/offspring/__init__.py (+5/-0)
lib/offspring/build/bin/offspring-build (+7/-1)
lib/offspring/build/scripts/archive-builds (+7/-3)
lib/offspring/config.py (+8/-4)
lib/offspring/slave/slave.py (+6/-2)
lib/offspring/slave/taskmonitor.py (+8/-2)
lib/offspring/web/queuemanager/models.py (+2/-1)
lib/offspring/web/settings.py (+0/-3)
lib/offspring/web/settings_production.py (+0/-3)
To merge this branch: bzr merge lp:~salgado/offspring/path-independence
Reviewer Review Type Date Requested Status
Offspring Hackers Pending
Review via email: mp+76277@code.launchpad.net

This proposal has been superseded by a proposal from 2011-09-21.

Description of the change

This is a second attempt at trying to get these changes merged; they simplify things a lot for developers.

Here's what Michael said in the original merge proposal:

This finally is an I think ready branch to allow offspring to be easily run from places other than /srv.

The code to find offspring-builder-config is a touch horrible, but it won't be needed in the slave.

To post a comment you must log in.
lp:~salgado/offspring/path-independence updated
29. By Guilherme Salgado

merge lp:~mwhudson/offspring/path-independence

30. By Guilherme Salgado

Fix one final place which was using offspring-builder-script instead of offspring-builder-config

31. By Guilherme Salgado

merge all-python-config

32. By Guilherme Salgado

merge all-python-config

33. By Guilherme Salgado

Update one remaining place which was still running offspring-builder-config

34. By Guilherme Salgado

merge trunk

35. By Guilherme Salgado

Some minor tweaks

36. By Guilherme Salgado

revert a few unnecessary changes

37. By Guilherme Salgado

remove a .bzrignore change that's no longer needed

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file '.bzrignore'
2--- .bzrignore 2010-12-03 04:46:30 +0000
3+++ .bzrignore 2011-09-21 20:44:26 +0000
4@@ -1,9 +1,11 @@
5 ./.download-cache/
6 ./.virtualenv/
7 ./bin/
8+./cron/
9 ./logs/
10 ./lib/offspring.egg-info/
11 ./lib/offspring/web/media/js/isomorphic/
12 ./config/launchpad.oauth
13 ./builds/
14+./config/offspring.cfg.*
15 ./scripts/
16
17=== modified file 'Makefile'
18--- Makefile 2011-09-01 14:42:20 +0000
19+++ Makefile 2011-09-21 20:44:26 +0000
20@@ -18,7 +18,7 @@
21 [ ! -d builds ] || chmod -r builds
22 ./.virtualenv/bin/python setup.py develop --script-dir=./bin/
23 [ ! -d builds ] || chmod +r builds
24- [ -f bin/offspring-build ] || ln -s `pwd`/lib/offspring/build/bin/* bin/
25+ [ -f bin/offspring-build ] || ln -s `pwd`/lib/offspring/build/bin/offspring-build bin/
26 mkdir -p logs
27 mkdir -p scripts && ln -s --force `pwd`/lib/offspring/build/scripts/* scripts/
28
29
30=== modified file 'config/offspring.cfg'
31--- config/offspring.cfg 2011-09-21 20:44:26 +0000
32+++ config/offspring.cfg 2011-09-21 20:44:26 +0000
33@@ -11,7 +11,15 @@
34 #===============================================================================
35 # Offspring Global Defaults
36 #
37+# Some options in this section are given defaults by code so that they
38+# can be referenced in subsitutions:
39+#
40 # base_dir: Base directory of offspring install
41+# hostname: The hostname of the running machine
42+# username: The name of the user running offspring
43+#
44+# The other options below are set normally:
45+#
46 # bin_dir: Directory to offspring binaries
47 # config_dir: Directory to offspring configuration files
48 # admin_email: E-mail admin notifications should be sent to by default
49@@ -25,7 +33,6 @@
50 #===============================================================================
51
52 [DEFAULT]
53-base_dir: /srv
54 bin_dir: %(base_dir)s/bin
55 config_dir: %(base_dir)s/config
56 log_dir: %(base_dir)s/logs
57@@ -178,6 +185,7 @@
58 poll_frequency: 20
59 logfile: %(log_dir)s/offspring-master.log
60 launchpad_oauth: %(config_dir)s/launchpad.oauth
61+result_directory: %(base_dir)s/builds
62
63 #===============================================================================
64 # Offspring publisher
65@@ -197,8 +205,8 @@
66
67 [publisher]
68 db: postgres://offspring:temp1234@localhost/offspring
69-development_pool: /srv/builds/
70-production_pool: /srv/partners/
71+development_pool: %(base_dir)s/builds/
72+production_pool: %(base_dir)s/partners/
73 signer_directory:
74 signer_files:
75 signer_results:
76
77=== modified file 'lib/offspring/__init__.py'
78--- lib/offspring/__init__.py 2010-11-29 08:27:24 +0000
79+++ lib/offspring/__init__.py 2011-09-21 20:44:26 +0000
80@@ -1,4 +1,9 @@
81 # Copyright 2010 Canonical Ltd. This software is licensed under the
82 # GNU Affero General Public License version 3 (see the file LICENSE).
83
84+import os
85+
86 __version__ = '0.0.1'
87+
88+offspring_root = os.path.abspath(
89+ os.path.join(os.path.dirname(__file__), '..', '..'))
90
91=== modified file 'lib/offspring/build/bin/offspring-build'
92--- lib/offspring/build/bin/offspring-build 2011-09-21 20:44:26 +0000
93+++ lib/offspring/build/bin/offspring-build 2011-09-21 20:44:26 +0000
94@@ -7,7 +7,13 @@
95
96 _Initialize() {
97 if [ -z "${OFFSPRING_CONFIGSCRIPT}" ]; then
98- OFFSPRING_CONFIGSCRIPT="/srv/bin/offspring-builder-config"
99+ SCRIPT_PATH="$(readlink -e "${0}" || which "${0}")"
100+ OFFSPRING_ROOT=$SCRIPT_PATH
101+ for i in 1 2 3 4 5;
102+ do
103+ OFFSPRING_ROOT="$(dirname "${OFFSPRING_ROOT}")"
104+ done
105+ OFFSPRING_CONFIGSCRIPT="${OFFSPRING_ROOT}/bin/offspring-builder-config"
106 fi
107
108 if [ \! -x "${OFFSPRING_CONFIGSCRIPT}" ]; then
109
110=== modified file 'lib/offspring/build/scripts/archive-builds'
111--- lib/offspring/build/scripts/archive-builds 2011-09-21 20:44:26 +0000
112+++ lib/offspring/build/scripts/archive-builds 2011-09-21 20:44:26 +0000
113@@ -9,7 +9,13 @@
114 #
115
116 if [ -z "${OFFSPRING_CONFIGSCRIPT}" ]; then
117- OFFSPRING_CONFIGSCRIPT="/srv/bin/offspring-builder-config"
118+ SCRIPT_PATH="$(readlink -e "${0}" || which "${0}")"
119+ OFFSPRING_ROOT=$SCRIPT_PATH
120+ for i in 1 2 3 4 5;
121+ do
122+ OFFSPRING_ROOT="$(dirname "${OFFSPRING_ROOT}")"
123+ done
124+ OFFSPRING_CONFIGSCRIPT="${OFFSPRING_ROOT}/bin/offspring-builder-config"
125 fi
126
127 if [ \! -x "${OFFSPRING_CONFIGSCRIPT}" ]; then
128@@ -24,8 +30,6 @@
129 exit 1
130 fi
131
132-source "$CONFIGSCRIPT"
133-
134 # calculateBuildAge DATE
135 calculateBuildAge() {
136 date2jd() {
137
138=== modified file 'lib/offspring/config.py'
139--- lib/offspring/config.py 2011-09-21 20:44:26 +0000
140+++ lib/offspring/config.py 2011-09-21 20:44:26 +0000
141@@ -8,16 +8,23 @@
142 import os
143 import sys
144
145+from offspring import offspring_root
146+
147 config = None
148
149 def _read_config():
150 global config
151 config = SafeConfigParser()
152+ hostname = os.popen('hostname').read().strip()
153+ config.set('DEFAULT', 'base_dir', offspring_root)
154+ config.set('DEFAULT', 'hostname', hostname)
155+ config.set('DEFAULT', 'username', os.environ['USERNAME'])
156
157 if os.environ.has_key("OFFSPRING_CONFIGFILE"):
158 CONFIGFILE_PATH = os.environ["OFFSPRING_CONFIGFILE"]
159 else:
160- CONFIGFILE_PATH = "/srv/config/offspring.cfg"
161+ CONFIGFILE_PATH = os.path.join(
162+ offspring_root, 'config', 'offspring.cfg')
163 configFile = None
164 try:
165 configFile = open(CONFIGFILE_PATH)
166@@ -27,9 +34,6 @@
167 % (CONFIGFILE_PATH, e) )
168 sys.exit(1)
169 else:
170- hostname = os.popen('hostname').read().strip()
171- config.set('DEFAULT', 'hostname', hostname)
172- config.set('DEFAULT', 'username', os.environ['USERNAME'])
173 config.read([CONFIGFILE_PATH + '.' + hostname])
174 finally:
175 if configFile is not None:
176
177=== modified file 'lib/offspring/slave/slave.py'
178--- lib/offspring/slave/slave.py 2011-05-30 22:17:10 +0000
179+++ lib/offspring/slave/slave.py 2011-09-21 20:44:26 +0000
180@@ -8,11 +8,12 @@
181 )
182
183 import logging
184+import os
185 import platform
186 import time
187 import xmlrpclib
188
189-from offspring import config
190+from offspring import config, offspring_root
191 from offspring.daemon import (
192 LexbuilderDaemon,
193 LexbuilderXMLRPCServer
194@@ -160,7 +161,10 @@
195
196 def __init__(self, supervisor, projectName, configBranchURL):
197 MonitoredProcess.__init__(
198- self, supervisor, [projectName, configBranchURL])
199+ self, supervisor, [projectName, configBranchURL],
200+ environment_updates={
201+ 'OFFSPRING_CONFIGSCRIPT': os.path.join(
202+ offspring_root, 'bin', 'offspring-builder-config')})
203 self.projectName = projectName
204
205 def setup(self):
206
207=== modified file 'lib/offspring/slave/taskmonitor.py'
208--- lib/offspring/slave/taskmonitor.py 2011-09-08 19:05:50 +0000
209+++ lib/offspring/slave/taskmonitor.py 2011-09-21 20:44:26 +0000
210@@ -2,6 +2,7 @@
211 # GNU Affero General Public License version 3 (see the file LICENSE).
212
213 import datetime
214+import os
215 import threading
216 import subprocess
217
218@@ -54,19 +55,24 @@
219
220 class MonitoredProcess(MonitoredThread):
221 def __init__(self, master=None, arguments=[], shell=False,
222- stdout=subprocess.PIPE, stderr=subprocess.PIPE):
223+ stdout=subprocess.PIPE, stderr=subprocess.PIPE,
224+ environment_updates={}):
225 self.process = None
226 self.shell = shell
227 self.stdout = stdout
228 self.stderr = stderr
229+ self.environment_updates = environment_updates
230 MonitoredThread.__init__(self, master, arguments)
231
232 def execute(self):
233+ child_environ = os.environ.copy()
234+ child_environ.update(self.environment_updates)
235 self.process = subprocess.Popen(
236 [self.getCommandPath()] + self.arguments,
237 shell=self.shell,
238 stdout=self.stdout,
239- stderr=self.stderr)
240+ stderr=self.stderr,
241+ env=child_environ)
242 self.stdoutdata, self.stderrdata = self.process.communicate()
243 return self.process.returncode
244
245
246=== modified file 'lib/offspring/web/queuemanager/models.py'
247--- lib/offspring/web/queuemanager/models.py 2011-02-22 09:46:13 +0000
248+++ lib/offspring/web/queuemanager/models.py 2011-09-21 20:44:26 +0000
249@@ -19,6 +19,7 @@
250 models
251 )
252
253+from offspring import config
254 from offspring.enums import ProjectBuildStates
255
256
257@@ -327,7 +328,7 @@
258 def result_directory(self):
259 if self.name:
260 try:
261- base_dir = getattr(settings, 'BUILDRESULTS_DIRECTORY', '/srv/builds/')
262+ base_dir = config.master('result_directory')
263 buildDate, buildId = self.name.rsplit('-', 1)
264 except:
265 return None
266
267=== modified file 'lib/offspring/web/settings.py'
268--- lib/offspring/web/settings.py 2011-08-11 00:16:09 +0000
269+++ lib/offspring/web/settings.py 2011-09-21 20:44:26 +0000
270@@ -130,6 +130,3 @@
271 CELERY_RESULT_BACKEND = 'djcelery.backends.database.DatabaseBackend'
272 CELERYBEAT_SCHEDULER = 'djcelery.schedulers.DatabaseScheduler'
273 djcelery.setup_loader()
274-
275-# Path to build results
276-BUILDRESULTS_DIRECTORY = '/srv/builds/'
277
278=== modified file 'lib/offspring/web/settings_production.py'
279--- lib/offspring/web/settings_production.py 2010-11-29 08:27:24 +0000
280+++ lib/offspring/web/settings_production.py 2011-09-21 20:44:26 +0000
281@@ -16,6 +16,3 @@
282 EMAIL_HOST = 'localhost'
283
284 MEDIA_URL = 'https://offspring.com/assets/'
285-
286-# Path to build results
287-BUILDRESULTS_DIRECTORY = '/srv/offspring.com/www/builds/'

Subscribers

People subscribed via source and target branches