Merge lp:~milo/linaro-ci-dashboard/android-build-config into lp:linaro-ci-dashboard
- android-build-config
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | 28 |
Proposed branch: | lp:~milo/linaro-ci-dashboard/android-build-config |
Merge into: | lp:linaro-ci-dashboard |
Diff against target: |
898 lines (+394/-129) 22 files modified
config_template/build_scripts/AndroidLoop.sh (+4/-0) config_template/build_scripts/IntegrationLoop.sh (+0/-8) config_template/jenkins-jobs/AndroidLoop.xml (+208/-0) config_template/jenkins-jobs/IntegrationLoop.xml (+0/-21) dashboard/frontend/android_build/forms/android_loop_form.py (+1/-1) dashboard/frontend/android_build/models/android_loop.py (+13/-6) dashboard/frontend/android_build/templates/android_loop_detail.html (+1/-1) dashboard/frontend/android_build/tests/test_android_build_clientresponse.py (+9/-10) dashboard/frontend/android_build/tests/test_android_build_views.py (+1/-1) dashboard/frontend/android_build/urls.py (+8/-4) dashboard/frontend/android_build/views/android_loop_build_view.py (+25/-0) dashboard/frontend/android_build/views/android_loop_detail_view.py (+11/-4) dashboard/frontend/integration_loop/forms/integration_loop_form.py (+1/-1) dashboard/frontend/integration_loop/models/integration_loop.py (+6/-9) dashboard/frontend/integration_loop/templates/integration_loop_detail.html (+1/-1) dashboard/frontend/kernel_build/models/kernel_loop.py (+2/-2) dashboard/frontend/migrations/0002_auto__add_field_loop_is_restricted__add_field_loop_is_official.py (+54/-0) dashboard/frontend/models/loop.py (+24/-3) dashboard/frontend/urls.py (+1/-1) dashboard/jenkinsserver/models/jenkins_server.py (+24/-14) dashboard/jenkinsserver/tests/test_jenkins_server.py (+0/-3) dashboard/lib/template.py (+0/-39) |
To merge this branch: | bzr merge lp:~milo/linaro-ci-dashboard/android-build-config |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Stevan Radaković | code | Approve | |
Paul Sokolovsky | Abstain | ||
Review via email: mp+120969@code.launchpad.net |
Commit message
Description of the change
Here there is preliminary support for android build of type "android".
We had to refactor a little bit the code, in particular:
- Added a new parameter to the schedule_build method in the Loop class, in order to be able to pass parameters to Jenkins. This was needed since using the same config.xml file taken from the build at android-
- Added a specialized AndroidLoopBuil
- Slightly modfied the method to create the job xml inside JenkinsServer. With android build we have right now 3 different templates for the jobs, we cannot rely solely on the class name to identify them. At the moment we added only one template.
There are also some PEP8 fixes.
- 30. By Milo Casagrande
-
Use only one android loop template.
Milo Casagrande (milo) wrote : | # |
Paul Sokolovsky (pfalcon) wrote : | # |
> - Slightly modfied the method to create the job xml inside JenkinsServer. With android build we have right now 3 different templates for the jobs, we cannot rely solely on the class name to identify them. At the moment we added only one template.
Well, that kinda hints that these should be 3 different model classes ;-I. (Unless there's general and universal approach how to deal with such cases for future, but having separate models is KISS, at least on model side (vs handling them)).
664 + if template_name == 'AndroidLoop':
665 + template_name = loop.build_type
666 + loop_params[
In the outlook I was having when introduced templating, this would be dealt with differently: AndroidLoop.json() would be overriden to add "base64_config" with needed value, then JenkinsServer wouldn't need to know about AndroidLoop existence and its peculiarities.
Re: other changes - it's a bit hard to review them, because it seems that diff is being made against older version of trunk, while tip of it already contains templating code, etc. Can you merge trunk into this branch and re-push it to get better diff?
Paul Sokolovsky (pfalcon) wrote : | # |
Well, imho "if template_name == 'AndroidLoop':" really Needs Fixing.
Milo Casagrande (milo) wrote : | # |
>
> Re: other changes - it's a bit hard to review them, because it seems that diff
> is being made against older version of trunk, while tip of it already contains
> templating code, etc. Can you merge trunk into this branch and re-push it to
> get better diff?
It is actually based on trunk, the problem was that we were already working on the template part, adding the directories as needed, but conflicts raised when merging trunk (conflicts on directory, with ".moved" attached to it). I thought that using "bzr resolve" would have clean it out, but that was not the case. It marked as removed and then added back the files.
Stevan Radaković (stevanr) wrote : | # |
> > - Slightly modfied the method to create the job xml inside JenkinsServer.
> With android build we have right now 3 different templates for the jobs, we
> cannot rely solely on the class name to identify them. At the moment we added
> only one template.
>
> Well, that kinda hints that these should be 3 different model classes ;-I.
> (Unless there's general and universal approach how to deal with such cases for
> future, but having separate models is KISS, at least on model side (vs
> handling them)).
>
> 664 + if template_name == 'AndroidLoop':
> 665 + template_name = loop.build_type
> 666 + loop_params[
>
> In the outlook I was having when introduced templating, this would be dealt
> with differently: AndroidLoop.json() would be overriden to add "base64_config"
> with needed value, then JenkinsServer wouldn't need to know about AndroidLoop
> existence and its peculiarities.
I can't agree with this. Moving template logic to the loop is not viable, as we discussed over and over again.
JenkinsServer need to know about the loops, since it can't be the other way around. It's not the Jenkins app that will be resusable by other applications in the future, but the frontend one.
>
> Re: other changes - it's a bit hard to review them, because it seems that diff
> is being made against older version of trunk, while tip of it already contains
> templating code, etc. Can you merge trunk into this branch and re-push it to
> get better diff?
Milo Casagrande (milo) wrote : | # |
I took a look and tried to use the AndroidLoop.json() approach, but I agree with Stevan, and that was our initial intention when we added the "if-clause" in JenkinsServer: storing nothing in the loop WRT the server.
Also, using the json() approach, I will really bond Jenkins specific values (like the name of the node, plus the possible base64 one) into the loop.
For a more general approach, I would also move the base64 calculation into JenkinsServer, and dealing with it only for the android loop (at the moment, since other jobs may use it).
Paul Sokolovsky (pfalcon) wrote : | # |
> JenkinsServer need to know about the loops, since it can't be the other way around.
My approach allows to decouple them even further and to follow data-driven approach: so neither JenkinsServer needs to know about adhoc peculiarities, nor Loops need to be tied to particular build system.
So, it all reduces to the question of where you draw the line regarding what is "including build system specific data into Loop". Let's start with saying that the fact that Jenkins uses XML for job config, and particular layout of XML is Jenkins-specific thing, and we can't include direct Jenkins job config generation as method of Loop.
Now let's think about culprits:
"like the name of the node" - well, that value is not really *name* of *specific Jenkins node*. It's a *tag* of node *type*. So, there's an abstract node, and it may have tags, and job (aka loop), may also have tags, and if tagsets of a node and job are compatible, the job can build on such node. That's fairly generic concept, not really tied to Jenkins, any sufficiently advanced build system would have similar concept. I don't see probs with that being a property of a loop.
base64-encoded entire job config - well, that's more peculiar, but looking "from the top", it's just that, implementation of following requirement: 1) being able to pass entire loop config to the build system; 2) that entire config being serialized in well-known format. So, those requirements and feature is not really Jenkins-specific, it fairly generic feature, which again can be reasonable assumed to be in need, and supported/parsable by any build system.
So, again it all depends on where you draw the line. I draw the line at the place which allows to use data-driven approach, to avoid patching code for each and every discovered peculiarities (with my outlook, it'll better to have {% if %}'s in templates, then in Jenkins server.
I don't say I'm right, just wanted to make sure that perspective is well understood. The strongest counter-argument I may imagine is you saying that all my approach is based on the fact that a job in build system can be created from a text config file which we prepare at once and submit. I would say that it covers our usecase so far (implemented in well-maintainable, declarative way) ;-). And would continue that if you want to make it fully generic (on code) and maintainable, then patching single method with more and more if's as we go won't scale. Then you'd need to add another layer (i.e.) class which algorithmically encodes process of creating a build from (particular type of) Loop, then have a registry (i.e. factory) which would allow to apply it to any type of Loop. Then would as beautiful as the solution I propose, though not declarative and data-driven.
Anyway, I change my vote to "Abstain" in this regard, and the rest looks good as far as I can tell.
Paul Sokolovsky (pfalcon) wrote : | # |
Whoa, that was long, sorry ;-)
Stevan Radaković (stevanr) wrote : | # |
Great work Milo.
354 + parameters = {'parameter': [{'delay': '0'}]}
355 self.server.
356 - super(AndroidLoop, self).schedule_
357 + return super(self.
'create_
535 - def schedule_
536 - # Every time before the build is scheduled, update the
537 - # loop config on the remote server.
538 - self.server.
539 - super(Integrati
540 -
Hm, why was this removed?
Regarding concerns Paul raised, we are going to discuss this on the meeting...
Milo Casagrande (milo) wrote : | # |
On Fri, Aug 24, 2012 at 11:47 AM, Stevan Radaković
<email address hidden> wrote:
>
> 354 + parameters = {'parameter': [{'delay': '0'}]}
> 355 self.server.
> 356 - super(AndroidLoop, self).schedule_
> 357 + return super(self.
>
> 'create_
Will refactor that into a "create_
> 535 - def schedule_
> 536 - # Every time before the build is scheduled, update the
> 537 - # loop config on the remote server.
> 538 - self.server.
> 539 - super(Integrati
> 540 -
>
> Hm, why was this removed?
This is actually not called, as long as you do not create your own
"build_view". We discovered that while trying to pass the "params"
dictionary to the jenkins server: it was always "None", the default
value defined. We had to temporary add logger calls everywhere to sort
it out.
When you click on the "schedule" button, what it calls is what is
found in the "loop/build/" URL, that matches to the "LoopBuildView"
view. This one, in turn, calls "self.object.
"self.object" in there is always of type Loop since the class is
defined against that model, and not an instance of the specific Loop.
If you want that method to be called, you need to define your own
"$LOOP_build_view", inherit from LoopBuildView, and in there just
define "model = $MODEL_CLASS" (plus add your "build" URL). In this way
that method is called, otherwise it can be safely removed since it
will never be called.
--
Milo Casagrande
Infrastructure Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
- 31. By Milo Casagrande
-
Merged from trunk.
- 32. By Milo Casagrande
-
Added new fields, updated migration.
Stevan Radaković (stevanr) wrote : | # |
> On Fri, Aug 24, 2012 at 11:47 AM, Stevan Radaković
> <email address hidden> wrote:
> >
> > 354 + parameters = {'parameter': [{'delay': '0'}]}
> > 355 self.server.
> > 356 - super(AndroidLoop, self).schedule_
> > 357 + return super(self.
> >
> > 'create_
> (jenkinsserver side as well probably)
>
> Will refactor that into a "create_
>
> > 535 - def schedule_
> > 536 - # Every time before the build is scheduled, update the
> > 537 - # loop config on the remote server.
> > 538 - self.server.
> > 539 - super(Integrati
> > 540 -
> >
> > Hm, why was this removed?
>
> This is actually not called, as long as you do not create your own
> "build_view". We discovered that while trying to pass the "params"
> dictionary to the jenkins server: it was always "None", the default
> value defined. We had to temporary add logger calls everywhere to sort
> it out.
>
> When you click on the "schedule" button, what it calls is what is
> found in the "loop/build/" URL, that matches to the "LoopBuildView"
> view. This one, in turn, calls "self.object.
> "self.object" in there is always of type Loop since the class is
> defined against that model, and not an instance of the specific Loop.
>
> If you want that method to be called, you need to define your own
> "$LOOP_build_view", inherit from LoopBuildView, and in there just
> define "model = $MODEL_CLASS" (plus add your "build" URL). In this way
> that method is called, otherwise it can be safely removed since it
> will never be called.
>
Ok this is a nice catch, but it was hardly the way to resolve the problem you noticed, since we need this create_or_update called.
I'll add this to my action list...
Milo Casagrande (milo) wrote : | # |
On Fri, Aug 24, 2012 at 1:16 PM, Stevan Radaković
<email address hidden> wrote:
>
> Ok this is a nice catch, but it was hardly the way to resolve the problem you noticed, since we need this create_or_update called.
> I'll add this to my action list...
You are right, my fault. I actually had to add it into the Loop class,
so that we do not need to call it in each derived class.
--
Milo Casagrande
Infrastructure Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
- 33. By Milo Casagrande
-
Moved call to updated config into Loop, refactored.
- 34. By Milo Casagrande
-
Reverted commit.
Milo Casagrande (milo) wrote : | # |
On Fri, Aug 24, 2012 at 1:45 PM, Milo Casagrande
<email address hidden> wrote:
>
> You are right, my fault. I actually had to add it into the Loop class,
> so that we do not need to call it in each derived class.
I just tried, but I reverted back. It is not only a matter of moving
the call, we need to define the specialized "build_view" also for the
other loops, otherwise it will not work, since what will be passed
will be the "Loop" object and not the actual derived one (like
KernelLoop or IntegrationLoop).
WRT IntegrationLoop, even removing it, but not adding the
"update_or_create" call into Loop, it can schedule a new build, but it
will never update the config. Same thing happens if we leave it there:
it is never called. If we add the call to "update_or_create" inside
Loop, it will fail with error 500 since we are not defining the
specialized "build_view".
I can take a look at it, it is not a problem, but in a separate
branch, so we have a clean diff of what is going on.
--
Milo Casagrande
Infrastructure Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs
Stevan Radaković (stevanr) wrote : | # |
Ok thanks for looking into it. Let's just rename the method to create_
Approve +1
- 35. By Milo Casagrande
-
Fixed problem with android detail view.
- 36. By Milo Casagrande
-
Fixed JenkinsServer to use new parameters.
- 37. By Milo Casagrande
-
Refactored method name.
Preview Diff
1 | === added file 'config_template/build_scripts/AndroidLoop.sh' |
2 | --- config_template/build_scripts/AndroidLoop.sh 1970-01-01 00:00:00 +0000 |
3 | +++ config_template/build_scripts/AndroidLoop.sh 2012-08-24 13:08:33 +0000 |
4 | @@ -0,0 +1,4 @@ |
5 | +rm -rf build-tools |
6 | +bzr get lp:linaro-android-build-tools build-tools |
7 | +build-tools/node/build us-east-1.ec2-git-mirror.linaro.org "$CONFIG" |
8 | +#build-tools/node/build "$HUDSON_URL" "$CONFIG" |
9 | |
10 | === added file 'config_template/build_scripts/IntegrationLoop.sh' |
11 | --- config_template/build_scripts/IntegrationLoop.sh 1970-01-01 00:00:00 +0000 |
12 | +++ config_template/build_scripts/IntegrationLoop.sh 2012-08-24 13:08:33 +0000 |
13 | @@ -0,0 +1,8 @@ |
14 | +rm -rf dest_dir |
15 | +export USER=`whoami`; |
16 | +export HOME=/home/$USER |
17 | +bzr branch {branch} dest_dir |
18 | +cur_dir=$WORKSPACE; cd ./dest_dir |
19 | +{precommand} |
20 | +{command} |
21 | +cd "$cur_dir" |
22 | |
23 | === removed file 'config_template/build_scripts/IntegrationLoop.sh' |
24 | --- config_template/build_scripts/IntegrationLoop.sh 2012-08-21 10:17:49 +0000 |
25 | +++ config_template/build_scripts/IntegrationLoop.sh 1970-01-01 00:00:00 +0000 |
26 | @@ -1,8 +0,0 @@ |
27 | -rm -rf dest_dir |
28 | -export USER=`whoami`; |
29 | -export HOME=/home/$USER |
30 | -bzr branch {branch} dest_dir |
31 | -cur_dir=$WORKSPACE; cd ./dest_dir |
32 | -{precommand} |
33 | -{command} |
34 | -cd "$cur_dir" |
35 | |
36 | === added file 'config_template/jenkins-jobs/AndroidLoop.xml' |
37 | --- config_template/jenkins-jobs/AndroidLoop.xml 1970-01-01 00:00:00 +0000 |
38 | +++ config_template/jenkins-jobs/AndroidLoop.xml 2012-08-24 13:08:33 +0000 |
39 | @@ -0,0 +1,208 @@ |
40 | +<?xml version="1.0" encoding="UTF-8"?> |
41 | +<project> |
42 | + <actions/> |
43 | + <description><!-- Automatically managed header - do not change, do not add anything before this! --> |
44 | +<table width="65%" align="center"> |
45 | +<tr><td> |
46 | +<p style="background: #FFCCFF; padding: 15px"> |
47 | +<span style="color: gray">New!</span><br /> |
48 | +We now provide <tt>linaro_android_build_cmds.sh</tt> script (see downloads) |
49 | +with exact commands to reproduce this build. Use it as a reference of |
50 | +commands you need to execute, or just run to perform a build automatically. |
51 | +There is also <tt>linaro_kernel_build_cmds.sh</tt> to rebuild just a kernel. |
52 | +Alternatively, you can try instructions below. |
53 | +</td></tr> |
54 | +</p> |
55 | +</table> |
56 | + |
57 | +<!-- End of automatically managed header --> |
58 | + |
59 | +</description> |
60 | + <logRotator> |
61 | + <daysToKeep>30</daysToKeep> |
62 | + <numToKeep>10</numToKeep> |
63 | + <artifactDaysToKeep>-1</artifactDaysToKeep> |
64 | + <artifactNumToKeep>-1</artifactNumToKeep> |
65 | + </logRotator> |
66 | + <keepDependencies>true</keepDependencies> |
67 | + <properties> |
68 | + <hudson.model.ParametersDefinitionProperty> |
69 | + <parameterDefinitions> |
70 | + <hudson.model.StringParameterDefinition> |
71 | + <name>CONFIG</name> |
72 | + <description/> |
73 | + <defaultValue>{base64_config}</defaultValue> |
74 | + </hudson.model.StringParameterDefinition> |
75 | + </parameterDefinitions> |
76 | + </hudson.model.ParametersDefinitionProperty> |
77 | + </properties> |
78 | + <scm class="hudson.scm.NullSCM"/> |
79 | + <assignedNode>{assigned_node}</assignedNode> |
80 | + <canRoam>true</canRoam> |
81 | + <disabled>false</disabled> |
82 | + <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> |
83 | + <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> |
84 | + <triggers class="vector"/> |
85 | + <concurrentBuild>true</concurrentBuild> |
86 | + <builders> |
87 | + <hudson.plugins.shell.ShellWithStatus> |
88 | + <command>{build_script}</command> |
89 | + </hudson.plugins.shell.ShellWithStatus> |
90 | + <jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin> |
91 | + <delegate> |
92 | + <consolePrefix>SSH: </consolePrefix> |
93 | + <delegate> |
94 | + <publishers> |
95 | + <jenkins.plugins.publish__over__ssh.BapSshPublisher> |
96 | + <configName>snapshots.linaro.org new</configName> |
97 | + <verbose>false</verbose> |
98 | + <transfers> |
99 | + <jenkins.plugins.publish__over__ssh.BapSshTransfer> |
100 | + <remoteDirectory>android/${{JOB_NAME}}/${{BUILD_NUMBER}}</remoteDirectory> |
101 | + <sourceFiles>build/out/target/*/*/*.img.bz2,build/out/target/*/*/*.tar.bz2,build/out/target/*/*/MD5SUMS,build/out/*.tar.bz2,build/out/*.xml,build/out/*_config,build/out/lava-job-info,build/out/linaro_kernel_build_cmds.sh,build/out/linaro_android_build_cmds.sh,build/out/**/*EULA*</sourceFiles> |
102 | + <excludes/> |
103 | + <removePrefix>build/out</removePrefix> |
104 | + <remoteDirectorySDF>false</remoteDirectorySDF> |
105 | + <flatten>false</flatten> |
106 | + <cleanRemote>false</cleanRemote> |
107 | + <execCommand/> |
108 | + <execTimeout>120000</execTimeout> |
109 | + <usePty>false</usePty> |
110 | + </jenkins.plugins.publish__over__ssh.BapSshTransfer> |
111 | + </transfers> |
112 | + <useWorkspaceInPromotion>false</useWorkspaceInPromotion> |
113 | + <usePromotionTimestamp>false</usePromotionTimestamp> |
114 | + </jenkins.plugins.publish__over__ssh.BapSshPublisher> |
115 | + <jenkins.plugins.publish__over__ssh.BapSshPublisher> |
116 | + <configName>snapshots.linaro.org file-move new</configName> |
117 | + <verbose>false</verbose> |
118 | + <transfers> |
119 | + <jenkins.plugins.publish__over__ssh.BapSshTransfer> |
120 | + <remoteDirectory/> |
121 | + <sourceFiles/> |
122 | + <excludes/> |
123 | + <removePrefix/> |
124 | + <remoteDirectorySDF>false</remoteDirectorySDF> |
125 | + <flatten>false</flatten> |
126 | + <cleanRemote>false</cleanRemote> |
127 | + <execCommand>reshuffle-files -t android -j ${{JOB_NAME}} -n ${{BUILD_NUMBER}} -m</execCommand> |
128 | + <execTimeout>120000</execTimeout> |
129 | + <usePty>false</usePty> |
130 | + </jenkins.plugins.publish__over__ssh.BapSshTransfer> |
131 | + </transfers> |
132 | + <useWorkspaceInPromotion>false</useWorkspaceInPromotion> |
133 | + <usePromotionTimestamp>false</usePromotionTimestamp> |
134 | + </jenkins.plugins.publish__over__ssh.BapSshPublisher> |
135 | + </publishers> |
136 | + <continueOnError>false</continueOnError> |
137 | + <failOnError>false</failOnError> |
138 | + <alwaysPublishFromMaster>true</alwaysPublishFromMaster> |
139 | + <hostConfigurationAccess class="jenkins.plugins.publish_over_ssh.BapSshPublisherPlugin" reference="../.."/> |
140 | + </delegate> |
141 | + </delegate> |
142 | + </jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin> |
143 | + <hudson.tasks.Shell> |
144 | + <command>build-tools/node/lava-submit "$CONFIG" |
145 | +</command> |
146 | + </hudson.tasks.Shell> |
147 | + <jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin> |
148 | + <delegate> |
149 | + <consolePrefix>SSH: </consolePrefix> |
150 | + <delegate> |
151 | + <publishers> |
152 | + <jenkins.plugins.publish__over__ssh.BapSshPublisher> |
153 | + <configName>snapshots.linaro.org new</configName> |
154 | + <verbose>false</verbose> |
155 | + <transfers> |
156 | + <jenkins.plugins.publish__over__ssh.BapSshTransfer> |
157 | + <remoteDirectory>android/${{JOB_NAME}}/${{BUILD_NUMBER}}</remoteDirectory> |
158 | + <sourceFiles>build/out/lava-job-info</sourceFiles> |
159 | + <excludes/> |
160 | + <removePrefix>build/out</removePrefix> |
161 | + <remoteDirectorySDF>false</remoteDirectorySDF> |
162 | + <flatten>false</flatten> |
163 | + <cleanRemote>false</cleanRemote> |
164 | + <execCommand/> |
165 | + <execTimeout>120000</execTimeout> |
166 | + <usePty>false</usePty> |
167 | + </jenkins.plugins.publish__over__ssh.BapSshTransfer> |
168 | + </transfers> |
169 | + <useWorkspaceInPromotion>false</useWorkspaceInPromotion> |
170 | + <usePromotionTimestamp>false</usePromotionTimestamp> |
171 | + </jenkins.plugins.publish__over__ssh.BapSshPublisher> |
172 | + </publishers> |
173 | + <continueOnError>false</continueOnError> |
174 | + <failOnError>false</failOnError> |
175 | + <alwaysPublishFromMaster>true</alwaysPublishFromMaster> |
176 | + <hostConfigurationAccess class="jenkins.plugins.publish_over_ssh.BapSshPublisherPlugin" reference="../.."/> |
177 | + </delegate> |
178 | + </delegate> |
179 | + </jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin> |
180 | + <jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin> |
181 | + <delegate> |
182 | + <consolePrefix>SSH: </consolePrefix> |
183 | + <delegate> |
184 | + <publishers> |
185 | + <jenkins.plugins.publish__over__ssh.BapSshPublisher> |
186 | + <configName>snapshots.linaro.org file-move new</configName> |
187 | + <verbose>false</verbose> |
188 | + <transfers> |
189 | + <jenkins.plugins.publish__over__ssh.BapSshTransfer> |
190 | + <remoteDirectory/> |
191 | + <sourceFiles/> |
192 | + <excludes/> |
193 | + <removePrefix/> |
194 | + <remoteDirectorySDF>false</remoteDirectorySDF> |
195 | + <flatten>false</flatten> |
196 | + <cleanRemote>false</cleanRemote> |
197 | + <execCommand>reshuffle-files -t android -j ${{JOB_NAME}} -n ${{BUILD_NUMBER}} -m</execCommand> |
198 | + <execTimeout>120000</execTimeout> |
199 | + <usePty>false</usePty> |
200 | + </jenkins.plugins.publish__over__ssh.BapSshTransfer> |
201 | + </transfers> |
202 | + <useWorkspaceInPromotion>false</useWorkspaceInPromotion> |
203 | + <usePromotionTimestamp>false</usePromotionTimestamp> |
204 | + </jenkins.plugins.publish__over__ssh.BapSshPublisher> |
205 | + </publishers> |
206 | + <continueOnError>false</continueOnError> |
207 | + <failOnError>false</failOnError> |
208 | + <alwaysPublishFromMaster>true</alwaysPublishFromMaster> |
209 | + <hostConfigurationAccess class="jenkins.plugins.publish_over_ssh.BapSshPublisherPlugin" reference="../.."/> |
210 | + </delegate> |
211 | + </delegate> |
212 | + </jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin> |
213 | + <hudson.tasks.Shell> |
214 | + <command>echo "Build finished" |
215 | +</command> |
216 | + </hudson.tasks.Shell> |
217 | + </builders> |
218 | + <publishers> |
219 | + <hudson.tasks.ArtifactArchiver> |
220 | + <artifacts>build/out/*.xml</artifacts> |
221 | + <latestOnly>false</latestOnly> |
222 | + </hudson.tasks.ArtifactArchiver> |
223 | + <hudson.plugins.logparser.LogParserPublisher> |
224 | + <unstableOnWarning>false</unstableOnWarning> |
225 | + <failBuildOnError>false</failBuildOnError> |
226 | + <parsingRulesPath>/var/lib/jenkins/userContent/android.parse</parsingRulesPath> |
227 | + </hudson.plugins.logparser.LogParserPublisher> |
228 | + <hudson.tasks.Fingerprinter> |
229 | + <targets>build/fingerprints/*</targets> |
230 | + <recordBuildArtifacts>false</recordBuildArtifacts> |
231 | + </hudson.tasks.Fingerprinter> |
232 | + </publishers> |
233 | + <buildWrappers> |
234 | + <hudson.plugins.build__timeout.BuildTimeoutWrapper> |
235 | + <timeoutMinutes>180</timeoutMinutes> |
236 | + <failBuild>false</failBuild> |
237 | + </hudson.plugins.build__timeout.BuildTimeoutWrapper> |
238 | + <com.michelin.cio.hudson.plugins.copytoslave.CopyToSlaveBuildWrapper> |
239 | + <includes>**/*</includes> |
240 | + <excludes/> |
241 | + <flatten>false</flatten> |
242 | + <includeAntExcludes>false</includeAntExcludes> |
243 | + <hudsonHomeRelative>false</hudsonHomeRelative> |
244 | + <relativeTo>copyToSlave</relativeTo> |
245 | + </com.michelin.cio.hudson.plugins.copytoslave.CopyToSlaveBuildWrapper> |
246 | + </buildWrappers> |
247 | +</project> |
248 | \ No newline at end of file |
249 | |
250 | === added file 'config_template/jenkins-jobs/IntegrationLoop.xml' |
251 | --- config_template/jenkins-jobs/IntegrationLoop.xml 1970-01-01 00:00:00 +0000 |
252 | +++ config_template/jenkins-jobs/IntegrationLoop.xml 2012-08-24 13:08:33 +0000 |
253 | @@ -0,0 +1,21 @@ |
254 | +<?xml version='1.0' encoding='UTF-8'?> |
255 | +<project> |
256 | + <actions/> |
257 | + <description></description> |
258 | + <keepDependencies>false</keepDependencies> |
259 | + <properties/> |
260 | + <scm class="hudson.scm.NullSCM"/> |
261 | + <canRoam>true</canRoam> |
262 | + <disabled>false</disabled> |
263 | + <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> |
264 | + <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> |
265 | + <triggers class="vector"/> |
266 | + <concurrentBuild>false</concurrentBuild> |
267 | + <builders> |
268 | + <hudson.tasks.Shell> |
269 | + <command>{build_script}</command> |
270 | + </hudson.tasks.Shell> |
271 | + </builders> |
272 | + <publishers/> |
273 | + <buildWrappers/> |
274 | +</project> |
275 | |
276 | === removed file 'config_template/jenkins-jobs/IntegrationLoop.xml' |
277 | --- config_template/jenkins-jobs/IntegrationLoop.xml 2012-08-21 10:17:49 +0000 |
278 | +++ config_template/jenkins-jobs/IntegrationLoop.xml 1970-01-01 00:00:00 +0000 |
279 | @@ -1,21 +0,0 @@ |
280 | -<?xml version='1.0' encoding='UTF-8'?> |
281 | -<project> |
282 | - <actions/> |
283 | - <description></description> |
284 | - <keepDependencies>false</keepDependencies> |
285 | - <properties/> |
286 | - <scm class="hudson.scm.NullSCM"/> |
287 | - <canRoam>true</canRoam> |
288 | - <disabled>false</disabled> |
289 | - <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding> |
290 | - <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding> |
291 | - <triggers class="vector"/> |
292 | - <concurrentBuild>false</concurrentBuild> |
293 | - <builders> |
294 | - <hudson.tasks.Shell> |
295 | - <command>{build_script}</command> |
296 | - </hudson.tasks.Shell> |
297 | - </builders> |
298 | - <publishers/> |
299 | - <buildWrappers/> |
300 | -</project> |
301 | |
302 | === modified file 'dashboard/frontend/android_build/forms/android_loop_form.py' |
303 | --- dashboard/frontend/android_build/forms/android_loop_form.py 2012-08-17 12:05:12 +0000 |
304 | +++ dashboard/frontend/android_build/forms/android_loop_form.py 2012-08-24 13:08:33 +0000 |
305 | @@ -26,7 +26,7 @@ |
306 | class AndroidLoopForm(ModelForm): |
307 | class Meta: |
308 | model = AndroidLoop |
309 | - exclude = ('server', 'config') |
310 | + exclude = ('server') |
311 | |
312 | def clean(self): |
313 | cleaned_data = super(AndroidLoopForm, self).clean() |
314 | |
315 | === modified file 'dashboard/frontend/android_build/models/android_loop.py' |
316 | --- dashboard/frontend/android_build/models/android_loop.py 2012-08-22 12:19:55 +0000 |
317 | +++ dashboard/frontend/android_build/models/android_loop.py 2012-08-24 13:08:33 +0000 |
318 | @@ -55,10 +55,16 @@ |
319 | DEFAULT_GCC = '4.4.0' |
320 | DEFAULT_IMAGE_SIZE = '2G' |
321 | |
322 | + EXCLUDE_LIST = ['id', 'name', 'server', 'loop_ptr'] |
323 | + |
324 | # BUILD_TYPE |
325 | - build_type = models.CharField(max_length=100, choices=ANDROID_BUILD_TYPES, verbose_name='Build type') |
326 | + build_type = models.CharField(max_length=100, |
327 | + choices=ANDROID_BUILD_TYPES, |
328 | + verbose_name='Build type') |
329 | # MANIFEST_REPO |
330 | - manifest_repo = models.CharField(max_length=255, default=DEFAULT_REPO, verbose_name='Manifest repo') |
331 | + manifest_repo = models.CharField(max_length=255, |
332 | + default=DEFAULT_REPO, |
333 | + verbose_name='Manifest repo') |
334 | # MANIFEST_BRANCH |
335 | manifest_branch = models.CharField(max_length=50, default=DEFAULT_BRANCH) |
336 | # REPO_QUIET |
337 | @@ -111,11 +117,11 @@ |
338 | |
339 | def save(self, *args, **kwargs): |
340 | self.server = JenkinsServer.objects.get(id=1) |
341 | - super(AndroidLoop, self).save(*args, **kwargs) |
342 | + super(self.__class__, self).save(*args, **kwargs) |
343 | # We do not want a failure in remote server to affect the |
344 | # CI Loop flow. |
345 | try: |
346 | - self.server.create_or_update_integration_loop(self) |
347 | + self.server.create_or_update_loop(self) |
348 | except: |
349 | # TODO: Log error. |
350 | pass |
351 | @@ -123,5 +129,6 @@ |
352 | def schedule_build(self): |
353 | # Every time before the build is scheduled, update the |
354 | # loop config on the remote server. |
355 | - self.server.create_or_update_integration_loop(self) |
356 | - super(AndroidLoop, self).schedule_build() |
357 | + parameters = {'parameter': [{'delay': '0'}]} |
358 | + self.server.create_or_update_loop(self) |
359 | + return super(self.__class__, self).schedule_build(parameters) |
360 | |
361 | === modified file 'dashboard/frontend/android_build/templates/android_loop_detail.html' |
362 | --- dashboard/frontend/android_build/templates/android_loop_detail.html 2012-08-20 09:10:05 +0000 |
363 | +++ dashboard/frontend/android_build/templates/android_loop_detail.html 2012-08-24 13:08:33 +0000 |
364 | @@ -38,7 +38,7 @@ |
365 | {% block scripts %} |
366 | <script type="text/javascript"> |
367 | $("#schedule_button").click(function() { |
368 | - $.get("{% if request.is_secure %}https{% else %}http{% endif %}://{{ request.get_host }}/loop/build/{{ android_loop_detail.name }}/", function(data) { |
369 | + $.get("{% if request.is_secure %}https{% else %}http{% endif %}://{{ request.get_host }}/android/build/{{ android_loop_detail.name }}/", function(data) { |
370 | data = $.parseJSON(data); |
371 | $("<div/>", { |
372 | "class": "test", |
373 | |
374 | === modified file 'dashboard/frontend/android_build/tests/test_android_build_clientresponse.py' |
375 | --- dashboard/frontend/android_build/tests/test_android_build_clientresponse.py 2012-08-21 06:55:57 +0000 |
376 | +++ dashboard/frontend/android_build/tests/test_android_build_clientresponse.py 2012-08-24 13:08:33 +0000 |
377 | @@ -23,15 +23,15 @@ |
378 | |
379 | def setUp(self): |
380 | super(AndroidBuildClientResponseTests, self).setUp() |
381 | - self.android_build_data = {"name" : "a-build", |
382 | - "build_type" : "build-android", |
383 | - "manifest_repo" : "repo", |
384 | - "manifest_branch" : "branch", |
385 | - "manifest_filename" : "filename", |
386 | - "make_jobs" : "2", |
387 | - "ramdisk_size" : "2", |
388 | - "binutils_version" : "1", |
389 | - "gcc_version" : "1" |
390 | + self.android_build_data = {"name": "a-build", |
391 | + "build_type": "build-android", |
392 | + "manifest_repo": "repo", |
393 | + "manifest_branch": "branch", |
394 | + "manifest_filename": "filename", |
395 | + "make_jobs": "2", |
396 | + "ramdisk_size": "2", |
397 | + "binutils_version": "1", |
398 | + "gcc_version": "1" |
399 | } |
400 | |
401 | def test_android_build_create_get(self): |
402 | @@ -91,4 +91,3 @@ |
403 | self.assertEquals(response.status_code, 200) |
404 | self.assertIn("Details", response.content) |
405 | self.assertIn("Build type", response.content) |
406 | - |
407 | |
408 | === modified file 'dashboard/frontend/android_build/tests/test_android_build_views.py' |
409 | --- dashboard/frontend/android_build/tests/test_android_build_views.py 2012-08-21 07:58:37 +0000 |
410 | +++ dashboard/frontend/android_build/tests/test_android_build_views.py 2012-08-24 13:08:33 +0000 |
411 | @@ -60,4 +60,4 @@ |
412 | context = AndroidLoopUpdateView.get_context_data( |
413 | android_loop_update_view) |
414 | self.assertEqual(context['request'], "some request data") |
415 | - self.assertEqual(context['form_name'], AndroidLoopUpdateView.form_name) |
416 | \ No newline at end of file |
417 | + self.assertEqual(context['form_name'], AndroidLoopUpdateView.form_name) |
418 | |
419 | === modified file 'dashboard/frontend/android_build/urls.py' |
420 | --- dashboard/frontend/android_build/urls.py 2012-08-20 09:10:05 +0000 |
421 | +++ dashboard/frontend/android_build/urls.py 2012-08-24 13:08:33 +0000 |
422 | @@ -15,13 +15,15 @@ |
423 | # You should have received a copy of the GNU Affero General Public License |
424 | # along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
425 | |
426 | -from django.conf.urls.defaults import patterns, include, url |
427 | -from frontend.android_build.views.android_loop_create_view\ |
428 | +from django.conf.urls.defaults import patterns, url |
429 | +from frontend.android_build.views.android_loop_create_view \ |
430 | import AndroidLoopCreateView |
431 | -from frontend.android_build.views.android_loop_update_view\ |
432 | +from frontend.android_build.views.android_loop_update_view \ |
433 | import AndroidLoopUpdateView |
434 | -from frontend.android_build.views.android_loop_detail_view\ |
435 | +from frontend.android_build.views.android_loop_detail_view \ |
436 | import AndroidLoopDetailView |
437 | +from frontend.android_build.views.android_loop_build_view \ |
438 | + import AndroidLoopBuildView |
439 | |
440 | |
441 | urlpatterns = patterns('', |
442 | @@ -31,4 +33,6 @@ |
443 | AndroidLoopDetailView.as_view(), name='AndroidLoopDetail'), |
444 | url(r'^android/update/(?P<slug>[\w|\W]+)/$', |
445 | AndroidLoopUpdateView.as_view(), name='AndroidLoopUpdate'), |
446 | + url(r'^android/build/(?P<slug>[\w|\W]+)/$', |
447 | + AndroidLoopBuildView.as_view(), name='AndroidLoopBuild'), |
448 | ) |
449 | |
450 | === added file 'dashboard/frontend/android_build/views/android_loop_build_view.py' |
451 | --- dashboard/frontend/android_build/views/android_loop_build_view.py 1970-01-01 00:00:00 +0000 |
452 | +++ dashboard/frontend/android_build/views/android_loop_build_view.py 2012-08-24 13:08:33 +0000 |
453 | @@ -0,0 +1,25 @@ |
454 | +#!/usr/bin/env python |
455 | +# Copyright (C) 2012 Linaro |
456 | +# |
457 | +# This file is part of linaro-ci-dashboard. |
458 | +# |
459 | +# linaro-ci-dashboard is free software: you can redistribute it and/or modify |
460 | +# it under the terms of the GNU Affero General Public License as published by |
461 | +# the Free Software Foundation, either version 3 of the License, or |
462 | +# (at your option) any later version. |
463 | +# |
464 | +# linaro-ci-dashboard is distributed in the hope that it will be useful, |
465 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
466 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
467 | +# GNU Affero General Public License for more details. |
468 | +# |
469 | +# You should have received a copy of the GNU Affero General Public License |
470 | +# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
471 | + |
472 | +from frontend.android_build.models.android_loop import AndroidLoop |
473 | +from frontend.views.loop_build_view import LoopBuildView |
474 | + |
475 | + |
476 | +class AndroidLoopBuildView(LoopBuildView): |
477 | + |
478 | + model = AndroidLoop |
479 | |
480 | === modified file 'dashboard/frontend/android_build/views/android_loop_detail_view.py' |
481 | --- dashboard/frontend/android_build/views/android_loop_detail_view.py 2012-08-20 16:00:44 +0000 |
482 | +++ dashboard/frontend/android_build/views/android_loop_detail_view.py 2012-08-24 13:08:33 +0000 |
483 | @@ -19,9 +19,11 @@ |
484 | from django.contrib.auth.decorators import login_required |
485 | from django.views.generic.detail import DetailView |
486 | from django.utils.decorators import method_decorator |
487 | -from frontend.android_build.models.android_loop \ |
488 | - import AndroidLoop |
489 | +from frontend.models.loop import Loop |
490 | +from frontend.android_build.models.android_loop import AndroidLoop |
491 | from django.core import serializers |
492 | +from itertools import chain |
493 | + |
494 | |
495 | class AndroidLoopDetailView(DetailView): |
496 | |
497 | @@ -40,6 +42,11 @@ |
498 | self).get_context_data(**kwargs) |
499 | context['request'] = self.request |
500 | context['builds'] = self.object.loop_ptr.loopbuild_set.all() |
501 | - context['data'] = serializers.serialize( "python", |
502 | - AndroidLoop.objects.filter(pk=self.object.loop_ptr.id) ) |
503 | + |
504 | + # Need to "merge" the two queryset for all the fields to appear |
505 | + querysets = list(chain(AndroidLoop.objects.filter( |
506 | + pk=self.object.loop_ptr.id), |
507 | + Loop.objects.filter(pk=self.object.loop_ptr.id))) |
508 | + context['data'] = serializers.serialize("python", querysets) |
509 | + |
510 | return context |
511 | |
512 | === modified file 'dashboard/frontend/integration_loop/forms/integration_loop_form.py' |
513 | --- dashboard/frontend/integration_loop/forms/integration_loop_form.py 2012-08-17 11:12:49 +0000 |
514 | +++ dashboard/frontend/integration_loop/forms/integration_loop_form.py 2012-08-24 13:08:33 +0000 |
515 | @@ -23,4 +23,4 @@ |
516 | class IntegrationLoopForm(ModelForm): |
517 | class Meta: |
518 | model = IntegrationLoop |
519 | - exclude = ('server', 'config') |
520 | + exclude = ('server') |
521 | |
522 | === modified file 'dashboard/frontend/integration_loop/models/integration_loop.py' |
523 | --- dashboard/frontend/integration_loop/models/integration_loop.py 2012-08-17 09:43:07 +0000 |
524 | +++ dashboard/frontend/integration_loop/models/integration_loop.py 2012-08-24 13:08:33 +0000 |
525 | @@ -29,22 +29,19 @@ |
526 | precommand = models.CharField(max_length=200) |
527 | command = models.CharField(max_length=200) |
528 | |
529 | + EXCLUDE_LIST = ['id', 'name', 'server', 'loop_ptr'] |
530 | + |
531 | def save(self, *args, **kwargs): |
532 | self.server = JenkinsServer.objects.get(id=1) |
533 | - super(IntegrationLoop, self).save(*args, **kwargs) |
534 | + super(self.__class__, self).save(*args, **kwargs) |
535 | # We do not want a failure in remote server to affect the |
536 | # CI Loop flow. |
537 | try: |
538 | - self.server.create_or_update_integration_loop(self) |
539 | + self.server.create_or_update_loop(self) |
540 | except: |
541 | # TODO: Log error. |
542 | pass |
543 | |
544 | - def schedule_build(self): |
545 | - # Every time before the build is scheduled, update the |
546 | - # loop config on the remote server. |
547 | - self.server.create_or_update_integration_loop(self) |
548 | - super(IntegrationLoop, self).schedule_build() |
549 | - |
550 | def __repr__(self): |
551 | - return "IntegrationLoop(id=%d, branch=%s, ...)" % (self.id, self.branch) |
552 | + return "IntegrationLoop(id=%d, branch=%s, ...)" % (self.id, |
553 | + self.branch) |
554 | |
555 | === modified file 'dashboard/frontend/integration_loop/templates/integration_loop_detail.html' |
556 | --- dashboard/frontend/integration_loop/templates/integration_loop_detail.html 2012-08-16 15:12:10 +0000 |
557 | +++ dashboard/frontend/integration_loop/templates/integration_loop_detail.html 2012-08-24 13:08:33 +0000 |
558 | @@ -48,7 +48,7 @@ |
559 | text: "Build: " + data[0].fields.build_number + " || " + |
560 | data[0].fields.status + " || " + |
561 | data[0].fields.duration + " || " + |
562 | - data[0].fields.remote_number, |
563 | + data[0].fields.remote_number |
564 | }) |
565 | .prependTo($("#builds")); |
566 | }); |
567 | |
568 | === modified file 'dashboard/frontend/kernel_build/models/kernel_loop.py' |
569 | --- dashboard/frontend/kernel_build/models/kernel_loop.py 2012-08-23 12:27:08 +0000 |
570 | +++ dashboard/frontend/kernel_build/models/kernel_loop.py 2012-08-24 13:08:33 +0000 |
571 | @@ -118,7 +118,7 @@ |
572 | # We do not want a failure in remote server to affect the |
573 | # CI Loop flow. |
574 | try: |
575 | - self.server.create_or_update_integration_loop(self) |
576 | + self.server.create_or_update_loop(self) |
577 | except: |
578 | # TODO: Log error. |
579 | pass |
580 | @@ -126,5 +126,5 @@ |
581 | def schedule_build(self): |
582 | # Every time before the build is scheduled, update the |
583 | # loop config on the remote server. |
584 | - self.server.create_or_update_integration_loop(self) |
585 | + self.server.create_or_update_loop(self) |
586 | super(KernelLoop, self).schedule_build() |
587 | |
588 | === added file 'dashboard/frontend/migrations/0002_auto__add_field_loop_is_restricted__add_field_loop_is_official.py' |
589 | --- dashboard/frontend/migrations/0002_auto__add_field_loop_is_restricted__add_field_loop_is_official.py 1970-01-01 00:00:00 +0000 |
590 | +++ dashboard/frontend/migrations/0002_auto__add_field_loop_is_restricted__add_field_loop_is_official.py 2012-08-24 13:08:33 +0000 |
591 | @@ -0,0 +1,54 @@ |
592 | +# encoding: utf-8 |
593 | +import datetime |
594 | +from south.db import db |
595 | +from south.v2 import SchemaMigration |
596 | +from django.db import models |
597 | + |
598 | +class Migration(SchemaMigration): |
599 | + |
600 | + def forwards(self, orm): |
601 | + |
602 | + # Adding field 'Loop.is_restricted' |
603 | + db.add_column('frontend_loop', 'is_restricted', self.gf('django.db.models.fields.BooleanField')(default=False), keep_default=False) |
604 | + |
605 | + # Adding field 'Loop.is_official' |
606 | + db.add_column('frontend_loop', 'is_official', self.gf('django.db.models.fields.BooleanField')(default=False), keep_default=False) |
607 | + |
608 | + |
609 | + def backwards(self, orm): |
610 | + |
611 | + # Deleting field 'Loop.is_restricted' |
612 | + db.delete_column('frontend_loop', 'is_restricted') |
613 | + |
614 | + # Deleting field 'Loop.is_official' |
615 | + db.delete_column('frontend_loop', 'is_official') |
616 | + |
617 | + |
618 | + models = { |
619 | + 'frontend.loop': { |
620 | + 'Meta': {'object_name': 'Loop'}, |
621 | + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), |
622 | + 'is_official': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), |
623 | + 'is_restricted': ('django.db.models.fields.BooleanField', [], {'default': 'False'}), |
624 | + 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '200'}), |
625 | + 'server': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['jenkinsserver.JenkinsServer']"}) |
626 | + }, |
627 | + 'frontend.loopbuild': { |
628 | + 'Meta': {'ordering': "['-id']", 'object_name': 'LoopBuild'}, |
629 | + 'build_number': ('django.db.models.fields.IntegerField', [], {'default': 'None'}), |
630 | + 'duration': ('django.db.models.fields.DecimalField', [], {'max_digits': '8', 'decimal_places': '2'}), |
631 | + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), |
632 | + 'loop': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['frontend.Loop']"}), |
633 | + 'remote_number': ('django.db.models.fields.IntegerField', [], {'null': 'True', 'blank': 'True'}), |
634 | + 'status': ('django.db.models.fields.CharField', [], {'max_length': '20'}) |
635 | + }, |
636 | + 'jenkinsserver.jenkinsserver': { |
637 | + 'Meta': {'object_name': 'JenkinsServer'}, |
638 | + 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}), |
639 | + 'password': ('django.db.models.fields.CharField', [], {'max_length': '100'}), |
640 | + 'url': ('django.db.models.fields.CharField', [], {'max_length': '255'}), |
641 | + 'username': ('django.db.models.fields.CharField', [], {'max_length': '100'}) |
642 | + } |
643 | + } |
644 | + |
645 | + complete_apps = ['frontend'] |
646 | |
647 | === modified file 'dashboard/frontend/models/loop.py' |
648 | --- dashboard/frontend/models/loop.py 2012-08-22 12:19:55 +0000 |
649 | +++ dashboard/frontend/models/loop.py 2012-08-24 13:08:33 +0000 |
650 | @@ -16,8 +16,10 @@ |
651 | # You should have received a copy of the GNU Affero General Public License |
652 | # along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
653 | |
654 | +import base64 |
655 | from django.db import models |
656 | from jenkinsserver.models.jenkins_server import JenkinsServer |
657 | +from lib.logger import Logger |
658 | |
659 | |
660 | class Loop(models.Model): |
661 | @@ -26,8 +28,16 @@ |
662 | |
663 | name = models.CharField(max_length=200, unique=True) |
664 | server = models.ForeignKey(JenkinsServer) |
665 | - |
666 | - def schedule_build(self): |
667 | + is_restricted = models.BooleanField(default=False, |
668 | + verbose_name='Build is restricted') |
669 | + is_official = models.BooleanField(default=False, |
670 | + verbose_name='Build is official') |
671 | + |
672 | + def __init__(self, *args, **kwargs): |
673 | + self.log = Logger.getClassLogger(self) |
674 | + super(Loop, self).__init__(*args, **kwargs) |
675 | + |
676 | + def schedule_build(self, parameters=None): |
677 | '''Add a LoopBuild record for this Loop. |
678 | |
679 | Also try to send a build job signal to the server, but if it fails, |
680 | @@ -40,7 +50,8 @@ |
681 | build.duration = 0.00 |
682 | build.save() |
683 | try: |
684 | - build.remote_number = self.server.schedule_job_build(self.name) |
685 | + build.remote_number = self.server.schedule_job_build(self.name, |
686 | + parameters) |
687 | build.save() |
688 | except: |
689 | # TODO: Log error. |
690 | @@ -57,3 +68,13 @@ |
691 | |
692 | def __repr__(self): |
693 | return "%s(id=%d, ...)" % (self.__class__.__name__, self.id) |
694 | + |
695 | + def base64_config(self): |
696 | + """ Return loop configuration as base64 encoded string. |
697 | + """ |
698 | + text_config = u'' |
699 | + for field in self._meta.fields: |
700 | + if not field.name in self.EXCLUDE_LIST: |
701 | + text_config += u'%s=%s\n' % (field.name.upper(), |
702 | + field.value_to_string(self)) |
703 | + return base64.b64encode(text_config) |
704 | |
705 | === modified file 'dashboard/frontend/urls.py' |
706 | --- dashboard/frontend/urls.py 2012-08-16 15:12:10 +0000 |
707 | +++ dashboard/frontend/urls.py 2012-08-24 13:08:33 +0000 |
708 | @@ -16,7 +16,7 @@ |
709 | # You should have received a copy of the GNU Affero General Public License |
710 | # along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
711 | |
712 | -from django.conf.urls.defaults import patterns, include, url |
713 | +from django.conf.urls.defaults import patterns, url |
714 | from frontend.views.home_page_view import HomePageView |
715 | from frontend.views.index_view import IndexView |
716 | from frontend.views.loop_build_view import LoopBuildView |
717 | |
718 | === modified file 'dashboard/jenkinsserver/models/jenkins_server.py' |
719 | --- dashboard/jenkinsserver/models/jenkins_server.py 2012-08-22 12:19:55 +0000 |
720 | +++ dashboard/jenkinsserver/models/jenkins_server.py 2012-08-24 13:08:33 +0000 |
721 | @@ -18,13 +18,13 @@ |
722 | |
723 | from django.db import models |
724 | from django.conf import settings |
725 | -from xml.dom.minidom import parse, parseString |
726 | import jenkins |
727 | import json |
728 | import urllib2 |
729 | from lib.logger import Logger |
730 | from lib.template import TextTemplate, XMLTemplate |
731 | |
732 | + |
733 | class JenkinsServer(models.Model): |
734 | class Meta: |
735 | app_label = 'jenkinsserver' |
736 | @@ -40,7 +40,7 @@ |
737 | self.username.encode('utf-8'), |
738 | self.password.encode('utf-8')) |
739 | |
740 | - def create_or_update_integration_loop(self, loop): |
741 | + def create_or_update_loop(self, loop): |
742 | """ Create integration loop if it doesn't exist on Jenkins. |
743 | Otherwise, update the existing one with new XML.""" |
744 | |
745 | @@ -100,22 +100,32 @@ |
746 | |
747 | def create_job_xml(self, loop): |
748 | "Prepare the config.xml to be used for the job." |
749 | - # Use loop's class name as the template name - JenkinsServer |
750 | - # still doesn't have any internal knowledge about loop |
751 | + # Get generic serialization data. |
752 | + loop_params = loop.json() |
753 | + |
754 | template_name = loop.__class__.__name__ |
755 | - # Get generic serialization data - Loop still doesn't know |
756 | - # about Jenkins |
757 | - loop_params = loop.json() |
758 | - build_script = TextTemplate(settings.BUILD_SCRIPT_TEMPLATE \ |
759 | - % template_name).render(loop_params) |
760 | - job_params = {"build_script": build_script} |
761 | - job_params.update(loop_params) |
762 | + |
763 | + # We need to handle the CONFIG parameter as a base64 encoded string. |
764 | + # At the moment used only by Android builds. |
765 | + loop_params['base64_config'] = loop.base64_config() |
766 | + |
767 | + # Check if the build is restricted, assign appropriate nodes. |
768 | + if loop.is_restricted: |
769 | + loop_params['assigned_node'] = 'private && natty' |
770 | + else: |
771 | + loop_params['assigned_node'] = 'ec2' |
772 | + |
773 | + build_script = TextTemplate(settings.BUILD_SCRIPT_TEMPLATE\ |
774 | + % template_name).render(loop_params) |
775 | + loop_params['build_script'] = build_script |
776 | + |
777 | xml = XMLTemplate(settings.JENKINS_JOB_TEMPLATE \ |
778 | - % template_name).render(job_params) |
779 | + % template_name).render(loop_params) |
780 | + |
781 | return xml |
782 | |
783 | - def schedule_job_build(self, jobname): |
784 | + def schedule_job_build(self, jobname, parameters=None): |
785 | """Trigger build job on jenkins. Return the next build number.""" |
786 | - self.jenkins.build_job(jobname) |
787 | + self.jenkins.build_job(jobname, parameters) |
788 | job_info = self.jenkins.get_job_info(jobname) |
789 | return job_info["nextBuildNumber"] |
790 | |
791 | === modified file 'dashboard/jenkinsserver/tests/test_jenkins_server.py' |
792 | --- dashboard/jenkinsserver/tests/test_jenkins_server.py 2012-08-21 17:47:38 +0000 |
793 | +++ dashboard/jenkinsserver/tests/test_jenkins_server.py 2012-08-24 13:08:33 +0000 |
794 | @@ -54,7 +54,6 @@ |
795 | loop.branch = 'lp:linaro-ci-dashboard' |
796 | loop.precommand = 'cd %s' % loop.branch |
797 | loop.command = 'make syncdb; make test' |
798 | - jbranch = 'bzr branch lp:%s dest_dir' % loop.branch |
799 | dest_dir = 'dest_dir' |
800 | env_cmds = "export USER=`whoami`;\nexport HOME=/home/$USER" |
801 | |
802 | @@ -85,8 +84,6 @@ |
803 | loop.branch = 'linaro-ci-dashboard' |
804 | loop.precommand = 'cd %s' % loop.branch |
805 | loop.command = 'make syncdb; make test' |
806 | - jbranch = 'bzr branch lp:%s' % loop.branch |
807 | - jcommands = '\n'.join([jbranch, loop.precommand, loop.command]) |
808 | jxml = self.server.create_job_xml(loop) |
809 | self.server.create_job(loop.name, jxml) |
810 | self.assertTrue(self.test_jenkins.job_exists(loop.name) == True) |
811 | |
812 | === added file 'dashboard/lib/template.py' |
813 | --- dashboard/lib/template.py 1970-01-01 00:00:00 +0000 |
814 | +++ dashboard/lib/template.py 2012-08-24 13:08:33 +0000 |
815 | @@ -0,0 +1,39 @@ |
816 | +# Copyright (C) 2012 Linaro |
817 | +# |
818 | +# This file is part of linaro-ci-dashboard. |
819 | +# |
820 | +# linaro-ci-dashboard is free software: you can redistribute it and/or modify |
821 | +# it under the terms of the GNU Affero General Public License as published by |
822 | +# the Free Software Foundation, either version 3 of the License, or |
823 | +# (at your option) any later version. |
824 | +# |
825 | +# linaro-ci-dashboard is distributed in the hope that it will be useful, |
826 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
827 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
828 | +# GNU Affero General Public License for more details. |
829 | +# |
830 | +# You should have received a copy of the GNU Affero General Public License |
831 | +# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
832 | +# Django settings for dashboard project. |
833 | + |
834 | +class AbstractTemplate(object): |
835 | + "Template class interface." |
836 | + |
837 | + def render(self, dict): |
838 | + pass |
839 | + |
840 | + |
841 | +class TextTemplate(AbstractTemplate): |
842 | + |
843 | + def __init__(self, filename): |
844 | + f = open(filename) |
845 | + self.tmpl = f.read() |
846 | + f.close() |
847 | + |
848 | + def render(self, dict): |
849 | + return self.tmpl.format(**dict) |
850 | + |
851 | + |
852 | +class XMLTemplate(TextTemplate): |
853 | + """Initial XML templating implementation is based in plain text |
854 | + templates, TODO: fix this!""" |
855 | |
856 | === removed file 'dashboard/lib/template.py' |
857 | --- dashboard/lib/template.py 2012-08-22 12:19:55 +0000 |
858 | +++ dashboard/lib/template.py 1970-01-01 00:00:00 +0000 |
859 | @@ -1,39 +0,0 @@ |
860 | -# Copyright (C) 2012 Linaro |
861 | -# |
862 | -# This file is part of linaro-ci-dashboard. |
863 | -# |
864 | -# linaro-ci-dashboard is free software: you can redistribute it and/or modify |
865 | -# it under the terms of the GNU Affero General Public License as published by |
866 | -# the Free Software Foundation, either version 3 of the License, or |
867 | -# (at your option) any later version. |
868 | -# |
869 | -# linaro-ci-dashboard is distributed in the hope that it will be useful, |
870 | -# but WITHOUT ANY WARRANTY; without even the implied warranty of |
871 | -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
872 | -# GNU Affero General Public License for more details. |
873 | -# |
874 | -# You should have received a copy of the GNU Affero General Public License |
875 | -# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
876 | -# Django settings for dashboard project. |
877 | - |
878 | -class AbstractTemplate(object): |
879 | - "Template class interface." |
880 | - |
881 | - def render(self, dict): |
882 | - pass |
883 | - |
884 | - |
885 | -class TextTemplate(AbstractTemplate): |
886 | - |
887 | - def __init__(self, filename): |
888 | - f = open(filename) |
889 | - self.tmpl = f.read() |
890 | - f.close() |
891 | - |
892 | - def render(self, dict): |
893 | - return self.tmpl.format(**dict) |
894 | - |
895 | - |
896 | -class XMLTemplate(TextTemplate): |
897 | - """Initial XML templating implementation is based in plain text |
898 | - templates, TODO: fix this!""" |
I updated the branch, after a better look at the XML files, they differ only in a small part.
We are now using the same approach of using the class name for the template file, modifying only one parameter in case the build is 'restricted'.