Merge lp:~milo/linaro-ci-dashboard/android-build-config into lp:linaro-ci-dashboard

Proposed by Milo Casagrande
Status: Merged
Merged at revision: 28
Proposed branch: lp:~milo/linaro-ci-dashboard/android-build-config
Merge into: lp:linaro-ci-dashboard
Diff against target: 898 lines (+394/-129)
22 files modified
config_template/build_scripts/AndroidLoop.sh (+4/-0)
config_template/build_scripts/IntegrationLoop.sh (+0/-8)
config_template/jenkins-jobs/AndroidLoop.xml (+208/-0)
config_template/jenkins-jobs/IntegrationLoop.xml (+0/-21)
dashboard/frontend/android_build/forms/android_loop_form.py (+1/-1)
dashboard/frontend/android_build/models/android_loop.py (+13/-6)
dashboard/frontend/android_build/templates/android_loop_detail.html (+1/-1)
dashboard/frontend/android_build/tests/test_android_build_clientresponse.py (+9/-10)
dashboard/frontend/android_build/tests/test_android_build_views.py (+1/-1)
dashboard/frontend/android_build/urls.py (+8/-4)
dashboard/frontend/android_build/views/android_loop_build_view.py (+25/-0)
dashboard/frontend/android_build/views/android_loop_detail_view.py (+11/-4)
dashboard/frontend/integration_loop/forms/integration_loop_form.py (+1/-1)
dashboard/frontend/integration_loop/models/integration_loop.py (+6/-9)
dashboard/frontend/integration_loop/templates/integration_loop_detail.html (+1/-1)
dashboard/frontend/kernel_build/models/kernel_loop.py (+2/-2)
dashboard/frontend/migrations/0002_auto__add_field_loop_is_restricted__add_field_loop_is_official.py (+54/-0)
dashboard/frontend/models/loop.py (+24/-3)
dashboard/frontend/urls.py (+1/-1)
dashboard/jenkinsserver/models/jenkins_server.py (+24/-14)
dashboard/jenkinsserver/tests/test_jenkins_server.py (+0/-3)
dashboard/lib/template.py (+0/-39)
To merge this branch: bzr merge lp:~milo/linaro-ci-dashboard/android-build-config
Reviewer Review Type Date Requested Status
Stevan Radaković code Approve
Paul Sokolovsky Abstain
Review via email: mp+120969@code.launchpad.net

Description of the change

Here there is preliminary support for android build of type "android".
We had to refactor a little bit the code, in particular:
- Added a new parameter to the schedule_build method in the Loop class, in order to be able to pass parameters to Jenkins. This was needed since using the same config.xml file taken from the build at android-build.linaro.org, we were not able to trigger a run.
 - Added a specialized AndroidLoopBuildView, in order to correctly pass the defined parameters via the overridden schedule_build method. This was necessary since that method is called inside the LoopBuildView class, in loop_build_view, and it considers the object passed only as Loop, not as a "specialized" one, and the method called was then the one inside Loop, not the specialized one.
 - Slightly modfied the method to create the job xml inside JenkinsServer. With android build we have right now 3 different templates for the jobs, we cannot rely solely on the class name to identify them. At the moment we added only one template.

There are also some PEP8 fixes.

To post a comment you must log in.
30. By Milo Casagrande

Use only one android loop template.

Revision history for this message
Milo Casagrande (milo) wrote :

I updated the branch, after a better look at the XML files, they differ only in a small part.
We are now using the same approach of using the class name for the template file, modifying only one parameter in case the build is 'restricted'.

Revision history for this message
Paul Sokolovsky (pfalcon) wrote :

> - Slightly modfied the method to create the job xml inside JenkinsServer. With android build we have right now 3 different templates for the jobs, we cannot rely solely on the class name to identify them. At the moment we added only one template.

Well, that kinda hints that these should be 3 different model classes ;-I. (Unless there's general and universal approach how to deal with such cases for future, but having separate models is KISS, at least on model side (vs handling them)).

664 + if template_name == 'AndroidLoop':
665 + template_name = loop.build_type
666 + loop_params['base64_config'] = loop.base64_config()

In the outlook I was having when introduced templating, this would be dealt with differently: AndroidLoop.json() would be overriden to add "base64_config" with needed value, then JenkinsServer wouldn't need to know about AndroidLoop existence and its peculiarities.

Re: other changes - it's a bit hard to review them, because it seems that diff is being made against older version of trunk, while tip of it already contains templating code, etc. Can you merge trunk into this branch and re-push it to get better diff?

Revision history for this message
Paul Sokolovsky (pfalcon) wrote :

Well, imho "if template_name == 'AndroidLoop':" really Needs Fixing.

review: Needs Fixing
Revision history for this message
Milo Casagrande (milo) wrote :

>
> Re: other changes - it's a bit hard to review them, because it seems that diff
> is being made against older version of trunk, while tip of it already contains
> templating code, etc. Can you merge trunk into this branch and re-push it to
> get better diff?

It is actually based on trunk, the problem was that we were already working on the template part, adding the directories as needed, but conflicts raised when merging trunk (conflicts on directory, with ".moved" attached to it). I thought that using "bzr resolve" would have clean it out, but that was not the case. It marked as removed and then added back the files.

Revision history for this message
Stevan Radaković (stevanr) wrote :

> > - Slightly modfied the method to create the job xml inside JenkinsServer.
> With android build we have right now 3 different templates for the jobs, we
> cannot rely solely on the class name to identify them. At the moment we added
> only one template.
>
> Well, that kinda hints that these should be 3 different model classes ;-I.
> (Unless there's general and universal approach how to deal with such cases for
> future, but having separate models is KISS, at least on model side (vs
> handling them)).
>
> 664 + if template_name == 'AndroidLoop':
> 665 + template_name = loop.build_type
> 666 + loop_params['base64_config'] = loop.base64_config()
>
> In the outlook I was having when introduced templating, this would be dealt
> with differently: AndroidLoop.json() would be overriden to add "base64_config"
> with needed value, then JenkinsServer wouldn't need to know about AndroidLoop
> existence and its peculiarities.

I can't agree with this. Moving template logic to the loop is not viable, as we discussed over and over again.
JenkinsServer need to know about the loops, since it can't be the other way around. It's not the Jenkins app that will be resusable by other applications in the future, but the frontend one.

>
> Re: other changes - it's a bit hard to review them, because it seems that diff
> is being made against older version of trunk, while tip of it already contains
> templating code, etc. Can you merge trunk into this branch and re-push it to
> get better diff?

Revision history for this message
Milo Casagrande (milo) wrote :

I took a look and tried to use the AndroidLoop.json() approach, but I agree with Stevan, and that was our initial intention when we added the "if-clause" in JenkinsServer: storing nothing in the loop WRT the server.

Also, using the json() approach, I will really bond Jenkins specific values (like the name of the node, plus the possible base64 one) into the loop.

For a more general approach, I would also move the base64 calculation into JenkinsServer, and dealing with it only for the android loop (at the moment, since other jobs may use it).

Revision history for this message
Paul Sokolovsky (pfalcon) wrote :

> JenkinsServer need to know about the loops, since it can't be the other way around.

My approach allows to decouple them even further and to follow data-driven approach: so neither JenkinsServer needs to know about adhoc peculiarities, nor Loops need to be tied to particular build system.

So, it all reduces to the question of where you draw the line regarding what is "including build system specific data into Loop". Let's start with saying that the fact that Jenkins uses XML for job config, and particular layout of XML is Jenkins-specific thing, and we can't include direct Jenkins job config generation as method of Loop.

Now let's think about culprits:

"like the name of the node" - well, that value is not really *name* of *specific Jenkins node*. It's a *tag* of node *type*. So, there's an abstract node, and it may have tags, and job (aka loop), may also have tags, and if tagsets of a node and job are compatible, the job can build on such node. That's fairly generic concept, not really tied to Jenkins, any sufficiently advanced build system would have similar concept. I don't see probs with that being a property of a loop.

base64-encoded entire job config - well, that's more peculiar, but looking "from the top", it's just that, implementation of following requirement: 1) being able to pass entire loop config to the build system; 2) that entire config being serialized in well-known format. So, those requirements and feature is not really Jenkins-specific, it fairly generic feature, which again can be reasonable assumed to be in need, and supported/parsable by any build system.

So, again it all depends on where you draw the line. I draw the line at the place which allows to use data-driven approach, to avoid patching code for each and every discovered peculiarities (with my outlook, it'll better to have {% if %}'s in templates, then in Jenkins server.

I don't say I'm right, just wanted to make sure that perspective is well understood. The strongest counter-argument I may imagine is you saying that all my approach is based on the fact that a job in build system can be created from a text config file which we prepare at once and submit. I would say that it covers our usecase so far (implemented in well-maintainable, declarative way) ;-). And would continue that if you want to make it fully generic (on code) and maintainable, then patching single method with more and more if's as we go won't scale. Then you'd need to add another layer (i.e.) class which algorithmically encodes process of creating a build from (particular type of) Loop, then have a registry (i.e. factory) which would allow to apply it to any type of Loop. Then would as beautiful as the solution I propose, though not declarative and data-driven.

Anyway, I change my vote to "Abstain" in this regard, and the rest looks good as far as I can tell.

review: Abstain
Revision history for this message
Paul Sokolovsky (pfalcon) wrote :

Whoa, that was long, sorry ;-)

Revision history for this message
Stevan Radaković (stevanr) wrote :

Great work Milo.

354 + parameters = {'parameter': [{'delay': '0'}]}
355 self.server.create_or_update_integration_loop(self)
356 - super(AndroidLoop, self).schedule_build()
357 + return super(self.__class__, self).schedule_build(parameters)

'create_or_update_integration_loop' ? Think we need some refactoring here (jenkinsserver side as well probably)

535 - def schedule_build(self):
536 - # Every time before the build is scheduled, update the
537 - # loop config on the remote server.
538 - self.server.create_or_update_integration_loop(self)
539 - super(IntegrationLoop, self).schedule_build()
540 -

Hm, why was this removed?

Regarding concerns Paul raised, we are going to discuss this on the meeting...

review: Needs Fixing (code)
Revision history for this message
Milo Casagrande (milo) wrote :

On Fri, Aug 24, 2012 at 11:47 AM, Stevan Radaković
<email address hidden> wrote:
>
> 354 + parameters = {'parameter': [{'delay': '0'}]}
> 355 self.server.create_or_update_integration_loop(self)
> 356 - super(AndroidLoop, self).schedule_build()
> 357 + return super(self.__class__, self).schedule_build(parameters)
>
> 'create_or_update_integration_loop' ? Think we need some refactoring here (jenkinsserver side as well probably)

Will refactor that into a "create_or_update_loop".

> 535 - def schedule_build(self):
> 536 - # Every time before the build is scheduled, update the
> 537 - # loop config on the remote server.
> 538 - self.server.create_or_update_integration_loop(self)
> 539 - super(IntegrationLoop, self).schedule_build()
> 540 -
>
> Hm, why was this removed?

This is actually not called, as long as you do not create your own
"build_view". We discovered that while trying to pass the "params"
dictionary to the jenkins server: it was always "None", the default
value defined. We had to temporary add logger calls everywhere to sort
it out.

When you click on the "schedule" button, what it calls is what is
found in the "loop/build/" URL, that matches to the "LoopBuildView"
view. This one, in turn, calls "self.object.schedule_build()", but
"self.object" in there is always of type Loop since the class is
defined against that model, and not an instance of the specific Loop.

If you want that method to be called, you need to define your own
"$LOOP_build_view", inherit from LoopBuildView, and in there just
define "model = $MODEL_CLASS" (plus add your "build" URL). In this way
that method is called, otherwise it can be safely removed since it
will never be called.

--
Milo Casagrande
Infrastructure Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs

31. By Milo Casagrande

Merged from trunk.

32. By Milo Casagrande

Added new fields, updated migration.

Revision history for this message
Stevan Radaković (stevanr) wrote :

> On Fri, Aug 24, 2012 at 11:47 AM, Stevan Radaković
> <email address hidden> wrote:
> >
> > 354 + parameters = {'parameter': [{'delay': '0'}]}
> > 355 self.server.create_or_update_integration_loop(self)
> > 356 - super(AndroidLoop, self).schedule_build()
> > 357 + return super(self.__class__, self).schedule_build(parameters)
> >
> > 'create_or_update_integration_loop' ? Think we need some refactoring here
> (jenkinsserver side as well probably)
>
> Will refactor that into a "create_or_update_loop".
>
> > 535 - def schedule_build(self):
> > 536 - # Every time before the build is scheduled, update the
> > 537 - # loop config on the remote server.
> > 538 - self.server.create_or_update_integration_loop(self)
> > 539 - super(IntegrationLoop, self).schedule_build()
> > 540 -
> >
> > Hm, why was this removed?
>
> This is actually not called, as long as you do not create your own
> "build_view". We discovered that while trying to pass the "params"
> dictionary to the jenkins server: it was always "None", the default
> value defined. We had to temporary add logger calls everywhere to sort
> it out.
>
> When you click on the "schedule" button, what it calls is what is
> found in the "loop/build/" URL, that matches to the "LoopBuildView"
> view. This one, in turn, calls "self.object.schedule_build()", but
> "self.object" in there is always of type Loop since the class is
> defined against that model, and not an instance of the specific Loop.
>
> If you want that method to be called, you need to define your own
> "$LOOP_build_view", inherit from LoopBuildView, and in there just
> define "model = $MODEL_CLASS" (plus add your "build" URL). In this way
> that method is called, otherwise it can be safely removed since it
> will never be called.
>

Ok this is a nice catch, but it was hardly the way to resolve the problem you noticed, since we need this create_or_update called.
I'll add this to my action list...

Revision history for this message
Milo Casagrande (milo) wrote :

On Fri, Aug 24, 2012 at 1:16 PM, Stevan Radaković
<email address hidden> wrote:
>
> Ok this is a nice catch, but it was hardly the way to resolve the problem you noticed, since we need this create_or_update called.
> I'll add this to my action list...

You are right, my fault. I actually had to add it into the Loop class,
so that we do not need to call it in each derived class.

--
Milo Casagrande
Infrastructure Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs

33. By Milo Casagrande

Moved call to updated config into Loop, refactored.

34. By Milo Casagrande

Reverted commit.

Revision history for this message
Milo Casagrande (milo) wrote :

On Fri, Aug 24, 2012 at 1:45 PM, Milo Casagrande
<email address hidden> wrote:
>
> You are right, my fault. I actually had to add it into the Loop class,
> so that we do not need to call it in each derived class.

I just tried, but I reverted back. It is not only a matter of moving
the call, we need to define the specialized "build_view" also for the
other loops, otherwise it will not work, since what will be passed
will be the "Loop" object and not the actual derived one (like
KernelLoop or IntegrationLoop).

WRT IntegrationLoop, even removing it, but not adding the
"update_or_create" call into Loop, it can schedule a new build, but it
will never update the config. Same thing happens if we leave it there:
it is never called. If we add the call to "update_or_create" inside
Loop, it will fail with error 500 since we are not defining the
specialized "build_view".

I can take a look at it, it is not a problem, but in a separate
branch, so we have a clean diff of what is going on.

--
Milo Casagrande
Infrastructure Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs

Revision history for this message
Stevan Radaković (stevanr) wrote :

Ok thanks for looking into it. Let's just rename the method to create_or_update_loop and get this landed.
Approve +1

review: Approve (code)
35. By Milo Casagrande

Fixed problem with android detail view.

36. By Milo Casagrande

Fixed JenkinsServer to use new parameters.

37. By Milo Casagrande

Refactored method name.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'config_template/build_scripts/AndroidLoop.sh'
2--- config_template/build_scripts/AndroidLoop.sh 1970-01-01 00:00:00 +0000
3+++ config_template/build_scripts/AndroidLoop.sh 2012-08-24 13:08:33 +0000
4@@ -0,0 +1,4 @@
5+rm -rf build-tools
6+bzr get lp:linaro-android-build-tools build-tools
7+build-tools/node/build us-east-1.ec2-git-mirror.linaro.org "$CONFIG"
8+#build-tools/node/build "$HUDSON_URL" "$CONFIG"
9
10=== added file 'config_template/build_scripts/IntegrationLoop.sh'
11--- config_template/build_scripts/IntegrationLoop.sh 1970-01-01 00:00:00 +0000
12+++ config_template/build_scripts/IntegrationLoop.sh 2012-08-24 13:08:33 +0000
13@@ -0,0 +1,8 @@
14+rm -rf dest_dir
15+export USER=`whoami`;
16+export HOME=/home/$USER
17+bzr branch {branch} dest_dir
18+cur_dir=$WORKSPACE; cd ./dest_dir
19+{precommand}
20+{command}
21+cd "$cur_dir"
22
23=== removed file 'config_template/build_scripts/IntegrationLoop.sh'
24--- config_template/build_scripts/IntegrationLoop.sh 2012-08-21 10:17:49 +0000
25+++ config_template/build_scripts/IntegrationLoop.sh 1970-01-01 00:00:00 +0000
26@@ -1,8 +0,0 @@
27-rm -rf dest_dir
28-export USER=`whoami`;
29-export HOME=/home/$USER
30-bzr branch {branch} dest_dir
31-cur_dir=$WORKSPACE; cd ./dest_dir
32-{precommand}
33-{command}
34-cd "$cur_dir"
35
36=== added file 'config_template/jenkins-jobs/AndroidLoop.xml'
37--- config_template/jenkins-jobs/AndroidLoop.xml 1970-01-01 00:00:00 +0000
38+++ config_template/jenkins-jobs/AndroidLoop.xml 2012-08-24 13:08:33 +0000
39@@ -0,0 +1,208 @@
40+<?xml version="1.0" encoding="UTF-8"?>
41+<project>
42+ <actions/>
43+ <description>&lt;!-- Automatically managed header - do not change, do not add anything before this! --&gt;
44+&lt;table width="65%" align="center"&gt;
45+&lt;tr&gt;&lt;td&gt;
46+&lt;p style="background: #FFCCFF; padding: 15px"&gt;
47+&lt;span style="color: gray"&gt;New!&lt;/span&gt;&lt;br /&gt;
48+We now provide &lt;tt&gt;linaro_android_build_cmds.sh&lt;/tt&gt; script (see downloads)
49+with exact commands to reproduce this build. Use it as a reference of
50+commands you need to execute, or just run to perform a build automatically.
51+There is also &lt;tt&gt;linaro_kernel_build_cmds.sh&lt;/tt&gt; to rebuild just a kernel.
52+Alternatively, you can try instructions below.
53+&lt;/td&gt;&lt;/tr&gt;
54+&lt;/p&gt;
55+&lt;/table&gt;
56+
57+&lt;!-- End of automatically managed header --&gt;
58+
59+</description>
60+ <logRotator>
61+ <daysToKeep>30</daysToKeep>
62+ <numToKeep>10</numToKeep>
63+ <artifactDaysToKeep>-1</artifactDaysToKeep>
64+ <artifactNumToKeep>-1</artifactNumToKeep>
65+ </logRotator>
66+ <keepDependencies>true</keepDependencies>
67+ <properties>
68+ <hudson.model.ParametersDefinitionProperty>
69+ <parameterDefinitions>
70+ <hudson.model.StringParameterDefinition>
71+ <name>CONFIG</name>
72+ <description/>
73+ <defaultValue>{base64_config}</defaultValue>
74+ </hudson.model.StringParameterDefinition>
75+ </parameterDefinitions>
76+ </hudson.model.ParametersDefinitionProperty>
77+ </properties>
78+ <scm class="hudson.scm.NullSCM"/>
79+ <assignedNode>{assigned_node}</assignedNode>
80+ <canRoam>true</canRoam>
81+ <disabled>false</disabled>
82+ <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
83+ <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
84+ <triggers class="vector"/>
85+ <concurrentBuild>true</concurrentBuild>
86+ <builders>
87+ <hudson.plugins.shell.ShellWithStatus>
88+ <command>{build_script}</command>
89+ </hudson.plugins.shell.ShellWithStatus>
90+ <jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin>
91+ <delegate>
92+ <consolePrefix>SSH: </consolePrefix>
93+ <delegate>
94+ <publishers>
95+ <jenkins.plugins.publish__over__ssh.BapSshPublisher>
96+ <configName>snapshots.linaro.org new</configName>
97+ <verbose>false</verbose>
98+ <transfers>
99+ <jenkins.plugins.publish__over__ssh.BapSshTransfer>
100+ <remoteDirectory>android/${{JOB_NAME}}/${{BUILD_NUMBER}}</remoteDirectory>
101+ <sourceFiles>build/out/target/*/*/*.img.bz2,build/out/target/*/*/*.tar.bz2,build/out/target/*/*/MD5SUMS,build/out/*.tar.bz2,build/out/*.xml,build/out/*_config,build/out/lava-job-info,build/out/linaro_kernel_build_cmds.sh,build/out/linaro_android_build_cmds.sh,build/out/**/*EULA*</sourceFiles>
102+ <excludes/>
103+ <removePrefix>build/out</removePrefix>
104+ <remoteDirectorySDF>false</remoteDirectorySDF>
105+ <flatten>false</flatten>
106+ <cleanRemote>false</cleanRemote>
107+ <execCommand/>
108+ <execTimeout>120000</execTimeout>
109+ <usePty>false</usePty>
110+ </jenkins.plugins.publish__over__ssh.BapSshTransfer>
111+ </transfers>
112+ <useWorkspaceInPromotion>false</useWorkspaceInPromotion>
113+ <usePromotionTimestamp>false</usePromotionTimestamp>
114+ </jenkins.plugins.publish__over__ssh.BapSshPublisher>
115+ <jenkins.plugins.publish__over__ssh.BapSshPublisher>
116+ <configName>snapshots.linaro.org file-move new</configName>
117+ <verbose>false</verbose>
118+ <transfers>
119+ <jenkins.plugins.publish__over__ssh.BapSshTransfer>
120+ <remoteDirectory/>
121+ <sourceFiles/>
122+ <excludes/>
123+ <removePrefix/>
124+ <remoteDirectorySDF>false</remoteDirectorySDF>
125+ <flatten>false</flatten>
126+ <cleanRemote>false</cleanRemote>
127+ <execCommand>reshuffle-files -t android -j ${{JOB_NAME}} -n ${{BUILD_NUMBER}} -m</execCommand>
128+ <execTimeout>120000</execTimeout>
129+ <usePty>false</usePty>
130+ </jenkins.plugins.publish__over__ssh.BapSshTransfer>
131+ </transfers>
132+ <useWorkspaceInPromotion>false</useWorkspaceInPromotion>
133+ <usePromotionTimestamp>false</usePromotionTimestamp>
134+ </jenkins.plugins.publish__over__ssh.BapSshPublisher>
135+ </publishers>
136+ <continueOnError>false</continueOnError>
137+ <failOnError>false</failOnError>
138+ <alwaysPublishFromMaster>true</alwaysPublishFromMaster>
139+ <hostConfigurationAccess class="jenkins.plugins.publish_over_ssh.BapSshPublisherPlugin" reference="../.."/>
140+ </delegate>
141+ </delegate>
142+ </jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin>
143+ <hudson.tasks.Shell>
144+ <command>build-tools/node/lava-submit "$CONFIG"
145+</command>
146+ </hudson.tasks.Shell>
147+ <jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin>
148+ <delegate>
149+ <consolePrefix>SSH: </consolePrefix>
150+ <delegate>
151+ <publishers>
152+ <jenkins.plugins.publish__over__ssh.BapSshPublisher>
153+ <configName>snapshots.linaro.org new</configName>
154+ <verbose>false</verbose>
155+ <transfers>
156+ <jenkins.plugins.publish__over__ssh.BapSshTransfer>
157+ <remoteDirectory>android/${{JOB_NAME}}/${{BUILD_NUMBER}}</remoteDirectory>
158+ <sourceFiles>build/out/lava-job-info</sourceFiles>
159+ <excludes/>
160+ <removePrefix>build/out</removePrefix>
161+ <remoteDirectorySDF>false</remoteDirectorySDF>
162+ <flatten>false</flatten>
163+ <cleanRemote>false</cleanRemote>
164+ <execCommand/>
165+ <execTimeout>120000</execTimeout>
166+ <usePty>false</usePty>
167+ </jenkins.plugins.publish__over__ssh.BapSshTransfer>
168+ </transfers>
169+ <useWorkspaceInPromotion>false</useWorkspaceInPromotion>
170+ <usePromotionTimestamp>false</usePromotionTimestamp>
171+ </jenkins.plugins.publish__over__ssh.BapSshPublisher>
172+ </publishers>
173+ <continueOnError>false</continueOnError>
174+ <failOnError>false</failOnError>
175+ <alwaysPublishFromMaster>true</alwaysPublishFromMaster>
176+ <hostConfigurationAccess class="jenkins.plugins.publish_over_ssh.BapSshPublisherPlugin" reference="../.."/>
177+ </delegate>
178+ </delegate>
179+ </jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin>
180+ <jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin>
181+ <delegate>
182+ <consolePrefix>SSH: </consolePrefix>
183+ <delegate>
184+ <publishers>
185+ <jenkins.plugins.publish__over__ssh.BapSshPublisher>
186+ <configName>snapshots.linaro.org file-move new</configName>
187+ <verbose>false</verbose>
188+ <transfers>
189+ <jenkins.plugins.publish__over__ssh.BapSshTransfer>
190+ <remoteDirectory/>
191+ <sourceFiles/>
192+ <excludes/>
193+ <removePrefix/>
194+ <remoteDirectorySDF>false</remoteDirectorySDF>
195+ <flatten>false</flatten>
196+ <cleanRemote>false</cleanRemote>
197+ <execCommand>reshuffle-files -t android -j ${{JOB_NAME}} -n ${{BUILD_NUMBER}} -m</execCommand>
198+ <execTimeout>120000</execTimeout>
199+ <usePty>false</usePty>
200+ </jenkins.plugins.publish__over__ssh.BapSshTransfer>
201+ </transfers>
202+ <useWorkspaceInPromotion>false</useWorkspaceInPromotion>
203+ <usePromotionTimestamp>false</usePromotionTimestamp>
204+ </jenkins.plugins.publish__over__ssh.BapSshPublisher>
205+ </publishers>
206+ <continueOnError>false</continueOnError>
207+ <failOnError>false</failOnError>
208+ <alwaysPublishFromMaster>true</alwaysPublishFromMaster>
209+ <hostConfigurationAccess class="jenkins.plugins.publish_over_ssh.BapSshPublisherPlugin" reference="../.."/>
210+ </delegate>
211+ </delegate>
212+ </jenkins.plugins.publish__over__ssh.BapSshBuilderPlugin>
213+ <hudson.tasks.Shell>
214+ <command>echo "Build finished"
215+</command>
216+ </hudson.tasks.Shell>
217+ </builders>
218+ <publishers>
219+ <hudson.tasks.ArtifactArchiver>
220+ <artifacts>build/out/*.xml</artifacts>
221+ <latestOnly>false</latestOnly>
222+ </hudson.tasks.ArtifactArchiver>
223+ <hudson.plugins.logparser.LogParserPublisher>
224+ <unstableOnWarning>false</unstableOnWarning>
225+ <failBuildOnError>false</failBuildOnError>
226+ <parsingRulesPath>/var/lib/jenkins/userContent/android.parse</parsingRulesPath>
227+ </hudson.plugins.logparser.LogParserPublisher>
228+ <hudson.tasks.Fingerprinter>
229+ <targets>build/fingerprints/*</targets>
230+ <recordBuildArtifacts>false</recordBuildArtifacts>
231+ </hudson.tasks.Fingerprinter>
232+ </publishers>
233+ <buildWrappers>
234+ <hudson.plugins.build__timeout.BuildTimeoutWrapper>
235+ <timeoutMinutes>180</timeoutMinutes>
236+ <failBuild>false</failBuild>
237+ </hudson.plugins.build__timeout.BuildTimeoutWrapper>
238+ <com.michelin.cio.hudson.plugins.copytoslave.CopyToSlaveBuildWrapper>
239+ <includes>**/*</includes>
240+ <excludes/>
241+ <flatten>false</flatten>
242+ <includeAntExcludes>false</includeAntExcludes>
243+ <hudsonHomeRelative>false</hudsonHomeRelative>
244+ <relativeTo>copyToSlave</relativeTo>
245+ </com.michelin.cio.hudson.plugins.copytoslave.CopyToSlaveBuildWrapper>
246+ </buildWrappers>
247+</project>
248\ No newline at end of file
249
250=== added file 'config_template/jenkins-jobs/IntegrationLoop.xml'
251--- config_template/jenkins-jobs/IntegrationLoop.xml 1970-01-01 00:00:00 +0000
252+++ config_template/jenkins-jobs/IntegrationLoop.xml 2012-08-24 13:08:33 +0000
253@@ -0,0 +1,21 @@
254+<?xml version='1.0' encoding='UTF-8'?>
255+<project>
256+ <actions/>
257+ <description></description>
258+ <keepDependencies>false</keepDependencies>
259+ <properties/>
260+ <scm class="hudson.scm.NullSCM"/>
261+ <canRoam>true</canRoam>
262+ <disabled>false</disabled>
263+ <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
264+ <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
265+ <triggers class="vector"/>
266+ <concurrentBuild>false</concurrentBuild>
267+ <builders>
268+ <hudson.tasks.Shell>
269+ <command>{build_script}</command>
270+ </hudson.tasks.Shell>
271+ </builders>
272+ <publishers/>
273+ <buildWrappers/>
274+</project>
275
276=== removed file 'config_template/jenkins-jobs/IntegrationLoop.xml'
277--- config_template/jenkins-jobs/IntegrationLoop.xml 2012-08-21 10:17:49 +0000
278+++ config_template/jenkins-jobs/IntegrationLoop.xml 1970-01-01 00:00:00 +0000
279@@ -1,21 +0,0 @@
280-<?xml version='1.0' encoding='UTF-8'?>
281-<project>
282- <actions/>
283- <description></description>
284- <keepDependencies>false</keepDependencies>
285- <properties/>
286- <scm class="hudson.scm.NullSCM"/>
287- <canRoam>true</canRoam>
288- <disabled>false</disabled>
289- <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>
290- <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
291- <triggers class="vector"/>
292- <concurrentBuild>false</concurrentBuild>
293- <builders>
294- <hudson.tasks.Shell>
295- <command>{build_script}</command>
296- </hudson.tasks.Shell>
297- </builders>
298- <publishers/>
299- <buildWrappers/>
300-</project>
301
302=== modified file 'dashboard/frontend/android_build/forms/android_loop_form.py'
303--- dashboard/frontend/android_build/forms/android_loop_form.py 2012-08-17 12:05:12 +0000
304+++ dashboard/frontend/android_build/forms/android_loop_form.py 2012-08-24 13:08:33 +0000
305@@ -26,7 +26,7 @@
306 class AndroidLoopForm(ModelForm):
307 class Meta:
308 model = AndroidLoop
309- exclude = ('server', 'config')
310+ exclude = ('server')
311
312 def clean(self):
313 cleaned_data = super(AndroidLoopForm, self).clean()
314
315=== modified file 'dashboard/frontend/android_build/models/android_loop.py'
316--- dashboard/frontend/android_build/models/android_loop.py 2012-08-22 12:19:55 +0000
317+++ dashboard/frontend/android_build/models/android_loop.py 2012-08-24 13:08:33 +0000
318@@ -55,10 +55,16 @@
319 DEFAULT_GCC = '4.4.0'
320 DEFAULT_IMAGE_SIZE = '2G'
321
322+ EXCLUDE_LIST = ['id', 'name', 'server', 'loop_ptr']
323+
324 # BUILD_TYPE
325- build_type = models.CharField(max_length=100, choices=ANDROID_BUILD_TYPES, verbose_name='Build type')
326+ build_type = models.CharField(max_length=100,
327+ choices=ANDROID_BUILD_TYPES,
328+ verbose_name='Build type')
329 # MANIFEST_REPO
330- manifest_repo = models.CharField(max_length=255, default=DEFAULT_REPO, verbose_name='Manifest repo')
331+ manifest_repo = models.CharField(max_length=255,
332+ default=DEFAULT_REPO,
333+ verbose_name='Manifest repo')
334 # MANIFEST_BRANCH
335 manifest_branch = models.CharField(max_length=50, default=DEFAULT_BRANCH)
336 # REPO_QUIET
337@@ -111,11 +117,11 @@
338
339 def save(self, *args, **kwargs):
340 self.server = JenkinsServer.objects.get(id=1)
341- super(AndroidLoop, self).save(*args, **kwargs)
342+ super(self.__class__, self).save(*args, **kwargs)
343 # We do not want a failure in remote server to affect the
344 # CI Loop flow.
345 try:
346- self.server.create_or_update_integration_loop(self)
347+ self.server.create_or_update_loop(self)
348 except:
349 # TODO: Log error.
350 pass
351@@ -123,5 +129,6 @@
352 def schedule_build(self):
353 # Every time before the build is scheduled, update the
354 # loop config on the remote server.
355- self.server.create_or_update_integration_loop(self)
356- super(AndroidLoop, self).schedule_build()
357+ parameters = {'parameter': [{'delay': '0'}]}
358+ self.server.create_or_update_loop(self)
359+ return super(self.__class__, self).schedule_build(parameters)
360
361=== modified file 'dashboard/frontend/android_build/templates/android_loop_detail.html'
362--- dashboard/frontend/android_build/templates/android_loop_detail.html 2012-08-20 09:10:05 +0000
363+++ dashboard/frontend/android_build/templates/android_loop_detail.html 2012-08-24 13:08:33 +0000
364@@ -38,7 +38,7 @@
365 {% block scripts %}
366 <script type="text/javascript">
367 $("#schedule_button").click(function() {
368- $.get("{% if request.is_secure %}https{% else %}http{% endif %}://{{ request.get_host }}/loop/build/{{ android_loop_detail.name }}/", function(data) {
369+ $.get("{% if request.is_secure %}https{% else %}http{% endif %}://{{ request.get_host }}/android/build/{{ android_loop_detail.name }}/", function(data) {
370 data = $.parseJSON(data);
371 $("<div/>", {
372 "class": "test",
373
374=== modified file 'dashboard/frontend/android_build/tests/test_android_build_clientresponse.py'
375--- dashboard/frontend/android_build/tests/test_android_build_clientresponse.py 2012-08-21 06:55:57 +0000
376+++ dashboard/frontend/android_build/tests/test_android_build_clientresponse.py 2012-08-24 13:08:33 +0000
377@@ -23,15 +23,15 @@
378
379 def setUp(self):
380 super(AndroidBuildClientResponseTests, self).setUp()
381- self.android_build_data = {"name" : "a-build",
382- "build_type" : "build-android",
383- "manifest_repo" : "repo",
384- "manifest_branch" : "branch",
385- "manifest_filename" : "filename",
386- "make_jobs" : "2",
387- "ramdisk_size" : "2",
388- "binutils_version" : "1",
389- "gcc_version" : "1"
390+ self.android_build_data = {"name": "a-build",
391+ "build_type": "build-android",
392+ "manifest_repo": "repo",
393+ "manifest_branch": "branch",
394+ "manifest_filename": "filename",
395+ "make_jobs": "2",
396+ "ramdisk_size": "2",
397+ "binutils_version": "1",
398+ "gcc_version": "1"
399 }
400
401 def test_android_build_create_get(self):
402@@ -91,4 +91,3 @@
403 self.assertEquals(response.status_code, 200)
404 self.assertIn("Details", response.content)
405 self.assertIn("Build type", response.content)
406-
407
408=== modified file 'dashboard/frontend/android_build/tests/test_android_build_views.py'
409--- dashboard/frontend/android_build/tests/test_android_build_views.py 2012-08-21 07:58:37 +0000
410+++ dashboard/frontend/android_build/tests/test_android_build_views.py 2012-08-24 13:08:33 +0000
411@@ -60,4 +60,4 @@
412 context = AndroidLoopUpdateView.get_context_data(
413 android_loop_update_view)
414 self.assertEqual(context['request'], "some request data")
415- self.assertEqual(context['form_name'], AndroidLoopUpdateView.form_name)
416\ No newline at end of file
417+ self.assertEqual(context['form_name'], AndroidLoopUpdateView.form_name)
418
419=== modified file 'dashboard/frontend/android_build/urls.py'
420--- dashboard/frontend/android_build/urls.py 2012-08-20 09:10:05 +0000
421+++ dashboard/frontend/android_build/urls.py 2012-08-24 13:08:33 +0000
422@@ -15,13 +15,15 @@
423 # You should have received a copy of the GNU Affero General Public License
424 # along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>.
425
426-from django.conf.urls.defaults import patterns, include, url
427-from frontend.android_build.views.android_loop_create_view\
428+from django.conf.urls.defaults import patterns, url
429+from frontend.android_build.views.android_loop_create_view \
430 import AndroidLoopCreateView
431-from frontend.android_build.views.android_loop_update_view\
432+from frontend.android_build.views.android_loop_update_view \
433 import AndroidLoopUpdateView
434-from frontend.android_build.views.android_loop_detail_view\
435+from frontend.android_build.views.android_loop_detail_view \
436 import AndroidLoopDetailView
437+from frontend.android_build.views.android_loop_build_view \
438+ import AndroidLoopBuildView
439
440
441 urlpatterns = patterns('',
442@@ -31,4 +33,6 @@
443 AndroidLoopDetailView.as_view(), name='AndroidLoopDetail'),
444 url(r'^android/update/(?P<slug>[\w|\W]+)/$',
445 AndroidLoopUpdateView.as_view(), name='AndroidLoopUpdate'),
446+ url(r'^android/build/(?P<slug>[\w|\W]+)/$',
447+ AndroidLoopBuildView.as_view(), name='AndroidLoopBuild'),
448 )
449
450=== added file 'dashboard/frontend/android_build/views/android_loop_build_view.py'
451--- dashboard/frontend/android_build/views/android_loop_build_view.py 1970-01-01 00:00:00 +0000
452+++ dashboard/frontend/android_build/views/android_loop_build_view.py 2012-08-24 13:08:33 +0000
453@@ -0,0 +1,25 @@
454+#!/usr/bin/env python
455+# Copyright (C) 2012 Linaro
456+#
457+# This file is part of linaro-ci-dashboard.
458+#
459+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
460+# it under the terms of the GNU Affero General Public License as published by
461+# the Free Software Foundation, either version 3 of the License, or
462+# (at your option) any later version.
463+#
464+# linaro-ci-dashboard is distributed in the hope that it will be useful,
465+# but WITHOUT ANY WARRANTY; without even the implied warranty of
466+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
467+# GNU Affero General Public License for more details.
468+#
469+# You should have received a copy of the GNU Affero General Public License
470+# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>.
471+
472+from frontend.android_build.models.android_loop import AndroidLoop
473+from frontend.views.loop_build_view import LoopBuildView
474+
475+
476+class AndroidLoopBuildView(LoopBuildView):
477+
478+ model = AndroidLoop
479
480=== modified file 'dashboard/frontend/android_build/views/android_loop_detail_view.py'
481--- dashboard/frontend/android_build/views/android_loop_detail_view.py 2012-08-20 16:00:44 +0000
482+++ dashboard/frontend/android_build/views/android_loop_detail_view.py 2012-08-24 13:08:33 +0000
483@@ -19,9 +19,11 @@
484 from django.contrib.auth.decorators import login_required
485 from django.views.generic.detail import DetailView
486 from django.utils.decorators import method_decorator
487-from frontend.android_build.models.android_loop \
488- import AndroidLoop
489+from frontend.models.loop import Loop
490+from frontend.android_build.models.android_loop import AndroidLoop
491 from django.core import serializers
492+from itertools import chain
493+
494
495 class AndroidLoopDetailView(DetailView):
496
497@@ -40,6 +42,11 @@
498 self).get_context_data(**kwargs)
499 context['request'] = self.request
500 context['builds'] = self.object.loop_ptr.loopbuild_set.all()
501- context['data'] = serializers.serialize( "python",
502- AndroidLoop.objects.filter(pk=self.object.loop_ptr.id) )
503+
504+ # Need to "merge" the two queryset for all the fields to appear
505+ querysets = list(chain(AndroidLoop.objects.filter(
506+ pk=self.object.loop_ptr.id),
507+ Loop.objects.filter(pk=self.object.loop_ptr.id)))
508+ context['data'] = serializers.serialize("python", querysets)
509+
510 return context
511
512=== modified file 'dashboard/frontend/integration_loop/forms/integration_loop_form.py'
513--- dashboard/frontend/integration_loop/forms/integration_loop_form.py 2012-08-17 11:12:49 +0000
514+++ dashboard/frontend/integration_loop/forms/integration_loop_form.py 2012-08-24 13:08:33 +0000
515@@ -23,4 +23,4 @@
516 class IntegrationLoopForm(ModelForm):
517 class Meta:
518 model = IntegrationLoop
519- exclude = ('server', 'config')
520+ exclude = ('server')
521
522=== modified file 'dashboard/frontend/integration_loop/models/integration_loop.py'
523--- dashboard/frontend/integration_loop/models/integration_loop.py 2012-08-17 09:43:07 +0000
524+++ dashboard/frontend/integration_loop/models/integration_loop.py 2012-08-24 13:08:33 +0000
525@@ -29,22 +29,19 @@
526 precommand = models.CharField(max_length=200)
527 command = models.CharField(max_length=200)
528
529+ EXCLUDE_LIST = ['id', 'name', 'server', 'loop_ptr']
530+
531 def save(self, *args, **kwargs):
532 self.server = JenkinsServer.objects.get(id=1)
533- super(IntegrationLoop, self).save(*args, **kwargs)
534+ super(self.__class__, self).save(*args, **kwargs)
535 # We do not want a failure in remote server to affect the
536 # CI Loop flow.
537 try:
538- self.server.create_or_update_integration_loop(self)
539+ self.server.create_or_update_loop(self)
540 except:
541 # TODO: Log error.
542 pass
543
544- def schedule_build(self):
545- # Every time before the build is scheduled, update the
546- # loop config on the remote server.
547- self.server.create_or_update_integration_loop(self)
548- super(IntegrationLoop, self).schedule_build()
549-
550 def __repr__(self):
551- return "IntegrationLoop(id=%d, branch=%s, ...)" % (self.id, self.branch)
552+ return "IntegrationLoop(id=%d, branch=%s, ...)" % (self.id,
553+ self.branch)
554
555=== modified file 'dashboard/frontend/integration_loop/templates/integration_loop_detail.html'
556--- dashboard/frontend/integration_loop/templates/integration_loop_detail.html 2012-08-16 15:12:10 +0000
557+++ dashboard/frontend/integration_loop/templates/integration_loop_detail.html 2012-08-24 13:08:33 +0000
558@@ -48,7 +48,7 @@
559 text: "Build: " + data[0].fields.build_number + " || " +
560 data[0].fields.status + " || " +
561 data[0].fields.duration + " || " +
562- data[0].fields.remote_number,
563+ data[0].fields.remote_number
564 })
565 .prependTo($("#builds"));
566 });
567
568=== modified file 'dashboard/frontend/kernel_build/models/kernel_loop.py'
569--- dashboard/frontend/kernel_build/models/kernel_loop.py 2012-08-23 12:27:08 +0000
570+++ dashboard/frontend/kernel_build/models/kernel_loop.py 2012-08-24 13:08:33 +0000
571@@ -118,7 +118,7 @@
572 # We do not want a failure in remote server to affect the
573 # CI Loop flow.
574 try:
575- self.server.create_or_update_integration_loop(self)
576+ self.server.create_or_update_loop(self)
577 except:
578 # TODO: Log error.
579 pass
580@@ -126,5 +126,5 @@
581 def schedule_build(self):
582 # Every time before the build is scheduled, update the
583 # loop config on the remote server.
584- self.server.create_or_update_integration_loop(self)
585+ self.server.create_or_update_loop(self)
586 super(KernelLoop, self).schedule_build()
587
588=== added file 'dashboard/frontend/migrations/0002_auto__add_field_loop_is_restricted__add_field_loop_is_official.py'
589--- dashboard/frontend/migrations/0002_auto__add_field_loop_is_restricted__add_field_loop_is_official.py 1970-01-01 00:00:00 +0000
590+++ dashboard/frontend/migrations/0002_auto__add_field_loop_is_restricted__add_field_loop_is_official.py 2012-08-24 13:08:33 +0000
591@@ -0,0 +1,54 @@
592+# encoding: utf-8
593+import datetime
594+from south.db import db
595+from south.v2 import SchemaMigration
596+from django.db import models
597+
598+class Migration(SchemaMigration):
599+
600+ def forwards(self, orm):
601+
602+ # Adding field 'Loop.is_restricted'
603+ db.add_column('frontend_loop', 'is_restricted', self.gf('django.db.models.fields.BooleanField')(default=False), keep_default=False)
604+
605+ # Adding field 'Loop.is_official'
606+ db.add_column('frontend_loop', 'is_official', self.gf('django.db.models.fields.BooleanField')(default=False), keep_default=False)
607+
608+
609+ def backwards(self, orm):
610+
611+ # Deleting field 'Loop.is_restricted'
612+ db.delete_column('frontend_loop', 'is_restricted')
613+
614+ # Deleting field 'Loop.is_official'
615+ db.delete_column('frontend_loop', 'is_official')
616+
617+
618+ models = {
619+ 'frontend.loop': {
620+ 'Meta': {'object_name': 'Loop'},
621+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
622+ 'is_official': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
623+ 'is_restricted': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
624+ 'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '200'}),
625+ 'server': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['jenkinsserver.JenkinsServer']"})
626+ },
627+ 'frontend.loopbuild': {
628+ 'Meta': {'ordering': "['-id']", 'object_name': 'LoopBuild'},
629+ 'build_number': ('django.db.models.fields.IntegerField', [], {'default': 'None'}),
630+ 'duration': ('django.db.models.fields.DecimalField', [], {'max_digits': '8', 'decimal_places': '2'}),
631+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
632+ 'loop': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['frontend.Loop']"}),
633+ 'remote_number': ('django.db.models.fields.IntegerField', [], {'null': 'True', 'blank': 'True'}),
634+ 'status': ('django.db.models.fields.CharField', [], {'max_length': '20'})
635+ },
636+ 'jenkinsserver.jenkinsserver': {
637+ 'Meta': {'object_name': 'JenkinsServer'},
638+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
639+ 'password': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
640+ 'url': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
641+ 'username': ('django.db.models.fields.CharField', [], {'max_length': '100'})
642+ }
643+ }
644+
645+ complete_apps = ['frontend']
646
647=== modified file 'dashboard/frontend/models/loop.py'
648--- dashboard/frontend/models/loop.py 2012-08-22 12:19:55 +0000
649+++ dashboard/frontend/models/loop.py 2012-08-24 13:08:33 +0000
650@@ -16,8 +16,10 @@
651 # You should have received a copy of the GNU Affero General Public License
652 # along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>.
653
654+import base64
655 from django.db import models
656 from jenkinsserver.models.jenkins_server import JenkinsServer
657+from lib.logger import Logger
658
659
660 class Loop(models.Model):
661@@ -26,8 +28,16 @@
662
663 name = models.CharField(max_length=200, unique=True)
664 server = models.ForeignKey(JenkinsServer)
665-
666- def schedule_build(self):
667+ is_restricted = models.BooleanField(default=False,
668+ verbose_name='Build is restricted')
669+ is_official = models.BooleanField(default=False,
670+ verbose_name='Build is official')
671+
672+ def __init__(self, *args, **kwargs):
673+ self.log = Logger.getClassLogger(self)
674+ super(Loop, self).__init__(*args, **kwargs)
675+
676+ def schedule_build(self, parameters=None):
677 '''Add a LoopBuild record for this Loop.
678
679 Also try to send a build job signal to the server, but if it fails,
680@@ -40,7 +50,8 @@
681 build.duration = 0.00
682 build.save()
683 try:
684- build.remote_number = self.server.schedule_job_build(self.name)
685+ build.remote_number = self.server.schedule_job_build(self.name,
686+ parameters)
687 build.save()
688 except:
689 # TODO: Log error.
690@@ -57,3 +68,13 @@
691
692 def __repr__(self):
693 return "%s(id=%d, ...)" % (self.__class__.__name__, self.id)
694+
695+ def base64_config(self):
696+ """ Return loop configuration as base64 encoded string.
697+ """
698+ text_config = u''
699+ for field in self._meta.fields:
700+ if not field.name in self.EXCLUDE_LIST:
701+ text_config += u'%s=%s\n' % (field.name.upper(),
702+ field.value_to_string(self))
703+ return base64.b64encode(text_config)
704
705=== modified file 'dashboard/frontend/urls.py'
706--- dashboard/frontend/urls.py 2012-08-16 15:12:10 +0000
707+++ dashboard/frontend/urls.py 2012-08-24 13:08:33 +0000
708@@ -16,7 +16,7 @@
709 # You should have received a copy of the GNU Affero General Public License
710 # along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>.
711
712-from django.conf.urls.defaults import patterns, include, url
713+from django.conf.urls.defaults import patterns, url
714 from frontend.views.home_page_view import HomePageView
715 from frontend.views.index_view import IndexView
716 from frontend.views.loop_build_view import LoopBuildView
717
718=== modified file 'dashboard/jenkinsserver/models/jenkins_server.py'
719--- dashboard/jenkinsserver/models/jenkins_server.py 2012-08-22 12:19:55 +0000
720+++ dashboard/jenkinsserver/models/jenkins_server.py 2012-08-24 13:08:33 +0000
721@@ -18,13 +18,13 @@
722
723 from django.db import models
724 from django.conf import settings
725-from xml.dom.minidom import parse, parseString
726 import jenkins
727 import json
728 import urllib2
729 from lib.logger import Logger
730 from lib.template import TextTemplate, XMLTemplate
731
732+
733 class JenkinsServer(models.Model):
734 class Meta:
735 app_label = 'jenkinsserver'
736@@ -40,7 +40,7 @@
737 self.username.encode('utf-8'),
738 self.password.encode('utf-8'))
739
740- def create_or_update_integration_loop(self, loop):
741+ def create_or_update_loop(self, loop):
742 """ Create integration loop if it doesn't exist on Jenkins.
743 Otherwise, update the existing one with new XML."""
744
745@@ -100,22 +100,32 @@
746
747 def create_job_xml(self, loop):
748 "Prepare the config.xml to be used for the job."
749- # Use loop's class name as the template name - JenkinsServer
750- # still doesn't have any internal knowledge about loop
751+ # Get generic serialization data.
752+ loop_params = loop.json()
753+
754 template_name = loop.__class__.__name__
755- # Get generic serialization data - Loop still doesn't know
756- # about Jenkins
757- loop_params = loop.json()
758- build_script = TextTemplate(settings.BUILD_SCRIPT_TEMPLATE \
759- % template_name).render(loop_params)
760- job_params = {"build_script": build_script}
761- job_params.update(loop_params)
762+
763+ # We need to handle the CONFIG parameter as a base64 encoded string.
764+ # At the moment used only by Android builds.
765+ loop_params['base64_config'] = loop.base64_config()
766+
767+ # Check if the build is restricted, assign appropriate nodes.
768+ if loop.is_restricted:
769+ loop_params['assigned_node'] = 'private &amp;&amp; natty'
770+ else:
771+ loop_params['assigned_node'] = 'ec2'
772+
773+ build_script = TextTemplate(settings.BUILD_SCRIPT_TEMPLATE\
774+ % template_name).render(loop_params)
775+ loop_params['build_script'] = build_script
776+
777 xml = XMLTemplate(settings.JENKINS_JOB_TEMPLATE \
778- % template_name).render(job_params)
779+ % template_name).render(loop_params)
780+
781 return xml
782
783- def schedule_job_build(self, jobname):
784+ def schedule_job_build(self, jobname, parameters=None):
785 """Trigger build job on jenkins. Return the next build number."""
786- self.jenkins.build_job(jobname)
787+ self.jenkins.build_job(jobname, parameters)
788 job_info = self.jenkins.get_job_info(jobname)
789 return job_info["nextBuildNumber"]
790
791=== modified file 'dashboard/jenkinsserver/tests/test_jenkins_server.py'
792--- dashboard/jenkinsserver/tests/test_jenkins_server.py 2012-08-21 17:47:38 +0000
793+++ dashboard/jenkinsserver/tests/test_jenkins_server.py 2012-08-24 13:08:33 +0000
794@@ -54,7 +54,6 @@
795 loop.branch = 'lp:linaro-ci-dashboard'
796 loop.precommand = 'cd %s' % loop.branch
797 loop.command = 'make syncdb; make test'
798- jbranch = 'bzr branch lp:%s dest_dir' % loop.branch
799 dest_dir = 'dest_dir'
800 env_cmds = "export USER=`whoami`;\nexport HOME=/home/$USER"
801
802@@ -85,8 +84,6 @@
803 loop.branch = 'linaro-ci-dashboard'
804 loop.precommand = 'cd %s' % loop.branch
805 loop.command = 'make syncdb; make test'
806- jbranch = 'bzr branch lp:%s' % loop.branch
807- jcommands = '\n'.join([jbranch, loop.precommand, loop.command])
808 jxml = self.server.create_job_xml(loop)
809 self.server.create_job(loop.name, jxml)
810 self.assertTrue(self.test_jenkins.job_exists(loop.name) == True)
811
812=== added file 'dashboard/lib/template.py'
813--- dashboard/lib/template.py 1970-01-01 00:00:00 +0000
814+++ dashboard/lib/template.py 2012-08-24 13:08:33 +0000
815@@ -0,0 +1,39 @@
816+# Copyright (C) 2012 Linaro
817+#
818+# This file is part of linaro-ci-dashboard.
819+#
820+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
821+# it under the terms of the GNU Affero General Public License as published by
822+# the Free Software Foundation, either version 3 of the License, or
823+# (at your option) any later version.
824+#
825+# linaro-ci-dashboard is distributed in the hope that it will be useful,
826+# but WITHOUT ANY WARRANTY; without even the implied warranty of
827+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
828+# GNU Affero General Public License for more details.
829+#
830+# You should have received a copy of the GNU Affero General Public License
831+# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>.
832+# Django settings for dashboard project.
833+
834+class AbstractTemplate(object):
835+ "Template class interface."
836+
837+ def render(self, dict):
838+ pass
839+
840+
841+class TextTemplate(AbstractTemplate):
842+
843+ def __init__(self, filename):
844+ f = open(filename)
845+ self.tmpl = f.read()
846+ f.close()
847+
848+ def render(self, dict):
849+ return self.tmpl.format(**dict)
850+
851+
852+class XMLTemplate(TextTemplate):
853+ """Initial XML templating implementation is based in plain text
854+ templates, TODO: fix this!"""
855
856=== removed file 'dashboard/lib/template.py'
857--- dashboard/lib/template.py 2012-08-22 12:19:55 +0000
858+++ dashboard/lib/template.py 1970-01-01 00:00:00 +0000
859@@ -1,39 +0,0 @@
860-# Copyright (C) 2012 Linaro
861-#
862-# This file is part of linaro-ci-dashboard.
863-#
864-# linaro-ci-dashboard is free software: you can redistribute it and/or modify
865-# it under the terms of the GNU Affero General Public License as published by
866-# the Free Software Foundation, either version 3 of the License, or
867-# (at your option) any later version.
868-#
869-# linaro-ci-dashboard is distributed in the hope that it will be useful,
870-# but WITHOUT ANY WARRANTY; without even the implied warranty of
871-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
872-# GNU Affero General Public License for more details.
873-#
874-# You should have received a copy of the GNU Affero General Public License
875-# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>.
876-# Django settings for dashboard project.
877-
878-class AbstractTemplate(object):
879- "Template class interface."
880-
881- def render(self, dict):
882- pass
883-
884-
885-class TextTemplate(AbstractTemplate):
886-
887- def __init__(self, filename):
888- f = open(filename)
889- self.tmpl = f.read()
890- f.close()
891-
892- def render(self, dict):
893- return self.tmpl.format(**dict)
894-
895-
896-class XMLTemplate(TextTemplate):
897- """Initial XML templating implementation is based in plain text
898- templates, TODO: fix this!"""

Subscribers

People subscribed via source and target branches