Merge ~sbaldassin/bileto:ust-bileto into bileto:master

Proposed by Santiago Baldassin
Status: Rejected
Rejected by: Robert Bruce Park
Proposed branch: ~sbaldassin/bileto:ust-bileto
Merge into: bileto:master
Diff against target: 856 lines (+742/-2)
7 files modified
.bzr-builddeb/default.conf (+2/-0)
bileto/models.py (+14/-1)
bileto/static/index.html (+285/-0)
bileto/static/style.css (+384/-0)
britney/iterate.py (+30/-1)
db_migrations.sh (+1/-0)
scripts/import.sh (+26/-0)
Reviewer Review Type Date Requested Status
Robert Bruce Park (community) Needs Information
Review via email: mp+306251@code.launchpad.net

Commit message

Adding ubuntu system tests to bileto

Description of the change

This change adds ubuntu system tests to bileto.

In order to deploy this, you'll need the qakit module available in the canonical-platform-qa ppa

To post a comment you must log in.
Revision history for this message
Robert Bruce Park (robru) wrote :

Thanks for rebasing your works Santiago, apologies again for the git history situation. Some comments inline.

Before you fix any of the issues I raise in my review, though, we need to consider that Steve and Martin both want this triggered from Britney, so that it can work again during proposed-migration.

Also, during the meeting, we agreed that we would transition this in slowly, such that at first we just trigger the tests without reporting them anywhere, so that results can be manually inspected. Today I'll be digging into the britney code to figure out the best way to trigger this.

review: Needs Information
Revision history for this message
Robert Bruce Park (robru) wrote :

Also, where is the code for qakit.ust.worker?

Thanks

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Thanks for your feedback Robert. I'll be working on your comments. In the meantime, here's the code for qakit: https://code.launchpad.net/qakit.

About triggering this from Britney during proposed migration, we would just need to re-use the code that I've added in bileto or we could just publish an amqp message (containing the ppa) to /ubuntu-system-tests. What we have to consider is that we want to avoid a message to be publish for every single package in ppa. From what I understood when I took a look at britney, a different amqp message is published for every single source package in the ppa and we need a single message for the whole ppa. The current worker is already listening amqp so I can give you a hand to test this if you want

Last but not least, I'd like to keep the reporting in bileto if you don't mind since it will help us to trace the results easily to our jenkins builds. Polling for results will generate a single http request which response is a 100 bytes message so the performance degradation in bileto should be minimum and I also make sure that if no response is received then an empty response is returned so that angularjs can parse it in any case

Revision history for this message
Robert Bruce Park (robru) wrote :

Ok, so I've had a chance to review the qakit source and I need to raise some concerns here. One thing is that the compute_ust_signoff as you've written it will be triggered every time the ticket is modified, for any reason. So it's true that the status job will update the ticket every 20 minutes, which will result in ust polling every 20 minutes, but it will also do a ust poll, eg, if a user edits a field on the ticket, and this is a really unacceptable slowdown for user experience, to do a syncronous poll and block the thread while the user is waiting for their change to save.

**IF** we go ahead with the idea of bileto polling ust results, we'll need to model it after britney/iterate.py, eg, there should be an independent script that runs from cron and then feeds the results into bileto. That way it would run much more consistently and performantly, without interfering with user edits of tickets.

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Good point Robert! There's no need to poll for results when the user is editing the form. Instead of using an independent script to get the results, I'd like to suggest doing it with an ajax call in the app.js which will be triggered every 2 minutes. That way users will have to wait only 2 minutes in the worst case scenario and we'll be using a non blocking asynchronous method. As long as we update the scope in the callback, angular will take care of reflecting that in the ui

Revision history for this message
Robert Bruce Park (robru) wrote :

Well the problem with doing it in app.js is that it then runs client side:
if two people are viewing the same ticket you'll get redundant polling of
results, and if nobody is viewing the ticket then the results never appear!
if app.js then attempts to save the result to the ticket, the audit log
will appear as if users are entering information manually, eg, the log
would show "sbalda: ust_signoff: approved" which would be quite strange. Is
this really the behavior you want?

I think really the only sensible approach is to model britney/iterate.py:
run it from cron periodically, query bileto for what tickets to submit for
testing, get results and save them to the ticket. Except for the first
phase we just want to submit requests and not poll or display results just
yet.

On Sep 21, 2016 6:59 AM, "Santiago Baldassin" <
<email address hidden>> wrote:

> Good point Robert! There's no need to poll for results when the user is
> editing the form. Instead of using an independent script to get the
> results, I'd like to suggest doing it with an ajax call in the app.js which
> will be triggered every 2 minutes. That way users will have to wait only 2
> minutes in the worst case scenario and we'll be using a non blocking
> asynchronous method. As long as we update the scope in the callback,
> angular will take care of reflecting that in the ui
> --
> https://code.launchpad.net/~sbaldassin/bileto/+git/bileto/+merge/306251
> You are reviewing the proposed merge of ~sbaldassin/bileto:ust-bileto into
> bileto:master.
>

Revision history for this message
Robert Bruce Park (robru) wrote :

Sorry, another question: Are you saying it's possible to trigger UST with an ajax request? Or is the ajax request only for querying the status after it's been triggered?

If the former, I'd strongly prefer that over importing this qakit module, which imports jenkins module. Even when we used jenkins extensively, bileto only ever spoke JSON to it, I never had to import jenkins module, so I'd prefer not to start using that now if it can be helped.

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Thanks Robert. I've addressed all your comments and replied to a couple of them inline. I've remove the polling results part for now and I'll be submitting a new mp for that

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

> Sorry, another question: Are you saying it's possible to trigger UST with an
> ajax request? Or is the ajax request only for querying the status after it's
> been triggered?
>
> If the former, I'd strongly prefer that over importing this qakit module,
> which imports jenkins module. Even when we used jenkins extensively, bileto
> only ever spoke JSON to it, I never had to import jenkins module, so I'd
> prefer not to start using that now if it can be helped.

I was thinking about an ajax request to get the results. About importing qakit, I'd like to keep it if you guys don't mind. Please take into account that qakit is not only talking to Jenkins but also to practitest and to launchpad. The beauty of using the jenkins lib is that you forget about dealing with http requests, catching errors, packing/unpacking requests and results, etc. By the way, the jenkins module is a well known and widely use module, so I'm not sure why we should not use it, that said, If there's any technical roadblock related to it, I'm open to change it

Revision history for this message
Robert Bruce Park (robru) wrote :

Hey Santiago, apologies I've been meaning to dedicate more time to this but the bileto.ubuntu.com rollout was not particularly smooth so I've had a few fires to put out.

The single greatest challenge we will face is the fact that britney runs in precise, which limits us to python 3.2, which is really showing its age. It's not clear to me that qakit or jenkins modules are going to work in precise, and backporting them may well be harder than just writing a few http requests.

In terms of the bileto side, I like to keep things as simple as possible, so loading the big jenkins module in my server process is really unappealing to me (even if you resolve the issue of polling jenkins every time a user edits a ticket, I won't accept any code that imports jenkins in bileto/*.py) keep in mind that our flask server has 15 worker processes so anything imported in bileto/*.py is going to be imported in 15 different processes and stick around forever.

The significantly better approach is to just have one script called periodically from cron. It can load jenkins, iterate over tickets, submit some tests, and then exit without being a huge burden on the system for a prolonged period of time.

Revision history for this message
Santiago Baldassin (sbaldassin) wrote :

Hi Robert, no need to apologize. I'll be working on getting rid of Jenkins in qakit and let you know when it's ready.

~sbaldassin/bileto:ust-bileto updated
0c5bc46... by Santiago Baldassin

Moving the trigger for ust to iterate.py

3560ed2... by Santiago Baldassin

Merge branch 'master' of git+ssh://git.launchpad.net/bileto

Unmerged commits

0c5bc46... by Santiago Baldassin

Moving the trigger for ust to iterate.py

cbf2617... by Santiago Baldassin

Adding ubuntu system tests signoff

This includes an additional signoff for the requests in bileto.
Once a silo is assigned to the requests and the lander approved
the changes, ubuntu system tests will be triggered

3560ed2... by Santiago Baldassin

Merge branch 'master' of git+ssh://git.launchpad.net/bileto

5f56daa... by Santiago Baldassin

Merge branch 'master' of https://git.launchpad.net/bileto

3f02943... by Robert Bruce Park

Retry git fetch a few times before giving up.

83f2a02... by Robert Bruce Park

PPAs are no longer Assigned, they are Active.

e1975b4... by Robert Bruce Park

Add multiverse dependency as per cjwatson.

51950d6... by Robert Bruce Park

Create dir before copying into it.

7096cc8... by Robert Bruce Park

Increase authorized_size.

0ba81f4... by Robert Bruce Park

Ephemeral PPAs phase two: Use them for new tickets.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
diff --git a/.bzr-builddeb/default.conf b/.bzr-builddeb/default.conf
0new file mode 1006440new file mode 100644
index 0000000..6c96a98
--- /dev/null
+++ b/.bzr-builddeb/default.conf
@@ -0,0 +1,2 @@
1[BUILDDEB]
2split = True
diff --git a/bileto/models.py b/bileto/models.py
index 5ea297f..a3a5bd2 100644
--- a/bileto/models.py
+++ b/bileto/models.py
@@ -3,7 +3,6 @@
3This file defines the two DB schemas, Request and Comment, and various3This file defines the two DB schemas, Request and Comment, and various
4metadata related to those.4metadata related to those.
5"""5"""
6
7import re6import re
87
9from os import environ8from os import environ
@@ -12,6 +11,7 @@ from datetime import datetime
12from functools import lru_cache11from functools import lru_cache
13from fcntl import LOCK_EX, LOCK_NB, flock12from fcntl import LOCK_EX, LOCK_NB, flock
1413
14from qakit.ust.worker import Worker, UstResult
15from sqlalchemy import BigInteger, desc, text15from sqlalchemy import BigInteger, desc, text
16from sqlalchemy.inspection import inspect16from sqlalchemy.inspection import inspect
17from sqlalchemy.orm import relationship17from sqlalchemy.orm import relationship
@@ -53,6 +53,11 @@ BRITNEY_SIGNOFF = """Automated Signoff
53Indicates if britney has approved the ticket based on autopkgtests & other \53Indicates if britney has approved the ticket based on autopkgtests & other \
54factors. Check the excuses page for details."""54factors. Check the excuses page for details."""
5555
56UST_SIGNOFF = """Ubuntu System Tests Signoff
57
58Indicates if ubuntu system tests has approved the silo"""
59
60
56CREATOR = """Request Creator61CREATOR = """Request Creator
5762
58Launchpad nickname of the person who created this request."""63Launchpad nickname of the person who created this request."""
@@ -103,6 +108,10 @@ QA = """QA Signoff
103108
104Indicates whether or not the ticket has been approved by QA team."""109Indicates whether or not the ticket has been approved by QA team."""
105110
111UST = """UST Signoff
112
113Indicates whether or not the silo has been approved by Ubuntu system tests."""
114
106SERIES = """Target Series115SERIES = """Target Series
107116
108What series to build for, eg, wily, vivid, or yakkety+xenial+vivid."""117What series to build for, eg, wily, vivid, or yakkety+xenial+vivid."""
@@ -239,6 +248,8 @@ class Request(db.Model, BiletoModel):
239 dest = db.Column(db.Text, default=EMPTY, doc=PPA)248 dest = db.Column(db.Text, default=EMPTY, doc=PPA)
240 lander_signoff = db.Column(db.Text, default=EMPTY, doc=LANDER_SIGNOFF)249 lander_signoff = db.Column(db.Text, default=EMPTY, doc=LANDER_SIGNOFF)
241 britney_signoff = db.Column(db.Text, default=EMPTY, doc=BRITNEY_SIGNOFF)250 britney_signoff = db.Column(db.Text, default=EMPTY, doc=BRITNEY_SIGNOFF)
251 ust_signoff = db.Column(db.Text, default=EMPTY, doc=UST_SIGNOFF)
252 ust_build_url = db.Column(db.Text, default=EMPTY, doc=UST_SIGNOFF)
242 qa_signoff = db.Column(db.Text, default=EMPTY, doc=QA)253 qa_signoff = db.Column(db.Text, default=EMPTY, doc=QA)
243254
244 search = db.Column(db.Text, default=EMPTY, doc='Searchable Field')255 search = db.Column(db.Text, default=EMPTY, doc='Searchable Field')
@@ -329,6 +340,8 @@ class Request(db.Model, BiletoModel):
329 if lander_fail or any(clear in status for clear in CLEAR):340 if lander_fail or any(clear in status for clear in CLEAR):
330 self.lander_signoff = EMPTY341 self.lander_signoff = EMPTY
331 self.britney_signoff = EMPTY342 self.britney_signoff = EMPTY
343 self.ust_signoff = EMPTY
344 self.ust_build_url = EMPTY
332 self.qa_signoff = 'Required'345 self.qa_signoff = 'Required'
333 self.autopkgtest = EMPTY346 self.autopkgtest = EMPTY
334347
diff --git a/bileto/static/index.html b/bileto/static/index.html
335new file mode 100644348new file mode 100644
index 0000000..9422e18
--- /dev/null
+++ b/bileto/static/index.html
@@ -0,0 +1,285 @@
1<!DOCTYPE html>
2<html ng-app="Bileto" ng-controller="appController">
3<head>
4<meta charset="utf-8" />
5<title>Bileto Landing Tickets</title>
6<link rel="stylesheet" type="text/css" href="/static/style.css">
7<link rel="icon" type="image/gif" href="/static/goldenticket.png">
8<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.0/angular.min.js"></script>
9<script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.5.0/angular-sanitize.min.js"></script>
10<script src="/static/app.js" charset="utf-8"></script>
11</head>
12<body>
13
14<datalist ng-repeat="(list, items) in meta.datalists" id="{{list}}">
15<option ng-repeat="item in items" value="{{item}}" />
16</datalist>
17
18<div id="nav">
19<div id="login">
20<a href="/logout" ng-if="data.nickname">Log out {{data.nickname}}</a>
21<form name="sso_login" action="/login" method=post ng-if="!data.nickname">
22<input type=submit value="Log in with Ubuntu SSO">
23<input type=hidden name=next value="/#{{here}}">
24</form>
25</div>
26<a href="#/create" ng-if="data.nickname">Create New Request</a>
27<a href="#" ng-click="editing = null">All Requests</a>
28<a href="#/silo/landing" ng-click="editing = null">{{data.assigned}} Active</a>
29<a href="#/user/{{data.ircnick}}" ng-click="editing = null" ng-if="data.ircnick">Mine</a>
30<a href="#/failures" ng-click="editing = null">Failures</a>
31<a href="#/tickets?active_only&amp;lander_signoff=Approved" ng-click="editing = null">Lander Approved</a>
32<a href="#/tickets?active_only&amp;qa_signoff=Ready" ng-click="editing = null">Ready for QA</a>
33<a href="#/publishable" ng-click="editing = null">Publishable</a>
34<a href="#/tickets?status=queue" ng-click="editing = null">Queued</a>
35<a href="#/tickets?status=pocket" ng-click="editing = null">Migrating</a>
36<a href="#/tickets?status=Landed" ng-click="editing = null">Landed</a>
37<a href="#/tickets?status=Abandoned" ng-click="editing = null">Abandoned</a>
38<form name="live_search">
39<input type="search" class="search" placeholder="Search..." ng-model="searchstring" />
40<input type="submit" ng-click="search(searchstring)" value="Search All Requests" />
41<input type="submit" ng-click="search(searchstring, true)" value="Search Active Requests" />
42<input type="submit" ng-click="siloname(searchstring)" value="Show PPA #" />
43<input type="submit" ng-click="request(searchstring)" value="Show Ticket #" />
44</form>
45</div>
46
47<div id="body">
48
49<div id="error" ng-if="error">
50<h2>{{error.header}}</h2>
51<img src="/static/error.gif">
52<p>{{error.message}}</p>
53</div>
54
55<div ng-if="data.prev" class="center">
56<a ng-click="prev()">... newer ...</a>
57</div>
58
59<div id="requests">
60<div class="req" ng-repeat="req in data.requests" ng-class="{card: !expand}">
61
62<div class="summary" ng-if="!expand" ng-click="request(req.request_id)">
63<p><span class="{{req.status_color}}" title="{{interpret_color(req.status_color)}}">&nbsp;</span>
64<span class="signoff_{{req.lander_signoff}}" title="Lander {{req.lander_signoff}}">&nbsp;</span>
65<span class="signoff_{{req.britney_signoff}}" title="Britney {{req.britney_signoff}}">&nbsp;</span>
66<span class="signoff_{{req.ust_signoff}}" title="UST {{req.britney_signoff}}">&nbsp;</span>
67<span class="signoff_{{req.qa_signoff}}" title="QA {{req.qa_signoff}}">&nbsp;</span></p>
68<p><a href="#/ticket/{{req.request_id}}">{{req.request_id}}.&nbsp;{{req.date | date:'yyyy-MM-dd'}}&nbsp;{{req.siloname}}</a></p>
69<p>{{req.sources}}</p>
70<p ng-bind-html="linkify(req.description)"></p>
71<p>{{req.landers}}</p>
72</div>
73
74<div class="expanded" ng-if="expand">
75<div class="display" ng-if="!editing">
76<form name="live_edit">
77<table class="details">
78
79<tr>
80<th class="signoff">
81<a target="_blank" href="{{req.job_log}}" ng-if="req.job_log" class="status_details" title="Click for Details">&#187; {{title(meta.docs.status)}}</a>
82<span ng-if="!req.job_log">{{title(meta.docs.status)}}</span>
83</th>
84<td ng-bind-html="display_status(req.status)" class="{{req.status_color}}"></td>
85</tr>
86
87<tr>
88<th class="signoff">{{title(meta.docs.lander_signoff)}}</th>
89<td class="signoff signoff_{{req.lander_signoff}}">
90<select class="signoff signoff_{{req.lander_signoff}}" name="lander_signoff" title="{{meta.docs.lander_signoff}}" ng-change="submit(live_edit, req)" ng-model="req.lander_signoff" ng-options="value for value in meta.datalists.lander_signoff" ng-disabled="!data.teams.length"></select>
91</td>
92</tr>
93
94<tr>
95<th class="signoff">{{title(meta.docs.britney_signoff)}}</th>
96<td class="signoff_{{req.britney_signoff}}">{{req.britney_signoff}}</td>
97</tr>
98
99<tr>
100<th class="signoff">{{title(meta.docs.ust_signoff)}}</th>
101<td>
102<a href="{{req.ust_build_url}}">
103<div class="signoff_{{req.ust_signoff}}">
104{{req.ust_signoff}}
105</div>
106</a>
107</td>
108</tr>
109
110<tr>
111<th class="signoff">{{title(meta.docs.qa_signoff)}}</th>
112<td class="signoff signoff_{{req.qa_signoff}}">
113<select class="signoff signoff_{{req.qa_signoff}}" name="qa_signoff" title="{{meta.docs.qa_signoff}}" ng-change="submit(live_edit, req)" ng-model="req.qa_signoff" ng-options="value for value in meta.datalists.qa_signoff" ng-disabled="!data.admin.length && !req.download_links"></select>
114</td>
115</tr>
116
117<tr>
118<th>{{title(meta.docs.artifacts)}}</th>
119<td ng-bind-html="linkify(req.artifacts)"></td>
120</tr>
121
122<tr ng-if="req.autopkgtest">
123<th>{{title(meta.docs.autopkgtest)}}</th>
124<td ng-bind-html="linkify(req.autopkgtest)"></td>
125</tr>
126
127<tr ng-repeat="field in long_fields">
128<th>{{title(meta.docs[field])}}</th>
129<td ng-bind-html="linkify(req[field])"></td>
130</tr>
131
132<tr>
133<th>{{title(meta.docs.siloname)}}</th>
134<td ng-bind-html="display_siloname(meta.ppa_team, req.siloname)"></td>
135</tr>
136
137<!-- FIXME: sync_request is bit-rotty and doesn't work. disabled for now.
138<tr>
139<th>{{title(meta.docs.sync_request)}}</th>
140<td>{{req.sync_request}}</td>
141</tr>
142-->
143
144<tr>
145<th>{{title(meta.docs.sources)}}</th>
146<td class="sources" ng-bind-html="display_sources(req.sources)"></td>
147</tr>
148
149<tr ng-repeat="field in dropdowns">
150<th>{{title(meta.docs[field])}}</th>
151<td>{{req[field]}}</td>
152</tr>
153
154<tr ng-if="req.series.indexOf('+') === -1">
155<th>{{title(meta.docs.dest)}}</th>
156<td ng-bind-html="display_dest(req.dest)"></td>
157</tr>
158
159<tr>
160<th>{{title(meta.docs.landers)}}</th>
161<td ng-bind-html="display_landers(req.landers)"></td>
162</tr>
163
164<tr>
165<th><a href="#/ticket/{{req.request_id}}">{{title(meta.docs.request_id)}}</a></th>
166<td><a href="#/ticket/{{req.request_id}}">
167<span ng-bind-html="easter_egg(req.request_id)"></span>
168{{req.request_id}}.&nbsp;{{req.date | date:'yyyy-MM-dd'}}
169</a></td>
170</tr>
171
172</table>
173</form>
174</div>
175</div>
176
177<div class="edit" ng-if="editing">
178<form name="edit_request">
179<p>
180<textarea rows="6" name="description"
181 placeholder="{{title(meta.docs.description)}}" title="{{meta.docs.description}}"
182 ng-model="req.description" required>{{req.description}}</textarea>
183</p>
184<p>
185<textarea rows="6" name="test_plan"
186 placeholder="{{title(meta.docs.test_plan)}}" title="{{meta.docs.test_plan}}"
187 ng-model="req.test_plan" required>{{req.test_plan}}</textarea>
188</p>
189<p>
190<textarea rows="6" wrap="off" name="merge_proposals"
191 placeholder="{{title(meta.docs.merge_proposals)}}" title="{{meta.docs.merge_proposals}}"
192 ng-model="req.merge_proposals" ng-pattern="/^((https?:\/\/code.launchpad.net\/~\S+\/\+merge\/\d+|#.*)[\s\n]*)*$/">{{req.merge_proposals}}</textarea>
193</p>
194<p>
195<textarea rows="6" wrap="off" name="download_links"
196 placeholder="{{title(meta.docs.download_links)}}" title="{{meta.docs.download_links}}"
197 ng-model="req.download_links" ng-pattern="/^(https?:\/\/\S+[\s\n]*)*$/">{{req.download_links}}</textarea>
198</p>
199<p>
200<input type="text" name="landers" list="landers" size="10"
201 placeholder="{{title(meta.docs.landers)}}" title="{{meta.docs.landers}}"
202 value="{{req.landers}}" ng-model="req.landers" />
203</p>
204<!-- FIXME: sync_request is bit-rotty and doesn't work. disabled for now.
205<p>
206<input type="text" name="sync_request" list="sync_request" size="10"
207 placeholder="{{title(meta.docs.sync_request)}}" title="{{meta.docs.sync_request}}"
208 value="{{req.sync_request}}" ng-model="req.sync_request"
209 ng-pattern="/^(\d+|\S+,[a-z0-9.]+)$/"/>
210</p>
211-->
212<p>
213<input type="text" name="sources" list="sources" size="10"
214 placeholder="{{title(meta.docs.sources)}}" title="{{meta.docs.sources}}"
215 value="{{req.sources}}" ng-model="req.sources" />
216</p>
217<p ng-repeat="field in dropdowns">
218<select name="{{field}}" title="{{meta.docs[field]}}" ng-model="req[field]" ng-options="value for value in meta.datalists[field]"></select>
219</p>
220<p ng-if="req.series.indexOf('+') === -1">
221<select name="dest" title="{{meta.docs.dest}}" ng-model="req.dest">
222<option value="">Ubuntu Archive</option>
223<option value="ci-train-ppa-service/ubuntu/stable-phone-overlay">Stable Overlay PPA</option>
224</select>
225</p>
226<p>
227<input type="hidden" name="distribution" value="ubuntu" ng-model="req.distribution" />
228<button ng-click="submit(edit_request, req)" ng-disabled="busy || !edit_request.$valid">
229<span ng-if="req.request_id">Save #{{req.request_id}}</span>
230<span ng-if="!req.request_id">Create New Request</span>
231<span ng-if="!edit_request.$valid">(mouse over pink fields for help)</span>
232</button>
233<div class="red" ng-if="display_sru_warning(req)">Warning: Are you sure you want to
234publish an SRU?</div>
235</p>
236</form>
237</div>
238
239<div class="actionbox" ng-if="req.request_id && expand">
240<table class="actions"><tr>
241<td ng-if="!editing"><a href="/log/{{req.request_id}}/build" target="_blank">Build</a></td>
242<td ng-if="req.siloname && !editing"><a href="/log/{{req.request_id}}/diff" target="_blank">Diff</a></td>
243<td ng-if="!editing"><a href="/log/{{req.request_id}}/revert" target="_blank">Revert</a></td>
244<td ng-if="req.siloname && !editing"><a href="/log/{{req.request_id}}/publish" target="_blank">Publish</a></td>
245<td ng-if="!editing"><a href="/log/{{req.request_id}}/finalize" target="_blank">Finalize</a></td>
246<td ng-if="!editing"><a href="/log/{{req.request_id}}/abandon" target="_blank">Abandon</a></td>
247<td ng-if="editing"><a ng-click="cancel()">Cancel Edit</a></td>
248<td ng-if="data.nickname && !editing"><a ng-click="edit(req)">Edit</a></td>
249</tr></table>
250</div>
251
252<div class="comments" ng-if="expand">
253<a ng-click="toggle_log()" ng-if="expand && data.requests[0].comments.length"><span ng-if="!full_log">&#9656;&nbsp;Show</span><span ng-if="full_log">&#9662;&nbsp;Hide</span> Audit Logs</a>
254<div class="comment" ng-repeat="comment in req.comments" ng-if="full_log || !comment.log_url">
255<strong>
256<a target="_blank" href="{{comment.log_url}}" ng-if="comment.log_url">{{comment.date | date:'yyyy-MM-dd HH:mm:ss Z'}} ({{comment.author}})</a>
257<span ng-if="!comment.log_url">{{comment.date | date:'yyyy-MM-dd HH:mm:ss Z'}} ({{comment.author}})</span>
258</strong>
259<span ng-bind-html="display_status(comment.text)"></span>
260</div>
261</div>
262
263<div class="new_comment" ng-if="req.request_id && data.nickname && expand && !editing">
264<form name="new_comment_form">
265<textarea class="commentbox" rows="2" name="comment_text" placeholder="Add a comment..." ng-model="new_comment.text"></textarea>
266<button ng-click="post_comment(req.request_id, new_comment.text)" ng-disabled="busy">Save Comment</button>
267</form>
268</div>
269
270</div>
271
272<div ng-if="data.next" class="center">
273<a ng-click="next()">... older ...</a>
274</div>
275
276<div id="admin">
277<a href="/static/logs.html">All Logs</a>
278<a href="/static/britney/index.html">Britney Files</a>
279<a href="/static/report.txt">Diagnostic Report</a>
280</div>
281
282<div id="busy" ng-if="busy"></div>
283
284</body>
285</html>
diff --git a/bileto/static/style.css b/bileto/static/style.css
0new file mode 100644286new file mode 100644
index 0000000..985e85f
--- /dev/null
+++ b/bileto/static/style.css
@@ -0,0 +1,384 @@
1html, body {
2 background-color: white;
3 color: black;
4 font-family: Ubuntu, Arial, sans-serif;
5 margin: 0;
6 padding: 0;
7}
8
9body {
10 padding-bottom: 1em;
11}
12
13input, select, button, textarea {
14 width: 100%;
15 font-size: medium;
16}
17
18a {
19 color: #dd4814;
20 background-color: white;
21}
22
23a, input[type=submit], select, button, .summary {
24 cursor: pointer;
25 text-decoration: none;
26 outline: 0;
27 margin: 0;
28 border: 0;
29 border-radius: 0;
30 font-size: medium;
31}
32
33#nav {
34 text-align: center;
35 width: auto;
36 float: right;
37}
38
39#body {
40 width: auto;
41 overflow: hidden;
42}
43
44form {
45 display: block;
46 margin: 0;
47 padding: 0;
48 width: 100%;
49}
50
51h1, #nav a, #nav input, .new_comment button, .actions a {
52 background-color: #dd4814;
53 border: 0;
54 color: white;
55 display: block;
56 margin: 0;
57 padding: 1em;
58 text-decoration: none;
59 text-align: center;
60}
61
62.status_details {
63 background-color: #dd4814;
64 color: white;
65 padding: 0.3em;
66}
67
68h1 {
69 font-size: large;
70 text-align: center;
71 width: auto;
72}
73
74th {
75 text-align: right;
76 vertical-align: top;
77 white-space: nowrap;
78 padding-right: 0.5em;
79}
80
81td {
82 vertical-align: top;
83 text-align: left;
84}
85
86#nav input, .new_comment input {
87 width: 100%;
88}
89
90#nav input[type=search] {
91 border-top: 1px solid white;
92}
93
94#nav a:hover, #nav input:hover, h1:hover,
95.edit input[type=submit]:hover, .edit button:hover,
96.new_comment input:hover, .new_comment button:hover,
97.actions a:hover {
98 text-decoration: none;
99 background-color: white;
100 color: #dd4814;
101}
102
103#requests, .display, .edit {
104 display: block;
105}
106
107#requests {
108 padding-top: 2em;
109}
110
111#requests > * {
112 vertical-align: top;
113}
114
115.req {
116 text-align: center;
117}
118
119.edit {
120 margin-top: -2em;
121}
122
123.summary {
124 padding: 0.5em 1em 1em 1em;
125 max-height: 20em;
126}
127
128.summary p {
129 margin: 0.5em;
130}
131
132.display {
133 padding: 1em;
134}
135
136.card {
137 width: 450px;
138 display: inline-block;
139 text-align: left;
140 border-top: 1px solid #ccc;
141 margin: 1em;
142 overflow: auto;
143}
144
145.card, .display {
146 margin-top: 0;
147 padding-top: 0;
148}
149
150.details {
151 width: auto;
152 margin: 0 auto;
153}
154
155.details td, .details th {
156 padding: 0.2em;
157}
158
159td.sources a {
160 display: block;
161}
162
163.center {
164 text-align: center;
165}
166
167input, textarea, button, select {
168 width: 100%;
169 display: block;
170 color: black;
171 background-color: #eee;
172 padding: 0.5em;
173 border: 0;
174 box-sizing: border-box;
175 -webkid-box-sizing: border-box;
176 -moz-box-sizing: border-box;
177}
178
179.edit input[type=submit], .edit button {
180 background-color: #dd4814;
181 color: white;
182 padding: 1em;
183 border: 0;
184}
185
186p {
187 margin: 1em 1em 0 1em;
188}
189
190.comments {
191 display: inline-block;
192 width: auto;
193 padding: 0 2em;
194 margin: 0 auto;
195 text-align: left;
196}
197
198.new_comment {
199 padding: 0;
200 margin: 1em;
201}
202
203table, tr, td {
204 border-collapse: collapse;
205 margin: 0;
206 padding: 0;
207}
208
209table {
210 width: 100%;
211}
212
213.commentbox {
214 width: 100%;
215}
216
217.actionbox {
218 margin: 1em;
219}
220
221table.actions {
222 table-layout: fixed;
223}
224
225table.actions tr, table.actions td {
226 height: 100%;
227 vertical-align: middle;
228}
229
230table.actions a {
231 display: block;
232 width: 100%;
233 height: 100%;
234 padding: 1em 0;
235 vertical-align: middle;
236 box-sizing: border-box;
237 -webkid-box-sizing: border-box;
238 -moz-box-sizing: border-box;
239}
240
241select.signoff {
242 width: auto;
243 display: inline-block;
244}
245
246/* These colors must be kept in sync with $scope.interpret_colors in app.js */
247.black::before, .signoff_::before, .signoff_Required::before,
248.red::before, .signoff_Failed::before,
249.orange::before, .signoff_Queued::before, .signoff_Running::before, .signoff_Ready::before,
250.purple::before,
251.green::before, .signoff_N\/A::before, .signoff_Approved::before,
252.olive::before,
253.blue::before,
254.grey::before {
255 font-size: xx-large;
256 line-height: 0;
257 display: inline-block;
258 text-align: center;
259 width: 1em;
260}
261
262.black, .signoff_, .signoff_Required {
263 color: black;
264}
265
266.black::before, .signoff_::before, .signoff_Required::before {
267 color: black;
268 content: "— ";
269}
270
271.red, .signoff_Failed {
272 color: red;
273 font-weight: bold;
274}
275
276.red::before, .signoff_Failed::before {
277 color: red;
278 content: "✘ ";
279}
280
281.orange, .signoff_Queued, .signoff_Running, .signoff_Ready {
282 color: orange;
283 font-weight: bold;
284}
285
286.orange::before, .signoff_Queued::before, .signoff_Running::before, .signoff_Ready::before {
287 color: orange;
288 content: "⚡ ";
289}
290
291.purple {
292 color: purple;
293}
294
295.purple::before {
296 color: purple;
297 content: "♨ ";
298}
299
300.green, .signoff_N\/A, .signoff_Approved {
301 color: green;
302}
303
304.green::before, .signoff_N\/A::before, .signoff_Approved::before {
305 color: green;
306 content: "✔ ";
307}
308
309.olive {
310 color: olive;
311 font-weight: bold;
312}
313
314.olive::before {
315 color: olive;
316 content: "✱ ";
317}
318
319.blue {
320 color: blue;
321}
322
323.blue::before {
324 color: blue;
325 content: "✔ ";
326}
327
328.grey {
329 color: grey;
330}
331
332.grey::before {
333 color: #888;
334 content: "◌ ";
335}
336
337#busy {
338 background-color: black;
339 color: black;
340 height: 100%;
341 left: 0;
342 margin: 0;
343 opacity: 0.3;
344 padding: 0;
345 position: absolute;
346 top: 0;
347 width: 100%;
348}
349
350.fifty {
351 width: 50%;
352 display: inline-block;
353}
354
355#error {
356 width: 100%;
357 text-align: center;
358}
359
360textarea.ng-invalid, input.ng-invalid {
361 background-color: #ffe5e5;
362}
363
364button[disabled], button[disabled]:hover, button[disabled]:focus, button[disabled]:active,
365select[disabled], select[disabled]:hover, select[disabled]:focus, select[disabled]:active {
366 color: #666;
367 background-color: #ccc;
368 cursor: default;
369}
370
371.signoff {
372 vertical-align: middle;
373}
374
375#admin {
376 margin: 3em 0 0 0;
377 text-align: center;
378 font-size: x-small;
379}
380
381#admin a {
382 margin: 1em;
383 font-size: x-small;
384}
diff --git a/britney/iterate.py b/britney/iterate.py
index edd9b35..08d3bae 100755
--- a/britney/iterate.py
+++ b/britney/iterate.py
@@ -20,6 +20,7 @@ from fcntl import LOCK_EX, LOCK_NB, flock
20from os import unlink, symlink20from os import unlink, symlink
21from os.path import abspath, dirname, exists21from os.path import abspath, dirname, exists
2222
23from qakit.ust.worker import UstResult, Worker
23from requests import get, post24from requests import get, post
2425
25HOME = dirname(abspath(__file__))26HOME = dirname(abspath(__file__))
@@ -179,6 +180,34 @@ def parallelize_fetching(all_series, runs):
179 update_request(ticket)180 update_request(ticket)
180181
181182
183def compute_ust_signoff(requests):
184 """Notify the ticket status in ubuntu system tests.
185 Once a silo is assigned to the ticket, ust is triggered only if
186 the lander has approved the ticket. After triggering the tests,
187 the worker is asked for results
188 :return UstResult which has two attributes: status and build url
189 which are both use in the UI"""
190 ust_signoff = None
191 for request in requests:
192 if not request['siloname']:
193 ust_worker = Worker()
194 if request['ust_signoff'] == "":
195 if request['lander_signoff'] == 'Approved':
196 try:
197 ust_worker.run_ust(request['siloname'])
198 ust_signoff = UstResult("Queued", "")
199 except Exception:
200 pass
201
202 if ust_signoff:
203 req = dict(
204 author='ust-bot',
205 request_id=request['request_id'],
206 ust_signoff=ust_signoff.status)
207 post(WRITE_API + Config.api_token, dumps(req, sort_keys=True),
208 headers={'content-type': 'application/json'})
209
210
182def print_proc(proc):211def print_proc(proc):
183 """Print the output from a subprocess."""212 """Print the output from a subprocess."""
184 sys.stdout.buffer.write(BNEWLINE.join(213 sys.stdout.buffer.write(BNEWLINE.join(
@@ -194,6 +223,7 @@ def main():
194 sys.stderr = sys.stdout = open(LOG_NAME, 'x')223 sys.stderr = sys.stdout = open(LOG_NAME, 'x')
195 reqs = get_requests()224 reqs = get_requests()
196 parallelize_fetching(*build_siloname_dict(reqs))225 parallelize_fetching(*build_siloname_dict(reqs))
226 compute_ust_signoff(reqs)
197 except BlockingIOError:227 except BlockingIOError:
198 return 0228 return 0
199 except (IOError, ConnectionRefusedError):229 except (IOError, ConnectionRefusedError):
@@ -205,6 +235,5 @@ def main():
205 cleanup(lock)235 cleanup(lock)
206 return 0236 return 0
207237
208
209if __name__ == '__main__': # pragma: no cover238if __name__ == '__main__': # pragma: no cover
210 sys.exit(main())239 sys.exit(main())
diff --git a/db_migrations.sh b/db_migrations.sh
index e346c14..f6a8b3f 100755
--- a/db_migrations.sh
+++ b/db_migrations.sh
@@ -32,6 +32,7 @@ exit 0
32# These ones have succeeded in production and only kept for reference32# These ones have succeeded in production and only kept for reference
33$PG "ALTER TABLE request ADD COLUMN lander_signoff text DEFAULT '';" || true33$PG "ALTER TABLE request ADD COLUMN lander_signoff text DEFAULT '';" || true
34$PG "ALTER TABLE request ADD COLUMN britney_signoff text DEFAULT '';" || true34$PG "ALTER TABLE request ADD COLUMN britney_signoff text DEFAULT '';" || true
35$PG "ALTER TABLE request ADD COLUMN ust_signoff text DEFAULT '';" || true
35$PG "ALTER TABLE request ADD COLUMN status_color text DEFAULT '';" || true36$PG "ALTER TABLE request ADD COLUMN status_color text DEFAULT '';" || true
36$PG "ALTER TABLE request ADD COLUMN artifacts text DEFAULT '';" || true37$PG "ALTER TABLE request ADD COLUMN artifacts text DEFAULT '';" || true
3738
diff --git a/scripts/import.sh b/scripts/import.sh
38new file mode 10075539new file mode 100755
index 0000000..60b8173
--- /dev/null
+++ b/scripts/import.sh
@@ -0,0 +1,26 @@
1#!/bin/sh
2
3# This is a history-destroying git-to-bzr converter,
4# for when you just want the code.
5
6repos="https://github.com/emacs-mirror/emacs.git"
7
8sleep "$(shuf --input-range 0-60 --head-count 1)"
9
10set -eux
11
12
13for repo in $repos; do
14 base="$(basename "$repo" .git)"
15 tmpdir="/tmp/$base"
16
17 [ -d "$tmpdir" ] || git clone --depth 1 "$repo" "$tmpdir"
18 cd "$tmpdir"
19 git pull
20
21 [ -d ".bzr" ] || bzr init .
22 echo '.git*' > .bzrignore
23 bzr add .
24 bzr commit --message "Auto commit $(date)"
25 bzr push "lp:~/$base/imported" --overwrite
26done

Subscribers

People subscribed via source and target branches