Merge lp:~mwhudson/lava-kernel-ci-views/add-board-type-model into lp:lava-kernel-ci-views

Proposed by Michael Hudson-Doyle
Status: Merged
Approved by: Michael Hudson-Doyle
Approved revision: 117
Merged at revision: 106
Proposed branch: lp:~mwhudson/lava-kernel-ci-views/add-board-type-model
Merge into: lp:lava-kernel-ci-views
Diff against target: 1521 lines (+812/-618)
10 files modified
lava_kernel_ci_views_app/admin.py (+4/-0)
lava_kernel_ci_views_app/helpers.py (+458/-0)
lava_kernel_ci_views_app/migrations/0001_initial.py (+37/-0)
lava_kernel_ci_views_app/models.py (+11/-0)
lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/compile_only.html (+13/-0)
lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/index.html (+46/-0)
lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/per_board.html (+13/-0)
lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/waterfall.html (+93/-125)
lava_kernel_ci_views_app/urls.py (+3/-0)
lava_kernel_ci_views_app/views.py (+134/-493)
To merge this branch: bzr merge lp:~mwhudson/lava-kernel-ci-views/add-board-type-model
Reviewer Review Type Date Requested Status
Paul Larson (community) Approve
Review via email: mp+89179@code.launchpad.net

Description of the change

I have some more tweaks I really want to make, but perhaps this is a good point to merge regardless.

This branch adds compile-only and board-specific views to the kernel ci views. The diff is probably largely meaningless, so I've updated staging to use this branch, so have a look at:

http://staging.validation.linaro.org/kernel-ci-views/index

Stuff I would like opinions/help with:

1) The wording on the landing page.
2) The icons/logos I've used for the boards.

Cheers,
mwh

To post a comment you must log in.
Revision history for this message
Paul Larson (pwlars) wrote :

Looks good to me, just a couple of comments. But nothing that I see as a blocker.
The image/board link seems a bit amorphous to me. Perhaps if we restricted the size of the logo? Or maybe put a border around it? It's something that I think we could improve on visually a bit, but it's not bad how it is. We just need to make sure to solicit good icons from everyone that *they* are happy with.

Also, on the compile status view, we will still have multiple builds with the same defconfig. We now know that these are really the same thing getting built, but are destined to be run on different boards. Metadata to help us determine that is now included, and we may want to consider referencing that somewhere. This has been raised before as something that users find confusing.

review: Approve
Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

On Tue, 24 Jan 2012 22:45:28 -0000, Paul Larson <email address hidden> wrote:
> Review: Approve
>
> Looks good to me, just a couple of comments. But nothing that I see as a blocker.
> The image/board link seems a bit amorphous to me. Perhaps if we
> restricted the size of the logo? Or maybe put a border around it?
> It's something that I think we could improve on visually a bit, but
> it's not bad how it is. We just need to make sure to solicit good
> icons from everyone that *they* are happy with.

Yeah, if we can get everyone to send us icons that are the same height
(ideally about 100px) and roughly square, that would be great.

> Also, on the compile status view, we will still have multiple builds
> with the same defconfig. We now know that these are really the same
> thing getting built, but are destined to be run on different boards.
> Metadata to help us determine that is now included, and we may want to
> consider referencing that somewhere. This has been raised before as
> something that users find confusing.

Yeah. I was thinking it wouldn't be too hard to suppress duplicate
results -- if they both built the same defconfig and had the same
outcome, just displaying one should be enough. (if they had a different
outcome, something pretty bizarre is going on).

Cheers,
mwh

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== added file 'lava_kernel_ci_views_app/admin.py'
2--- lava_kernel_ci_views_app/admin.py 1970-01-01 00:00:00 +0000
3+++ lava_kernel_ci_views_app/admin.py 2012-01-19 01:20:31 +0000
4@@ -0,0 +1,4 @@
5+from django.contrib import admin
6+from lava_kernel_ci_views_app.models import BoardType
7+
8+admin.site.register(BoardType)
9
10=== added file 'lava_kernel_ci_views_app/helpers.py'
11--- lava_kernel_ci_views_app/helpers.py 1970-01-01 00:00:00 +0000
12+++ lava_kernel_ci_views_app/helpers.py 2012-01-19 01:20:31 +0000
13@@ -0,0 +1,458 @@
14+from collections import defaultdict, namedtuple
15+import contextlib
16+import datetime
17+
18+from dashboard_app.models import DataView, TestResult
19+
20+DAY_DELTA = datetime.timedelta(days=1)
21+WEEK_DELTA = datetime.timedelta(days=7)
22+
23+def fetchnamed(cursor):
24+ class cls(object):
25+ pass
26+ names = ' '.join(col[0] for col in cursor.description)
27+ typ = namedtuple('result', names)
28+ for row in cursor.fetchall():
29+ yield typ(*row)
30+
31+
32+index_sql = """
33+select softwaresource.branch_url as git_url,
34+ softwaresource.branch_revision as git_commit_id,
35+ coalesce(namedattribute_git_describe.value, softwaresource.branch_revision) as git_describe,
36+ coalesce(namedattribute_git_log_info.value, softwaresource.branch_revision) as git_log_info,
37+ namedattribute_gitweb_url.value as gitweb_url,
38+ softwaresource.commit_timestamp as commit_timestamp,
39+ testrun.analyzer_assigned_date as build_date,
40+ namedattribute_kernelconfig.value as config,
41+ testresult.result as result,
42+ namedattribute_kernelbuild_url.value as build_url,
43+ bundle.content_sha1 as bundle_sha1
44+ from dashboard_app_bundlestream as bundlestream,
45+ dashboard_app_bundle as bundle,
46+ dashboard_app_testresult as testresult,
47+ dashboard_app_testrun as testrun
48+inner join dashboard_app_namedattribute AS namedattribute_kernelconfig
49+ on (namedattribute_kernelconfig.object_id = testrun.id
50+ and namedattribute_kernelconfig.name = 'kernel.config'
51+ and namedattribute_kernelconfig.content_type_id = (
52+ select django_content_type.id from django_content_type
53+ where app_label = 'dashboard_app' and model='testrun')
54+ )
55+inner join dashboard_app_namedattribute AS namedattribute_kernelbuild_url
56+ on (namedattribute_kernelbuild_url.object_id = testrun.id
57+ and namedattribute_kernelbuild_url.name = 'kernel.build_url'
58+ and namedattribute_kernelbuild_url.content_type_id = (
59+ select django_content_type.id from django_content_type
60+ where app_label = 'dashboard_app' and model='testrun')
61+ )
62+left outer join dashboard_app_namedattribute as namedattribute_git_describe
63+ on (namedattribute_git_describe.object_id = testrun.id
64+ and namedattribute_git_describe.name = 'kernel.git_describe_info'
65+ and namedattribute_git_describe.content_type_id = (
66+ select django_content_type.id from django_content_type
67+ where app_label = 'dashboard_app' and model='testrun')
68+ )
69+left outer join dashboard_app_namedattribute as namedattribute_git_log_info
70+ on (namedattribute_git_log_info.object_id = testrun.id
71+ and namedattribute_git_log_info.name = 'kernel.git_log_info'
72+ and namedattribute_git_log_info.content_type_id = (
73+ select django_content_type.id from django_content_type
74+ where app_label = 'dashboard_app' and model='testrun')
75+ )
76+left outer join dashboard_app_namedattribute as namedattribute_gitweb_url
77+ on (namedattribute_gitweb_url.object_id = testrun.id
78+ and namedattribute_gitweb_url.name = 'kernel.gitweb_url'
79+ and namedattribute_gitweb_url.content_type_id = (
80+ select django_content_type.id from django_content_type
81+ where app_label = 'dashboard_app' and model='testrun')
82+ )
83+left outer join dashboard_app_namedattribute as namedattribute_target_device_type
84+ on (namedattribute_target_device_type.object_id = testrun.id
85+ and namedattribute_target_device_type.name = 'target.device_type'
86+ and namedattribute_target_device_type.content_type_id = (
87+ select django_content_type.id from django_content_type
88+ where app_label = 'dashboard_app' and model='testrun')
89+ ),
90+ dashboard_app_softwaresource as softwaresource,
91+ dashboard_app_testrun_sources as tr_ss_link
92+ where bundle.bundle_stream_id = bundlestream.id
93+ and testrun.bundle_id = bundle.id
94+ and softwaresource.id = tr_ss_link.softwaresource_id
95+ and testrun.id = tr_ss_link.testrun_id
96+ and bundlestream.slug like 'ci-linux%%-build'
97+ and testresult.test_run_id = testrun.id
98+ and %s < testrun.analyzer_assigned_date and testrun.analyzer_assigned_date < %s
99+"""
100+
101+find_builds_sql = """
102+select testresult.result as result,
103+ namedattribute_kernelbuild_url.value as build_url,
104+ namedattribute_targethostname.value as targethostname,
105+ testcase.test_case_id as test_case_id,
106+ bundle.content_sha1 as bundle_sha1
107+ from dashboard_app_bundlestream as bundlestream,
108+ dashboard_app_bundle as bundle,
109+ dashboard_app_testresult as testresult,
110+ dashboard_app_testcase as testcase,
111+ dashboard_app_testrun as testrun
112+inner join dashboard_app_namedattribute AS namedattribute_targethostname
113+ on (namedattribute_targethostname.object_id = testrun.id
114+ and namedattribute_targethostname.name = 'target.hostname'
115+ and namedattribute_targethostname.content_type_id = (
116+ select django_content_type.id from django_content_type
117+ where app_label = 'dashboard_app' and model='testrun')
118+ )
119+inner join dashboard_app_namedattribute AS namedattribute_kernelbuild_url
120+ on (namedattribute_kernelbuild_url.object_id = testrun.id
121+ and namedattribute_kernelbuild_url.name = 'kernel.build_url'
122+ and namedattribute_kernelbuild_url.content_type_id = (
123+ select django_content_type.id from django_content_type
124+ where app_label = 'dashboard_app' and model='testrun')
125+ )
126+ where bundle.bundle_stream_id = bundlestream.id
127+ and testrun.bundle_id = bundle.id
128+ and bundlestream.slug like 'ci-linux%%'
129+ and testresult.test_run_id = testrun.id
130+ and testresult.test_case_id = testcase.id
131+ and namedattribute_kernelbuild_url.value = ANY(%s)
132+"""
133+
134+class Test(object):
135+ def __init__(self, board_class, sha1, build):
136+ self._board_class = board_class
137+ self.sha1 = sha1
138+ self.build = build
139+ self.results = defaultdict(set)
140+ @property
141+ def passes(self):
142+ return len(self.results[TestResult.RESULT_PASS])
143+ @property
144+ def fails(self):
145+ return len(self.results[TestResult.RESULT_FAIL])
146+ def json_ready(self):
147+ if self.fails > 0:
148+ result = 'fail'
149+ else:
150+ result = 'pass'
151+ if self.prev:
152+ pass_to_fail = self.results[TestResult.RESULT_FAIL] & \
153+ self.prev.results[TestResult.RESULT_PASS]
154+ pass_to_fail = sorted(pass_to_fail)
155+ fail_to_pass = self.results[TestResult.RESULT_PASS] & \
156+ self.prev.results[TestResult.RESULT_FAIL]
157+ fail_to_pass = sorted(fail_to_pass)
158+ else:
159+ pass_to_fail = []
160+ fail_to_pass = []
161+ return {
162+ 'board_class': self._board_class,
163+ 'result': result,
164+ 'passes': self.passes,
165+ 'fails': self.fails,
166+ 'test_count': self.passes + self.fails,
167+ 'sha1': self.sha1,
168+ 'pass_to_fail': len(pass_to_fail),
169+ 'fail_to_pass': len(fail_to_pass),
170+ }
171+
172+
173+class Build(object):
174+ def __init__(self, config, result, sha1, commit):
175+ self._config = config
176+ self._result = result
177+ self._tests = {}
178+ self.sha1 = sha1
179+ self.commit = commit
180+ @property
181+ def tests(self):
182+ return self._tests.values()
183+ def json_ready(self):
184+ tests = [test.json_ready()
185+ for (board_class, test) in sorted(self._tests.iteritems())]
186+ if self._result:
187+ result = 'fail'
188+ else:
189+ result = 'pass'
190+ return {
191+ 'config': self._config,
192+ 'result': result,
193+ 'tests': tests,
194+ 'sha1': self.sha1,
195+ }
196+ def add_test(self, board_class, test_case, result, sha1):
197+ if board_class in self._tests:
198+ test = self._tests[board_class]
199+ else:
200+ test = self._tests[board_class] = Test(board_class, sha1, self)
201+ test.results[result].add(test_case)
202+ return test
203+ def find_test(self, board_class):
204+ return self._tests.get(board_class)
205+
206+
207+class Commit(object):
208+ def __init__(self, sha1, commit_timestamp, describe, log_info, gitweb_url, tree):
209+ self.sha1 = sha1
210+ self.commit_timestamp = commit_timestamp
211+ self.describe = describe
212+ self._builds = []
213+ self.log_info = log_info
214+ self.gitweb_url = gitweb_url
215+ self.tree = tree
216+ def add_build(self, config, result, sha1):
217+ build = Build(config, result, sha1, self)
218+ self.tree.day.daycollection._configs.add(config)
219+ self._builds.append(build)
220+ return build
221+ def find_builds(self, config):
222+ for build in self._builds:
223+ if build._config == config:
224+ yield build
225+ @property
226+ def builds(self):
227+ return list(self._builds)
228+ def json_ready(self):
229+ builds = [build.json_ready() for build in self._builds]
230+ def sort_key(build):
231+ has_test = len(build['tests']) > 0
232+ if has_test:
233+ board_class = build['tests'][0]['board_class']
234+ else:
235+ board_class = None
236+ return (-has_test, build['config'], board_class)
237+ builds.sort(key=sort_key)
238+ for build in builds:
239+ build['width'] = (100.0 - len(builds) + 1)/len(builds)
240+ if self.prev:
241+ prev = self.prev.sha1
242+ else:
243+ prev = None
244+ commit_url = shortlog_url = None
245+ if self.gitweb_url:
246+ if 'github' in self.gitweb_url:
247+ base_url = self.gitweb_url
248+ if base_url.endswith('.git'):
249+ base_url = base_url[:-len('.git')]
250+ commit_url = base_url + '/commit/' + self.sha1
251+ else:
252+ commit_url = self.gitweb_url + ';a=commit;h=' + self.sha1
253+ if self.prev:
254+ shortlog_url = '%s;a=shortlog;h=%s;hp=%s' % (
255+ self.gitweb_url, self.sha1, self.prev.sha1)
256+ return {
257+ 'sha1': self.sha1,
258+ 'describe': self.describe,
259+ 'builds': builds,
260+ 'test_count': sum(len(build['tests']) for build in builds),
261+ 'prev': prev,
262+ 'log_info': self.log_info,
263+ 'commit_url': commit_url,
264+ 'shortlog_url': shortlog_url,
265+ }
266+
267+
268+class Tree(object):
269+ def __init__(self, git_url, index, day):
270+ self._git_url = git_url
271+ self._commits = {}
272+ self.day = day
273+ self.index = index
274+ def get_commit(self, sha1, commit_timestamp, describe, log_info, gitweb_url):
275+ if sha1 in self._commits:
276+ # XXX if self._commits[sha1].commit_timestamp != commit_timestamp ...
277+ return self._commits[sha1]
278+ else:
279+ commit_obj = self._commits[sha1] = Commit(
280+ sha1, commit_timestamp, describe, log_info, gitweb_url, self)
281+ return commit_obj
282+ @property
283+ def commits(self):
284+ return self._commits.values()
285+ def json_ready(self):
286+ def key(commit):
287+ return commit.commit_timestamp
288+ commits = self._commits.values()
289+ commits.sort(key=key)
290+ commits.reverse()
291+ commits = [commit.json_ready() for commit in commits]
292+ return {
293+ 'name': self._git_url,
294+ 'commits': commits,
295+ 'test_count': sum(commit['test_count'] for commit in commits),
296+ 'build_count': sum(len(commit['builds']) for commit in commits),
297+ 'index': self.index,
298+ }
299+
300+
301+class Day(object):
302+ def __init__(self, day, daycollection):
303+ self._day = day
304+ self._trees = {}
305+ self.daycollection = daycollection
306+ def get_tree(self, git_url, index):
307+ if git_url in self._trees:
308+ return self._trees[git_url]
309+ else:
310+ tree_obj = self._trees[git_url] = Tree(git_url, index, self)
311+ return tree_obj
312+ @property
313+ def trees(self):
314+ return self._trees.values()
315+ def json_ready(self):
316+ # XXX how do we order trees?
317+ trees = [tree.json_ready() for tree in self._trees.values()]
318+ for tree in trees:
319+ tree['width'] = (100.0 - len(trees) + 1)/len(trees)
320+ trees.sort(key=lambda tree:tree['name'])
321+ return {
322+ 'date': self._day.strftime('%A %B %d %Y'),
323+ 'trees': trees,
324+ 'build_count': sum(tree['build_count'] for tree in trees),
325+ 'test_count': sum(tree['test_count'] for tree in trees),
326+ 'tree_count': len([tree for tree in trees if tree['build_count'] > 0]),
327+ }
328+
329+
330+class DayCollection(object):
331+ def __init__(self, board_type, start, finish):
332+ self.board_type = board_type
333+ self.start = start
334+ self.finish = finish
335+ self._days = {}
336+ # We create a Day object for every day we're interested in, plus one
337+ # day older so that we can find the previous build for the last row of
338+ # builds displayed. This isn't quite enough, because the previous
339+ # build might have been more than one day before the last one
340+ # displayed, but for now it's a reasonable hack.
341+ d = start - DAY_DELTA
342+ while d < finish:
343+ self._days[d] = Day(d, self)
344+ d += DAY_DELTA
345+ self._board_classes = set()
346+ self._configs = set()
347+ self._urls_to_builds = {}
348+ self._tree2index = {}
349+ self.all_trees = set()
350+
351+ def get_day(self, datestamp):
352+ day = datestamp.date()
353+ return self._days[day]
354+
355+ def add_build(self, build_results):
356+ day = self.get_day(build_results.build_date)
357+ git_url = build_results.git_url
358+ if git_url not in self._tree2index:
359+ self._tree2index[git_url] = len(self._tree2index)
360+ tree = day.get_tree(git_url, self._tree2index[git_url])
361+ self.all_trees.add(tree)
362+ commit = tree.get_commit(
363+ build_results.git_commit_id, build_results.commit_timestamp,
364+ build_results.git_describe, build_results.git_log_info, build_results.gitweb_url)
365+ config = build_results.config
366+ if config.endswith('_defconfig'):
367+ config = config[:-len('_defconfig')]
368+ self._configs.add(config)
369+ build = commit.add_build(
370+ config, build_results.result, build_results.bundle_sha1)
371+ build_url = build_results.build_url
372+ if build_url != 'unknown':
373+ self._urls_to_builds[build_url] = build
374+
375+ def add_test(self, test_result):
376+ board_class = test_result.targethostname.strip('0123456789')
377+ self._board_classes.add(board_class)
378+ self._urls_to_builds[test_result.build_url].add_test(
379+ board_class, test_result.test_case_id, test_result.result,
380+ test_result.bundle_sha1)
381+
382+ def compute_prevs(self):
383+ # tree_commits maps git urls to Commit objects in the order they were
384+ # built. Notice that it's not only possibly but likely that there
385+ # will be more than one Commit for a given git_commit_id (and hence
386+ # commit_timestamp) because we run builds daily even if the tip hasn't
387+ # changed.
388+
389+ tree_commits = defaultdict(list)
390+ for day in sorted(self.days, key=lambda day:day._day):
391+ for tree in day.trees:
392+ commits = tree.commits
393+ commits.sort(key=lambda commit: commit.commit_timestamp)
394+ tree_commits[tree._git_url].extend(commits)
395+ for commits_list in tree_commits.values():
396+ if commits_list:
397+ commits_list.sort(key=lambda commit: commit.commit_timestamp)
398+ commits_list[0].prev = None
399+ for i in range(1, len(commits_list)):
400+ commits_list[i].prev = commits_list[i-1]
401+
402+ for day in self.days:
403+ for tree in day.trees:
404+ for commit in tree.commits:
405+ for build in commit.builds:
406+ for test in build.tests:
407+ test.prev = None
408+ prev_commit = test.build.commit.prev
409+ while prev_commit is not None:
410+ for prev_build in prev_commit.find_builds(test.build._config):
411+ prev_test = prev_build.find_test(test._board_class)
412+ if prev_test:
413+ test.prev = prev_test
414+ break
415+ if test.prev:
416+ break
417+ prev_commit = prev_commit.prev
418+
419+ def evaluate(self, fetch_builds):
420+ connection = DataView.get_connection()
421+ with contextlib.closing(connection.cursor()) as cursor:
422+ sql = index_sql
423+ sql_args = (self.start - DAY_DELTA, self.finish)
424+ if self.board_type:
425+ sql += ' and namedattribute_target_device_type.value = %s'
426+ sql_args += (self.board_type,)
427+ cursor.execute(sql, sql_args)
428+
429+ for build_result in fetchnamed(cursor):
430+ self.add_build(build_result)
431+
432+ if fetch_builds:
433+ cursor.execute(find_builds_sql, (self._urls_to_builds.keys(),))
434+
435+ for test_result in fetchnamed(cursor):
436+ self.add_test(test_result)
437+
438+ self.compute_prevs()
439+
440+ self.fill_in_missing_trees()
441+
442+ def fill_in_missing_trees(self):
443+ for day in self.days:
444+ missing_trees = self.all_trees - set(day.trees)
445+ for tree in missing_trees:
446+ day.get_tree(tree._git_url, self._tree2index[tree._git_url])
447+
448+ @property
449+ def days(self):
450+ return self._days.values()
451+
452+ def json_ready(self):
453+ def key(day_obj):
454+ return day_obj._day
455+ day_objs = self._days.values()
456+ day_objs.sort(key=key)
457+ day_objs.reverse()
458+ # Remove the extra day (see __init__ for the comment about this).
459+ del day_objs[-1]
460+ trees = []
461+ for url, index in sorted(self._tree2index.items()):
462+ trees.append({
463+ 'url': url,
464+ 'index': index,
465+ 'width': (100.0 - len(self._tree2index) + 1)/len(self._tree2index)
466+ })
467+ return {
468+ 'days': [day_obj.json_ready() for day_obj in day_objs],
469+ 'trees': trees,
470+ }
471+
472
473=== added directory 'lava_kernel_ci_views_app/migrations'
474=== added file 'lava_kernel_ci_views_app/migrations/0001_initial.py'
475--- lava_kernel_ci_views_app/migrations/0001_initial.py 1970-01-01 00:00:00 +0000
476+++ lava_kernel_ci_views_app/migrations/0001_initial.py 2012-01-19 01:20:31 +0000
477@@ -0,0 +1,37 @@
478+# encoding: utf-8
479+import datetime
480+from south.db import db
481+from south.v2 import SchemaMigration
482+from django.db import models
483+
484+class Migration(SchemaMigration):
485+
486+ def forwards(self, orm):
487+
488+ # Adding model 'BoardType'
489+ db.create_table('lava_kernel_ci_views_app_boardtype', (
490+ ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
491+ ('slug', self.gf('django.db.models.fields.SlugField')(max_length=50, db_index=True)),
492+ ('display_name', self.gf('django.db.models.fields.TextField')()),
493+ ('icon', self.gf('django.db.models.fields.files.FileField')(max_length=100)),
494+ ))
495+ db.send_create_signal('lava_kernel_ci_views_app', ['BoardType'])
496+
497+
498+ def backwards(self, orm):
499+
500+ # Deleting model 'BoardType'
501+ db.delete_table('lava_kernel_ci_views_app_boardtype')
502+
503+
504+ models = {
505+ 'lava_kernel_ci_views_app.boardtype': {
506+ 'Meta': {'object_name': 'BoardType'},
507+ 'display_name': ('django.db.models.fields.TextField', [], {}),
508+ 'icon': ('django.db.models.fields.files.FileField', [], {'max_length': '100'}),
509+ 'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
510+ 'slug': ('django.db.models.fields.SlugField', [], {'max_length': '50', 'db_index': 'True'})
511+ }
512+ }
513+
514+ complete_apps = ['lava_kernel_ci_views_app']
515
516=== added file 'lava_kernel_ci_views_app/migrations/__init__.py'
517=== modified file 'lava_kernel_ci_views_app/models.py'
518--- lava_kernel_ci_views_app/models.py 2011-09-20 04:16:10 +0000
519+++ lava_kernel_ci_views_app/models.py 2012-01-19 01:20:31 +0000
520@@ -19,3 +19,14 @@
521 from django.db import models
522
523 # Create your models here.
524+
525+class BoardType(models.Model):
526+
527+ slug = models.SlugField()
528+
529+ display_name = models.TextField()
530+
531+ icon = models.FileField(upload_to="board_icons")
532+
533+ def __unicode__(self):
534+ return "%s board type" % self.slug
535
536=== added file 'lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/compile_only.html'
537--- lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/compile_only.html 1970-01-01 00:00:00 +0000
538+++ lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/compile_only.html 2012-01-19 01:20:31 +0000
539@@ -0,0 +1,13 @@
540+{% extends "layouts/content.html" %}
541+
542+{% block extrahead %}
543+{{ block.super }}
544+<script type="text/javascript" src="{{ STATIC_URL }}lava_kernel_ci_views_app/js/nested-render.js"></script>
545+<link rel="stylesheet" type="text/css" href="{{ STATIC_URL }}lava_kernel_ci_views_app/css/nested-render.css"></style>
546+{% endblock %}
547+
548+{% block content %}
549+
550+{% include "lava_kernel_ci_views_app/waterfall.html" %}
551+
552+{% endblock %}
553
554=== added file 'lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/index.html'
555--- lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/index.html 1970-01-01 00:00:00 +0000
556+++ lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/index.html 2012-01-19 01:20:31 +0000
557@@ -0,0 +1,46 @@
558+{% extends "layouts/content.html" %}
559+
560+{% block extrahead %}
561+{{ block.super }}
562+{% endblock %}
563+
564+{% block content %}
565+
566+<h1>Kernel CI</h1>
567+
568+<p>
569+ This is the landing page for our automated kernel testing efforts.
570+ We build multiple defconfigs from multiple trees every day and test
571+ the built kernels in LAVA. For more
572+ details <a href="http://lava-kernel-ci.rtfd.org/">see the
573+ documentation</a>.
574+</p>
575+
576+<h2>Compile status view</h2>
577+
578+<p>
579+ To see the build status of every defconfig we build, see
580+ the <a href="{% url lava_kernel_ci_views_app.views.compile_view %}">compile results</a>.
581+</p>
582+
583+<h2>Board views</h2>
584+
585+<p>
586+ Alternatively, see the results of testing on the boards we have in
587+ the lab:
588+</p>
589+
590+<table>
591+ <tr>
592+ {% for board_type in board_types %}
593+ <td style="text-align: center">
594+ <a href="{% url lava_kernel_ci_views_app.views.per_board board_type=board_type.slug %}">
595+ <img src="{% url lava_kernel_ci_views_app.views.board_icon board_type=board_type.slug%}" />
596+ <br />
597+ {{ board_type.display_name }}
598+ </a>
599+ </td>
600+ {% endfor %}
601+ </tr>
602+</table>
603+{% endblock %}
604
605=== added file 'lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/per_board.html'
606--- lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/per_board.html 1970-01-01 00:00:00 +0000
607+++ lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/per_board.html 2012-01-19 01:20:31 +0000
608@@ -0,0 +1,13 @@
609+{% extends "layouts/content.html" %}
610+
611+{% block extrahead %}
612+{{ block.super }}
613+<script type="text/javascript" src="{{ STATIC_URL }}lava_kernel_ci_views_app/js/nested-render.js"></script>
614+<link rel="stylesheet" type="text/css" href="{{ STATIC_URL }}lava_kernel_ci_views_app/css/nested-render.css"></style>
615+{% endblock %}
616+
617+{% block content %}
618+
619+{% include "lava_kernel_ci_views_app/waterfall.html" %}
620+
621+{% endblock %}
622
623=== renamed file 'lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/index.html' => 'lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/waterfall.html'
624--- lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/index.html 2012-01-12 02:07:21 +0000
625+++ lava_kernel_ci_views_app/templates/lava_kernel_ci_views_app/waterfall.html 2012-01-19 01:20:31 +0000
626@@ -1,143 +1,111 @@
627-{% extends "layouts/content.html" %}
628-
629-{% block extrahead %}
630-{{ block.super }}
631-<script type="text/javascript" src="{{ STATIC_URL }}lava_kernel_ci_views_app/js/nested-render.js"></script>
632-<link rel="stylesheet" type="text/css" href="{{ STATIC_URL }}lava_kernel_ci_views_app/css/nested-render.css"></style>
633-{% endblock %}
634-
635-{% block breadcrumbs %}
636-{{ block.super }}
637-<li><a href="{% url lava_kernel_ci_views_app.views.index %}">Kernel CI Views</a></li>
638-{% endblock %}
639-
640-{% block content %}
641- <body>
642- <p>
643- <b>Show tests run on:</b>
644- {% for board_class in board_classes %}
645- <input class="showhide test" type="checkbox" value=".test-{{ board_class }}" checked="true"/>{{ board_class }}
646- {% endfor %}
647- </p>
648- <p>
649- <b>Show builds of configs:</b>
650- {% for config in configs %}
651- <input class="showhide config" type="checkbox" value=".build-{{ config }}" checked="true"/>{{ config }}
652- {% endfor %}
653- </p>
654- <p>
655- <b>Show trees:</b>
656- {% for tree in trees %}
657- <input class="showhide tree" type="checkbox" value=".tree-{{ tree.1 }}" checked="true"/>{{ tree.0 }}
658- {% endfor %}
659- </p>
660- <ul class="container days">
661+<p>
662+ <ul class="container days">
663+ <ul class="container trees">
664+ {% for tree in data.trees %}
665+ <li class="cell tree tree-{{ tree.index }}" style="width: {{ tree.width }}%">
666+ <div class="item tree">
667+ {{ tree.url }}
668+ </div>
669+ </li>
670+ {% endfor %}
671+ </ul>
672+ {% for day in data.days %}
673+ <li class="day">
674+ <div class="item day">
675+ <div class="item-inner day">
676+ {{ day.date }} &ndash;
677+ {% if include_tests %}
678+ {{ day.test_count }} tests of
679+ {% endif %}
680+ {{ day.build_count }} builds on {{ day.tree_count }} trees
681+ </div>
682+ </div>
683 <ul class="container trees">
684- {% for tree in data.trees %}
685- <li class="cell tree tree-{{ tree.index }}" style="width: {{ tree.width }}%">
686- <div class="item tree">
687- {{ tree.url }}
688- </div>
689- </li>
690- {% endfor %}
691- </ul>
692- {% for day in data.days %}
693- <li class="day">
694- <div class="item day">
695- <div class="item-inner day">
696- {{ day.date }} &ndash; {{ day.test_count }} tests of
697- {{ day.build_count }} builds on {{ day.tree_count }} trees
698- </div>
699- </div>
700- <ul class="container trees">
701- {% for tree in day.trees %}
702- <li class="tree tree-{{ tree.index }} cell" style="width: {{ tree.width }}%">
703- <ul class="container commits">
704- {% for commit in tree.commits %}
705- <li class="commit">
706- <li class="dummy build cell build-dummy">
707+ {% for tree in day.trees %}
708+ <li class="tree tree-{{ tree.index }} cell" style="width: {{ tree.width }}%">
709+ <ul class="container commits">
710+ {% for commit in tree.commits %}
711+ <li class="commit">
712+ <li class="dummy build cell build-dummy">
713+ {% if include_tests %}
714+ <ul class="container tests">
715+ <li class="cell test dummy">
716+ <div class="item test test-dummy result result-dummy">
717+ <div class="item-inner test test-dummy result result-dummy">
718+ &nbsp;<br/>&nbsp;
719+ </div>
720+ </div>
721+ </li>
722+ </ul>
723+ {% endif %}
724+ <div class="item build build-dummy result result-dummy">
725+ <div class="item-inner build build-dummy result result-dummy">
726+ &nbsp;<br/>&nbsp;
727+ </div>
728+ </div>
729+ </li>
730+ <ul class="container builds">
731+ {% for build in commit.builds %}
732+ <li class="build cell build-{{ build.config }}" style="width: {{ build.width }}%">
733+ {% if include_tests %}
734 <ul class="container tests">
735 <li class="cell test dummy">
736 <div class="item test test-dummy result result-dummy">
737 <div class="item-inner test test-dummy result result-dummy">
738- &nbsp;<br/>&nbsp;
739+ &nbsp;<br/>&nbsp;<br/>&nbsp;<br/>&nbsp;
740 </div>
741 </div>
742 </li>
743- </ul>
744- <div class="item build build-dummy result result-dummy">
745- <div class="item-inner build build-dummy result result-dummy">
746- &nbsp;<br/>&nbsp;
747- </div>
748- </div>
749- </li>
750- <ul class="container builds">
751- {% for build in commit.builds %}
752- <li class="build cell build-{{ build.config }}" style="width: {{ build.width }}%">
753- <ul class="container tests">
754- <li class="cell test dummy">
755- <div class="item test test-dummy result result-dummy">
756- <div class="item-inner test test-dummy result result-dummy">
757- &nbsp;<br/>&nbsp;<br/>&nbsp;<br/>&nbsp;
758- </div>
759- </div>
760- </li>
761- {% for test in build.tests %}
762- <li class="cell test" style="width: 100%">
763- <div class="item test test-{{ test.board_class }} result result-{{ test.result }}">
764- <div class="item-inner test test-{{ test.board_class }} result result-{{ test.result }}">
765- <a href="{{ link_prefix }}{{ test.sha1 }}">{{ test.board_class }}</a>&nbsp;test:<br />
766- {{ test.test_count }}&nbsp;total<br />
767- {{ test.passes }}&nbsp;passed{% if test.fail_to_pass %}&nbsp;({{test.fail_to_pass}}&nbsp;fixed){% endif %}<br />
768- {{ test.fails }}&nbsp;failed{% if test.pass_to_fail %}&nbsp;({{test.pass_to_fail}}&nbsp;regressed){% endif %}
769- </div>
770- </div>
771- </li>
772- {% endfor %}
773- </ul>
774- <div class="item build build-{{ build.config }} result result-{{ build.result }}">
775- <div class="item-inner build build-{{ build.config }} result result-{{ build.result }}">
776- Build&nbsp;of<br /><a href="{{ link_prefix }}{{ build.sha1 }}">{{ build.config }}</a>
777+ {% for test in build.tests %}
778+ <li class="cell test" style="width: 100%">
779+ <div class="item test test-{{ test.board_class }} result result-{{ test.result }}">
780+ <div class="item-inner test test-{{ test.board_class }} result result-{{ test.result }}">
781+ <a href="{{ link_prefix }}{{ test.sha1 }}">{{ test.board_class }}</a>&nbsp;test:<br />
782+ {{ test.test_count }}&nbsp;total<br />
783+ {{ test.passes }}&nbsp;passed{% if test.fail_to_pass %}&nbsp;({{test.fail_to_pass}}&nbsp;fixed){% endif %}<br />
784+ {{ test.fails }}&nbsp;failed{% if test.pass_to_fail %}&nbsp;({{test.pass_to_fail}}&nbsp;regressed){% endif %}
785 </div>
786 </div>
787 </li>
788 {% endfor %}
789 </ul>
790- <div class="item commit">
791- <div class="item-inner commit"
792- title="{{ commit.log_info }}">
793- {% if commit.commit_url %}
794- <a href="{{ commit.commit_url }}">{{ commit.describe }}</a>
795- {% else %}
796- {{ commit.describe }}
797- {% endif %}
798- {% if commit.shortlog_url %}
799- &nbsp<a href="{{ commit.shortlog_url }}">shortlog</a>
800- {% endif %}
801+ {% endif %}
802+ <div class="item build build-{{ build.config }} result result-{{ build.result }}">
803+ <div class="item-inner build build-{{ build.config }} result result-{{ build.result }}">
804+ Build&nbsp;of<br /><a href="{{ link_prefix }}{{ build.sha1 }}">{{ build.config }}</a>
805 </div>
806 </div>
807 </li>
808 {% endfor %}
809 </ul>
810- </li>
811- {% endfor %}
812- </ul>
813- </li>
814- {% endfor %}
815- </ul>
816- <p style="clear:both">
817- {% if newer_results_link %}
818- <a href="{{ newer_results_link }}">&lsaquo;&lsaquo; Newer results</a>
819- {% endif %}
820- &nbsp;
821- {% if older_results_link %}
822- <a style="float:right" href="{{ older_results_link }}">Older results &rsaquo;&rsaquo;</a>
823- {% endif %}
824- </p>
825- <script>
826- $('input.showhide.config').click(function () { hideCells(this, '.builds.container'); });
827- $('input.showhide.test').click(function () { hideCells(this, '.tests.container'); });
828- $('input.showhide.tree').click(function () { hideCells(this, '.trees.container'); });
829- </script>
830-
831-{% endblock %}
832+ <div class="item commit">
833+ <div class="item-inner commit"
834+ title="{{ commit.log_info }}">
835+ {% if commit.commit_url %}
836+ <a href="{{ commit.commit_url }}">{{ commit.describe }}</a>
837+ {% else %}
838+ {{ commit.describe }}
839+ {% endif %}
840+ {% if commit.shortlog_url %}
841+ &nbsp<a href="{{ commit.shortlog_url }}">shortlog</a>
842+ {% endif %}
843+ </div>
844+ </div>
845+ </li>
846+ {% endfor %}
847+ </ul>
848+ </li>
849+ {% endfor %}
850+ </ul>
851+ </li>
852+ {% endfor %}
853+ </ul>
854+<p style="clear:both">
855+ {% if newer_results_link %}
856+ <a href="{{ newer_results_link }}">&lsaquo;&lsaquo; Newer results</a>
857+ {% endif %}
858+ &nbsp;
859+ {% if older_results_link %}
860+ <a style="float:right" href="{{ older_results_link }}">Older results &rsaquo;&rsaquo;</a>
861+ {% endif %}
862+</p>
863
864=== modified file 'lava_kernel_ci_views_app/urls.py'
865--- lava_kernel_ci_views_app/urls.py 2011-10-05 04:00:28 +0000
866+++ lava_kernel_ci_views_app/urls.py 2012-01-19 01:20:31 +0000
867@@ -21,4 +21,7 @@
868 urlpatterns = patterns(
869 'lava_kernel_ci_views_app.views',
870 url(r'index$', 'index'),
871+ url(r'compile$', 'compile_view'),
872+ url(r'per_board/(?P<board_type>[a-zA-Z0-9]+)$', 'per_board'),
873+ url(r'board_icon/(?P<board_type>[a-zA-Z0-9]+)$', 'board_icon'),
874 )
875
876=== modified file 'lava_kernel_ci_views_app/views.py'
877--- lava_kernel_ci_views_app/views.py 2012-01-13 03:00:48 +0000
878+++ lava_kernel_ci_views_app/views.py 2012-01-19 01:20:31 +0000
879@@ -16,457 +16,41 @@
880 # You should have received a copy of the GNU Affero General Public License
881 # along with LAVA Kernel CI Views. If not, see <http://www.gnu.org/licenses/>.
882
883-from collections import defaultdict, namedtuple
884-import contextlib
885 import datetime
886-import json
887
888+from django import forms
889 from django.core.urlresolvers import reverse
890-from django import forms
891+from django.http import (
892+ HttpResponse,
893+ )
894 from django.template import RequestContext
895-from django.shortcuts import render_to_response
896-
897-from dashboard_app.models import DataView, TestResult
898-
899-
900-DAY_DELTA = datetime.timedelta(days=1)
901-WEEK_DELTA = datetime.timedelta(days=7)
902-
903-def fetchnamed(cursor):
904- class cls(object):
905- pass
906- names = ' '.join(col[0] for col in cursor.description)
907- typ = namedtuple('result', names)
908- for row in cursor.fetchall():
909- yield typ(*row)
910-
911-
912-index_sql = """
913-select softwaresource.branch_url as git_url,
914- softwaresource.branch_revision as git_commit_id,
915- coalesce(namedattribute_git_describe.value, softwaresource.branch_revision) as git_describe,
916- coalesce(namedattribute_git_log_info.value, softwaresource.branch_revision) as git_log_info,
917- namedattribute_gitweb_url.value as gitweb_url,
918- softwaresource.commit_timestamp as commit_timestamp,
919- testrun.analyzer_assigned_date as build_date,
920- namedattribute_kernelconfig.value as config,
921- testresult.result as result,
922- namedattribute_kernelbuild_url.value as build_url,
923- bundle.content_sha1 as bundle_sha1
924- from dashboard_app_bundlestream as bundlestream,
925- dashboard_app_bundle as bundle,
926- dashboard_app_testresult as testresult,
927- dashboard_app_testrun as testrun
928-inner join dashboard_app_namedattribute AS namedattribute_kernelconfig
929- on (namedattribute_kernelconfig.object_id = testrun.id
930- and namedattribute_kernelconfig.name = 'kernel.config'
931- and namedattribute_kernelconfig.content_type_id = (
932- select django_content_type.id from django_content_type
933- where app_label = 'dashboard_app' and model='testrun')
934- )
935-inner join dashboard_app_namedattribute AS namedattribute_kernelbuild_url
936- on (namedattribute_kernelbuild_url.object_id = testrun.id
937- and namedattribute_kernelbuild_url.name = 'kernel.build_url'
938- and namedattribute_kernelbuild_url.content_type_id = (
939- select django_content_type.id from django_content_type
940- where app_label = 'dashboard_app' and model='testrun')
941- )
942-left outer join dashboard_app_namedattribute as namedattribute_git_describe
943- on (namedattribute_git_describe.object_id = testrun.id
944- and namedattribute_git_describe.name = 'kernel.git_describe_info'
945- and namedattribute_git_describe.content_type_id = (
946- select django_content_type.id from django_content_type
947- where app_label = 'dashboard_app' and model='testrun')
948- )
949-left outer join dashboard_app_namedattribute as namedattribute_git_log_info
950- on (namedattribute_git_log_info.object_id = testrun.id
951- and namedattribute_git_log_info.name = 'kernel.git_log_info'
952- and namedattribute_git_log_info.content_type_id = (
953- select django_content_type.id from django_content_type
954- where app_label = 'dashboard_app' and model='testrun')
955- )
956-left outer join dashboard_app_namedattribute as namedattribute_gitweb_url
957- on (namedattribute_gitweb_url.object_id = testrun.id
958- and namedattribute_gitweb_url.name = 'kernel.gitweb_url'
959- and namedattribute_gitweb_url.content_type_id = (
960- select django_content_type.id from django_content_type
961- where app_label = 'dashboard_app' and model='testrun')
962- ),
963- dashboard_app_softwaresource as softwaresource,
964- dashboard_app_testrun_sources as tr_ss_link
965- where bundle.bundle_stream_id = bundlestream.id
966- and testrun.bundle_id = bundle.id
967- and softwaresource.id = tr_ss_link.softwaresource_id
968- and testrun.id = tr_ss_link.testrun_id
969- and bundlestream.slug like 'ci-linux%%-build'
970- and testresult.test_run_id = testrun.id
971- and %s < testrun.analyzer_assigned_date and testrun.analyzer_assigned_date < %s
972-"""
973-
974-find_builds_sql = """
975-select testresult.result as result,
976- namedattribute_kernelbuild_url.value as build_url,
977- namedattribute_targethostname.value as targethostname,
978- testcase.test_case_id as test_case_id,
979- bundle.content_sha1 as bundle_sha1
980- from dashboard_app_bundlestream as bundlestream,
981- dashboard_app_bundle as bundle,
982- dashboard_app_testresult as testresult,
983- dashboard_app_testcase as testcase,
984- dashboard_app_testrun as testrun
985-inner join dashboard_app_namedattribute AS namedattribute_targethostname
986- on (namedattribute_targethostname.object_id = testrun.id
987- and namedattribute_targethostname.name = 'target.hostname'
988- and namedattribute_targethostname.content_type_id = (
989- select django_content_type.id from django_content_type
990- where app_label = 'dashboard_app' and model='testrun')
991- )
992-inner join dashboard_app_namedattribute AS namedattribute_kernelbuild_url
993- on (namedattribute_kernelbuild_url.object_id = testrun.id
994- and namedattribute_kernelbuild_url.name = 'kernel.build_url'
995- and namedattribute_kernelbuild_url.content_type_id = (
996- select django_content_type.id from django_content_type
997- where app_label = 'dashboard_app' and model='testrun')
998- )
999- where bundle.bundle_stream_id = bundlestream.id
1000- and testrun.bundle_id = bundle.id
1001- and bundlestream.slug like 'ci-linux%%'
1002- and testresult.test_run_id = testrun.id
1003- and testresult.test_case_id = testcase.id
1004- and namedattribute_kernelbuild_url.value = ANY(%s)
1005-"""
1006-
1007-class Test(object):
1008- def __init__(self, board_class, sha1, build):
1009- self._board_class = board_class
1010- self.sha1 = sha1
1011- self.build = build
1012- self.results = defaultdict(set)
1013- @property
1014- def passes(self):
1015- return len(self.results[TestResult.RESULT_PASS])
1016- @property
1017- def fails(self):
1018- return len(self.results[TestResult.RESULT_FAIL])
1019- def json_ready(self):
1020- if self.fails > 0:
1021- result = 'fail'
1022- else:
1023- result = 'pass'
1024- if self.prev:
1025- pass_to_fail = self.results[TestResult.RESULT_FAIL] & \
1026- self.prev.results[TestResult.RESULT_PASS]
1027- pass_to_fail = sorted(pass_to_fail)
1028- fail_to_pass = self.results[TestResult.RESULT_PASS] & \
1029- self.prev.results[TestResult.RESULT_FAIL]
1030- fail_to_pass = sorted(fail_to_pass)
1031- else:
1032- pass_to_fail = []
1033- fail_to_pass = []
1034- return {
1035- 'board_class': self._board_class,
1036- 'result': result,
1037- 'passes': self.passes,
1038- 'fails': self.fails,
1039- 'test_count': self.passes + self.fails,
1040- 'sha1': self.sha1,
1041- 'pass_to_fail': len(pass_to_fail),
1042- 'fail_to_pass': len(fail_to_pass),
1043- }
1044-
1045-
1046-class Build(object):
1047- def __init__(self, config, result, sha1, commit):
1048- self._config = config
1049- self._result = result
1050- self._tests = {}
1051- self.sha1 = sha1
1052- self.commit = commit
1053- @property
1054- def tests(self):
1055- return self._tests.values()
1056- def json_ready(self):
1057- tests = [test.json_ready()
1058- for (board_class, test) in sorted(self._tests.iteritems())]
1059- if self._result:
1060- result = 'fail'
1061- else:
1062- result = 'pass'
1063- return {
1064- 'config': self._config,
1065- 'result': result,
1066- 'tests': tests,
1067- 'sha1': self.sha1,
1068- }
1069- def add_test(self, board_class, test_case, result, sha1):
1070- if board_class in self._tests:
1071- test = self._tests[board_class]
1072- else:
1073- test = self._tests[board_class] = Test(board_class, sha1, self)
1074- test.results[result].add(test_case)
1075- return test
1076- def find_test(self, board_class):
1077- return self._tests.get(board_class)
1078-
1079-
1080-class Commit(object):
1081- def __init__(self, sha1, commit_timestamp, describe, log_info, gitweb_url, tree):
1082- self.sha1 = sha1
1083- self.commit_timestamp = commit_timestamp
1084- self.describe = describe
1085- self._builds = []
1086- self.log_info = log_info
1087- self.gitweb_url = gitweb_url
1088- self.tree = tree
1089- def add_build(self, config, result, sha1):
1090- build = Build(config, result, sha1, self)
1091- self.tree.day.daycollection._configs.add(config)
1092- self._builds.append(build)
1093- return build
1094- def find_builds(self, config):
1095- for build in self._builds:
1096- if build._config == config:
1097- yield build
1098- @property
1099- def builds(self):
1100- return list(self._builds)
1101- def json_ready(self):
1102- builds = [build.json_ready() for build in self._builds]
1103- def sort_key(build):
1104- has_test = len(build['tests']) > 0
1105- if has_test:
1106- board_class = build['tests'][0]['board_class']
1107- else:
1108- board_class = None
1109- return (-has_test, build['config'], board_class)
1110- builds.sort(key=sort_key)
1111- for build in builds:
1112- build['width'] = (100.0 - len(builds) + 1)/len(builds)
1113- if self.prev:
1114- prev = self.prev.sha1
1115- else:
1116- prev = None
1117- commit_url = shortlog_url = None
1118- if self.gitweb_url:
1119- if 'github' in self.gitweb_url:
1120- base_url = self.gitweb_url
1121- if base_url.endswith('.git'):
1122- base_url = base_url[:-len('.git')]
1123- commit_url = base_url + '/commit/' + self.sha1
1124- else:
1125- commit_url = self.gitweb_url + ';a=commit;h=' + self.sha1
1126- if self.prev:
1127- shortlog_url = '%s;a=shortlog;h=%s;hp=%s' % (
1128- self.gitweb_url, self.sha1, self.prev.sha1)
1129- return {
1130- 'sha1': self.sha1,
1131- 'describe': self.describe,
1132- 'builds': builds,
1133- 'test_count': sum(len(build['tests']) for build in builds),
1134- 'prev': prev,
1135- 'log_info': self.log_info,
1136- 'commit_url': commit_url,
1137- 'shortlog_url': shortlog_url,
1138- }
1139-
1140-
1141-class Tree(object):
1142- def __init__(self, git_url, index, day):
1143- self._git_url = git_url
1144- self._commits = {}
1145- self.day = day
1146- self.index = index
1147- def get_commit(self, sha1, commit_timestamp, describe, log_info, gitweb_url):
1148- if sha1 in self._commits:
1149- # XXX if self._commits[sha1].commit_timestamp != commit_timestamp ...
1150- return self._commits[sha1]
1151- else:
1152- commit_obj = self._commits[sha1] = Commit(
1153- sha1, commit_timestamp, describe, log_info, gitweb_url, self)
1154- return commit_obj
1155- @property
1156- def commits(self):
1157- return self._commits.values()
1158- def json_ready(self):
1159- def key(commit):
1160- return commit.commit_timestamp
1161- commits = self._commits.values()
1162- commits.sort(key=key)
1163- commits.reverse()
1164- commits = [commit.json_ready() for commit in commits]
1165- return {
1166- 'name': self._git_url,
1167- 'commits': commits,
1168- 'test_count': sum(commit['test_count'] for commit in commits),
1169- 'build_count': sum(len(commit['builds']) for commit in commits),
1170- 'index': self.index,
1171- }
1172-
1173-
1174-class Day(object):
1175- def __init__(self, day, daycollection):
1176- self._day = day
1177- self._trees = {}
1178- self.daycollection = daycollection
1179- def get_tree(self, git_url, index):
1180- if git_url in self._trees:
1181- return self._trees[git_url]
1182- else:
1183- tree_obj = self._trees[git_url] = Tree(git_url, index, self)
1184- return tree_obj
1185- @property
1186- def trees(self):
1187- return self._trees.values()
1188- def json_ready(self):
1189- # XXX how do we order trees?
1190- trees = [tree.json_ready() for tree in self._trees.values()]
1191- for tree in trees:
1192- tree['width'] = (100.0 - len(trees) + 1)/len(trees)
1193- trees.sort(key=lambda tree:tree['name'])
1194- return {
1195- 'date': self._day.strftime('%A %B %d %Y'),
1196- 'trees': trees,
1197- 'build_count': sum(tree['build_count'] for tree in trees),
1198- 'test_count': sum(tree['test_count'] for tree in trees),
1199- 'tree_count': len([tree for tree in trees if tree['build_count'] > 0]),
1200- }
1201-
1202-
1203-class DayCollection(object):
1204- def __init__(self, start, finish):
1205- self.start = start
1206- self.finish = finish
1207- self._days = {}
1208- # We create a Day object for every day we're interested in, plus one
1209- # day older so that we can find the previous build for the last row of
1210- # builds displayed. This isn't quite enough, because the previous
1211- # build might have been more than one day before the last one
1212- # displayed, but for now it's a reasonable hack.
1213- d = start - DAY_DELTA
1214- while d < finish:
1215- self._days[d] = Day(d, self)
1216- d += DAY_DELTA
1217- self._board_classes = set()
1218- self._configs = set()
1219- self._urls_to_builds = {}
1220- self._tree2index = {}
1221- self.all_trees = set()
1222-
1223- def get_day(self, datestamp):
1224- day = datestamp.date()
1225- return self._days[day]
1226-
1227- def add_build(self, build_results):
1228- day = self.get_day(build_results.build_date)
1229- git_url = build_results.git_url
1230- if git_url not in self._tree2index:
1231- self._tree2index[git_url] = len(self._tree2index)
1232- tree = day.get_tree(git_url, self._tree2index[git_url])
1233- self.all_trees.add(tree)
1234- print repr(build_results.gitweb_url)
1235- commit = tree.get_commit(
1236- build_results.git_commit_id, build_results.commit_timestamp,
1237- build_results.git_describe, build_results.git_log_info, build_results.gitweb_url)
1238- config = build_results.config
1239- if config.endswith('_defconfig'):
1240- config = config[:-len('_defconfig')]
1241- self._configs.add(config)
1242- build = commit.add_build(
1243- config, build_results.result, build_results.bundle_sha1)
1244- build_url = build_results.build_url
1245- if build_url != 'unknown':
1246- self._urls_to_builds[build_url] = build
1247-
1248- def add_test(self, test_result):
1249- board_class = test_result.targethostname.strip('0123456789')
1250- self._board_classes.add(board_class)
1251- self._urls_to_builds[test_result.build_url].add_test(
1252- board_class, test_result.test_case_id, test_result.result,
1253- test_result.bundle_sha1)
1254-
1255- def compute_prevs(self):
1256- # tree_commits maps git urls to Commit objects in the order they were
1257- # built. Notice that it's not only possibly but likely that there
1258- # will be more than one Commit for a given git_commit_id (and hence
1259- # commit_timestamp) because we run builds daily even if the tip hasn't
1260- # changed.
1261-
1262- tree_commits = defaultdict(list)
1263- for day in sorted(self.days, key=lambda day:day._day):
1264- for tree in day.trees:
1265- commits = tree.commits
1266- commits.sort(key=lambda commit: commit.commit_timestamp)
1267- tree_commits[tree._git_url].extend(commits)
1268- for commits_list in tree_commits.values():
1269- if commits_list:
1270- commits_list.sort(key=lambda commit: commit.commit_timestamp)
1271- commits_list[0].prev = None
1272- for i in range(1, len(commits_list)):
1273- commits_list[i].prev = commits_list[i-1]
1274-
1275- for day in self.days:
1276- for tree in day.trees:
1277- for commit in tree.commits:
1278- for build in commit.builds:
1279- for test in build.tests:
1280- test.prev = None
1281- prev_commit = test.build.commit.prev
1282- while prev_commit is not None:
1283- for prev_build in prev_commit.find_builds(test.build._config):
1284- prev_test = prev_build.find_test(test._board_class)
1285- if prev_test:
1286- test.prev = prev_test
1287- break
1288- if test.prev:
1289- break
1290- prev_commit = prev_commit.prev
1291-
1292- def evaluate(self):
1293- connection = DataView.get_connection()
1294- with contextlib.closing(connection.cursor()) as cursor:
1295- cursor.execute(index_sql, (self.start - DAY_DELTA, self.finish))
1296-
1297- for build_result in fetchnamed(cursor):
1298- self.add_build(build_result)
1299-
1300- cursor.execute(find_builds_sql, (self._urls_to_builds.keys(),))
1301-
1302- for test_result in fetchnamed(cursor):
1303- self.add_test(test_result
1304+from django.shortcuts import (
1305+ get_object_or_404,
1306+ render_to_response,
1307+ )
1308+
1309+from lava_server.views import index as lava_index
1310+from lava_server.bread_crumbs import (
1311+ BreadCrumb,
1312+ BreadCrumbTrail,
1313 )
1314- self.compute_prevs()
1315-
1316- self.fill_in_missing_trees()
1317-
1318- def fill_in_missing_trees(self):
1319- for day in self.days:
1320- missing_trees = self.all_trees - set(day.trees)
1321- for tree in missing_trees:
1322- day.get_tree(tree._git_url, self._tree2index[tree._git_url])
1323-
1324- @property
1325- def days(self):
1326- return self._days.values()
1327-
1328- def json_ready(self):
1329- def key(day_obj):
1330- return day_obj._day
1331- day_objs = self._days.values()
1332- day_objs.sort(key=key)
1333- day_objs.reverse()
1334- # Remove the extra day (see __init__ for the comment about this).
1335- del day_objs[-1]
1336- trees = []
1337- for url, index in sorted(self._tree2index.items()):
1338- trees.append({
1339- 'url': url,
1340- 'index': index,
1341- 'width': (100.0 - len(self._tree2index) + 1)/len(self._tree2index)
1342- })
1343- return {
1344- 'days': [day_obj.json_ready() for day_obj in day_objs],
1345- 'trees': trees,
1346- }
1347+
1348+from lava_kernel_ci_views_app.helpers import (
1349+ DayCollection,
1350+ DAY_DELTA,
1351+ WEEK_DELTA,
1352+ )
1353+from lava_kernel_ci_views_app.models import BoardType
1354+
1355+
1356+@BreadCrumb("Kernel CI", parent=lava_index)
1357+def index(request):
1358+ return render_to_response(
1359+ "lava_kernel_ci_views_app/index.html",
1360+ {
1361+ 'board_types': BoardType.objects.all().order_by('slug'),
1362+ 'bread_crumb_trail': BreadCrumbTrail.leading_to(index),
1363+ }, RequestContext(request))
1364
1365
1366 class DateRange(forms.Form):
1367@@ -474,50 +58,107 @@
1368 end = forms.DateField(required=False)
1369
1370
1371-def index(request):
1372-
1373- form = DateRange(request.GET)
1374- form.is_valid()
1375- start = form.cleaned_data['start']
1376- if start is None:
1377- start = datetime.date.today() - WEEK_DELTA + DAY_DELTA
1378-
1379- end = form.cleaned_data['end']
1380- if end is None:
1381- end = datetime.date.today() + DAY_DELTA
1382-
1383- day_collection = DayCollection(start, end)
1384-
1385- day_collection.evaluate()
1386-
1387- link_prefix = reverse(
1388- 'dashboard_app.views.redirect_to_bundle',
1389- kwargs={'content_sha1':'aaaaaaaaaaaaaaaaaaaaaaaaaaaa'}
1390- ).replace('aaaaaaaaaaaaaaaaaaaaaaaaaaaa/', '')
1391-
1392- data = day_collection.json_ready()
1393- newer_results_link = None
1394- if end <= datetime.date.today() + DAY_DELTA:
1395- newer_start = end
1396- newer_end = max(
1397- newer_start + WEEK_DELTA, datetime.date.today())
1398- newer_results_link = (
1399- reverse(index) + '?start=' + newer_start.strftime('%Y-%m-%d') +
1400- '&end=' + newer_end.strftime('%Y-%m-%d'))
1401- older_end = start
1402- older_start = older_end - WEEK_DELTA
1403- older_results_link = (
1404- reverse(index) + '?start=' + older_start.strftime('%Y-%m-%d') +
1405- '&end=' + older_end.strftime('%Y-%m-%d'))
1406- return render_to_response(
1407- "lava_kernel_ci_views_app/index.html",
1408- {
1409- 'data': data,
1410- 'link_prefix': link_prefix,
1411- 'board_classes': sorted(day_collection._board_classes),
1412- 'configs': sorted(day_collection._configs),
1413- 'trees': sorted(day_collection._tree2index.items()),
1414- 'newer_results_link': newer_results_link,
1415- 'older_results_link': older_results_link,
1416- }, RequestContext(request))
1417-
1418+@BreadCrumb("Tests run on {board_desc}", parent=index)
1419+def per_board(request, board_type):
1420+
1421+ board_type = get_object_or_404(BoardType, slug=board_type)
1422+
1423+ form = DateRange(request.GET)
1424+ form.is_valid()
1425+ start = form.cleaned_data['start']
1426+ if start is None:
1427+ start = datetime.date.today() - WEEK_DELTA + DAY_DELTA
1428+
1429+ end = form.cleaned_data['end']
1430+ if end is None:
1431+ end = datetime.date.today() + DAY_DELTA
1432+
1433+ day_collection = DayCollection(board_type.slug, start, end)
1434+
1435+ day_collection.evaluate(fetch_builds=True)
1436+
1437+ link_prefix = reverse(
1438+ 'dashboard_app.views.redirect_to_bundle',
1439+ kwargs={'content_sha1':'aaaaaaaaaaaaaaaaaaaaaaaaaaaa'}
1440+ ).replace('aaaaaaaaaaaaaaaaaaaaaaaaaaaa/', '')
1441+
1442+ data = day_collection.json_ready()
1443+ newer_results_link = None
1444+ self_link = reverse(per_board, kwargs=dict(board_type=board_type.slug))
1445+ if end <= datetime.date.today() + DAY_DELTA:
1446+ newer_start = end
1447+ newer_end = max(
1448+ newer_start + WEEK_DELTA, datetime.date.today())
1449+ newer_results_link = (
1450+ self_link + '?start=' + newer_start.strftime('%Y-%m-%d') +
1451+ '&end=' + newer_end.strftime('%Y-%m-%d'))
1452+ older_end = start
1453+ older_start = older_end - WEEK_DELTA
1454+ older_results_link = (
1455+ self_link + '?start=' + older_start.strftime('%Y-%m-%d') +
1456+ '&end=' + older_end.strftime('%Y-%m-%d'))
1457+ return render_to_response(
1458+ "lava_kernel_ci_views_app/compile_only.html",
1459+ {
1460+ 'data': data,
1461+ 'link_prefix': link_prefix,
1462+ 'newer_results_link': newer_results_link,
1463+ 'older_results_link': older_results_link,
1464+ 'bread_crumb_trail': BreadCrumbTrail.leading_to(
1465+ per_board, board_desc=board_type.display_name),
1466+ 'include_tests': True,
1467+ }, RequestContext(request))
1468+
1469+
1470+@BreadCrumb("Compile-only Results", parent=index)
1471+def compile_view(request):
1472+
1473+ form = DateRange(request.GET)
1474+ form.is_valid()
1475+ start = form.cleaned_data['start']
1476+ if start is None:
1477+ start = datetime.date.today() - WEEK_DELTA + DAY_DELTA
1478+
1479+ end = form.cleaned_data['end']
1480+ if end is None:
1481+ end = datetime.date.today() + DAY_DELTA
1482+
1483+ day_collection = DayCollection(None, start, end)
1484+
1485+ day_collection.evaluate(fetch_builds=False)
1486+
1487+ link_prefix = reverse(
1488+ 'dashboard_app.views.redirect_to_bundle',
1489+ kwargs={'content_sha1':'aaaaaaaaaaaaaaaaaaaaaaaaaaaa'}
1490+ ).replace('aaaaaaaaaaaaaaaaaaaaaaaaaaaa/', '')
1491+
1492+ data = day_collection.json_ready()
1493+ newer_results_link = None
1494+ self_link = reverse(compile_view)
1495+ if end <= datetime.date.today() + DAY_DELTA:
1496+ newer_start = end
1497+ newer_end = max(
1498+ newer_start + WEEK_DELTA, datetime.date.today())
1499+ newer_results_link = (
1500+ self_link + '?start=' + newer_start.strftime('%Y-%m-%d') +
1501+ '&end=' + newer_end.strftime('%Y-%m-%d'))
1502+ older_end = start
1503+ older_start = older_end - WEEK_DELTA
1504+ older_results_link = (
1505+ self_link + '?start=' + older_start.strftime('%Y-%m-%d') +
1506+ '&end=' + older_end.strftime('%Y-%m-%d'))
1507+ return render_to_response(
1508+ "lava_kernel_ci_views_app/per_board.html",
1509+ {
1510+ 'data': data,
1511+ 'link_prefix': link_prefix,
1512+ 'newer_results_link': newer_results_link,
1513+ 'older_results_link': older_results_link,
1514+ 'bread_crumb_trail': BreadCrumbTrail.leading_to(compile_view),
1515+ 'include_tests': False,
1516+ }, RequestContext(request))
1517+
1518+def board_icon(request, board_type):
1519+ board_type = get_object_or_404(BoardType, slug=board_type)
1520+ response = HttpResponse(board_type.icon, content_type='image/png')
1521+ return response

Subscribers

People subscribed via source and target branches

to all changes: